dragonegg in FSF gcc?

dragonegg in FSF gcc?

[Jack Howarth]

   What are the opinions here about merging
dragonegg into FSF gcc? It is in the odd position
of straddling two projects so perhaps it could
reside in both the LLVM and FSF gcc projects
with regularly remerging. Certainly it would be
an interesting addition to FSF gcc. For instance,
without even attempting to target gfortran, in
the last year, the llvm code performance has
improved about 15% for the Polyhedron 2005
benchmarks...

http://lists.cs.uiuc.edu/pipermail/llvmdev/2010-April/030800.html

For the darwin target, this would be a particularly
big win since we would gain access to the llvm LTO.
Thanks in advance for any comments.
                        Jack

Re: dragonegg in FSF gcc?

[Basile Starynkevitch]

Jack Howarth wrote:
>    What are the opinions here about merging
> dragonegg into FSF gcc? It is in the odd position
> of straddling two projects so perhaps it could
> reside in both the LLVM and FSF gcc projects
> with regularly remerging. Certainly it would be
> an interesting addition to FSF gcc. 

What are the alternatives?

I tend to be quite happy with the idea of dragonegg being a good GCC 
plugin, since it is a good illustration of the plugin feature.

What exactly is wrong with the approach of keeping dragonegg as a GCC 
plugin? In principle, I like it very much.

(I admit I did not compile dragonegg, so maybe there are issues in using 
it as a plugin).

And I could imagine that the difference in the licenses and the 
copyright owners between DragonEgg & GCC might make the merging legally 
difficult, but of course I am not a lawyer and know nothing about these 
issues.

Regards
-- 
Basile STARYNKEVITCH         http://starynkevitch.net/Basile/
email: basile<at>starynkevitch<dot>net mobile: +33 6 8501 2359
8, rue de la Faiencerie, 92340 Bourg La Reine, France
*** opinions {are only mines, sont seulement les miennes} ***

Re: dragonegg in FSF gcc?

[Duncan Sands]

Hi Basile,

> I tend to be quite happy with the idea of dragonegg being a good GCC
> plugin, since it is a good illustration of the plugin feature.

I think Jack wasn't suggesting that dragonegg should be changed to not be
a plugin any more.  I think he was suggesting that it should live in the gcc
repository rather than the LLVM repository.

Ciao,

Duncan.

Re: dragonegg in FSF gcc?

[Steven Bosscher]

On Sat, Apr 10, 2010 at 3:03 PM, Duncan Sands <baldrick@free.fr> wrote:
> Hi Basile,
>
>> I tend to be quite happy with the idea of dragonegg being a good GCC
>> plugin, since it is a good illustration of the plugin feature.
>
> I think Jack wasn't suggesting that dragonegg should be changed to not be
> a plugin any more.  I think he was suggesting that it should live in the gcc
> repository rather than the LLVM repository.

So, no offense, but the suggestion here is to make this subversive
(for FSF GCC) plugin part of FSF GCC? What is the benefit of this for
GCC? I don't see any. I just see a plugin trying to piggy-back on the
hard work of GCC front-end developers and negating the efforts of
those working on the middle ends and back ends.

Ciao!
Steven

Re: dragonegg in FSF gcc?

[Duncan Sands]

Hi Steven,

>> I think Jack wasn't suggesting that dragonegg should be changed to not be
>> a plugin any more.  I think he was suggesting that it should live in the gcc
>> repository rather than the LLVM repository.
>
> So, no offense, but the suggestion here is to make this subversive
> (for FSF GCC) plugin part of FSF GCC? What is the benefit of this for
> GCC? I don't see any. I just see a plugin trying to piggy-back on the
> hard work of GCC front-end developers and negating the efforts of
> those working on the middle ends and back ends.

I'm sorry you see the dragonegg project so negatively.  I think it is useful
for gcc (though not hugely useful), since it makes it easy to compare the gcc
and LLVM optimizers and code generators, not to mention the gcc and LLVM
approaches to LTO.  If LLVM manages to produce better code than gcc for some
testcase, then it is a convenient tool for the gcc devs to find out why, and
improve gcc.  If gcc is consistently better than LLVM then there's nothing to
worry about!  Of course, right now it is LLVM that is mostly playing catchup
with gcc, so for the moment it is principally the LLVM devs that get to learn
from gcc, but as LLVM improves the other direction is likely to occur more
often.

As for "negating the efforts of those working on the middle ends and back ends",
would you complain if someone came up with a new register allocator because it
negates the efforts of those who work on the old one?  If LLVM is technically
superior, then that's a fact and a good thing, not subversion, and hopefully
will encourage the gcc devs to either improve gcc or migrate to LLVM.  If GCC
is technically superior, then hopefully the dragonegg project will help people
see this, by making it easier to compare the two technologies, and result in
them giving up on LLVM and working on or using gcc instead.

In my opinion a bit of friendly competition from LLVM is on the whole a good
thing for gcc.

That said, maybe your worry is that dragonegg makes it easier to undermine the
GPL, or perhaps you don't like LLVM's BSD style license?  I really have no
understanding of the legal issues involved with "undermining the GPL", but I
know that some of the gcc devs have thought hard about this so perhaps they can
comment.  I'm personally not at all interested in undermining the GPL.  As for
licenses, the dragonegg plugin, as a combined work of GPLv3 code (gcc), GPLv2
or later (the plugin) and GPL compatible code (LLVM), is as far as I can see
GPLv3 and as such no different to gcc itself.

Finally, I don't see much point in dragonegg being moved to the gcc repository.
It wasn't I who suggested it.

Ciao,

Duncan.

Re: dragonegg in FSF gcc?

[Jack Howarth]

On Sun, Apr 11, 2010 at 01:19:06PM +0200, Steven Bosscher wrote:
> On Sat, Apr 10, 2010@3:03 PM, Duncan Sands <baldrick@free.fr> wrote:
> > Hi Basile,
> >
> >> I tend to be quite happy with the idea of dragonegg being a good GCC
> >> plugin, since it is a good illustration of the plugin feature.
> >
> > I think Jack wasn't suggesting that dragonegg should be changed to not be
> > a plugin any more.  I think he was suggesting that it should live in the gcc
> > repository rather than the LLVM repository.
> 
> So, no offense, but the suggestion here is to make this subversive
> (for FSF GCC) plugin part of FSF GCC? What is the benefit of this for
> GCC? I don't see any. I just see a plugin trying to piggy-back on the
> hard work of GCC front-end developers and negating the efforts of
> those working on the middle ends and back ends.
> 
> Ciao!
> Steven
>                

Steven,
   Invoking the term 'subversive' seems rather strong
for utilizing a feature added by the FSF gcc developers
themselves. Rather than viewing dragon-egg as some sort
of lamprey which is feeding off of the FSF gcc project,
we should welcome the competition from a direct comparison
of alternative back/middle ends (not fear it).
   One could also make an argument that it is in the best
interest of FSF gcc to do so. Isn't better to keep all
of the alternative front-end developers (gfortran, ada,
etc) within the FSF gcc tent rather than forcing the
creation of competing clang fortran and ada projects.
Your view seems a tad short-sighted.
         Jack
ps I've watched FSF gcc development for awhile now
and have become a bit concerned that it is slowing
tending towards a gnu-linux mono-culture (through
no real fault of its own). There should be every effort
made to keep as many alternative platforms in the
picture (even if these end up being supported through
plugins).

Re: dragonegg in FSF gcc?

[Jack Howarth]

Steven,
   One other comment. I've felt for awhile that a major
strength of FSF gcc was the fact that its support for
so many targets insured that latent bugs tended to be
found in the compiler. Likewise graphite has recently 
exposed certain latent bugs as well. Why should we not
expect the same to be true for the front-end if an
alternative middle/end is used via dragon-egg? It may
well tickle unique flaws in the front-end.
             Jack

Re: dragonegg in FSF gcc?

[Eric Botcazou]

> As for "negating the efforts of those working on the middle ends and back
> ends", would you complain if someone came up with a new register allocator
> because it negates the efforts of those who work on the old one?  If LLVM
> is technically superior, then that's a fact and a good thing, not
> subversion, and hopefully will encourage the gcc devs to either improve gcc
> or migrate to LLVM.

Well, the last point is very likely precisely what Steven is talking about.
GCC doesn't have to shoot itself in the foot by encouraging its developers to 
migrate to LLVM.

-- 
Eric Botcazou

Re: dragonegg in FSF gcc?

[David Edelsohn]

On Sun, Apr 11, 2010 at 10:30 AM, Jack Howarth <howarth@bromo.med.uc.edu> wrote:
> Steven,
>   One other comment. I've felt for awhile that a major
> strength of FSF gcc was the fact that its support for
> so many targets insured that latent bugs tended to be
> found in the compiler. Likewise graphite has recently
> exposed certain latent bugs as well. Why should we not
> expect the same to be true for the front-end if an
> alternative middle/end is used via dragon-egg? It may
> well tickle unique flaws in the front-end.

Jack,

The Graphite project and the various GCC targets participate in GCC
development.  Helping fix GCC bugs affecting those features, supports
and grows the GCC developer base.  There needs to be some mutualistic
relationship.  I don't see members of the LLVM community arguing that
they should contribute to GCC to improve performance comparisons.

As Steven mentioned, LLVM has been extremely effective at utilizing
FSF technology while its community complains about the FSF, GCC, GCC's
leadership and GCC's developer community.  If GCC is so helpful and
useful and effective, then work on it as well and give it credit; if
GCC is so bad, then why rely on it?  The rhetoric is disconnected from
the actions.

David

Re: dragonegg in FSF gcc?

[Jack Howarth]

On Sun, Apr 11, 2010 at 10:56:55AM -0400, David Edelsohn wrote:
> On Sun, Apr 11, 2010 at 10:30 AM, Jack Howarth <howarth@bromo.med.uc.edu> wrote:
> > Steven,
> >   One other comment. I've felt for awhile that a major
> > strength of FSF gcc was the fact that its support for
> > so many targets insured that latent bugs tended to be
> > found in the compiler. Likewise graphite has recently
> > exposed certain latent bugs as well. Why should we not
> > expect the same to be true for the front-end if an
> > alternative middle/end is used via dragon-egg? It may
> > well tickle unique flaws in the front-end.
> 
> Jack,
> 
> The Graphite project and the various GCC targets participate in GCC
> development.  Helping fix GCC bugs affecting those features, supports
> and grows the GCC developer base.  There needs to be some mutualistic
> relationship.  I don't see members of the LLVM community arguing that
> they should contribute to GCC to improve performance comparisons.
> 
> As Steven mentioned, LLVM has been extremely effective at utilizing
> FSF technology while its community complains about the FSF, GCC, GCC's
> leadership and GCC's developer community.  If GCC is so helpful and
> useful and effective, then work on it as well and give it credit; if
> GCC is so bad, then why rely on it?  The rhetoric is disconnected from
> the actions.
> 
> David

David,
   While this may all be true, dragon-egg does represent an opportunity
for the two communities to engage each other in a cooperative endeaver
rather than retreating into competing camps. I still believe, for the
secondary language front-ends in FSF gcc, forcing their development
communities to eventually fork into the same competing camps (rather
than pool efforts in the same front-end) will not be in the best
interests of either group.
            Jack

Re: dragonegg in FSF gcc?

[Manuel López-Ibáñez]

On 11 April 2010 16:17, Jack Howarth <howarth@bromo.med.uc.edu> wrote:
> ps I've watched FSF gcc development for awhile now
> and have become a bit concerned that it is slowing
> tending towards a gnu-linux mono-culture (through
> no real fault of its own). There should be every effort
> made to keep as many alternative platforms in the
> picture (even if these end up being supported through
> plugins).

Do you have any real fact or measure that substantiates such claim? Or
is this just a "feeling"?

I can think many reasons why such thing would happen but I would like
to know how are you sure that it is happening in the first place.

Cheers,

Manuel.

Re: dragonegg in FSF gcc?

[Duncan Sands]

Hi Eric,

>> As for "negating the efforts of those working on the middle ends and back
>> ends", would you complain if someone came up with a new register allocator
>> because it negates the efforts of those who work on the old one?  If LLVM
>> is technically superior, then that's a fact and a good thing, not
>> subversion, and hopefully will encourage the gcc devs to either improve gcc
>> or migrate to LLVM.
>
> Well, the last point is very likely precisely what Steven is talking about.
> GCC doesn't have to shoot itself in the foot by encouraging its developers to
> migrate to LLVM.

I hope it was clear from my email that by "gcc" I was talking about the gcc
optimizers and code generators and not the gcc frontends.  If the dragonegg
project shows that feeding the output of the gcc frontends into the LLVM
optimizers and code generators results in better code, then gcc can always
change to using the LLVM optimizers and code generators, resulting in a better
compiler.  I don't see how this is gcc the compiler shooting itself in the foot.

Of course, some gcc devs have invested a lot in the gcc middle and back ends,
and moving to LLVM might be personally costly for them.  Thus they might be
shooting themselves in the foot by helping the LLVM project, but this should
not be confused with gcc the compiler shooting itself in the foot.

All this is predicated on gcc-frontends+LLVM producing better code than the
current gcc-frontends+gcc-middle/backends.  As I mentioned, dragonegg makes
it easier, even trivial, to test this.  So those who think that LLVM is all
hype should be cheering on the dragonegg project, because now they have a
great way to prove that gcc does a better job!

Ciao,

Duncan.

Re: dragonegg in FSF gcc?

[Robert Dewar]

Duncan Sands wrote:

> I hope it was clear from my email that by "gcc" I was talking about the gcc
> optimizers and code generators and not the gcc frontends.  If the dragonegg
> project shows that feeding the output of the gcc frontends into the LLVM
> optimizers and code generators results in better code, then gcc can always
> change to using the LLVM optimizers and code generators, resulting in a better
> compiler.  I don't see how this is gcc the compiler shooting itself in the foot.

Note that better code is just one aspect of compiler quality :-) One 
should be sure to make this judgment over a significant body of code,
and not just on some bogus benchmarks :-)

RE: dragonegg in FSF gcc?

[Grigori Fursin]

Hello,

Hope my question will not completely divert the topic of this discussion -
just curious what do you mean by better code? Better execution time, code size, 
compilation time?..

If yes, then why not to compare different compilers by just compiling multiple programs
with GCC, LLVM, Open64, ICC, etc, separately to compare those characteristics and then 
find missing optimizations or better combinations of optimizations to achieve the result? 

By the way, using iterative feedback-directed compilation (searching for best combinations 
of optimizations) can considerably improve default optimization heuristic of nearly any 
compiler (look at ACOVEA or cTuning.org results) so it may not be so straightforward
to answer a question which compiler is better when just using default optimization heuristic ...

Cheers,
Grigori

*************************************************************
Grigori Fursin, Exascale Research Center, France
http://unidapt.org/people/gfursin
*************************************************************




-----Original Message-----
From: gcc-owner@gcc.gnu.org [mailto:gcc-owner@gcc.gnu.org] On Behalf Of Duncan Sands
Sent: Sunday, April 11, 2010 5:36 PM
To: Eric Botcazou
Cc: gcc@gcc.gnu.org; Steven Bosscher
Subject: Re: dragonegg in FSF gcc?

Hi Eric,

>> As for "negating the efforts of those working on the middle ends and back
>> ends", would you complain if someone came up with a new register allocator
>> because it negates the efforts of those who work on the old one?  If LLVM
>> is technically superior, then that's a fact and a good thing, not
>> subversion, and hopefully will encourage the gcc devs to either improve gcc
>> or migrate to LLVM.
>
> Well, the last point is very likely precisely what Steven is talking about.
> GCC doesn't have to shoot itself in the foot by encouraging its developers to
> migrate to LLVM.

I hope it was clear from my email that by "gcc" I was talking about the gcc
optimizers and code generators and not the gcc frontends.  If the dragonegg
project shows that feeding the output of the gcc frontends into the LLVM
optimizers and code generators results in better code, then gcc can always
change to using the LLVM optimizers and code generators, resulting in a better
compiler.  I don't see how this is gcc the compiler shooting itself in the foot.

Of course, some gcc devs have invested a lot in the gcc middle and back ends,
and moving to LLVM might be personally costly for them.  Thus they might be
shooting themselves in the foot by helping the LLVM project, but this should
not be confused with gcc the compiler shooting itself in the foot.

All this is predicated on gcc-frontends+LLVM producing better code than the
current gcc-frontends+gcc-middle/backends.  As I mentioned, dragonegg makes
it easier, even trivial, to test this.  So those who think that LLVM is all
hype should be cheering on the dragonegg project, because now they have a
great way to prove that gcc does a better job!

Ciao,

Duncan.

Re: dragonegg in FSF gcc?

[Robert Dewar]

Grigori Fursin wrote:
> Hello,
> 
> Hope my question will not completely divert the topic of this discussion -
> just curious what do you mean by better code? Better execution time, code size, 
> compilation time?..

Could mean all these things as well as other issues

a) better realiability
b) better behavior for undefined cases
c) better source-object mapping for coverage/certification purposes
d) better predictability
e) better debugging information

I am sure I could thing up many additions to this list if I
spent more time :-)

And of course there are many other quality issues for compilers
that do not come under the "better code" category. Comparing
compilers is not an easy thing to do :-)

Sure you can run some benchmarks and look for missed optimization
opportunities, that's always worth while, for instance people
regularly compare gcc and icc to look for cases where the gcc
optimization can be improved

Robert Dewar

Re: dragonegg in FSF gcc?

[Duncan Sands]

Hi David,

> The Graphite project and the various GCC targets participate in GCC
> development.  Helping fix GCC bugs affecting those features, supports
> and grows the GCC developer base.  There needs to be some mutualistic
> relationship.  I don't see members of the LLVM community arguing that
> they should contribute to GCC to improve performance comparisons.

as I mentioned in my email, I see dragonegg as being a useful tool for
comparing the gcc and LLVM optimizers and code generators.  That sounds
like the kind of thing you are asking for, but perhaps I misunderstood?

> As Steven mentioned, LLVM has been extremely effective at utilizing
> FSF technology while its community complains about the FSF, GCC, GCC's
> leadership and GCC's developer community.

It is true that plenty of people disaffected with gcc can be found in the
LLVM community.  Dislike of gcc or its license seems a common motivation
for looking into the clang compiler for example.  It seems to me that this
is a natural phenomenon - where else would such people go?  It would be a
mistake to think that the LLVM community consists principally of gcc haters
though.

If GCC is so helpful and
> useful and effective, then work on it as well and give it credit; if
> GCC is so bad, then why rely on it?  The rhetoric is disconnected from
> the actions.

I'm not sure what you mean.  Working on an LLVM middle-end/back-end for
gcc doesn't mean I despise the gcc middle-end and back-ends, it just means
that I think this is an interesting project with the potential to result
in a better gcc in the long term.

Ciao,

Duncan.

Re: dragonegg in FSF gcc?

[Jack Howarth]

On Sun, Apr 11, 2010 at 06:02:38PM +0200, Duncan Sands wrote:
>> useful and effective, then work on it as well and give it credit; if
>> GCC is so bad, then why rely on it?  The rhetoric is disconnected from
>> the actions.
>
> I'm not sure what you mean.  Working on an LLVM middle-end/back-end for
> gcc doesn't mean I despise the gcc middle-end and back-ends, it just means
> that I think this is an interesting project with the potential to result
> in a better gcc in the long term.
>
> Ciao,
>
> Duncan.

  I would also add that some of this seems like deja vu from the
egcs days. Granted it is extremely unlikely, but who is to say that
at some future date, if the license conflicts subside, that FSF gcc
might decide that llvm wasn't so bad for the middle/back-ends. Wouldn't
having the front-end running entirely via the plugin be a huge help
in that case.
            Jack

Re: dragonegg in FSF gcc?

[Duncan Sands]

Hi Grigori,

> Hope my question will not completely divert the topic of this discussion -
> just curious what do you mean by better code? Better execution time, code size,
> compilation time?..

this depends on each persons needs of course.  The dragonegg plugin makes it
easy for people to see if the LLVM optimizers and code generators are helpful
for their projects.  Evaluating whether replacing whole-sale the gcc middle and
backends with LLVM (which I consider pretty unlikely) is an overall win is much
harder, but I doubt anyone on this mailing list needs to be told that.

> If yes, then why not to compare different compilers by just compiling multiple programs
> with GCC, LLVM, Open64, ICC, etc, separately to compare those characteristics and then
> find missing optimizations or better combinations of optimizations to achieve the result?

how do you compile a program with LLVM?  It's not a compiler, it's a set of
optimization and codegen libraries.  You also need a front-end, which takes
the users code and turns it into the LLVM intermediate representation [IR].  The
dragonegg plugin takes the output of the gcc-4.5 front-ends, turns it into LLVM
IR and runs the LLVM optimizers and code generators on it.  In other words, it
is exactly what you need in order to compile programs with LLVM.  There is also
llvm-gcc, which is a hacked version of gcc-4.2 that does much the same thing,
and for C and C++ there is now the clang front-end to LLVM.  The big advantage
of dragonegg is that it isolates the effect of the LLVM optimizers and code
generators by removing the effect of having a different front-end.  For example,
if llvm-gcc produces slower code than gcc-4.5, this might be due to front-end
changes between gcc-4.2 and gcc-4.5 rather than because the gcc optimizers are
doing a better job.  This confounding factor goes away with the dragonegg
plugin.

Ciao,

Duncan.

Re: dragonegg in FSF gcc?

[Robert Dewar]

Duncan Sands wrote:

> how do you compile a program with LLVM?  It's not a compiler, it's a set of
> optimization and codegen libraries.  You also need a front-end, which takes
> the users code and turns it into the LLVM intermediate representation [IR].  The
> dragonegg plugin takes the output of the gcc-4.5 front-ends, turns it into LLVM
> IR and runs the LLVM optimizers and code generators on it.  In other words, it
> is exactly what you need in order to compile programs with LLVM.  There is also
> llvm-gcc, which is a hacked version of gcc-4.2 that does much the same thing,
> and for C and C++ there is now the clang front-end to LLVM.  The big advantage
> of dragonegg is that it isolates the effect of the LLVM optimizers and code
> generators by removing the effect of having a different front-end.  For example,
> if llvm-gcc produces slower code than gcc-4.5, this might be due to front-end
> changes between gcc-4.2 and gcc-4.5 rather than because the gcc optimizers are
> doing a better job.  This confounding factor goes away with the dragonegg
> plugin.

Goes away is a bit strong. In practice, front ends know about their back 
ends and are tuned in various ways for things to work well.
> 
> Ciao,
> 
> Duncan.

Re: dragonegg in FSF gcc?

[Duncan Sands]

Hi Robert,

> b) better behavior for undefined cases

this is one of the problems with using LLVM with the Ada front-end.  LLVM makes
pretty aggressive deductions when it sees undefined behaviour, which can result
in (for example) validity checks being removed exactly in the cases when they
are most needed.  There are various ways of solving this problem, but I didn't
implement any of them yet.

Ciao,

Duncan.

Re: dragonegg in FSF gcc?

[Robert Dewar]

Duncan Sands wrote:
> Hi Robert,
> 
>> b) better behavior for undefined cases
> 
> this is one of the problems with using LLVM with the Ada front-end.  LLVM makes
> pretty aggressive deductions when it sees undefined behaviour, which can result
> in (for example) validity checks being removed exactly in the cases when they
> are most needed.  There are various ways of solving this problem, but I didn't
> implement any of them yet.

Validity models are tricky. The model in Ada is importantly different
from the model in C, and indeed a C back end is likely to have troubles
in this area. We have had to step carefully in the gcc back end to get
proper Ada semantics, and there are still some compromises (not ones
that affect functionality, but ones that affect efficiency).
> 
> Ciao,
> 
> Duncan.

Re: dragonegg in FSF gcc?

[Basile Starynkevitch]

Duncan Sands wrote:
> 
> In my opinion a bit of friendly competition from LLVM is on the whole a 
> good thing for gcc.
> 

I definitely agree with that position. Real competition between LLVM & 
GCC is good for both projects, and is good for free software as a whole.

Cheers.


-- 
Basile STARYNKEVITCH         http://starynkevitch.net/Basile/
email: basile<at>starynkevitch<dot>net mobile: +33 6 8501 2359
8, rue de la Faiencerie, 92340 Bourg La Reine, France
*** opinions {are only mines, sont seulement les miennes} ***

Re: dragonegg in FSF gcc?

[Dave Korn]

On 11/04/2010 16:23, Manuel López-Ibáñez wrote:
> On 11 April 2010 16:17, Jack Howarth wrote:
>> ps I've watched FSF gcc development for awhile now
>> and have become a bit concerned that it is slowing
>> tending towards a gnu-linux mono-culture (through
>> no real fault of its own). There should be every effort
>> made to keep as many alternative platforms in the
>> picture (even if these end up being supported through
>> plugins).
> 
> Do you have any real fact or measure that substantiates such claim? Or
> is this just a "feeling"?

  Here's a very crude indicator:

> $ wget http://gcc.gnu.org/ml/gcc-testresults/2010-04/
> --2010-04-11 17:45:09--  http://gcc.gnu.org/ml/gcc-testresults/2010-04/
> Resolving gcc.gnu.org... 209.132.180.131
> Connecting to gcc.gnu.org|209.132.180.131|:80... connected.
> HTTP request sent, awaiting response... 200 OK
> Length: 245044 (239K) [text/html]
> Saving to: `index.html.6'
> 
> 100%[======================================>] 245,044     91.5K/s   in 2.6s
> 
> 2010-04-11 17:45:12 (91.5 KB/s) - `index.html.6' saved [245044/245044]
> 
> 
> $ grep 'Results' index.html.6 > results
> 
> $ wc -l results
> 753 results
> 
> $ grep -i linux results  | wc -l
> 482
> 
> $ grep -vi linux results  | wc -l
> 271
> 
> $

  Grepping the -patches archives to see which platforms submitted patches get
testing on would also be interesting, but somewhat harder owing to the more
free-form nature of the text there.  Still, a two-to-one ratio of linux to
rest-of-the-world would be in line with my subjective impression: it's not
overwhelming the rest, but it's substantially the best tended-to.

  So, I certainly have the same feeling, but I think it's just inevitable that
the most popular platform gets the most support.

    cheers,
      DaveK

Re: dragonegg in FSF gcc?

[Duncan Sands]

> Goes away is a bit strong. In practice, front ends know about their back
> ends and are tuned in various ways for things to work well.

Likewise, back-ends are tuned for their front-ends.

Ciao,

Duncan.

RE: dragonegg in FSF gcc?

[Grigori Fursin]

Hi Duncan,

>how do you compile a program with LLVM?  It's not a compiler, it's a set of
>optimization and codegen libraries.  You also need a front-end, which takes
>the users code and turns it into the LLVM intermediate representation [IR].  The
>dragonegg plugin takes the output of the gcc-4.5 front-ends, turns it into LLVM
>IR and runs the LLVM optimizers and code generators on it.  In other words, it
>is exactly what you need in order to compile programs with LLVM.  There is also
>llvm-gcc, which is a hacked version of gcc-4.2 that does much the same thing,
>and for C and C++ there is now the clang front-end to LLVM.  The big advantage
>of dragonegg is that it isolates the effect of the LLVM optimizers and code
>generators by removing the effect of having a different front-end.  For example,
>if llvm-gcc produces slower code than gcc-4.5, this might be due to front-end
>changes between gcc-4.2 and gcc-4.5 rather than because the gcc optimizers are
>doing a better job.  This confounding factor goes away with the dragonegg
>plugin.

Ok. I see what you mean. We simply used llvm-gcc so that's why the confusion ;) ...
Cheers,
Grigori

Re: dragonegg in FSF gcc?

[Robert Dewar]

Duncan Sands wrote:
>> Goes away is a bit strong. In practice, front ends know about their back
>> ends and are tuned in various ways for things to work well.
> 
> Likewise, back-ends are tuned for their front-ends.
> 
> Ciao,
> 
> Duncan.

Yes indeed, in particular, there is often a substantial covert
channel between the front end and back end (information passed
with restrictions or details that are not properly documented).
We try to avoid this for GNAT, but not with 100% success.

Re: dragonegg in FSF gcc?

[Andi Kleen]

Robert Dewar <dewar@adacore.com> writes:
>
> Sure you can run some benchmarks and look for missed optimization
> opportunities, that's always worth while, for instance people
> regularly compare gcc and icc to look for cases where the gcc
> optimization can be improved

OT, but there's lots of cool data on all of this on 

http://embed.cs.utah.edu/embarrassing/dec_09/

I spent some time some time ago to file a few "missed optimizations"
bugzillas based on examples there and some got addressed. I'm sure
there are lots more nuggets in there (as in easy cases to improve
gcc), but analyzing each example takes time.

The reason each sample takes time to analyze is that it is sometimes
a bug in the other compiler or different target options or so.
Sometimes it's also undefined code.

Yes there are plenty of bugs in there, but I didn't find any for gcc
at least (but only looked at a relatively limited set)

-Andi

-- 
ak@linux.intel.com -- Speaking for myself only.

Re: dragonegg in FSF gcc?

[Toon Moene]

Duncan Sands wrote:

> I hope it was clear from my email that by "gcc" I was talking about the gcc
> optimizers and code generators and not the gcc frontends.  If the dragonegg
> project shows that feeding the output of the gcc frontends into the LLVM
> optimizers and code generators results in better code, then gcc can always
> change to using the LLVM optimizers and code generators, resulting in a 
> better
> compiler.  I don't see how this is gcc the compiler shooting itself in 
> the foot.

Someone already showed a couple of years ago that if you write a 
"backend" that generates C code, and feed that code back into GCC, you 
can get a 2 times speedup on some Fortran code.

This excercise is not useful, unless you can point out exactly what's 
wrong with today's GCC optimization passes.

Cheers,

-- 
Toon Moene - e-mail: toon@moene.org - phone: +31 346 214290
Saturnushof 14, 3738 XG  Maartensdijk, The Netherlands
At home: http://moene.org/~toon/
Progress of GNU Fortran: http://gcc.gnu.org/gcc-4.5/changes.html

Re: dragonegg in FSF gcc?

[Laurent GUERBY]

On Sun, 2010-04-11 at 17:50 +0100, Dave Korn wrote:
> On 11/04/2010 16:23, Manuel López-Ibáñez wrote:
> > On 11 April 2010 16:17, Jack Howarth wrote:
> >> ps I've watched FSF gcc development for awhile now
> >> and have become a bit concerned that it is slowing
> >> tending towards a gnu-linux mono-culture (through
> >> no real fault of its own). There should be every effort
> >> made to keep as many alternative platforms in the
> >> picture (even if these end up being supported through
> >> plugins).
> > 
> > Do you have any real fact or measure that substantiates such claim? Or
> > is this just a "feeling"?
> 
>   Here's a very crude indicator:
> 
> > $ wget http://gcc.gnu.org/ml/gcc-testresults/2010-04/
> > --2010-04-11 17:45:09--  http://gcc.gnu.org/ml/gcc-testresults/2010-04/
> > Resolving gcc.gnu.org... 209.132.180.131
> > Connecting to gcc.gnu.org|209.132.180.131|:80... connected.
> > HTTP request sent, awaiting response... 200 OK
> > Length: 245044 (239K) [text/html]
> > Saving to: `index.html.6'
> > 
> > 100%[======================================>] 245,044     91.5K/s   in 2.6s
> > 
> > 2010-04-11 17:45:12 (91.5 KB/s) - `index.html.6' saved [245044/245044]
> > 
> > 
> > $ grep 'Results' index.html.6 > results
> > 
> > $ wc -l results
> > 753 results
> > 
> > $ grep -i linux results  | wc -l
> > 482
> > 
> > $ grep -vi linux results  | wc -l
> > 271
> > 
> > $
> 
>   Grepping the -patches archives to see which platforms submitted patches get
> testing on would also be interesting, but somewhat harder owing to the more
> free-form nature of the text there.  Still, a two-to-one ratio of linux to
> rest-of-the-world would be in line with my subjective impression: it's not
> overwhelming the rest, but it's substantially the best tended-to.
> 
>   So, I certainly have the same feeling, but I think it's just inevitable that
> the most popular platform gets the most support.

Quick grepping on 2010-03 gcc-testresults shows more than 70% of the
results are submitted by only 5 sources:

    106 ghazi
    267 regress at geoffk dot org
    333 Laurent GUERBY
    387 Mike Stein
    905 H.J. Lu

Of these five sources three are running on the GCC compile farm (which
BTW is not limited to GCC testing :). The compile farm operates mostly
out of donated hardware and hosting. We currently cover all non
proprietary OS primary and secondary platforms except s390-linux. 

We're of course open to accept proprietary platforms donations (machine
and/or hosting) provided they come with the right to open public shell
accounts on them (which is the way we operate the project, ~ 160
accounts to date).

Sincerely,

Laurent
http://gcc.gnu.org/wiki/CompileFarm

Re: dragonegg in FSF gcc?

[Eric Botcazou]

> I don't see how this is gcc the compiler shooting itself in the foot.

Simply because LLVM isn't part of the GNU project.

-- 
Eric Botcazou

Re: dragonegg in FSF gcc?

[Manuel López-Ibáñez]

On 11 April 2010 18:50, Dave Korn <dave.korn.cygwin@googlemail.com> wrote:
>
>  Here's a very crude indicator:

No, it is not. Apart from all the points that Laurent raised in a
previous email, lack of test results in some platforms does not mean
that GCC developers are uninterested on supporting those platforms and
much less that they are against supporting those platforms. The GCC
community haven't forbidden anyone from contributing to support any
platform in GCC.

GCC supports whatever the people that are willing to contribute (or
are paid to contribute) want to support. Exactly like LLVM. It is easy
to see why gnu/linux is more supported than other platforms in GCC,
and why the same is true for darwin in LLVM.

Cheers,

Manuel.

Re: dragonegg in FSF gcc?

[Dave Korn]

On 11/04/2010 22:42, Manuel López-Ibáñez wrote:

> [ ... ] lack of test results in some platforms does not mean
> that GCC developers are uninterested on supporting those platforms and
> much less that they are against supporting those platforms. The GCC
> community haven't forbidden anyone from contributing to support any
> platform in GCC.

  I don't know who you're talking to, but it sure isn't to me or about
anything remotely like anything I said.  Put your straw man away.

    cheers,
      DaveK

Re: dragonegg in FSF gcc?

[Jonathan Wakely]

On 11 April 2010 17:17, Jack Howarth wrote:
>
>  I would also add that some of this seems like deja vu from the
> egcs days. Granted it is extremely unlikely, but who is to say that
> at some future date, if the license conflicts subside, that FSF gcc
> might decide that llvm wasn't so bad for the middle/back-ends. Wouldn't
> having the front-end running entirely via the plugin be a huge help
> in that case.

egcs code was always license-compatible with GCC and was always
assigned to the FSF

The difference is quite significant.

Re: dragonegg in FSF gcc?

[Manuel López-Ibáñez]

On 12 April 2010 00:38, Dave Korn <dave.korn.cygwin@googlemail.com> wrote:
> On 11/04/2010 22:42, Manuel López-Ibáñez wrote:
>
>> [ ... ] lack of test results in some platforms does not mean
>> that GCC developers are uninterested on supporting those platforms and
>> much less that they are against supporting those platforms. The GCC
>> community haven't forbidden anyone from contributing to support any
>> platform in GCC.
>
>  I don't know who you're talking to, but it sure isn't to me or about
> anything remotely like anything I said.  Put your straw man away.

I am just trying to negate what a casual reader might conclude from
Jack's original comment about gnulinux mono-culture and your analysis.
I understand that you (and perhaps even Jack) do not actually mean to
say that but the above sentiment is out there, specially among
bsd/darwin users, so let's try not to reinforce it.

Cheers,

Manuel.

Re: dragonegg in FSF gcc?

[Duncan Sands]

Hi Jonathan,

> egcs code was always license-compatible with GCC and was always
> assigned to the FSF
>
> The difference is quite significant.

both dragonegg and LLVM are license-compatible with GCC.  The dragonegg
code is licensed under GPLv2 or later, while LLVM is licensed under the
University of Illinois/NCSA Open Source License, which is GPL compatible
according to

  http://www.gnu.org/licenses/license-list.html#GPLCompatibleLicenses

The dragonegg plugin, being a combination of these plus GCC, is therefore
GPLv3.

You are of course quite right that neither LLVM nor dragonegg has its
copyright assigned to the FSF.

Ciao,

Duncan.

Re: dragonegg in FSF gcc?

[Jack Howarth]

On Mon, Apr 12, 2010 at 08:47:54AM +0200, Manuel López-Ibáñez wrote:
> On 12 April 2010 00:38, Dave Korn <dave.korn.cygwin@googlemail.com> wrote:
> > On 11/04/2010 22:42, Manuel López-Ibáñez wrote:
> >
> >> [ ... ] lack of test results in some platforms does not mean
> >> that GCC developers are uninterested on supporting those platforms and
> >> much less that they are against supporting those platforms. The GCC
> >> community haven't forbidden anyone from contributing to support any
> >> platform in GCC.
> >
> >  I don't know who you're talking to, but it sure isn't to me or about
> > anything remotely like anything I said.  Put your straw man away.
> 
> I am just trying to negate what a casual reader might conclude from
> Jack's original comment about gnulinux mono-culture and your analysis.
> I understand that you (and perhaps even Jack) do not actually mean to
> say that but the above sentiment is out there, specially among
> bsd/darwin users, so let's try not to reinforce it.
> 
> Cheers,
> 
> Manuel.

Manuel,
   What I meant to say was that the reality of the situation is
that 90%+ of the code (by line) is probably created by paid
employees and the way things have shaken out has placed the bulk
of those on linux. So FSF gcc will have to go out of its way to
try to limit the monoculture tendencies that this will naturally
cause. The llvm project has this issue probably less for linux
than for other surviving platforms (like Solaris, etc).
            Jack

Re: dragonegg in FSF gcc?

[Robert Dewar]

Jack Howarth wrote:

> Manuel,
>    What I meant to say was that the reality of the situation is
> that 90%+ of the code (by line) is probably created by paid
> employees and the way things have shaken out has placed the bulk
> of those on linux. 

Just a note, AdaCore is certainly not Linux-only-centric :-)

Re: dragonegg in FSF gcc?

[Richard Guenther]

On Mon, Apr 12, 2010 at 3:35 PM, Jack Howarth <howarth@bromo.med.uc.edu> wrote:
> On Mon, Apr 12, 2010 at 08:47:54AM +0200, Manuel López-Ibáñez wrote:
>> On 12 April 2010 00:38, Dave Korn <dave.korn.cygwin@googlemail.com> wrote:
>> > On 11/04/2010 22:42, Manuel López-Ibáñez wrote:
>> >
>> >> [ ... ] lack of test results in some platforms does not mean
>> >> that GCC developers are uninterested on supporting those platforms and
>> >> much less that they are against supporting those platforms. The GCC
>> >> community haven't forbidden anyone from contributing to support any
>> >> platform in GCC.
>> >
>> >  I don't know who you're talking to, but it sure isn't to me or about
>> > anything remotely like anything I said.  Put your straw man away.
>>
>> I am just trying to negate what a casual reader might conclude from
>> Jack's original comment about gnulinux mono-culture and your analysis.
>> I understand that you (and perhaps even Jack) do not actually mean to
>> say that but the above sentiment is out there, specially among
>> bsd/darwin users, so let's try not to reinforce it.
>>
>> Cheers,
>>
>> Manuel.
>
> Manuel,
>   What I meant to say was that the reality of the situation is
> that 90%+ of the code (by line) is probably created by paid
> employees and the way things have shaken out has placed the bulk
> of those on linux. So FSF gcc will have to go out of its way to
> try to limit the monoculture tendencies that this will naturally
> cause. The llvm project has this issue probably less for linux
> than for other surviving platforms (like Solaris, etc).

Err, well.  I do not see how most of the code is OS specific
anyway - there is _very_ little code in GCC that is OS specific.

Richard.

>            Jack
>
>

Re: dragonegg in FSF gcc?

[Dave Korn]

On 12/04/2010 07:47, Manuel López-Ibáñez wrote:
> On 12 April 2010 00:38, Dave Korn <dave.korn.cygwin@> wrote:
>> On 11/04/2010 22:42, Manuel López-Ibáñez wrote:
>>
>>> [ ... ] lack of test results in some platforms does not mean
>>> that GCC developers are uninterested on supporting those platforms and
>>> much less that they are against supporting those platforms. The GCC
>>> community haven't forbidden anyone from contributing to support any
>>> platform in GCC.
>>  I don't know who you're talking to, but it sure isn't to me or about
>> anything remotely like anything I said.  Put your straw man away.
> 
> I am just trying to negate what a casual reader might conclude from
> Jack's original comment about gnulinux mono-culture and your analysis.
> I understand that you (and perhaps even Jack) do not actually mean to
> say that but the above sentiment is out there, specially among
> bsd/darwin users, so let's try not to reinforce it.

  I had never even heard such a suggestion, and wouldn't have known it was out
there if you hadn't raised it!

  Could anyone really believe that without being a grade A tinfoil-hat wearing
crazy?  More precisely, could anyone capable of the kind of rational thought
processes that they'd need to have in order to be able to make any kind of
contribution to the GCC project really believe that?  I'm not convinced that
we need to worry much about what generic non-contributing internet kooks,
trolls and idiots think.

  Nope, as far as I'm concerned, there's a preponderance of
gnu-linux-centricity just because that's where most of the people who can be
bothered to contribute are at, and if other platforms feel neglected, there's
absolutely nothing to stop them stepping up to the plate and getting involved.
 Heh, I work on Windows, if any OS was getting excluded for political reasons
I surely ought to be the first to know!

  As Richard points out elsethread, GCC is not very OS specific.  There *is*
occasionally some tendency towards all-the-world's-an-ELF-ism, although even
that isn't any deliberate policy but just the result of people not being aware
of the attributes of other platforms or the semantic differences between their
otherwise-similar concepts.  LTO is the first big example, but there are
numerous minor things that rely implicitly on such features as (e.g.) leaving
undefined references to be resolved at load-time.  (Yes, it makes vague
linkage a right PITA not being able to do that, sigh.  Don't think we'll ever
be able to avoid violating the ODR with typeinfos on Windows and having to
rely on full name-string comparisons always.)


    cheers,
      DaveK

Re: dragonegg in FSF gcc?

[Jack Howarth]

On Mon, Apr 12, 2010 at 03:42:39PM +0200, Richard Guenther wrote:
> On Mon, Apr 12, 2010 at 3:35 PM, Jack Howarth <howarth@bromo.med.uc.edu> wrote:
> > On Mon, Apr 12, 2010 at 08:47:54AM +0200, Manuel López-Ibáñez wrote:
> >> On 12 April 2010 00:38, Dave Korn <dave.korn.cygwin@googlemail.com> wrote:
> >> > On 11/04/2010 22:42, Manuel López-Ibáñez wrote:
> >> >
> >> >> [ ... ] lack of test results in some platforms does not mean
> >> >> that GCC developers are uninterested on supporting those platforms and
> >> >> much less that they are against supporting those platforms. The GCC
> >> >> community haven't forbidden anyone from contributing to support any
> >> >> platform in GCC.
> >> >
> >> >  I don't know who you're talking to, but it sure isn't to me or about
> >> > anything remotely like anything I said.  Put your straw man away.
> >>
> >> I am just trying to negate what a casual reader might conclude from
> >> Jack's original comment about gnulinux mono-culture and your analysis.
> >> I understand that you (and perhaps even Jack) do not actually mean to
> >> say that but the above sentiment is out there, specially among
> >> bsd/darwin users, so let's try not to reinforce it.
> >>
> >> Cheers,
> >>
> >> Manuel.
> >
> > Manuel,
> >   What I meant to say was that the reality of the situation is
> > that 90%+ of the code (by line) is probably created by paid
> > employees and the way things have shaken out has placed the bulk
> > of those on linux. So FSF gcc will have to go out of its way to
> > try to limit the monoculture tendencies that this will naturally
> > cause. The llvm project has this issue probably less for linux
> > than for other surviving platforms (like Solaris, etc).
> 
> Err, well.  I do not see how most of the code is OS specific
> anyway - there is _very_ little code in GCC that is OS specific.
> 
> Richard.

Richard,
   Except for LTO (for which dragon-egg represents a way around
since we can use the llvm's libLTO with that). In fact, dragon-egg
should be welcomed as a way to directly compare the two approaches
to LTO from within a single compiler (and perhaps prove FSF gcc's choice 
superior).
             Jack

> 
> >            Jack
> >
> >

Re: dragonegg in FSF gcc?

[Manuel López-Ibáñez]

On 12 April 2010 16:18, Dave Korn <dave.korn.cygwin@googlemail.com> wrote:
>
>  Could anyone really believe that without being a grade A tinfoil-hat wearing
> crazy?  More precisely, could anyone capable of the kind of rational thought

Then, you do not read LWN comments, OS news or BSD mailing lists.  You
probably do well for your sanity ;-).

But I do not think they are crazy, trolls or idiots, just uninformed,
misinformed, disfranchised or unwilling to understand.

The fact is that it is (perceived as) more difficult to contribute to
GCC than LLVM/Clang for the reasons we all know already. And the
LLVM/Clang project has done an excellent job at presenting itself as
an alternative to GCC for those "neglected" platforms. I am not saying
this in a negative tone. I honestly think GCC could learn a lot about
marketing and usability details from LLVM.

Cheers,

Manuel.

Re: dragonegg in FSF gcc?

[Alfred M. Szmidt]

If the dragonegg and/or LLVM copyright was assigned to the FSF, which
is a prerequisit for anything included in GCC and not what license the
program is under currently, then I'm quite sure that the GCC
maintainers would be more than happy to include both.

Re: dragonegg in FSF gcc?

[Jack Howarth]

On Mon, Apr 12, 2010 at 11:00:13AM -0400, Alfred M. Szmidt wrote:
> If the dragonegg and/or LLVM copyright was assigned to the FSF, which
> is a prerequisit for anything included in GCC and not what license the
> program is under currently, then I'm quite sure that the GCC
> maintainers would be more than happy to include both.

I don't think anyone was proposing llvm be added to FSF gcc but
rather just dragon-egg (so the interfaces to FSF gcc could be
kept updated more easily). The dependency on llvm would be treated
as any other (ppl, cloog, gmp, etc) and the user would have to
provide their own copy out of tree (although an in-tree build 
could be supported).
                  Jack

Re: dragonegg in FSF gcc?

[Steven Bosscher]

On Mon, Apr 12, 2010 at 3:52 PM, Jack Howarth <howarth@bromo.med.uc.edu> wrote:
>> Err, well.  I do not see how most of the code is OS specific
>> anyway - there is _very_ little code in GCC that is OS specific.
>>
>> Richard.
>
> Richard,
>   Except for LTO (for which dragon-egg represents a way around
> since we can use the llvm's libLTO with that).

No, LTO is in fact not very OS specific at all. Just because your
favorite platform isn't supported, does not mean that something in GCC
is linux-specific. LTO works on all targets with ELF binaries, and
patches exist to make it work with COFF binaries too. You could add
MACH-O support, it shouldn't be very difficult to do if you can follow
Dave's example.

But instead you go to LLVM, which is, bottom line, not a solution for
GCC -- and that's what this thread is all about to me.

Ciao!
Steven

Re: dragonegg in FSF gcc?

[Jack Howarth]

On Mon, Apr 12, 2010 at 05:45:52PM +0200, Steven Bosscher wrote:
> On Mon, Apr 12, 2010 at 3:52 PM, Jack Howarth <howarth@bromo.med.uc.edu> wrote:
> >> Err, well.  I do not see how most of the code is OS specific
> >> anyway - there is _very_ little code in GCC that is OS specific.
> >>
> >> Richard.
> >
> > Richard,
> >   Except for LTO (for which dragon-egg represents a way around
> > since we can use the llvm's libLTO with that).
> 
> No, LTO is in fact not very OS specific at all. Just because your
> favorite platform isn't supported, does not mean that something in GCC
> is linux-specific. LTO works on all targets with ELF binaries, and
> patches exist to make it work with COFF binaries too. You could add
> MACH-O support, it shouldn't be very difficult to do if you can follow
> Dave's example.
> 
> But instead you go to LLVM, which is, bottom line, not a solution for
> GCC -- and that's what this thread is all about to me.

  I have opened PR43729, "MachO LTO support needed for darwin", to discuss
this. Can you point me at Dave's previous patches that added LTO-suppport
to a non-ELF platform? Also, I was unaware that this feat had been performed
on a target which both is non-ELF and non-binutils.
                   Jack

> 
> Ciao!
> Steven

Re: dragonegg in FSF gcc?

[Steven Bosscher]

On Mon, Apr 12, 2010 at 5:59 PM, Jack Howarth <howarth@bromo.med.uc.edu> wrote:

>  I have opened PR43729, "MachO LTO support needed for darwin", to discuss
> this. Can you point me at Dave's previous patches that added LTO-suppport
> to a non-ELF platform?

I've linked your new PR to the existing "LTO doesn't work on non-ELF
platform" PR. We even discussed Mach-O already there.

> Also, I was unaware that this feat had been performed
> on a target which both is non-ELF and non-binutils.

AFAIK cygwin also uses binutils, but no changes are needed to make LTO
work with the collect2 approach (Dave is that correct?).

Ciao!
Steven

RE: dragonegg in FSF gcc?

[Weddington, Eric]

 

> -----Original Message-----
> From: Manuel López-Ibáñez [mailto:lopezibanez@gmail.com] 
> Sent: Monday, April 12, 2010 8:27 AM
> To: Dave Korn
> Cc: Jack Howarth; Steven Bosscher; Duncan Sands; gcc@gcc.gnu.org
> Subject: Re: dragonegg in FSF gcc?
> 
> The fact is that it is (perceived as) more difficult to contribute to
> GCC than LLVM/Clang for the reasons we all know already. And the
> LLVM/Clang project has done an excellent job at presenting itself as
> an alternative to GCC for those "neglected" platforms. I am not saying
> this in a negative tone. I honestly think GCC could learn a lot about
> marketing and usability details from LLVM.

From my perspective (and someone correct me if I'm wrong) it is easier for LLVM to do such marketing and focus on usability details because they seem to have a central driver to the project, whether person/group (founder(s)/champion(s)). GCC is, what I would call, aggressively decentralized; Who would do such marketing? Who decides what marketing to do? Who decides the usability details? Who would work on it? GCC is the epitome of the saying "If you want something done, do it yourself." There is no central authority who can, or will, make a decision about this. There is a Steering Committee for GCC, but as they keep saying at the GCC Summits, their power and scope is very limited.

Eric Weddington

Re: dragonegg in FSF gcc?

[Dave Korn]

On 12/04/2010 17:03, Steven Bosscher wrote:
> On Mon, Apr 12, 2010 at 5:59 PM, Jack Howarth wrote:
> 
>>  I have opened PR43729, "MachO LTO support needed for darwin", to discuss
>> this. Can you point me at Dave's previous patches that added LTO-suppport
>> to a non-ELF platform?
> 
> I've linked your new PR to the existing "LTO doesn't work on non-ELF
> platform" PR. We even discussed Mach-O already there.
> 
>> Also, I was unaware that this feat had been performed
>> on a target which both is non-ELF and non-binutils.
> 
> AFAIK cygwin also uses binutils, but no changes are needed to make LTO
> work with the collect2 approach (Dave is that correct?).

  Binutils for COFF targets needed a patch to allow sections to be
byte-aligned and byte-packed, as it wasn't originally possible to use any
alignment directive to reduce the section alignment below the default, and the
zip-compressed data sections need to be exactly sized to the data they contain
rather than padded up to the default section alignment of 4.

  If MachO can do that already, it won't need any changes.  Or it could be
fixed in GCC by modifying the format of the compressed sections to be
self-describing w.r.t valid data length in some way - this would probably be
the better thing to do in the long run.

    cheers,
      DaveK

Re: dragonegg in FSF gcc?

[Toon Moene]

Weddington, Eric wrote:

>> From: Manuel López-Ibáñez [mailto:lopezibanez@gmail.com] 

>> The fact is that it is (perceived as) more difficult to contribute to
>> GCC than LLVM/Clang for the reasons we all know already.
> 
> From my perspective (and someone correct me if I'm wrong) it is easier for LLVM to do such marketing
> and focus on usability details because they seem to have a central driver to the project,
> whether person/group (founder(s)/champion(s)). GCC is, what I would call, aggressively decentralized;
> Who would do such marketing? Who decides what marketing to do? Who decides the usability details?
> Who would work on it? GCC is the epitome of the saying "If you want something done, do it yourself."
> There is no central authority who can, or will, make a decision about this.
> There is a Steering Committee for GCC, but as they keep saying at the GCC Summits,
> their power and scope is very limited.

Well, it is an open question (at least to me) whether you *want* a 
central driver.

In late 2003, three national laboratories in the US wrote up a position 
paper on their needs in Fortran-land, and lamented the lack of a free 
Fortran compiler (they noted that there was g77, but it wasn't up to 
speed in Fortran-95 land, which they needed).

This was my reply:

http://gcc.gnu.org/ml/fortran/2003-11/msg00052.html

With that answer, I essentially tossed $ 5 million away (the amount they 
estimated to be needed for a free Fortran 95 compiler, because, in a 
followup mail, I said "Anyway, we will produce a Fortran 95 compiler, 
regardless."

-- 
Toon Moene - e-mail: toon@moene.org - phone: +31 346 214290
Saturnushof 14, 3738 XG  Maartensdijk, The Netherlands
At home: http://moene.org/~toon/
Progress of GNU Fortran: http://gcc.gnu.org/gcc-4.5/changes.html

Re: dragonegg in FSF gcc?

[Ian Lance Taylor]

"Weddington, Eric" <Eric.Weddington@atmel.com> writes:

From my perspective (and someone correct me if I'm wrong) it is
>easier for LLVM to do such marketing and focus on usability details
>because they seem to have a central driver to the project, whether
>person/group (founder(s)/champion(s)). GCC is, what I would call,
>aggressively decentralized; Who would do such marketing? Who decides
>what marketing to do? Who decides the usability details? Who would
>work on it? GCC is the epitome of the saying "If you want something
>done, do it yourself." There is no central authority who can, or
>will, make a decision about this. There is a Steering Committee for
>GCC, but as they keep saying at the GCC Summits, their power and
>scope is very limited.

Having a central driver would certainly help--though only to the
extent that anybody listened.

I have seen people complain that the gcc developers are ornery and
difficult to work with.  I've been reading the mailing lists with that
in mind, and I actually don't see that very much.  However, it only
takes a very small number of mean-spirited messages to give that
impression.  What I do see is that relatively few gcc developers take
the time to reach out to new people and help them become part of the
community.  I also see a lot of external patches not reviewed, and I
see a lot of back-and-forth about patches which is simply confusing
and offputting to those trying to contribute.  Joining the gcc
community requires a lot of self-motivation, or it takes being paid
enough to get over the obstacles.


There is also the matter of the old code base, the lack of a clean
separation between passes, and, most important, weak internal
documentation.

For example, in my view of internal documentation:

How to write a new backend: good.
Details of RTL IR: adequate.
Details of GIMPLE IR: poor.
Details of tree IR: poor.
How to write a new optimization pass: poor.
How to write a new frontend: nonexistent.
General overview of compiler source: nonexistent.
Overview of internal compiler datastructures: nonexistent.


I am as responsible for this state of affairs as anybody.

Ian

Re: dragonegg in FSF gcc?

[Paolo Bonzini]

On 04/11/2010 06:26 PM, Duncan Sands wrote:
> Hi Robert,
>
>> b) better behavior for undefined cases
>
> this is one of the problems with using LLVM with the Ada front-end.
> LLVM makes pretty aggressive deductions when it sees undefined
> behaviour

These do not seem to point at LLVM's optimizer bugs/aggressiveness, but 
rather at expressiveness of the IR.  GENERIC benefited from years of 
working on the Ada front-end, and though it was indeed a good deal of 
work to iron out all the bugs, it is likely that a different compiler 
infrastructure does not have all the nuances.

Paolo

Re: dragonegg in FSF gcc?

[Paolo Bonzini]

On 04/11/2010 06:50 PM, Dave Korn wrote:
> Grepping the -patches archives to see which platforms submitted
> patches get testing on would also be interesting, but somewhat
> harder owing to the more free-form nature of the text there.  Still,
> a two-to-one ratio of linux to rest-of-the-world would be in line
> with my subjective impression: it's not overwhelming the rest, but
> it's substantially the best tended-to.

The fact that testing under Cygwin is so much slower certainly has an
impact.  A lot of people used to develop under Darwin, but unfortunately
it tended to break quite often, and it often became very hard to
bootstrap without the latest Mac OS X and Xcode.  The fact that Apple is
not maintaining the port anymore didn't help, as Jack knows.

Paolo

Re: dragonegg in FSF gcc?

[Paolo Bonzini]

On 04/12/2010 04:18 PM, Dave Korn wrote:
> Could anyone really believe that without being a grade A tinfoil-hat
> wearing crazy?  More precisely, could anyone capable of the kind of
> rational thought processes that they'd need to have in order to be
> able to make any kind of contribution to the GCC project really
> believe that?  I'm not convinced that we need to worry much about
> what generic non-contributing internet kooks, trolls and idiots
> think.

Unfortunately there is a great deal of people that are ready to drop
sh*t on projects just because they are copyleft, and that doesn't help GCC.

Of course, those are the same people that wonder "why" when a random 
company promises contributing some cool technology to a BSD-licensed 
project, and then changes their mind.

We can indeed choose to ignore trolls; still, they exist.

Paolo

Re: dragonegg in FSF gcc?

[Jack Howarth]

On Tue, Apr 13, 2010 at 07:05:17PM +0200, Paolo Bonzini wrote:
> On 04/12/2010 04:18 PM, Dave Korn wrote:
>> Could anyone really believe that without being a grade A tinfoil-hat
>> wearing crazy?  More precisely, could anyone capable of the kind of
>> rational thought processes that they'd need to have in order to be
>> able to make any kind of contribution to the GCC project really
>> believe that?  I'm not convinced that we need to worry much about
>> what generic non-contributing internet kooks, trolls and idiots
>> think.
>
> Unfortunately there is a great deal of people that are ready to drop
> sh*t on projects just because they are copyleft, and that doesn't help GCC.
>
> Of course, those are the same people that wonder "why" when a random  
> company promises contributing some cool technology to a BSD-licensed  
> project, and then changes their mind.
>
> We can indeed choose to ignore trolls; still, they exist.

Paolo,
   I hope you don't think I was trolling in my initial post. Assuming
plugins will eventually be welcomed into the FSF gcc source tree in
general, there is a valid argument for having dragon-egg present with
a configure option that builds it if the proper llvm is available.
               Jack

>
> Paolo

Re: dragonegg in FSF gcc?

[Paolo Bonzini]

On 04/13/2010 07:14 PM, Jack Howarth wrote:
> Paolo,
>     I hope you don't think I was trolling in my initial post. Assuming
> plugins will eventually be welcomed into the FSF gcc source tree in
> general, there is a valid argument for having dragon-egg present with
> a configure option that builds it if the proper llvm is available.

I was absolutely not thinking of anyone in this thread.

FWIW, my opinion is that I think it would be nice to have dragonegg 
build out-of-the-box with FSF GCC (and I would very much welcome that my 
distro applied the patch, if not) however I don't see a reason to have 
it live in the FSF repository.

On the other hand, I would welcome a gcc-clang front-end in the FSF 
repository.  16 months ago I probably would have even given it a shot.

Paolo

Re: dragonegg in FSF gcc?

[Steven Bosscher]

On Tue, Apr 13, 2010 at 7:14 PM, Jack Howarth <howarth@bromo.med.uc.edu> wrote:

>   I hope you don't think I was trolling in my initial post. Assuming
> plugins will eventually be welcomed into the FSF gcc source tree in
> general, there is a valid argument for having dragon-egg present with
> a configure option that builds it if the proper llvm is available.

There are already plugins in the FSF gcc source tree. Well, OK, just
one (lto-plugin) but there aren't very many plugins at the moment that
are suitable for inclusion in the FSF tree (i.e. not as tightly tied
to a GCC feature that GCC itself can't work fully without it).

IMHO the nature of the DragonEgg plugin makes it unsuitable for
inclusion in the FSF gcc source tree, ever.

Ciao!
Steven

Re: dragonegg in FSF gcc?

[Jack Howarth]

On Tue, Apr 13, 2010 at 07:18:12PM +0200, Paolo Bonzini wrote:
> On 04/13/2010 07:14 PM, Jack Howarth wrote:
>> Paolo,
>>     I hope you don't think I was trolling in my initial post. Assuming
>> plugins will eventually be welcomed into the FSF gcc source tree in
>> general, there is a valid argument for having dragon-egg present with
>> a configure option that builds it if the proper llvm is available.
>
> I was absolutely not thinking of anyone in this thread.
>
> FWIW, my opinion is that I think it would be nice to have dragonegg  
> build out-of-the-box with FSF GCC (and I would very much welcome that my  
> distro applied the patch, if not) however I don't see a reason to have  
> it live in the FSF repository.

Paolo,
   It is unclear to me what the original intentions were when the
plugin infrastructure was created. That is, was it envisioned that
FSF could accumulate the plugins directly in the source tree to
ensure they were well maintained across FSF gcc releases?
          Jack

>
> On the other hand, I would welcome a gcc-clang front-end in the FSF  
> repository.  16 months ago I probably would have even given it a shot.
>
> Paolo

Re: dragonegg in FSF gcc?

[Andrew Pinski]

On Tue, Apr 13, 2010 at 10:22 AM, Steven Bosscher <stevenb.gcc@gmail.com> wrote:
> There are already plugins in the FSF gcc source tree. Well, OK, just
> one (lto-plugin) but there aren't very many plugins at the moment that
> are suitable for inclusion in the FSF tree (i.e. not as tightly tied
> to a GCC feature that GCC itself can't work fully without it).

Except lto-plugin is a plugin for the gold linker and not for GCC.  Oh
and the linker has a more stable ABI already because of the way
plugins are implemented there.

I think most plugins should be done just to experiment with and real
passes should become integrated fully and not a plugin at all.  This
was the same argument I had the last time about plugin database.

>
> IMHO the nature of the DragonEgg plugin makes it unsuitable for
> inclusion in the FSF gcc source tree, ever.

It belongs with LLVM sources if anywhere.

Thanks,
Andrew Pinski

Re: dragonegg in FSF gcc?

[Jack Howarth]

On Tue, Apr 13, 2010 at 12:21:09PM -0700, Andrew Pinski wrote:
> On Tue, Apr 13, 2010 at 10:22 AM, Steven Bosscher <stevenb.gcc@gmail.com> wrote:
> > There are already plugins in the FSF gcc source tree. Well, OK, just
> > one (lto-plugin) but there aren't very many plugins at the moment that
> > are suitable for inclusion in the FSF tree (i.e. not as tightly tied
> > to a GCC feature that GCC itself can't work fully without it).
> 
> Except lto-plugin is a plugin for the gold linker and not for GCC.  Oh
> and the linker has a more stable ABI already because of the way
> plugins are implemented there.
> 
> I think most plugins should be done just to experiment with and real
> passes should become integrated fully and not a plugin at all.  This
> was the same argument I had the last time about plugin database.
> 
> >
> > IMHO the nature of the DragonEgg plugin makes it unsuitable for
> > inclusion in the FSF gcc source tree, ever.
> 
> It belongs with LLVM sources if anywhere.

  I think the idea was that it would live in both repositories. The
dragon-egg in FSF gcc would be focused on using the stable llvm
and adapting to the FSF gcc trunk changes. The dragon-egg in llvm
would use the stable gcc release and be focused on adapting to llvm
trunk changes. The two could be re-merged on each llvm or gcc release.
My view anyway.
         Jack

> 
> Thanks,
> Andrew Pinski

Re: dragonegg in FSF gcc?

[Paolo Bonzini]

On 04/13/2010 09:16 PM, Jack Howarth wrote:
> Paolo,
>     It is unclear to me what the original intentions were when the
> plugin infrastructure was created. That is, was it envisioned that
> FSF could accumulate the plugins directly in the source tree to
> ensure they were well maintained across FSF gcc releases?

Not really.  Plugins were added for people to be able to extend GCC in 
novel ways.  It was clear that some of these would circumvent GCC's 
copyleft, and so the runtime license exception was devised.  In this 
sense, dragonegg is not harmful to GCC; if you wrote a proprietary LLVM 
pass, the result would be not eligible for the runtime license exception.

However, I think there is another reason why dragonegg belongs in the 
LLVM repository.  It is very unlikely to be used by anyone who has no 
interest in LLVM, and vice versa many LLVM users will want to try it 
out.  GCC developers who otherwise won't use LLVM are the exception, in 
my opinion.

Similarly, if a gcc-clang existed it would belong in the FSF repository 
(probably on a branch) because (I think) hardly any LLVM user would be 
interested in it.

Paolo

Re: dragonegg in FSF gcc?

[Diego Novillo]

On Tue, Apr 13, 2010 at 15:16, Jack Howarth <howarth@bromo.med.uc.edu> wrote:

>   It is unclear to me what the original intentions were when the
> plugin infrastructure was created. That is, was it envisioned that
> FSF could accumulate the plugins directly in the source tree to
> ensure they were well maintained across FSF gcc releases?

My idea was (and still is) that we could host a directory of available
plugins, but each plugin would be a separate project with its own
schedules, home pages and source trees.  The directory is hosted at
http://gcc.gnu.org/wiki/plugins.  Dragonegg is already there.

The plugin API will likely be very volatile at first, but it may
converge at some point.

Personally, I would love to see DragonEgg used to bridge between gcc
and llvm.  This will bring lots of benefits to both compilers, since
we'll be able to easily compare and take from each other.


Diego.

Re: dragonegg in FSF gcc?

[Steven Bosscher]

On Tue, Apr 13, 2010 at 10:19 PM, Diego Novillo <dnovillo@google.com> wrote:
> Personally, I would love to see DragonEgg used to bridge between gcc
> and llvm.  This will bring lots of benefits to both compilers, since
> we'll be able to easily compare and take from each other.

You say you see benefits for both compilers. What benefits do you see
for GCC then, if I may ask? And what can GCC take from LLVM? (And I
mean the FSF GCC, long term.) This is an honest question, because I
personally really don't see any benefit for GCC.

Ciao!
Steven

Testing GCC on Cygwin made substantially easier [was Re: dragonegg in FSF gcc?]

[Dave Korn]

On 13/04/2010 17:57, Paolo Bonzini wrote:

> The fact that testing under Cygwin is so much slower certainly has an
> impact.  

  It gets more manageable at -j levels, but there's an underlying bug in the
Cygwin DLL's process info management that causes expect to fall into
cpu-spinning lockups and cascading process fork collapse syndrome.

  Until I find time to do a more major rewrite, anyone who wants to do testing
on Cygwin could do worse than apply the sticking-plaster patch that I posted at:

  http://www.mail-archive.com/cygwin-patches@cygwin.com/msg04677.html

and build themselves a locally modified version of the Cygwin DLL that will
happily run make check at significant -j levels (I think I tried 12 at most;
I've only got a dual-core cpu so it wasn't exactly efficient, but it proved
that the patch holds up under substantial load).

    cheers,
      DaveK

Re: dragonegg in FSF gcc?

[Diego Novillo]

On Tue, Apr 13, 2010 at 16:51, Steven Bosscher <stevenb.gcc@gmail.com> wrote:

> You say you see benefits for both compilers. What benefits do you see
> for GCC then, if I may ask? And what can GCC take from LLVM? (And I
> mean the FSF GCC, long term.) This is an honest question, because I
> personally really don't see any benefit for GCC.

If comparisons between the two compilers are easy to make, then it's
easy to determine what one compiler is doing better than the other and
do the necessary port.

In terms of internal structure, LLVM is more modular, which simplifies
maintenance (e.g., the automatic bug finder, unit tests).  The various
components of the pipeline have better separation and stronger APIs.
GCC has been slowly moving in that direction, but it still have ways
to go.  LLVM has already proven that organizing the compiler that way
is advantageous (additionally, other research compilers were
structured similarly: Sage++, SUIF), so emulating that structure
sounds like a reasonable approach.

Another example where GCC may want to operate with LLVM is in JIT
compilation.  Clearly, LLVM has made a significant investment in this
area.  If GCC were to generate LLVM IR, it could just use all the JIT
machinery without having to replicate it.

There may be other things GCC could take advantage of.

OTOH, GCC has optimizer and codegen features that LLVM may want to
incorporate.  I don't have specific examples, since I am not very
familiar with LLVM.


Diego.

Re: dragonegg in FSF gcc?

[Steven Bosscher]

On Sun, Apr 11, 2010 at 4:17 PM, Jack Howarth <howarth@bromo.med.uc.edu> wrote:
> Rather than viewing dragon-egg as some sort
> of lamprey which is feeding off of the FSF gcc project,
> we should welcome the competition from a direct comparison
> of alternative back/middle ends (not fear it).

FWIW, this sounds great and all... but I haven't actually seen any
comparisons of GCC vs. LLVM with DragonEgg. A search with Google
doesn't give me any results.

Can you point out some postings where people actually made a
comparison between GCC and LLVM with DragonEgg?


However, I found your posting of "llvm-gfortran" Polyhedron
comparisons of different LLVM versions
(http://lists.cs.uiuc.edu/pipermail/llvmdev/2010-April/030799.html).
But that comparison does not including gfortran results.

I also found comparisons of llvm-gfortran and FSF gfortran
(http://lists.cs.uiuc.edu/pipermail/llvmdev/2008-June/015441.html and
http://lists.cs.uiuc.edu/pipermail/llvmdev/2009-August/025119.html).
But unfortunately I can't find these results posted on the GCC fortran
mailing list, or Bugzilla bug reports for cases where llvm-gfortran
performed better than FSF gfortran. So there's not much benefit from
these comparisons for GCC.

Where these results generated using DragonEgg? Or can you make these
comparisons without DragonEgg too?


>   One could also make an argument that it is in the best
> interest of FSF gcc to do so. Isn't better to keep all
> of the alternative front-end developers (gfortran, ada,
> etc) within the FSF gcc tent rather than forcing the
> creation of competing clang fortran and ada projects.

Why would that be better? And, why would that be better only for the
front ends, but not for the middle ends and back ends? You don't seem
to have a problem with the creation of competing back ends. If you so
believe in competition between compilers, then why are you against
competition at the level of front ends? You should encourage a clang
fortran project.

But apparently, you don't. That doesn't look like a desire for
competition or comparisons, but only a desire to rip the parts of GCC
that LLVM doesn't already provide itself.

Ciao!
Steven

Re: dragonegg in FSF gcc?

[Jack Howarth]

On Wed, Apr 14, 2010 at 12:57:41AM +0200, Steven Bosscher wrote:
> On Sun, Apr 11, 2010 at 4:17 PM, Jack Howarth <howarth@bromo.med.uc.edu> wrote:
> > Rather than viewing dragon-egg as some sort
> > of lamprey which is feeding off of the FSF gcc project,
> > we should welcome the competition from a direct comparison
> > of alternative back/middle ends (not fear it).
> 
> FWIW, this sounds great and all... but I haven't actually seen any
> comparisons of GCC vs. LLVM with DragonEgg. A search with Google
> doesn't give me any results.
> 
> Can you point out some postings where people actually made a
> comparison between GCC and LLVM with DragonEgg?
> 
> 
> However, I found your posting of "llvm-gfortran" Polyhedron
> comparisons of different LLVM versions
> (http://lists.cs.uiuc.edu/pipermail/llvmdev/2010-April/030799.html).
> But that comparison does not including gfortran results.
> 
> I also found comparisons of llvm-gfortran and FSF gfortran
> (http://lists.cs.uiuc.edu/pipermail/llvmdev/2008-June/015441.html and
> http://lists.cs.uiuc.edu/pipermail/llvmdev/2009-August/025119.html).
> But unfortunately I can't find these results posted on the GCC fortran
> mailing list, or Bugzilla bug reports for cases where llvm-gfortran
> performed better than FSF gfortran. So there's not much benefit from
> these comparisons for GCC.
> 
> Where these results generated using DragonEgg? Or can you make these
> comparisons without DragonEgg too?

Those benchmarks were made with just the stock gfortran from the
llvm-gcc-4.2 source tree from the release_27 branch. Of course, the
benchmarks for gcc 4.5 (or gcc 4.4) are signficantly faster (gcc 4.2.1
isn't the best release to be trapped on for gfortran performance).
On the same machine, I am seeing these numbers for the gcc 4.5 release...

================================================================================
Date & Time     : 25 Mar 2010  0:08:05
Test Name       : gfortran_lin_p4
Compile Command : gfortran -ffast-math -funroll-loops -msse3 -O3 %n.f90 -o %n
Benchmarks      : ac aermod air capacita channel doduc fatigue gas_dyn induct linpk mdbx nf protein rnflow test_fpu tfft
Maximum Times   :     2000.0
Target Error %  :      0.100
Minimum Repeats :    10
Maximum Repeats :   100

   Benchmark   Compile  Executable   Ave Run  Number   Estim
        Name    (secs)     (bytes)    (secs) Repeats   Err %
   ---------   -------  ----------   ------- -------  ------
          ac      0.86       10000     10.59      10  0.0069
      aermod     39.15       10000     21.35      10  0.0086
         air      2.24       10000      5.68      12  0.0392
    capacita      1.70       10000     33.19      10  0.0110
     channel      0.56       10000      1.83      10  0.0257
       doduc      4.58       10000     27.72      10  0.0122
     fatigue      1.54       10000      8.04      10  0.0382
     gas_dyn      2.83       10000      4.39      18  0.0965
      induct      3.55       10000     12.67      10  0.0127
       linpk      0.70       10000     15.41      10  0.0458
        mdbx      1.51       10000     11.33      10  0.0059
          nf      1.90       10000     27.96      17  0.0967
     protein      3.48       10000     37.02      10  0.0058
      rnflow      5.20       10000     23.64      10  0.0243
    test_fpu      4.19       10000      8.68      10  0.0494
        tfft      0.49       10000      1.88      10  0.0766

Geometric Mean Execution Time =      11.27 seconds

================================================================================

I intend to do some benchmarks with dragon-egg shortly. At the moment, I am
finishing up a gcc45 fink package capable of building dragon-egg. My goal
is to have fink packages for gcc45, llvm-2.7 and current dragon-egg so
that everything can be tested against the other. My current plan is to
have compiler wrappers, degcc-4, etc, which will automatically call the
plugin. It will be particularly interesting to see how dragon-egg manages
libLTO usage. Hopefully by gcc-4.5.1 we will have the Mach-O LTO working
and some direct comparisons can be made between the two.
         Jack

> 
> 
> >   One could also make an argument that it is in the best
> > interest of FSF gcc to do so. Isn't better to keep all
> > of the alternative front-end developers (gfortran, ada,
> > etc) within the FSF gcc tent rather than forcing the
> > creation of competing clang fortran and ada projects.
> 
> Why would that be better? And, why would that be better only for the
> front ends, but not for the middle ends and back ends? You don't seem
> to have a problem with the creation of competing back ends. If you so
> believe in competition between compilers, then why are you against
> competition at the level of front ends? You should encourage a clang
> fortran project.
> 
> But apparently, you don't. That doesn't look like a desire for
> competition or comparisons, but only a desire to rip the parts of GCC
> that LLVM doesn't already provide itself.
> 
> Ciao!
> Steven

Re: dragonegg in FSF gcc?

[Duncan Sands]

Hi Steven,

> FWIW, this sounds great and all... but I haven't actually seen any
> comparisons of GCC vs. LLVM with DragonEgg. A search with Google
> doesn't give me any results.
>
> Can you point out some postings where people actually made a
> comparison between GCC and LLVM with DragonEgg?

I gave some comparisons in my talk at the 2009 LLVM developers meeting.
See the links at the bottom of http://dragonegg.llvm.org/

Since then I've been working on completeness and correctness, and didn't
do any additional benchmarking yet.  I don't know if anyone else did any
benchmarking.  If so they didn't inform me.

Ciao,

Duncan.

Re: dragonegg in FSF gcc?

[Jack Howarth]

On Wed, Apr 14, 2010 at 08:24:35AM +0200, Duncan Sands wrote:
> Hi Steven,
>
>> FWIW, this sounds great and all... but I haven't actually seen any
>> comparisons of GCC vs. LLVM with DragonEgg. A search with Google
>> doesn't give me any results.
>>
>> Can you point out some postings where people actually made a
>> comparison between GCC and LLVM with DragonEgg?
>
> I gave some comparisons in my talk at the 2009 LLVM developers meeting.
> See the links at the bottom of http://dragonegg.llvm.org/
>
> Since then I've been working on completeness and correctness, and didn't
> do any additional benchmarking yet.  I don't know if anyone else did any
> benchmarking.  If so they didn't inform me.

Duncan,
   It would be interesting to know what the Sqlite3 -O3/-O2 benchmarks
show for llvm 2.7. I benchmarked about a 14% improvement in the
Polyhedron 2005 benchmarks over a 13 month period of llvm development.
Perhaps the current numbers look a bit better.
                Jack

>
> Ciao,
>
> Duncan.

Re: dragonegg in FSF gcc?

[Paolo Bonzini]

On 04/14/2010 03:36 PM, Jack Howarth wrote:
> On Wed, Apr 14, 2010 at 08:24:35AM +0200, Duncan Sands wrote:
>> Hi Steven,
>>
>>> FWIW, this sounds great and all... but I haven't actually seen any
>>> comparisons of GCC vs. LLVM with DragonEgg. A search with Google
>>> doesn't give me any results.
>>>
>>> Can you point out some postings where people actually made a
>>> comparison between GCC and LLVM with DragonEgg?
>>
>> I gave some comparisons in my talk at the 2009 LLVM developers meeting.
>> See the links at the bottom of http://dragonegg.llvm.org/
>>
>> Since then I've been working on completeness and correctness, and didn't
>> do any additional benchmarking yet.  I don't know if anyone else did any
>> benchmarking.  If so they didn't inform me.
>
> Duncan,
>     It would be interesting to know what the Sqlite3 -O3/-O2 benchmarks
> show for llvm 2.7. I benchmarked about a 14% improvement in the
> Polyhedron 2005 benchmarks over a 13 month period of llvm development.
> Perhaps the current numbers look a bit better.

This kind of summarizes where the interest for dragonegg lies. :-P

Paolo

ps: look at smiley before clicking "reply"

RE: dragonegg in FSF gcc?

[Grigori Fursin]

Hi Diego,

I agree with what you said. As a researcher I started using GCC instead of Open64 in 2005
after I saw some steps towards modularity when pass manager has been introduced since it
was really simplifying my life when working on iterative/collective compilation. We have
been also trying to propose further modularization/API-zation using plugins and interactive compilation
interface to provide more abstractions to GCC but the acceptance was far too slow (6+ years).

Up to now, LLVM is quite behind in terms of optimizations, but it's modularity simplifies
adding new optimization, instrumentation and analysis passes among other things. I still use or 
plan to GCC for many reasons but I also use LLVM and I see some of my colleagues 
moving from GCC to LLVM mainly due to modularity and simplicity-of-use reasons. I still sometimes hear 
comments that GCC shouldn't be driven at all by the needs of researchers but lots of advanced optimizations 
actually came from academic research, so I think this can be a bit short-sighted. If GCC will not move
towards modularization and clean APIs (by the way, I am not saying that it's easy), it doesn't mean that 
it will disappear, but it will change the role and will have to catch up. So, I think having 2 good 
open-source compilers and a healthy competition is not bad ;) ... We also heard many similar comments 
from our colleagues at GROW'09 and GROW'10 workshops...

Cheers,
Grigori



-----Original Message-----
From: gcc-owner@gcc.gnu.org [mailto:gcc-owner@gcc.gnu.org] On Behalf Of Diego Novillo
Sent: Tuesday, April 13, 2010 11:06 PM
To: Steven Bosscher
Cc: Jack Howarth; Paolo Bonzini; Dave Korn; Manuel López-Ibáñez; Duncan Sands; gcc@gcc.gnu.org
Subject: Re: dragonegg in FSF gcc?

On Tue, Apr 13, 2010 at 16:51, Steven Bosscher <stevenb.gcc@gmail.com> wrote:

> You say you see benefits for both compilers. What benefits do you see
> for GCC then, if I may ask? And what can GCC take from LLVM? (And I
> mean the FSF GCC, long term.) This is an honest question, because I
> personally really don't see any benefit for GCC.

If comparisons between the two compilers are easy to make, then it's
easy to determine what one compiler is doing better than the other and
do the necessary port.

In terms of internal structure, LLVM is more modular, which simplifies
maintenance (e.g., the automatic bug finder, unit tests).  The various
components of the pipeline have better separation and stronger APIs.
GCC has been slowly moving in that direction, but it still have ways
to go.  LLVM has already proven that organizing the compiler that way
is advantageous (additionally, other research compilers were
structured similarly: Sage++, SUIF), so emulating that structure
sounds like a reasonable approach.

Another example where GCC may want to operate with LLVM is in JIT
compilation.  Clearly, LLVM has made a significant investment in this
area.  If GCC were to generate LLVM IR, it could just use all the JIT
machinery without having to replicate it.

There may be other things GCC could take advantage of.

OTOH, GCC has optimizer and codegen features that LLVM may want to
incorporate.  I don't have specific examples, since I am not very
familiar with LLVM.


Diego.

Re: Some benchmark comparison of gcc4.5 and dragonegg (was dragonegg in FSF gcc?)

[Vladimir Makarov]

Duncan Sands wrote:
> Hi Steven,
>
>>> I think Jack wasn't suggesting that dragonegg should be changed to 
>>> not be
>>> a plugin any more.  I think he was suggesting that it should live in 
>>> the gcc
>>> repository rather than the LLVM repository.
>>
>> So, no offense, but the suggestion here is to make this subversive
>> (for FSF GCC) plugin part of FSF GCC? What is the benefit of this for
>> GCC? I don't see any. I just see a plugin trying to piggy-back on the
>> hard work of GCC front-end developers and negating the efforts of
>> those working on the middle ends and back ends.
>
> I'm sorry you see the dragonegg project so negatively.  I think it is 
> useful
> for gcc (though not hugely useful), since it makes it easy to compare 
> the gcc
> and LLVM optimizers and code generators, not to mention the gcc and LLVM
> approaches to LTO.  If LLVM manages to produce better code than gcc 
> for some
> testcase, then it is a convenient tool for the gcc devs to find out 
> why, and
> improve gcc.  If gcc is consistently better than LLVM then there's 
> nothing to
> worry about!  Of course, right now it is LLVM that is mostly playing 
> catchup
> with gcc, so for the moment it is principally the LLVM devs that get 
> to learn
> from gcc, but as LLVM improves the other direction is likely to occur 
> more
> often.
I've tried to compare gcc4.5 and dragonegg a week ago on SPEC2000 on a 
Core I7.
Here are some results.

  Only SPECIn2000 for x86_64 has been compiled fully successfully by
dragonegg.  There were a few compiler crashes including some in LLVM
itself for SPECFP2000 and for SPECINT2000 for x86.

So here is SPECInt2000 for x86_64 comparison:

dragonegg: -O3 (with LLVM release build)
gcc4.5: -O3 -flto (--enable-checking=release)

          Compilation Time  SPECINT2000
Dragonegg 122.85user         2572
gcc-4.5   283.49user         2841

  On integer benchmarks, dragonegg generates about 11% slower code.
One interesting thing is that dragonegg is a really fast compiler.  It
is 2.3 times faster than gcc.

Draggonegg generates smaller text sections but bigger data sections.
Unfortunately, my scripts measure and compare only text sections.  Therefore
I am not posting this text code size comparison because it has no 
sense.  But looking
at small benchmarks, I got an impression that gcc generates smaller code 
(text + data)
in general than dragonegg.

Re: Some benchmark comparison of gcc4.5 and dragonegg (was dragonegg in FSF gcc?)

[Robert Dewar]

Vladimir Makarov wrote:
> Duncan Sands wrote:
>> Hi Steven,
>>
>>>> I think Jack wasn't suggesting that dragonegg should be changed to 
>>>> not be
>>>> a plugin any more.  I think he was suggesting that it should live in 
>>>> the gcc
>>>> repository rather than the LLVM repository.
>>> So, no offense, but the suggestion here is to make this subversive
>>> (for FSF GCC) plugin part of FSF GCC? What is the benefit of this for
>>> GCC? I don't see any. I just see a plugin trying to piggy-back on the
>>> hard work of GCC front-end developers and negating the efforts of
>>> those working on the middle ends and back ends.
>> I'm sorry you see the dragonegg project so negatively.  I think it is 
>> useful
>> for gcc (though not hugely useful), since it makes it easy to compare 
>> the gcc
>> and LLVM optimizers and code generators, not to mention the gcc and LLVM
>> approaches to LTO.  If LLVM manages to produce better code than gcc 
>> for some
>> testcase, then it is a convenient tool for the gcc devs to find out 
>> why, and
>> improve gcc.  If gcc is consistently better than LLVM then there's 
>> nothing to
>> worry about!  Of course, right now it is LLVM that is mostly playing 
>> catchup
>> with gcc, so for the moment it is principally the LLVM devs that get 
>> to learn
>> from gcc, but as LLVM improves the other direction is likely to occur 
>> more
>> often.
> I've tried to compare gcc4.5 and dragonegg a week ago on SPEC2000 on a 
> Core I7.
> Here are some results.
> 
>   Only SPECIn2000 for x86_64 has been compiled fully successfully by
> dragonegg.  There were a few compiler crashes including some in LLVM
> itself for SPECFP2000 and for SPECINT2000 for x86.
> 
> So here is SPECInt2000 for x86_64 comparison:
> 
> dragonegg: -O3 (with LLVM release build)
> gcc4.5: -O3 -flto (--enable-checking=release)
> 
>           Compilation Time  SPECINT2000
> Dragonegg 122.85user         2572
> gcc-4.5   283.49user         2841
> 
>   On integer benchmarks, dragonegg generates about 11% slower code.
> One interesting thing is that dragonegg is a really fast compiler.  It
> is 2.3 times faster than gcc.

Actually for my taste, you have to get a MUCH bigger factor in compile
time before you can call yourself a fast compiler (Realia COBOL by
comparison compiles millions of lines a minute of code on current
PC's, using just one core). GCC has taken a decision to favor
performance of the code absolutely over compiler performance.
That's not such a bad bet given how fast machines are getting.
So I think this compile time advantage is not that interesting.
> 
> Draggonegg generates smaller text sections but bigger data sections.
> Unfortunately, my scripts measure and compare only text sections.  Therefore
> I am not posting this text code size comparison because it has no 
> sense.  But looking
> at small benchmarks, I got an impression that gcc generates smaller code 
> (text + data)
> in general than dragonegg.

Usually you will find that to a first order approximation, speed and
size are linearly related.

Re: Some benchmark comparison of gcc4.5 and dragonegg (was dragonegg in FSF gcc?)

[Steven Bosscher]

On Wed, Apr 21, 2010 at 6:53 PM, Vladimir Makarov <vmakarov@redhat.com> wrote:
> One interesting thing is that dragonegg is a really fast compiler.  It
> is 2.3 times faster than gcc.

Yes, well, this is one thing "the crowd out there" complains about all
the time. It just appears to be almost impossible for GCC (the
project) to turn this around.

Do you also have per-benchmark compile time measurements, by any chance?

Ciao!
Steven

Re: Some benchmark comparison of gcc4.5 and dragonegg (was dragonegg in FSF gcc?)

[Steven Bosscher]

On Wed, Apr 21, 2010 at 6:56 PM, Robert Dewar <dewar@adacore.com> wrote:
> Actually for my taste, you have to get a MUCH bigger factor in compile
> time before you can call yourself a fast compiler (Realia COBOL by
> comparison compiles millions of lines a minute of code on current
> PC's, using just one core).

Heh, you always bring up the Realia compiler when there's a compile
time discussion. Must have been a really impressive piece of work,
that it was so fast :-)

> GCC has taken a decision to favor
> performance of the code absolutely over compiler performance.
> That's not such a bad bet given how fast machines are getting.
> So I think this compile time advantage is not that interesting.

I disagree (how unexpected is that? :-).

I think you are right that it is not per se a bad decision to favor
performance of the code over performance of the compiler itself. But a
quick investigation of, say, compile times for a linux kernel from GCC
3.1 to GCC 4.5 shows that GCC slows down faster than what Moore's law
compensates for. And people do complain about this. I'll admit it is
hard to say if the complainers are a significant number of GCC users
or just a loud minority...

Ciao!
Steven

Re: Some benchmark comparison of gcc4.5 and dragonegg (was dragonegg in FSF gcc?)

[Vladimir Makarov]

Steven Bosscher wrote:
> On Wed, Apr 21, 2010 at 6:53 PM, Vladimir Makarov <vmakarov@redhat.com> wrote:
>   
>> One interesting thing is that dragonegg is a really fast compiler.  It
>> is 2.3 times faster than gcc.
>>     
>
> Yes, well, this is one thing "the crowd out there" complains about all
> the time. It just appears to be almost impossible for GCC (the
> project) to turn this around.
>
>   
I don't think we should be too much worried about it.  GCC looks good in 
comparison with other industrial compiler with compile time point (and 
code size too) of view (e.g. SunStudio compiler is about 2 times slower 
and generates worse code on x86/x86_64 according to my benchmarking 2 
years ago, Intel is also slower but generates much better code than gcc).
> Do you also have per-benchmark compile time measurements, by any chance?
>
>   
No, I measure only overall compiler time for SPECInt2000 and SPECFP2000.

Re: Some benchmark comparison of gcc4.5 and dragonegg (was dragonegg in FSF gcc?)

[Vladimir Makarov]

Robert Dewar wrote:
> Vladimir Makarov wrote:
>> Duncan Sands wrote:
>>> Hi Steven,
>>>
>>>>> I think Jack wasn't suggesting that dragonegg should be changed to 
>>>>> not be
>>>>> a plugin any more.  I think he was suggesting that it should live 
>>>>> in the gcc
>>>>> repository rather than the LLVM repository.
>>>> So, no offense, but the suggestion here is to make this subversive
>>>> (for FSF GCC) plugin part of FSF GCC? What is the benefit of this for
>>>> GCC? I don't see any. I just see a plugin trying to piggy-back on the
>>>> hard work of GCC front-end developers and negating the efforts of
>>>> those working on the middle ends and back ends.
>>> I'm sorry you see the dragonegg project so negatively.  I think it 
>>> is useful
>>> for gcc (though not hugely useful), since it makes it easy to 
>>> compare the gcc
>>> and LLVM optimizers and code generators, not to mention the gcc and 
>>> LLVM
>>> approaches to LTO.  If LLVM manages to produce better code than gcc 
>>> for some
>>> testcase, then it is a convenient tool for the gcc devs to find out 
>>> why, and
>>> improve gcc.  If gcc is consistently better than LLVM then there's 
>>> nothing to
>>> worry about!  Of course, right now it is LLVM that is mostly playing 
>>> catchup
>>> with gcc, so for the moment it is principally the LLVM devs that get 
>>> to learn
>>> from gcc, but as LLVM improves the other direction is likely to 
>>> occur more
>>> often.
>> I've tried to compare gcc4.5 and dragonegg a week ago on SPEC2000 on 
>> a Core I7.
>> Here are some results.
>>
>>   Only SPECIn2000 for x86_64 has been compiled fully successfully by
>> dragonegg.  There were a few compiler crashes including some in LLVM
>> itself for SPECFP2000 and for SPECINT2000 for x86.
>>
>> So here is SPECInt2000 for x86_64 comparison:
>>
>> dragonegg: -O3 (with LLVM release build)
>> gcc4.5: -O3 -flto (--enable-checking=release)
>>
>>           Compilation Time  SPECINT2000
>> Dragonegg 122.85user         2572
>> gcc-4.5   283.49user         2841
>>
>>   On integer benchmarks, dragonegg generates about 11% slower code.
>> One interesting thing is that dragonegg is a really fast compiler.  It
>> is 2.3 times faster than gcc.
>
> Actually for my taste, you have to get a MUCH bigger factor in compile
> time before you can call yourself a fast compiler (Realia COBOL by
> comparison compiles millions of lines a minute of code on current
> PC's, using just one core). GCC has taken a decision to favor
> performance of the code absolutely over compiler performance.
> That's not such a bad bet given how fast machines are getting.
> So I think this compile time advantage is not that interesting.
For me definitely.  Also as I wrote I would not be too much worried 
about it.  GCC looks good in comparison with other industrial compiler 
with compile time (and code size too) point of view. Here I mean Intel, 
SunStudio, PathScale, Open64 ones.
>
>>
>> Draggonegg generates smaller text sections but bigger data sections.
>> Unfortunately, my scripts measure and compare only text sections.  
>> Therefore
>> I am not posting this text code size comparison because it has no 
>> sense.  But looking
>> at small benchmarks, I got an impression that gcc generates smaller 
>> code (text + data)
>> in general than dragonegg.
>
> Usually you will find that to a first order approximation, speed and
> size are linearly related.
>
I am agree with this for moderately optimizing compilers.  But for 
highly optimizing compilers it might be no true.  Intel generates much 
better and bigger code than gcc.  Although it might be mostly because of 
code versioning  (including one for different subtargets).

Re: Some benchmark comparison of gcc4.5 and dragonegg (was dragonegg in FSF gcc?)

[Chris Lattner]

On Apr 21, 2010, at 9:53 AM, Vladimir Makarov wrote:

> Only SPECIn2000 for x86_64 has been compiled fully successfully by
> dragonegg.  There were a few compiler crashes including some in LLVM
> itself for SPECFP2000 and for SPECINT2000 for x86.
> 
> So here is SPECInt2000 for x86_64 comparison:
> 
> dragonegg: -O3 (with LLVM release build)
> gcc4.5: -O3 -flto (--enable-checking=release)
> 
>         Compilation Time  SPECINT2000
> Dragonegg 122.85user         2572
> gcc-4.5   283.49user         2841
> 
> On integer benchmarks, dragonegg generates about 11% slower code.
> One interesting thing is that dragonegg is a really fast compiler.  It
> is 2.3 times faster than gcc.

This is definitely interesting, but you're also comparing apples and oranges here (for both compile time and performance).  Can you get numbers showing GCC -O3 and dragonegg with LTO to get a better comparison?

-Chris

Re: Some benchmark comparison of gcc4.5 and dragonegg (was dragonegg in FSF gcc?)

[Manuel López-Ibáñez]

On 21 April 2010 19:11, Vladimir Makarov <vmakarov@redhat.com> wrote:
> I don't think we should be too much worried about it.  GCC looks good in
> comparison with other industrial compiler with compile time point (and code
> size too) of view (e.g. SunStudio compiler is about 2 times slower and
> generates worse code on x86/x86_64 according to my benchmarking 2 years ago,
> Intel is also slower but generates much better code than gcc).

There is the perception that GCC is too slow and every release it gets
much slower for not significant gain. At some point one has to start
asking whether there is something that can be done to alleviate this.
Either by showing that in fact there is a significant gain, or by
improving compilation speed. But we should be worried.

We have to wait until clang can compile as much C++ code as GCC and
implement a similar feature set, but the differences are going to be
much larger when LLVM uses Clang. [*] This is a major selling point of
Clang/LLVM against GCC. You can choose to ignore it but it is out
there unchallenged and GCC users are listening to it. And it will
probably show that reimplementing GCC FEs using LLVM infrastructure is
an expensive but rewarding project. In fact, given the LLVM/Clang
already has many features that GCC has not, it is not clear what is
the overhead of implementing those features in GCC.

So do you think that the differences in compilation speed can be
explained mostly by lack of optimization features in LLVM?

Cheers,

Manuel.

[*] I also would be very interested on knowing what is the impact of
the integrated assembler approach in compile time:
http://blog.llvm.org/2010/04/intro-to-llvm-mc-project.html

Re: Some benchmark comparison of gcc4.5 and dragonegg (was dragonegg in FSF gcc?)

[Paolo Bonzini]

On 04/21/2010 07:04 PM, Steven Bosscher wrote:
> On Wed, Apr 21, 2010 at 6:56 PM, Robert Dewar<dewar@adacore.com>  wrote:
>> Actually for my taste, you have to get a MUCH bigger factor in compile
>> time before you can call yourself a fast compiler (Realia COBOL by
>> comparison compiles millions of lines a minute of code on current
>> PC's, using just one core).
>
> Heh, you always bring up the Realia compiler when there's a compile
> time discussion. Must have been a really impressive piece of work,
> that it was so fast :-)

It was fast, I used it for my first summer job, and spent some time 
looking at its output too.  :-)

And actually I'm impressed especially because it wasn't (as far as I 
remember) an optimizing compiler, yet it was written in itself _and_ so 
fast.

Paolo

Re: Some benchmark comparison of gcc4.5 and dragonegg (was dragonegg in FSF gcc?)

[Duncan Sands]

Hi Vladimir, thank you for doing this benchmarking.

> Only SPECIn2000 for x86_64 has been compiled fully successfully by
> dragonegg. There were a few compiler crashes including some in LLVM
> itself for SPECFP2000 and for SPECINT2000 for x86.

Sorry about that.  Can you please send me preprocessed code for the
spec tests that crashed the plugin (unless you are not allowed to).
By the way, if you target something (eg: i386) that doesn't have SSE
support then I've noticed that the plugin tends to crash on code that
does vector operations.  If you have assertions turned on in LLVM then
you get something like:

Assertion `TLI.isTypeLegal(Op.getValueType()) && "Intrinsic uses a non-legal 
type?"' failed.
Stack dump:
0.	Running pass 'X86 DAG->DAG Instruction Selection' on function '@_ada_sse_nolib'

So if the compile failures are of that kind, no need to send testcases, I
already have several.

Best wishes,

Duncan.

Re: Some benchmark comparison of gcc4.5 and dragonegg (was dragonegg in FSF gcc?)

[Robert Dewar]

Steven Bosscher wrote:
> On Wed, Apr 21, 2010 at 6:56 PM, Robert Dewar <dewar@adacore.com> wrote:
>> Actually for my taste, you have to get a MUCH bigger factor in compile
>> time before you can call yourself a fast compiler (Realia COBOL by
>> comparison compiles millions of lines a minute of code on current
>> PC's, using just one core).
> 
> Heh, you always bring up the Realia compiler when there's a compile
> time discussion. Must have been a really impressive piece of work,
> that it was so fast :-)

Well if you look at the parameters ... that compiler was written
in the early 80's (the first machine on which we ran it was a
PC-1 with diskettes, and it achieved about 10,000 lines/minute
in that environment. Why was compilation time important? Because
COBOL programmers did and still do routinely write very large
programs (a COBOL program = a C function roughly). A COBOL
run-unit (= a C program roughly) is composed of one or more
programs, and very often it was the style for there to be
only a few really large programs, so even back in 1980, COBOL
programmers routinely compiled files consisting of tens of
thousands of lines of code. So compile time speed was a factor.

And indeed Realia COBOL was much faster than the major
competitor Microfocus.

But the interesting thing is that over time, that advantage
evaporated. It was very interesting to compile 10,000 lines
a minute when the competition does only 1,000 lpm, but it
is no longer so exciting to compile 1,000,000 lpm with the
competition managing 100,000 lpm, since both are fast
enough in practice.
> 
>> GCC has taken a decision to favor
>> performance of the code absolutely over compiler performance.
>> That's not such a bad bet given how fast machines are getting.
>> So I think this compile time advantage is not that interesting.
> 
> I disagree (how unexpected is that? :-).
> 
> I think you are right that it is not per se a bad decision to favor
> performance of the code over performance of the compiler itself. But a
> quick investigation of, say, compile times for a linux kernel from GCC
> 3.1 to GCC 4.5 shows that GCC slows down faster than what Moore's law
> compensates for.

You are ignoring the multi-core effect!

It's interesting to look at the time it takes to run our internal
gnat test suite (tens of millions of lines of code, hundreds
of thousands of files, 12000 distinct test cases).

This used to take about an hour for many many years, the test
suite seemed to grow fast enough to compensate for improved
processor performance.

But with the advent of multi-core machines, the picture has
changed entirely, although the test suite has continued to
grow rapidly in size, the time to run it is now down to
15 minutes when we run on a multi-core machine, and we just
got a machine with 2x6 cores, each hyperthreaded, which will
probably cut this in half again.

Given it is so easy to take advantage of multi-cores when
compiling large projects, the overall effect is definitely
that GCC 4.5 is much faster than GCC 3.1 for handling large
projects with latest hardware.

I do realize that some people are running gcc on very old
machines, that particularly happens say in developing
countries or with students or hobbyists using old cast
off machines. And for those compile time is a problem,
but for out Ada users, many of whom have absolutely giant
programs of millions of lines, compile time speed has not
been an issue (we would know if it was, people would
tell us, they tell us of ANY problems they have).

The one exception is that native compilation on VMS
is slow, but that's a consequence of the VMS file
system, where opening lots of small files is slow.
We are planning to encourage people using VMS to
switch to using PC-based cross-compilation.

Re: Some benchmark comparison of gcc4.5 and dragonegg (was dragonegg in FSF gcc?)

[Vladimir Makarov]

Chris Lattner wrote:
> On Apr 21, 2010, at 9:53 AM, Vladimir Makarov wrote:
>
>   
>> Only SPECIn2000 for x86_64 has been compiled fully successfully by
>> dragonegg.  There were a few compiler crashes including some in LLVM
>> itself for SPECFP2000 and for SPECINT2000 for x86.
>>
>> So here is SPECInt2000 for x86_64 comparison:
>>
>> dragonegg: -O3 (with LLVM release build)
>> gcc4.5: -O3 -flto (--enable-checking=release)
>>
>>         Compilation Time  SPECINT2000
>> Dragonegg 122.85user         2572
>> gcc-4.5   283.49user         2841
>>
>> On integer benchmarks, dragonegg generates about 11% slower code.
>> One interesting thing is that dragonegg is a really fast compiler.  It
>> is 2.3 times faster than gcc.
>>     
>
> This is definitely interesting, but you're also comparing apples and oranges here (for both compile time and performance).  Can you get numbers showing GCC -O3 and dragonegg with LTO to get a better comparison?
>
>   
Dragonegg does not work with -flto.  It generates assembler code on 
which gas complaints (a lot of non-assembler code like target 
data-layout which are not in comments).

So I'll do gcc -O3 without -flto.  I don't think it will change average 
SPECINT2000 rate significantly (although it can change separte benchmark 
significantly)  but it will make gcc compiler much faster (may be 2 
times because I did not use -fwhole-program).  I'll post the results in 
an hour.

Re: Some benchmark comparison of gcc4.5 and dragonegg (was dragonegg in FSF gcc?)

[Vladimir Makarov]

Duncan Sands wrote:
> Hi Vladimir, thank you for doing this benchmarking.
>
>> Only SPECIn2000 for x86_64 has been compiled fully successfully by
>> dragonegg. There were a few compiler crashes including some in LLVM
>> itself for SPECFP2000 and for SPECINT2000 for x86.
>
> Sorry about that. Can you please send me preprocessed code for the
> spec tests that crashed the plugin (unless you are not allowed to).
> By the way, if you target something (eg: i386) that doesn't have SSE
> support then I've noticed that the plugin tends to crash on code that
> does vector operations. If you have assertions turned on in LLVM then
> you get something like:
>
> Assertion `TLI.isTypeLegal(Op.getValueType()) && "Intrinsic uses a 
> non-legal type?"' failed.
> Stack dump:
> 0. Running pass 'X86 DAG->DAG Instruction Selection' on function 
> '@_ada_sse_nolib'
>
> So if the compile failures are of that kind, no need to send testcases, I
> already have several.
>
I have one different crash on galgel:

/home/vmakarov/build/dragonegg/64/bin/gfortran -c -o bifg21.o  -ffixed-form -mpc64          -O3 -m32 -mpc64 -fplugin=/home/cygnus/vmakarov/build/dragonegg/dragonegg/dragonegg.so   bifg21.f90
specmake: *** Warning: File `Makefile.spec' has modification time in the future (1271359622 > 1271358843)
f951: /to/scratch/vmakarov/build/dragonegg/llvm/lib/VMCore/Instructions.cpp:320: void llvm::CallInst::init(llvm::Value*, llvm::Value* const*, unsigned int): Assertion `(NumParams == FTy->getNumParams() || (FTy->isVarArg() && NumParams > FTy->getNumParams())) && "Calling a function with bad signature!"' failed.
*** WARNING *** there are active plugins, do not report this as a bug unless you can reproduce it without enabling any plugins.
Event                            | Plugins
PLUGIN_FINISH_UNIT               | dragonegg
PLUGIN_FINISH                    | dragonegg
PLUGIN_START_UNIT                | dragonegg
bifg21.f90: In function ‘bifg21_’:
bifg21.f90:21:0: internal compiler error: Aborted
Please submit a full bug report,
with preprocessed source if appropriate.
See <http://gcc.gnu.org/bugs.html> for instructions.
specmake: *** [bifg21.o] Error 1

It is impossible (as I know) to send a preprocessed file for fortran90.  It needs other program files to compile too.  But may be this diagnostic is still useful for you.

Re: Some benchmark comparison of gcc4.5 and dragonegg (was dragonegg in FSF gcc?)

[Robert Dewar]

> I am agree with this for moderately optimizing compilers.  But for 
> highly optimizing compilers it might be no true.  Intel generates much 
> better and bigger code than gcc.  Although it might be mostly because of 
> code versioning  (including one for different subtargets).

I don't think this is true if you select the appropriate option in
ICC to generate code for just one target, but of course if you let
ICC generate code for multiple targets (e.g. GenuineIntel with SSE
vs AuthenticAMD without SSE), then of course you get larger objects,
since you have a run time test and then essentially two separate
compilations of the same code in the same object.

Re: Some benchmark comparison of gcc4.5 and dragonegg (was dragonegg in FSF gcc?)

[Chris Lattner]

On Apr 21, 2010, at 11:11 AM, Vladimir Makarov wrote:

>> 
>> This is definitely interesting, but you're also comparing apples and oranges here (for both compile time and performance). Can you get numbers showing GCC -O3 and dragonegg with LTO to get a better comparison?
>> 
>>  
> Dragonegg does not work with -flto.  It generates assembler code on which gas complaints (a lot of non-assembler code like target data-layout which are not in comments).

Ok, I didn't know that didn't get wired up.  I'm not familiar with dragonegg, it might require gold with the llvm lto gold plugin or something.

> So I'll do gcc -O3 without -flto.  I don't think it will change average SPECINT2000 rate significantly (although it can change separte benchmark significantly)  but it will make gcc compiler much faster (may be 2 times because I did not use -fwhole-program).  I'll post the results in an hour.

Sounds good, thanks!  I suspect the gcc build times will improve.

-Chris

Re: Some benchmark comparison of gcc4.5 and dragonegg (was dragonegg in FSF gcc?)

[Robert Dewar]

Paolo Bonzini wrote:
> On 04/21/2010 07:04 PM, Steven Bosscher wrote:
>> On Wed, Apr 21, 2010 at 6:56 PM, Robert Dewar<dewar@adacore.com>  wrote:
>>> Actually for my taste, you have to get a MUCH bigger factor in compile
>>> time before you can call yourself a fast compiler (Realia COBOL by
>>> comparison compiles millions of lines a minute of code on current
>>> PC's, using just one core).
>> Heh, you always bring up the Realia compiler when there's a compile
>> time discussion. Must have been a really impressive piece of work,
>> that it was so fast :-)
> 
> It was fast, I used it for my first summer job, and spent some time 
> looking at its output too.  :-)
> 
> And actually I'm impressed especially because it wasn't (as far as I 
> remember) an optimizing compiler, yet it was written in itself _and_ so 
> fast.

It did not do what we would call global optimization, but it had very
good local optimization, and very good handling of jumps and effective
inlining of PERFORMS which made a big difference. For example

    HANDLE-BALANCE.
      IF BALANCE IS NEGATIVE THEN
         PERFORM SEND-BILL
      ELSE
         PERFORM RECORD-CREDIT
      END-IF.

    SEND-BILL.
      <<code to send bill>>

    RECORD-CREDIT.
      <<code to record credit>>

(see you can read COBOL even if you never saw it before :-) :-))

was compiled as though the two performs were inlined. This is
valuable in COBOL (and not done by the IBM mainframe compiler
at the time), since it is the style in COBOL to do these small
named local refinements, COBOL programmers consider nesting
constructs such as IF's to be bad style, preferring instead
to name the refined blocks as shown above (see what a very
nice compact syntax COBOL has for this kind of refinement,
much better than the algol or fortran style languages :-)
still shouldn't start a COBOL flame war here, sorry!)
> 
> Paolo

Re: Some benchmark comparison of gcc4.5 and dragonegg (was dragonegg in FSF gcc?)

[Robert Dewar]

Manuel López-Ibáñez wrote:
> On 21 April 2010 19:11, Vladimir Makarov <vmakarov@redhat.com> wrote:
>> I don't think we should be too much worried about it.  GCC looks good in
>> comparison with other industrial compiler with compile time point (and code
>> size too) of view (e.g. SunStudio compiler is about 2 times slower and
>> generates worse code on x86/x86_64 according to my benchmarking 2 years ago,
>> Intel is also slower but generates much better code than gcc).
> 
> There is the perception that GCC is too slow and every release it gets
> much slower for not significant gain. At some point one has to start
> asking whether there is something that can be done to alleviate this.
> Either by showing that in fact there is a significant gain, or by
> improving compilation speed. But we should be worried.

We (here we = the commercial company AdaCore) would be worried if
ANY of our customers were worried, but they are not, they see a
continuous effective improvement in compile speed using the latest
available hardware, and it's not a factor for them.

> So do you think that the differences in compilation speed can be
> explained mostly by lack of optimization features in LLVM?

Nobody said that, the explanation is of course FAR more complex,
and to some extent it may be a matter of orientation and
enthusiasm. There is more enthusiasm in the gcc community for
implementation new optimizations to improve performance, than
in speeding up the existing compiler, which is quite
understandable.

Re: Some benchmark comparison of gcc4.5 and dragonegg (was dragonegg in FSF gcc?)

[Basile Starynkevitch]

Steven Bosscher wrote:
> On Wed, Apr 21, 2010 at 6:56 PM, Robert Dewar <dewar@adacore.com> wrote:
>> Actually for my taste, you have to get a MUCH bigger factor in compile
>> time before you can call yourself a fast compiler (Realia COBOL by
>> comparison compiles millions of lines a minute of code on current
>> PC's, using just one core).
> 
> Heh, you always bring up the Realia compiler when there's a compile
> time discussion. Must have been a really impressive piece of work,
> that it was so fast :-)

Another example of a compiler which compiles quickly but produces slow 
code is tinycc	http://www.tinycc.org/  http://repo.or.cz/w/tinycc.git
(the program is called tcc)

In my very small & rusty experience, it did happen that tcc used to 
generate incorrect machine code, at least some old version of tcc did 
compile some old version of MELT generated code incorrectly on x86-64 
[the tcc-generated *.so crashed, while the *.so generated by GCC from 
same source did run correctly].

Now, it is indeed true that TCC probably evolved since (& MELT also), 
and I don't know where and how to get the newest TCC source (is the 
"git clone git://repo.or.cz/tinycc.git" command enough?, the version 
number seems to be 0.9.25 since more than a year...).

A useless measure of compile time (within the MELT branch, subdirectory 
gcc of the build directory. warmelt-first.1.c is a generated C file of 
96KLOC)

% time gcc-4.5 -g -DIN_GCC -DHAVE_CONFIG_H  \
   -I melt-private-build-include -I. -fPIC -c -o warmelt-first.1.pic.o 
warmelt-first.1.c

gcc-4.5 -g -DIN_GCC -DHAVE_CONFIG_H -I melt-private-build-include -I. 
-fPIC -  10.29s user 0.41s system 100% cpu 10.695 total

  % time tcc -g -DIN_GCC -DHAVE_CONFIG_H -I melt-private-build-include 
-I. -fPIC -c -o warmelt-first.1.pic.o warmelt-first.1.c
tcc -g -DIN_GCC -DHAVE_CONFIG_H -I melt-private-build-include -I. -fPIC 
-c -o  0.63s user 0.03s system 99% cpu 0.660 total

The current tcc is not really usable for me, I am not able to do a melt 
bootstrap (that is to compile warmelt-*.0.c into MELT modules 
warmelt*0.so, use them to generate warmelt*1.c, compile them to 
warmelt*1.so, and use them to generate warmelt*2.c). This MELT bootstrap 
is routinely done with GCC 4.4 & GCC 4.5 (the warmelt*1.c is generated 
but does not work ok).

Regards.

PS. About GCC MELT see http://gcc.gnu.org/wiki/MiddleEndLispTranslator
-- 
Basile STARYNKEVITCH         http://starynkevitch.net/Basile/
email: basile<at>starynkevitch<dot>net mobile: +33 6 8501 2359
8, rue de la Faiencerie, 92340 Bourg La Reine, France
*** opinions {are only mines, sont seulement les miennes} ***

Re: Some benchmark comparison of gcc4.5 and dragonegg (was dragonegg in FSF gcc?)

[Robert Dewar]

 From the early days, WATFOR was an impressively fast compiler,
and then there is always Borland Pascal.

I once gave a talk at the SIGPLAN compiler conference whose
theme was the great successes we were having in managing
to dramatically slow down compilers :-)

Re: Some benchmark comparison of gcc4.5 and dragonegg (was dragonegg in FSF gcc?)

[Vladimir Makarov]

Chris Lattner wrote:
> On Apr 21, 2010, at 11:11 AM, Vladimir Makarov wrote:
>
>   
>>> This is definitely interesting, but you're also comparing apples and oranges here (for both compile time and performance). Can you get numbers showing GCC -O3 and dragonegg with LTO to get a better comparison?
>>>
>>>  
>>>       
>> Dragonegg does not work with -flto.  It generates assembler code on which gas complaints (a lot of non-assembler code like target data-layout which are not in comments).
>>     
>
> Ok, I didn't know that didn't get wired up.  I'm not familiar with dragonegg, it might require gold with the llvm lto gold plugin or something.
>
>   
>> So I'll do gcc -O3 without -flto.  I don't think it will change average SPECINT2000 rate significantly (although it can change separte benchmark significantly)  but it will make gcc compiler much faster (may be 2 times because I did not use -fwhole-program).  I'll post the results in an hour.
>>     
>
> Sounds good, thanks!  I suspect the gcc build times will improve.
>   
Here the results of SPECINT2000 on x86_64 for dragonegg -O3 vs gcc-4.5 -O3.

dragonegg: -O3 (release build)
gcc4.5: -O3 (--enable-checking=release)

          Compilation Time  SPECINT2000
Dragonegg 122.85user         2572
gcc-4.5   142.76user         2784

  Dragonegg generates about 9% slower code (vs 11% for gcc with
-flto).  Without -flto, gcc4.5 is only 16% slower than dragonegg.

Re: Some benchmark comparison of gcc4.5 and dragonegg (was dragonegg in FSF gcc?)

[Eric Botcazou]

> We (here we = the commercial company AdaCore) would be worried if
> ANY of our customers were worried, but they are not, they see a
> continuous effective improvement in compile speed using the latest
> available hardware, and it's not a factor for them.

The Ada compiler is a little special here because our internal measures show
that GCC 4.x based Ada compilers are faster than GCC 3.x based ones, all other 
things being equal, at least on x86/Linux.

GCC 4.5 hasn't been evaluated yet though.

-- 
Eric Botcazou

Re: Some benchmark comparison of gcc4.5 and dragonegg (was dragonegg in FSF gcc?)

[Duncan Sands]

Hi Vladimir,

> Dragonegg does not work with -flto. It generates assembler code on which
> gas complaints (a lot of non-assembler code like target data-layout
> which are not in comments).

actually it does work with -flto, in an awkward way.  When you use -flto
it spits out LLVM IR.  You need to use -S, otherwise the system assembler
tries (and fails) to compile this.  You need to then use llvm-as to turn
this into LLVM bitcode.  You can then link and optimize the bitcode either
by hand (using llvm-ld) or using the gold plugin, as described in
   http://llvm.org/docs/GoldPlugin.html

It is annoying that gcc insists on running the system assembler when passed
-c.  Not running the assembler isn't only good for avoiding the -S + llvm-as
rigmarole mentioned above.  LLVM is now capable of writing out object files
directly, i.e. without having to pass via an assembler at all.  It would be
neat if I could have the plugin immediately write out the final object file
if -c is passed.  I didn't work out how to do this yet.  It probably requires
some gcc modifications, so maybe something can be done for gcc-4.6.

For transparent LTO another possibility is to encode LLVM bitcode in the
assembler in the same way as gcc does for gimple when passed -flto.  I didn't
investigate this yet.

Ciao,

Duncan.

Re: Some benchmark comparison of gcc4.5 and dragonegg (was dragonegg in FSF gcc?)

[Vladimir Makarov]

Robert Dewar wrote:
>
>> I am agree with this for moderately optimizing compilers.  But for 
>> highly optimizing compilers it might be no true.  Intel generates 
>> much better and bigger code than gcc.  Although it might be mostly 
>> because of code versioning  (including one for different subtargets).
>
> I don't think this is true if you select the appropriate option in
> ICC to generate code for just one target, but of course if you let
> ICC generate code for multiple targets (e.g. GenuineIntel with SSE
> vs AuthenticAMD without SSE), then of course you get larger objects,
> since you have a run time test and then essentially two separate
> compilations of the same code in the same object.
>
>
It is hard to find appropriate options even if we put mutliple targets 
code generation away.  For example, if you use -fast for ICC it means 
using -static libraries which makes code much bigger.

Although it is not right argument to what you mean.  But example about 
vectorization would be right.  ICC vectorizes many more loops than gcc 
does.  Vectorized loops is much bigger in size than their non-vectorized 
variants.  So faster code does not mean smaller code in general.  There 
are a lot of optimization which makes code bigger and faster: like 
function versioning (based on argument values), aggressive inlining, 
modulo scheduling, vectorization, loop unrolling, loop versioning, loop 
tiling etc.  So even if the both compiler do the same optimizations and 
if one compiler is more successful in such optimizations, the generated 
code will be bigger and faster.

Re: Some benchmark comparison of gcc4.5 and dragonegg (was dragonegg in FSF gcc?)

[Toon Moene]

Robert Dewar wrote:

> Actually for my taste, you have to get a MUCH bigger factor in compile
> time before you can call yourself a fast compiler (Realia COBOL by
> comparison compiles millions of lines a minute of code on current
> PC's, using just one core).

Obviously, apart from comparing a sufficiently large set of compilers on 
this, "speed of compilation" is mostly in the eye of the beholder.

Subjectively, as of gcc/gfortran 4.4, our (roughly 1 million lines of 
Fortran + 30,000 lines of C) code gets compiled (optimized and 
vectorized at -O3) in about 5 minutes on a quad core machine (using make 
-j8).

As an absolute number, this tells you nothing.  But as a measure of 
usefulness, it means that from 4.4 onwards, it is possible to recompile 
our complete weather forecasting suite at *every* new run, 4 times a day.

You bet that's sometimes useful ...

-- 
Toon Moene - e-mail: toon@moene.org - phone: +31 346 214290
Saturnushof 14, 3738 XG  Maartensdijk, The Netherlands
At home: http://moene.org/~toon/
Progress of GNU Fortran: http://gcc.gnu.org/gcc-4.5/changes.html#Fortran

Re: Some benchmark comparison of gcc4.5 and dragonegg (was dragonegg in FSF gcc?)

[Robert Dewar]

Vladimir Makarov wrote:

> Although it is not right argument to what you mean.  But example about 
> vectorization would be right.  ICC vectorizes many more loops than gcc 
> does.  Vectorized loops is much bigger in size than their non-vectorized 
> variants.  So faster code does not mean smaller code in general.  There 
> are a lot of optimization which makes code bigger and faster: like 
> function versioning (based on argument values), aggressive inlining, 
> modulo scheduling, vectorization, loop unrolling, loop versioning, loop 
> tiling etc.  So even if the both compiler do the same optimizations and 
> if one compiler is more successful in such optimizations, the generated 
> code will be bigger and faster.

Sure, we can all find such examples, but if you take a large program,
(say hundreds of thousands of lines), you will find that the speed
vs size relation holds pretty well.

Re: Some benchmark comparison of gcc4.5 and dragonegg (was dragonegg in FSF gcc?)

[Robert Dewar]

Toon Moene wrote:
> Robert Dewar wrote:
> 
>> Actually for my taste, you have to get a MUCH bigger factor in compile
>> time before you can call yourself a fast compiler (Realia COBOL by
>> comparison compiles millions of lines a minute of code on current
>> PC's, using just one core).
> 
> Obviously, apart from comparing a sufficiently large set of compilers on 
> this, "speed of compilation" is mostly in the eye of the beholder.
> 
> Subjectively, as of gcc/gfortran 4.4, our (roughly 1 million lines of 
> Fortran + 30,000 lines of C) code gets compiled (optimized and 
> vectorized at -O3) in about 5 minutes on a quad core machine (using make 
> -j8).
> 
> As an absolute number, this tells you nothing.  But as a measure of 
> usefulness, it means that from 4.4 onwards, it is possible to recompile 
> our complete weather forecasting suite at *every* new run, 4 times a day.
> 
> You bet that's sometimes useful ...


Right, and I think once you get down to 5 minutes, then you are good
enough, it is in the minor convenience category to get down to 2 minutes.

The Realia COBOL compiler, written entirely in COBOL, could compile
itself in less than a minute on a 25 megahertz 386, but 5 minutes
would have been fine (of course the compiler was a small program
compared to many of the customer programs, less than a hundred thousand
lines of COBOL).

Re: Some benchmark comparison of gcc4.5 and dragonegg (was dragonegg in FSF gcc?)

[Laurent GUERBY]

On Wed, 2010-04-21 at 14:03 -0400, Robert Dewar wrote:
> I do realize that some people are running gcc on very old
> machines, that particularly happens say in developing
> countries or with students or hobbyists using old cast
> off machines. 

For those developping free software the compile farm
has some nice iron:

http://gcc.gnu.org/wiki/CompileFarm

> And for those compile time is a problem,
> but for out Ada users, many of whom have absolutely giant
> programs of millions of lines, compile time speed has not
> been an issue (we would know if it was, people would
> tell us, they tell us of ANY problems they have).
> 
> The one exception is that native compilation on VMS
> is slow, but that's a consequence of the VMS file
> system, where opening lots of small files is slow.
> We are planning to encourage people using VMS to
> switch to using PC-based cross-compilation.

In my (limited) experience for daily development link times on the
Windows platform for big Ada applications are an issue too, not compile
times.

Laurent

Re: Some benchmark comparison of gcc4.5 and dragonegg (was dragonegg in FSF gcc?)

[Vladimir Makarov]

Robert Dewar wrote:
> Vladimir Makarov wrote:
>
>> Although it is not right argument to what you mean.  But example 
>> about vectorization would be right.  ICC vectorizes many more loops 
>> than gcc does.  Vectorized loops is much bigger in size than their 
>> non-vectorized variants.  So faster code does not mean smaller code 
>> in general.  There are a lot of optimization which makes code bigger 
>> and faster: like function versioning (based on argument values), 
>> aggressive inlining, modulo scheduling, vectorization, loop 
>> unrolling, loop versioning, loop tiling etc.  So even if the both 
>> compiler do the same optimizations and if one compiler is more 
>> successful in such optimizations, the generated code will be bigger 
>> and faster.
>
> Sure, we can all find such examples, but if you take a large program,
> (say hundreds of thousands of lines), you will find that the speed
> vs size relation holds pretty well.
>
  Definitely not for Intel compiler and not for modern x86_64 processors 
(although it is most probably true for some other processors like ARM).  
ICC really generates much bigger code than GCC even we take subtarget 
versioning away.  The closest analog on x86_64 for gcc -O3 would be -O3 
-xT for icc.  -xT means generating code only one subtarget which is Core2.

  I tried to compile the biggest one file program I have (about 500K 
lines).   ICC crashed on it because there is not enough memory (8GB) in 
comparison with gcc which is doing fine with 2GB memory.   So I had to 
check SPEC2006.  In average the code increase on all SPEC2006 for ICC 
was 34%.  But because you mentioned programs with hundreds of thousands 
of lines, I am also giving numbers for some programs from SPEC2006.

                  lines     Intel code size increase
gromacs    400K    23%
tonto         125K    29%
wrf            115K    44%
gobmk       197K    -2%

Already long ago I got impression that ICC is good mostly for FP 
programs (for integer benchmarks gcc frequently generates a better code) 
but if icc stays its course, gcc will be always a better system compiler.

Re: Poor internal documentation (was: dragonegg in FSF gcc?)

[Philipp Thomas]

* Ian Lance Taylor (iant@google.com) [20100413 00:41]:

> Details of GIMPLE IR: poor.
> Details of tree IR: poor.
> How to write a new optimization pass: poor.
> How to write a new frontend: nonexistent.
> General overview of compiler source: nonexistent.
> Overview of internal compiler datastructures: nonexistent.

I'd say these these warrant an additional bullet "Documentation" under
"Improving GCC" on the GCC wiki that then lists (at least) these points.
It's not much, but it at least shows the GCC developers are aware and just
maybe it does attract the interest of someone.

Philipp

Re: Poor internal documentation (was: dragonegg in FSF gcc?)

[Manuel López-Ibáñez]

On 23 April 2010 15:05, Philipp Thomas <pth@suse.de> wrote:
> * Ian Lance Taylor (iant@google.com) [20100413 00:41]:
>
>> Details of GIMPLE IR: poor.
>> Details of tree IR: poor.
>> How to write a new optimization pass: poor.
>> How to write a new frontend: nonexistent.
>> General overview of compiler source: nonexistent.
>> Overview of internal compiler datastructures: nonexistent.
>
> I'd say these these warrant an additional bullet "Documentation" under
> "Improving GCC" on the GCC wiki that then lists (at least) these points.
> It's not much, but it at least shows the GCC developers are aware and just
> maybe it does attract the interest of someone.

Great! Go ahead, please. The wiki is easy to edit. Bonus points if you
collect there links to the existing documentation, so anyone wishing
to help has the many sources at hand.

Cheers,

Manuel.

Documentation legal issues (Was: Re: Poor internal documentation)

[Joern Rennecke]

Quoting Manuel López-Ibáñez <lopezibanez@gmail.com>:

> On 23 April 2010 15:05, Philipp Thomas <pth@suse.de> wrote:
>> * Ian Lance Taylor (iant@google.com) [20100413 00:41]:
>>
>>> Details of GIMPLE IR: poor.
>>> Details of tree IR: poor.
>>> How to write a new optimization pass: poor.
>>> How to write a new frontend: nonexistent.
>>> General overview of compiler source: nonexistent.
>>> Overview of internal compiler datastructures: nonexistent.
>>
>> I'd say these these warrant an additional bullet "Documentation" under
>> "Improving GCC" on the GCC wiki that then lists (at least) these points.
>> It's not much, but it at least shows the GCC developers are aware and just
>> maybe it does attract the interest of someone.
>
> Great! Go ahead, please. The wiki is easy to edit. Bonus points if you
> collect there links to the existing documentation, so anyone wishing
> to help has the many sources at hand.

However, is that not putting well-meaning contributers in peril of infringing
on the Copyright of the FSF in GCC by putting GPLed code into a GFDLed
document?  Often, you want to use some snippets of code from the sources as
an example, or lift some explanation from a comment in order to write
documentation.
If the legal entity doing this is not the one who contributed the code in
the first place, the only right they have to the code is what is granted
under the GPL.  Posting a patch with such code to the GCC mailing list
without a GFDL license grant would be Copyright infringement, unless
the poster could be construed to act on behalf of the FSF due to a
maintainership held, or the post is considered internal to the FSF -
I'm not sure if either of these would apply, but I don't think that
could possibly apply for a new contributer who has not at least
write-after-approval status.
Or will there be a license grant to cover such uses of GPLed code under
the GPL?

Is the steering committee empowered and willing to do that?

Re: Testing GCC on Cygwin made substantially easier [was Re: dragonegg in FSF gcc?]

[Laurynas Biveinis]

2010/4/13 Dave Korn <dave.korn.cygwin@googlemail.com>:
>  Until I find time to do a more major rewrite, anyone who wants to do testing
> on Cygwin could do worse than apply the sticking-plaster patch that I posted at:
>
>  http://www.mail-archive.com/cygwin-patches@cygwin.com/msg04677.html
>
> and build themselves a locally modified version of the Cygwin DLL that will
> happily run make check at significant -j levels (I think I tried 12 at most;
> I've only got a dual-core cpu so it wasn't exactly efficient, but it proved
> that the patch holds up under substantial load).

I do not test on Cygwin these days, but previously I did and I wish I
knew this back then. I have added a note to
http://gcc.gnu.org/wiki/Testing_GCC . Thanks for the info!

-- 
Laurynas

Re: Poor internal documentation (was: dragonegg in FSF gcc?)

[Philipp Thomas]

On Fri, 23 Apr 2010 16:23:29 +0200, Manuel López-Ibáñez
<lopezibanez@gmail.com> wrote:

>Great! Go ahead, please. The wiki is easy to edit.

Finally I got around to do it.

Editing is easy ... kind of :) Creating the Links was easy but I
failed do discover how I could actually make them point to other wiki
pages. 

> Bonus points if you collect there links to the existing documentation,
> so anyone wishing to help has the many sources at hand.

Maybe I will find the time but I doubt that.

Philipp

Re: Poor internal documentation (was: dragonegg in FSF gcc?)

[Jonathan Wakely]

On 5 June 2010 04:53, Philipp Thomas <Philipp.Thomas2@gmx.net> wrote:
> On Fri, 23 Apr 2010 16:23:29 +0200, Manuel López-Ibáñez
> <lopezibanez@gmail.com> wrote:
>
>>Great! Go ahead, please. The wiki is easy to edit.
>
> Finally I got around to do it.
>
> Editing is easy ... kind of :) Creating the Links was easy but I
> failed do discover how I could actually make them point to other wiki
> pages.

http://gcc.gnu.org/wiki/HelpOnMoinWikiSyntax#Internal_Links