Re: A quick summary of gcc compilation speed from a political point of view

Re: A quick summary of gcc compilation speed from a political point of view

[Benjamin Kosnik]

> So parhaps you can work with GGC developers on providing better
> development tools for that or provide some good way to measure
> compilation speed (SPEC tests do have compilation time graphs but they
> are not too usefull as they are with checking enabled and SPECs are too
> small to make changes visible in the noise). Or you can find some other
> way to contribute to GCC.

You know, I strongly agree with this. One of the big problems with the
speed issue is that it's so invisible: every time there is a new
release, somebody invariably complains it's slower than the last release.

Until then, nobody seems to notice, even though there are ostensibly
minimal speed requirements in the release criteria.

What's really needed is some kind of regular posting to gcc-testresults
that indicate the time it took to do a make check, or the time it took
to compile some known sources, etc. Perhaps contrib/test_summary could
be modified to do something like report the time to run make check?

If we had this, then people would have an easier time pinpointing
slowdowns. Also, the developers would be able to see exactly what
compilation performance was for various platforms, which would be an
excellent starting point.

Personally, I always run 

time make check

Just to make sure that PCH is not broken. But, I digress.

-benjamin

Re: A quick summary of gcc compilation speed from a political point of view

[Zack Weinberg]

Marc Espie <espie@nerim.net> writes:

> For what it's worth, I notice side_effects_p as being rather high there.
> Did someone try to provide an inlined version of the simple, non recursive
> part of it ? does that make any sense.

I don't know what you mean here...

> I'm also wondering about the `type' cases. I'll admit the letters might
> be nice for debug (e.g., #if DEBUG), but it would probably be more
> efficient to have simple, small codes. Judging from the number of switch()
> in gcc, those must take a larger amount of time, won't they ?

The TREE_CODE_CLASS letters?  That might be useful to try, but it
would be a huge patch ... are you interested in developing such?

> I also see make_node first filling a tmp node to get its size.
> Maybe retrieving a size from a code might be better ?

Yeah, good idea.  I am trying such a patch now.

zw

Re: A quick summary of gcc compilation speed from a political point of view

[Marc Espie]

I've built a 3.3.2 profiling compiler, and built a kernel with it.
I'll try doing the same with 2.95.3, probably tomorrow, and also -current,
time-willing.

ftp://quatramaran.ens.fr/pub/espie/gcc/profile-kernel-gcc-3.3.2-1.gz

you'll notice a few calls to propolice, which don't interfere with
the timings.

For what it's worth, I notice side_effects_p as being rather high there.
Did someone try to provide an inlined version of the simple, non recursive
part of it ? does that make any sense.

I'm also wondering about the `type' cases. I'll admit the letters might
be nice for debug (e.g., #if DEBUG), but it would probably be more
efficient to have simple, small codes. Judging from the number of switch()
in gcc, those must take a larger amount of time, won't they ?

I also see make_node first filling a tmp node to get its size.
Maybe retrieving a size from a code might be better ?

And the memset() in make_node is quite obvious.

Re: A quick summary of gcc compilation speed from a political point of view

[Nathanael Nerode]

Marc Espie wrote:
>Most of my work on GCC happens in other places. If you want to find
>significant work, look into the OpenBSD repository.  I've spent a long
>time catching up with recent GCC development.  One good thing is that,
>these days, GCC compiles reasonably well on OpenBSD. This was not the
>case three years ago. I also finally have enough room on my development
>machine to be able to work with GCC current.
>
>And so, I'm working on fixing things.  

Great!

Any progress on porting that configuration patch to mainline, and addressing
the various questions I asked about the config.gcc entries?  Conceptually
separate pieces of it can be submitted piecemeal as it gets ready; so, for
instance, the libc_r/libpthread bit as one change; any fixincludes issues
as another; etc.  (That's actually preferable, in fact, since it's easier
to review, and problems in one part don't necessarily hold up other parts.)

As a configury maintainer, I would really like GCC to configure correctly
for OpenBSD out of the box, so I appreciate this work.

-- 
Nathanael Nerode  <neroden at gcc.gnu.org>
http://home.twcny.rr.com/nerode/neroden/fdl.html

Re: A quick summary of gcc compilation speed from a political point of view

[Diego Novillo]

On Tue, 2004-01-20 at 08:55, Jan Hubicka wrote:

> (SPEC tests do have compilation time graphs but they
> are not too usefull as they are with checking enabled and SPECs are too
> small to make changes visible in the noise). Or you can find some other
> way to contribute to GCC.
> 
Not the ones at http://people.redhat.com/dnovillo/spec2000/
I run SPEC with checking disabled.


Diego.

Re: A quick summary of gcc compilation speed from a political point of view

[Gabriel Dos Reis]

"Giovanni Bajo" <giovannibajo@libero.it> writes:

| Gabriel Dos Reis wrote:
| 
| > Yeah, but more and more people now seem to be swearing only by
| > bugzilla
| 
| It's a fact that our releases and regression fixes are driven by the
| bugreports we get.

Bug reports are key factors, but they are not the only one.  And
basing GCC releases _only_ on things one sees in bugzilla would be
inappropriate. 

The compile-time regression is not a new topic. It was noticed before
we released GCC-3.3, but then we made a clear commitment that it will
be a release criteria for GCC-3.4.  Now, some people seem to narrow
their views only on bugzilla.  Bugzilla is a great tool, but
somethings are just not that easy to fill in there.

-- Gaby

Re: A quick summary of gcc compilation speed from a political point of view

[Giovanni Bajo]

Gabriel Dos Reis wrote:

> Yeah, but more and more people now seem to be swearing only by
> bugzilla

It's a fact that our releases and regression fixes are driven by the bugreports
we get. If we had 20-30 more 3.4 regressions about compile-time issues,
probably we would have not branched at this point, and people would be working
on at least some of them. Jan did an amazing job and fixed many compile-hog and
memory-hog reports in time for 3.4 - but he knew about it because we had a
report with a testcase.

I apologize with Marc, he was surely the wrong person for my rant. It's just
that there are too many generic complaints on the list and only a few
bugreports. As someone else said, a complaint on the list is forgotten in less
than a week. I should do like most developers seem to do - just ignore and
forget the complaints and go back fixing bugreports.

Giovanni Bajo

Re: A quick summary of gcc compilation speed from a political point of view

[Gabriel Dos Reis]

"Giovanni Bajo" <giovannibajo@libero.it> writes:

| Gabriel Dos Reis wrote:
| 
| > Bugzilla is not all about it ;-)  I just tried simple google to
| > see whether Marc did never actually report any bug about GCC and the
| > first link gives
| >
| >    http://gcc.gnu.org/ml/gcc/2003-11/msg01303.html
| >
| > with a simple patch
| >
| >    http://gcc.gnu.org/ml/gcc/2003-11/msg01506.html
| >
| > I did not go further looking other hits.
| > (I hope we won't end up classifying people based on their hits in
| > bugzilla).
| 
| I didn't mean to classify anybody.

Yeah, but more and more people now seem to be swearing only by
bugzilla and the way you phrased your previous message could be
misleading.

| It was just a way to note that we don't get
| enough compile-time regression bugreports, but we do get a lot of compile-time
| regression complaints. If the OpenBSD community now wants to switch to 3.3.x,

Marc and I have had a conversation in the last couple of months where I
have been talking him into moving forward to 3.3.x adoption.  I suppose
his disappointment is partly my fault -- I did not anticipate that
"you can buy a new hardware for $400" or "your bugzilla hit is zero"
will welcome him :-/

And of course, I've also encouraged people to submit reports. 

-- Gaby

Re: A quick summary of gcc compilation speed from a political point of view

[Marc Espie]

On Tue, Jan 20, 2004 at 02:32:22PM +0100, Gabriel Dos Reis wrote:
> "Giovanni Bajo" <giovannibajo@libero.it> writes:
> 
> | Some of them, like you, never filed a single bug report since we
> | started our bug database (4 years ago or something). 
> | At least, this is what Bugzilla told me.
> 
> Bugzilla is not all about it ;-)  I just tried simple google to
> see whether Marc did never actually report any bug about GCC and the
> first link gives
> 
>    http://gcc.gnu.org/ml/gcc/2003-11/msg01303.html
> 
> with a simple patch 
> 
>    http://gcc.gnu.org/ml/gcc/2003-11/msg01506.html
> 
> I did not go further looking other hits.
> (I hope we won't end up classifying people based on their hits in
> bugzilla). 

Most of my work on GCC happens in other places. If you want to find
significant work, look into the OpenBSD repository.  I've spent a long
time catching up with recent GCC development.  One good thing is that,
these days, GCC compiles reasonably well on OpenBSD. This was not the
case three years ago. I also finally have enough room on my development
machine to be able to work with GCC current.

And so, I'm working on fixing things.  

Re: A quick summary of gcc compilation speed from a political point of view

[Jan Hubicka]

To correct myself,
> > >From the little discussion that I inadvertently started, I believe I can
> > start to draw conclusions.
> > 
> > There is a large number of people who apparently do not care at all that
> > gcc is getting slower, because they say that we can buy faster hardware,
> > and gcc is not getting slower THAT fast.
> > 
> > If that's the consensus, it would be a good idea to seriously amend the
> > GCC description.
> Mark,
> 
> it is definitly not concensus and what I wanted to point out by my
> little kernel compilation benchmark is that the situation is now taken
> quite seriously and the compiler is _not_ getting slower with each
> release anymore. All GCC developers noticed that compile time
> performance is important and it is now common to provide compilation
> time infromation for each change and try hard to minimize the compile
			^ this is not quite the case, we do it only for
			changes that looks dangerous, like adding new
			pass or so.    Still things may get missed so
			it can be nice to good periodic tester doing
			usefull testing on compile time perofmrance.
			Our SPEC testers are not as good on it as it can
			be.
Honza

Re: A quick summary of gcc compilation speed from a political point of view

[Jan Hubicka]

> >From the little discussion that I inadvertently started, I believe I can
> start to draw conclusions.
> 
> There is a large number of people who apparently do not care at all that
> gcc is getting slower, because they say that we can buy faster hardware,
> and gcc is not getting slower THAT fast.
> 
> If that's the consensus, it would be a good idea to seriously amend the
> GCC description.
Mark,

it is definitly not concensus and what I wanted to point out by my
little kernel compilation benchmark is that the situation is now taken
quite seriously and the compiler is _not_ getting slower with each
release anymore. All GCC developers noticed that compile time
performance is important and it is now common to provide compilation
time infromation for each change and try hard to minimize the compile
time costs.

The fact that we see comparable speeds in between 3.2.0 done about two
years go and today is kind of sucess, especially if you compare it with
regular slowdown in between release in the past.  We clearly can try
harder and do better.

There is little value from sending desperate emails once new version has
been feature freezed and there is little that can be do to fact twofold
slowodn relative to 2.95.  In the case the perofmrance is really dear to
you, there are many ways you can help to improve it for future releases.

For instance, providing way to test the changes can be very cool.  You
can take a look at graphs at http://www.suse.de/~aj/SPEC.  At the day
Diego and Andreas started the testers, the code quality started to
improve.  Similarly we do have CSiBE benchmarks for code size that are
very usefull.

Also when I started to work on cgraph code I wasn't interested in
solving the C++ inliner problem as I hardly ever use C++, but the fact
that Gerald provided a good testcases, benchmark and a lot of assistance
it made it challenging and seemingly important project for me and I
spent about 5 months working on C++ frontend and other areas I never was
interested before.

Problem of performance is that each idividual change makes too little
effect on compiler to be measurable in isolation and many changes cause
disaster.  If we had good way to measure the factors such as memory
usage or page faults, we would know better how to look at.

So parhaps you can work with GGC developers on providing better
development tools for that or provide some good way to measure
compilation speed (SPEC tests do have compilation time graphs but they
are not too usefull as they are with checking enabled and SPECs are too
small to make changes visible in the noise). Or you can find some other
way to contribute to GCC.

We are are trying hard to make wide range of GCC users happy  and we
hope to find a way to make GCC both faster and producing better code,
but this is a lot of effort and a lot of things may get missed.

Honza

Re: A quick summary of gcc compilation speed from a political point of view

[Giovanni Bajo]

Gabriel Dos Reis wrote:

> Bugzilla is not all about it ;-)  I just tried simple google to
> see whether Marc did never actually report any bug about GCC and the
> first link gives
>
>    http://gcc.gnu.org/ml/gcc/2003-11/msg01303.html
>
> with a simple patch
>
>    http://gcc.gnu.org/ml/gcc/2003-11/msg01506.html
>
> I did not go further looking other hits.
> (I hope we won't end up classifying people based on their hits in
> bugzilla).

I didn't mean to classify anybody. It was just a way to note that we don't get
enough compile-time regression bugreports, but we do get a lot of compile-time
regression complaints. If the OpenBSD community now wants to switch to 3.3.x,
I'm more than happy about it. If they want to file bugreports, they're very
welcome. But doing it is much much more effective than speaking of it.

Giovanni Bajo

Re: A quick summary of gcc compilation speed from a political point of view

[Gabriel Dos Reis]

"Giovanni Bajo" <giovannibajo@libero.it> writes:

| Some of them, like you, never filed a single bug report since we
| started our bug database (4 years ago or something). 
| At least, this is what Bugzilla told me.

Bugzilla is not all about it ;-)  I just tried simple google to
see whether Marc did never actually report any bug about GCC and the
first link gives

   http://gcc.gnu.org/ml/gcc/2003-11/msg01303.html

with a simple patch 

   http://gcc.gnu.org/ml/gcc/2003-11/msg01506.html

I did not go further looking other hits.
(I hope we won't end up classifying people based on their hits in
bugzilla). 

-- Gaby

Re: A quick summary of gcc compilation speed from a political point of view

[Giovanni Bajo]

Marc Espie wrote:

> But that's not the point.
>
> The point is that, right now, GCC is each day becoming more unusable
> as a compiler on any host that isn't ix86.

A point *I* would like to make is that there are a lot of people interested in
different areas of development. Some of them actually do files bug reports (for
instance, we get all kind of advanced C++ compatibility bug reports from the
Boost people - and we actually fix them). Some of them, like you, never filed a
single bug report since we started our bug database (4 years ago or something).
At least, this is what Bugzilla told me.

Marc, compile time regressions are seriously taken by GCC developers. But
they're not something that are *easily* noticed. It's not like a crash in a
testsuite which is immediatly noticed by the tests. We strongly need bug
reports to be able to fix them, because they're more hideous.

Are you preparing a bug report for the OpenBSD kernel compile-time regression
you showed us?

Giovanni Bajo

A quick summary of gcc compilation speed from a political point of view

[Marc Espie]

From the little discussion that I inadvertently started, I believe I can
start to draw conclusions.

There is a large number of people who apparently do not care at all that
gcc is getting slower, because they say that we can buy faster hardware,
and gcc is not getting slower THAT fast.

If that's the consensus, it would be a good idea to seriously amend the
GCC description.

Because I don't think GCC fits its former description.

If that's true, and speed doesn't count that much, then GCC has become
a compiler that can should only be hosted on ix86 systems.

All over systems are either becoming too expensive or too slow for almost
anyone... Frankly, I don't really see myself getting a distcc farm of
sparc64 machines in the future...

So, the target audience of GCC consists of:
- people who want native ix86 compilers that create good code;
- people who want a cross-compiler for another system that's hosted on an
ix86.


Other combinations don't look reasonable to me.   In any case, when I
pointed out other combinations that looked useful and that would be nice to
have, I completely got shooted down by various people, stating cost issues,
or the excellence of GCC as a cross-compiler.

[ From personal experience, I can tell you that cross-compiling an entire
distribution is hard, and yields a whole other set of bugs to fix.
I can also tell you that native compiling the same distribution on a little
used system will invariably expose bugs that are not visible in a
cross-compiling setting. ]


But that's not the point.

The point is that, right now, GCC is each day becoming more unusable as a
compiler on any host that isn't ix86.


-- 
	Marc