Re: GCC still getting a lot slower

Re: GCC still getting a lot slower

[Ritzert]

Nathanael Nerode wrote:
> Can we test compile time change of some *fixed* piece of code,
perhaps?

Have a look at http://www.globe-tec.de/~ritzert/STLport.png . The plot
shows the time it takes to compile STLport on a Pentium II 400. The raw
data is at
http://www.globe-tec.de/~ritzert/STLport.times .

The latest jump is here:
16/12/2002 1901.45
17/12/2002 1981.24
a 4.2% increase.

Happy new year everyone!
Michael

Re: GCC still getting a lot slower

[Jan Hubicka]

> On 02-Jan-2003, Jan Hubicka <jh@suse.cz> wrote:
> > 
> > For C performance I think it can be practical to just benchmark linux
> > kernel build - it is fixed piece as it does not include any external
> > headers, large enought to make measurements hopefully stable enought and
> > tests performance of "usual hand writen C program" that (for C) is about
> > the most important.
> 
> The trouble is, which version?
> The Linux kernel itself is not stable.  To make a useful benchmark,
> you need to pick a specific version of the Linux kernel.
> 
> Old versions of the Linux kernel don't work with current versions of GCC,
> and probably current versions of the Linux kernel won't work with future
> versions of GCC.
Yes, but I think there is large enought overlap to make the measurements
usefull.  In casee we are interested in compilation speed only, we can
simply fix new syntax errors.

Honza
> 
> -- 
> Fergus Henderson <fjh@cs.mu.oz.au>  |  "I have always known that the pursuit
> The University of Melbourne         |  of excellence is a lethal habit"
> WWW: <http://www.cs.mu.oz.au/~fjh>  |     -- the last words of T. S. Garp.

Re: GCC still getting a lot slower

[Ben Elliston]

>>>>> "Jan" == Jan Hubicka <jh@suse.cz> writes:

  >> > Can we test compile time change of some *fixed* piece of code,
  >> > perhaps?

  Jan> For C++ I am not aware of something like that.

We have found SID to be a good test case for the C++ compiler (see
http://sources.redhat.com/sid).

Ben

Re: GCC still getting a lot slower

[Fergus Henderson]

On 02-Jan-2003, Jan Hubicka <jh@suse.cz> wrote:
> 
> For C performance I think it can be practical to just benchmark linux
> kernel build - it is fixed piece as it does not include any external
> headers, large enought to make measurements hopefully stable enought and
> tests performance of "usual hand writen C program" that (for C) is about
> the most important.

The trouble is, which version?
The Linux kernel itself is not stable.  To make a useful benchmark,
you need to pick a specific version of the Linux kernel.

Old versions of the Linux kernel don't work with current versions of GCC,
and probably current versions of the Linux kernel won't work with future
versions of GCC.

-- 
Fergus Henderson <fjh@cs.mu.oz.au>  |  "I have always known that the pursuit
The University of Melbourne         |  of excellence is a lethal habit"
WWW: <http://www.cs.mu.oz.au/~fjh>  |     -- the last words of T. S. Garp.

Re: GCC still getting a lot slower

[Jan Hubicka]

> Neil Booth wrote:
> 
> > Any idea what killed GCC bootstrap time in Mid-Dec?  This really must
> > stop happening; we're out of control.
> > 
> > http://www.suse.de/~aj/SPEC/CINT/d-permanent/times.html
> 
> Just curious:
> 
> When the B-I-B branch was announced in
> http://gcc.gnu.org/ml/gcc/2002-08/msg01575.html, the announcement also
> spoke of the "faster-compiler-branch", but it seems that this branch was
> never used (at least, I can't find any mails about it). The announcement
> mentions a few bottlenecks, and IIRC from some other mails by Zack at
> least some bottlenecks are known. Could somebody make a list of those (I
> can make that+whatever else into a nice XHTML page for this branch...)? 
> This would also look like the right branch to play with Dan's memory
> work...
> 
> There are two SPEC testers and a regression tester, but is there a
> cachegrind tester, or a tester that collects profile data?

I have simple scripts to collect oprofile data (including the anotated
disassembly of hot spots) I am using to tune compiler and have data
usually for few weeks back.  The data are quite usefull to easilly fix
the problem, but somewhat large to be practical to collect regularry.
Even the current SPEC testers has problem with disc capacity

Honza
> 
> Greetz
> Steven
> 

Re: GCC still getting a lot slower

[Jan Hubicka]

> On Tue, 31 Dec 2002, Nathanael Nerode wrote:
> > Can we test compile time change of some *fixed* piece of code, perhaps?
> 
> PR 8361 (http://gcc.gnu.org/cgi-bin/gnatsweb.pl?cmd=view&pr=8361),
> which is basically the old PR 3083, is here for anyone to use. ;-)
> 
> Gerald
> 
> PS: It's no longer a completely fixed piece of code, as it's not possible
> to have preprocessed source that is forwards or backwards compatible with
> or from current mainline, but I have included two preprocessed files now.
It would be perhaps good idea to include the performance of preprocesing
as well.  That would fix the compatibility too.

For C performance I think it can be practical to just benchmark linux
kernel build - it is fixed piece as it does not include any external
headers, large enought to make measurements hopefully stable enought and
tests performance of "usual hand writen C program" that (for C) is about
the most important.  Fixing performance of computer generated code is
important too, but it usually has different problem (like computed
jumps, large switch statements and so on) and I tend to believe that
still most of code is hand writen.
I did some oprofiling in the past on this but didn't find anything
obvious to fix.

For C++ I am not aware of something like that.
> -- 
> Gerald "Jerry"   pfeifer@dbai.tuwien.ac.at   http://www.pfeifer.com/gerald/

Re: GCC still getting a lot slower

[Gerald Pfeifer]

On Tue, 31 Dec 2002, Nathanael Nerode wrote:
> Can we test compile time change of some *fixed* piece of code, perhaps?

PR 8361 (http://gcc.gnu.org/cgi-bin/gnatsweb.pl?cmd=view&pr=8361),
which is basically the old PR 3083, is here for anyone to use. ;-)

Gerald

PS: It's no longer a completely fixed piece of code, as it's not possible
to have preprocessed source that is forwards or backwards compatible with
or from current mainline, but I have included two preprocessed files now.
-- 
Gerald "Jerry"   pfeifer@dbai.tuwien.ac.at   http://www.pfeifer.com/gerald/

Re: GCC still getting a lot slower

[David Edelsohn]

>>>>> Neil Booth writes:

Neil> It's only going to get fixed by refusing a release until it's as fast
Neil> as the prior one.  Otherwise it appears that people just don't care
Neil> enough that each release is 30-50% slower than the previous one.

	I strongly believe that we need to have performance goals for GCC
releases, not just release schedules.  Previously when people have
complained about performance, the response has been that it is too late in
the release schedule.  Well, we're now in Stage 1 of development for GCC
3.4, so it definitely cannot be too late.

	As a start, how about some comment about Daniel Berlin's
Generational Copying Garbage Collector?  Complaining about the problem and
then ignoring developers who try to address the problem discourages
further effort.  It isn't perfect, but nothing in GCC is perfect.  If it
needs to be improved, the GCC developer community is invited to assist.
Silence from the peanut gallery is abdication of moral authority to
complain in the future.

David

Re: GCC still getting a lot slower

[Diego Novillo]

On Tue, 31 Dec 2002, Nathanael Nerode wrote:

> Can we test compile time change of some *fixed* piece of code, perhaps?
> 
Fair enough.  Look at SPEC build times then.

http://people.redhat.com/dnovillo/spec95/gcc/global-build-secs_elapsed.html


		  Dec16		  Dec17	    slowdown

SPECint		272 secs	290 secs 	6%
SPECfp		 64 secs	 68 secs	6%


The configury changes have indeed slowed down the bootstrap
process significantly.  Is this easily fixable?

OTOH, 6% compile time slowdown is not good news.


Diego.

Re: GCC still getting a lot slower

[Paul Jarc]

Nathanael Nerode <neroden@twcny.rr.com> wrote:
> Paul Jarc said:
>> Nathanael Nerode <neroden@twcny.rr.com> wrote:
>>> Frankly, I don't think we should look at bootstrap time.  A number of
>>> the largest changes in B-I-B were necessary build process changes,  which
>>> are quite likely to slow down the build *of* the compiler, but not the
>>> performance of the compiler.  Or, in other words, "so what?". ;-)
>>
>> As someone else noted a while ago, bootstrap times are important
>> because changes are tested by bootstrapping.  So slow bootstraps slow
>> down all other progress.
>
> Yeah.  But when I said 'necessary', I meant it; these changes have been
> on the TODO list forever, and they're correctness issues.

I wasn't disputing that - just pointing out that bootstrap times are
indeed important, in answer to your "so what".


paul

Re: GCC still getting a lot slower

[Nathanael Nerode]

Paul Jarc said:
>Nathanael Nerode <neroden@twcny.rr.com> wrote:
>> Frankly, I don't think we should look at bootstrap time.  A number of
>> the largest changes in B-I-B were necessary build process changes,  which
>> are quite likely to slow down the build *of* the compiler, but not the
>> performance of the compiler.  Or, in other words, "so what?". ;-)
>
>As someone else noted a while ago, bootstrap times are important
>because changes are tested by bootstrapping.  So slow bootstraps slow
>down all other progress.
Yeah.  But when I said 'necessary', I meant it; these changes have been 
on the TODO list forever, and they're correctness issues.

>> Can we test compile time change of some *fixed* piece of code, 
>>perhaps?
>
>That would also be useful, in its own way.

If on the other hand the bootstrap slowdown is due to changes in the way 
the compiler works -- which would be evidenced by compile time change in 
a fixed piece of code -- then it's likely to be a bad change which we 
*can* track down and fix by binary search on the b-i-b branch.  The 
changes made in this regard on B-I-B were fewer, smaller, and less invasive.

For instance, the change to every single C file on the tree was a 
build-process change which shouldn't have affected compiler behavior.
(If it did, that's another matter.)

--Nathanael

Re: GCC still getting a lot slower

[Paul Jarc]

Nathanael Nerode <neroden@twcny.rr.com> wrote:
> Frankly, I don't think we should look at bootstrap time.  A number of
> the largest changes in B-I-B were necessary build process changes, which
> are quite likely to slow down the build *of* the compiler, but not the
> performance of the compiler.  Or, in other words, "so what?". ;-)

As someone else noted a while ago, bootstrap times are important
because changes are tested by bootstrapping.  So slow bootstraps slow
down all other progress.

> Can we test compile time change of some *fixed* piece of code, perhaps?

That would also be useful, in its own way.


paul

Re: GCC still getting a lot slower

[Nathanael Nerode]

Diego Novillo wrote:
>On Tue, 31 Dec 2002, Paolo Carlini wrote:
>
>> Neil Booth wrote:
>> 
>> >Any idea what killed GCC bootstrap time in Mid-Dec?
>> >
>> Wild guess
>> 
>>    http://gcc.gnu.org/ml/gcc-cvs/2002-12/msg00481.html
>> 
>>Very likely.  Bootstrap times for C and Fortran went from 2,000
>secs on 2002-12-16 to 2,500 secs on 2002-12-17 (25% slowdown).
>
>http://people.redhat.com/dnovillo/spec95/gcc/gcc-stats.html

Frankly, I don't think we should look at bootstrap time.  A number of 
the largest changes in B-I-B were necessary build process changes, which 
are quite likely to slow down the build *of* the compiler, but not the 
performance of the compiler.  Or, in other words, "so what?". ;-)

Can we test compile time change of some *fixed* piece of code, perhaps?

--Nathanael

Re: GCC still getting a lot slower

[Neil Booth]

Paolo Carlini wrote:-

> Yes :-(
> And unfortuntely the culprit is exactly the merge itself. See:
> 
>    http://www.suse.de/~aj/SPEC/CINT/sandbox-bib/Bootstrap-time_big.png

It's only going to get fixed by refusing a release until it's as fast
as the prior one.  Otherwise it appears that people just don't care
enough that each release is 30-50% slower than the previous one.

Neil.

Re: GCC still getting a lot slower

[Diego Novillo]

On Tue, 31 Dec 2002, Paolo Carlini wrote:

> Neil Booth wrote:
> 
> >Any idea what killed GCC bootstrap time in Mid-Dec?
> >
> Wild guess
> 
>    http://gcc.gnu.org/ml/gcc-cvs/2002-12/msg00481.html
> 
Very likely.  Bootstrap times for C and Fortran went from 2,000
secs on 2002-12-16 to 2,500 secs on 2002-12-17 (25% slowdown).

http://people.redhat.com/dnovillo/spec95/gcc/gcc-stats.html

The diff is, of course, huge.  Ignore the second jump you see to
6,000 secs.  I started bootstrapping all the default front ends
on Dec22.


Diego.

Re: GCC still getting a lot slower

[Paolo Carlini]

Neil Booth wrote:

>>Wild guess
>>
>>   http://gcc.gnu.org/ml/gcc-cvs/2002-12/msg00481.html
>>
>>???
>>    
>>
>If so, ugh.  Fat chance of finding out exactly what it was.
>
Yes :-(
And unfortuntely the culprit is exactly the merge itself. See:

    http://www.suse.de/~aj/SPEC/CINT/sandbox-bib/Bootstrap-time_big.png

Paolo.

Re: GCC still getting a lot slower

[Neil Booth]

Paolo Carlini wrote:-

> >Any idea what killed GCC bootstrap time in Mid-Dec?
> >
> Wild guess
> 
>    http://gcc.gnu.org/ml/gcc-cvs/2002-12/msg00481.html
> 
> ???

If so, ugh.  Fat chance of finding out exactly what it was.

Neil.

Re: GCC still getting a lot slower

[Paolo Carlini]

Neil Booth wrote:

>Any idea what killed GCC bootstrap time in Mid-Dec?
>
Wild guess

    http://gcc.gnu.org/ml/gcc-cvs/2002-12/msg00481.html

???

Paolo.

GCC still getting a lot slower

[Neil Booth]

Any idea what killed GCC bootstrap time in Mid-Dec?  This really must
stop happening; we're out of control.

http://www.suse.de/~aj/SPEC/CINT/d-permanent/times.html

Neil.