Speed issues taken seriously?

Speed issues taken seriously?

[Lars Gullik Bjønnes]

I just thought that I'd see how gcc 3.3.2 (CVS) and 3.4(CVS) compared,
I was hoping (expecting) the recent months long threads on compile
performance to have had an effect... the result was a bit
disappointing...

This is the results from two LyX compiles, both xforms and qt
frontends. Identical setup: (LyX is a c++ program)

Configure line used for LyX:
./configure --without-pspell --without-apsell --disable-debug
--with-frontend="xforms qt" --enable-maintainer-mode


G++ 3.3:
gcc (GCC) 3.3.2 20030923 (prerelease)
real    21m23.975s
user    19m25.020s
sys     1m44.680s

ls -l src/lyx-xforms src/lyx-qt
-rwxrwxr-x    1 larsbj   larsbj    5019271 Oct 10 16:13 src/lyx-qt
-rwxrwxr-x    1 larsbj   larsbj    4732606 Oct 10 16:13 src/lyx-xforms

size src/lyx-xforms src/lyx-qt
   text    data     bss     dec     hex filename
3651516   18852   53532 3723900  38d27c src/lyx-xforms
3863606   17100   28636 3909342  3ba6de src/lyx-qt


G++ 3.4:
gcc (GCC) 3.4 20031009 (experimental)
real    36m27.137s
user    34m26.340s
sys     1m37.250s

ls -l src/lyx-xforms src/lyx-qt
-rwxrwxr-x    1 larsbj   larsbj    5837180 Oct 10 15:44 src/lyx-qt
-rwxrwxr-x    1 larsbj   larsbj    5513473 Oct 10 15:44 src/lyx-xforms

size src/lyx-xforms src/lyx-qt
   text    data     bss     dec     hex filename
3907336   18948   53664 3979948  3cbaac src/lyx-xforms
4088916   17180   28736 4134832  3f17b0 src/lyx-qt


-- 
	Lgb

Re: Speed issues taken seriously?

[Steven Bosscher]

Op vr 10-10-2003, om 16:49 schreef Lars Gullik Bjønnes:
> I just thought that I'd see how gcc 3.3.2 (CVS) and 3.4(CVS) compared,
> I was hoping (expecting) the recent months long threads on compile
> performance to have had an effect... the result was a bit
> disappointing...

Did you compile with checking disabled?

Gr.
Steven

Re: Speed issues taken seriously?

[Lars Gullik Bjønnes]

Steven Bosscher <s.bosscher@student.tudelft.nl> writes:

| Op vr 10-10-2003, om 16:49 schreef Lars Gullik Bjønnes:
>> I just thought that I'd see how gcc 3.3.2 (CVS) and 3.4(CVS) compared,
>> I was hoping (expecting) the recent months long threads on compile
>> performance to have had an effect... the result was a bit
>> disappointing...
>
| Did you compile with checking disabled?

Probably not. I guess this will be turned off before 3.4. release?

-- 
	Lgb

Re: Speed issues taken seriously?

[Andreas Jaeger]

[-- Attachment #1: Type: text/plain, Size: 820 bytes --]

larsbj@gullik.net (Lars Gullik Bjønnes) writes:

> Steven Bosscher <s.bosscher@student.tudelft.nl> writes:
>
> | Op vr 10-10-2003, om 16:49 schreef Lars Gullik Bjønnes:
>>> I just thought that I'd see how gcc 3.3.2 (CVS) and 3.4(CVS) compared,
>>> I was hoping (expecting) the recent months long threads on compile
>>> performance to have had an effect... the result was a bit
>>> disappointing...
>>
> | Did you compile with checking disabled?
>
> Probably not. I guess this will be turned off before 3.4. release?

Yes, it will be turned off for the release but we have it on for
development,

Andreas
-- 
 Andreas Jaeger, aj@suse.de, http://www.suse.de/~aj
  SuSE Linux AG, Deutschherrnstr. 15-19, 90429 Nürnberg, Germany
   GPG fingerprint = 93A3 365E CE47 B889 DF7F  FED1 389A 563C C272 A126

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

Re: Speed issues taken seriously?

[Andreas Jaeger]

[-- Attachment #1: Type: text/plain, Size: 627 bytes --]

larsbj@gullik.net (Lars Gullik Bjønnes) writes:

> I just thought that I'd see how gcc 3.3.2 (CVS) and 3.4(CVS) compared,
> I was hoping (expecting) the recent months long threads on compile
> performance to have had an effect... the result was a bit
> disappointing...

How did you configure GCC exactly?  Please use --disable-checking so
that the checking code is disabled, it causes some run-time overhead.

Andreas
-- 
 Andreas Jaeger, aj@suse.de, http://www.suse.de/~aj
  SuSE Linux AG, Deutschherrnstr. 15-19, 90429 Nürnberg, Germany
   GPG fingerprint = 93A3 365E CE47 B889 DF7F  FED1 389A 563C C272 A126

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

Re: Speed issues taken seriously?

[Lars Gullik Bjønnes]

Andreas Jaeger <aj@suse.de> writes:

| larsbj@gullik.net (Lars Gullik Bjønnes) writes:
>
>> I just thought that I'd see how gcc 3.3.2 (CVS) and 3.4(CVS) compared,
>> I was hoping (expecting) the recent months long threads on compile
>> performance to have had an effect... the result was a bit
>> disappointing...
>
| How did you configure GCC exactly?  Please use --disable-checking so
| that the checking code is disabled, it causes some run-time overhead.

I'll redo my tests with this.

-- 
	Lgb

Re: Speed issues taken seriously?

[Lars Gullik Bjønnes]

larsbj@gullik.net (Lars Gullik Bjønnes) writes:

| Andreas Jaeger <aj@suse.de> writes:
>
| | larsbj@gullik.net (Lars Gullik Bjønnes) writes:
>>
>>> I just thought that I'd see how gcc 3.3.2 (CVS) and 3.4(CVS) compared,
>>> I was hoping (expecting) the recent months long threads on compile
>>> performance to have had an effect... the result was a bit
>>> disappointing...
>>
| | How did you configure GCC exactly?  Please use --disable-checking so
| | that the checking code is disabled, it causes some run-time overhead.
>
| I'll redo my tests with this.

Test redone.

g++ (GCC) 3.4 20031010 (experimental)

real    24m20.049s
user    22m17.930s
sys     1m45.420s

ls -l src/lyx-xforms src/lyx-qt
-rwxrwxr-x    1 larsbj   larsbj    5837084 Oct 10 18:18 src/lyx-qt
-rwxrwxr-x    1 larsbj   larsbj    5513413 Oct 10 18:18 src/lyx-xforms

size src/lyx-xforms src/lyx-qt
   text    data     bss     dec     hex filename
3907253   18948   53660 3979861  3cba55 src/lyx-xforms
4088833   17180   28732 4134745  3f1759 src/lyx-qt

So a lot better, but still a bit disappointing, since the compilation
time is still increasing.

PS. Gcc configured like this:

../configure --prefix=/opt/gcc34 --enable-languages=c,c++ --disable-checking

-- 
	Lgb

Re: Speed issues taken seriously?

[Andreas Jaeger]

[-- Attachment #1: Type: text/plain, Size: 1195 bytes --]

larsbj@gullik.net (Lars Gullik Bjønnes) writes:

> larsbj@gullik.net (Lars Gullik Bjønnes) writes:
>
> | Andreas Jaeger <aj@suse.de> writes:
>>
> | | larsbj@gullik.net (Lars Gullik Bjønnes) writes:
>>>
>>>> I just thought that I'd see how gcc 3.3.2 (CVS) and 3.4(CVS) compared,
>>>> I was hoping (expecting) the recent months long threads on compile
>>>> performance to have had an effect... the result was a bit
>>>> disappointing...
>>>
> | | How did you configure GCC exactly?  Please use --disable-checking so
> | | that the checking code is disabled, it causes some run-time overhead.
>>
> | I'll redo my tests with this.
>
> Test redone.
>
> g++ (GCC) 3.4 20031010 (experimental)
>
> real    24m20.049s
> user    22m17.930s
> sys     1m45.420s

This gives a degradion of 13 per cent in realtime compared with your
earlier results of:

G++ 3.3:
gcc (GCC) 3.3.2 20030923 (prerelease)
real    21m23.975s
user    19m25.020s
sys     1m44.680s

Thanks,
Andreas
-- 
 Andreas Jaeger, aj@suse.de, http://www.suse.de/~aj
  SuSE Linux AG, Deutschherrnstr. 15-19, 90429 Nürnberg, Germany
   GPG fingerprint = 93A3 365E CE47 B889 DF7F  FED1 389A 563C C272 A126

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

Re: Speed issues taken seriously?

[Jan Hubicka]

> larsbj@gullik.net (Lars Gullik Bj?nnes) writes:
> 
> | Andreas Jaeger <aj@suse.de> writes:
> >
> | | larsbj@gullik.net (Lars Gullik Bj?nnes) writes:
> >>
> >>> I just thought that I'd see how gcc 3.3.2 (CVS) and 3.4(CVS) compared,
> >>> I was hoping (expecting) the recent months long threads on compile
> >>> performance to have had an effect... the result was a bit
> >>> disappointing...
> >>
> | | How did you configure GCC exactly?  Please use --disable-checking so
> | | that the checking code is disabled, it causes some run-time overhead.
> >
> | I'll redo my tests with this.
> 
> Test redone.
> 
> g++ (GCC) 3.4 20031010 (experimental)
> 
> real    24m20.049s
> user    22m17.930s
> sys     1m45.420s
> 
> ls -l src/lyx-xforms src/lyx-qt
> -rwxrwxr-x    1 larsbj   larsbj    5837084 Oct 10 18:18 src/lyx-qt
> -rwxrwxr-x    1 larsbj   larsbj    5513413 Oct 10 18:18 src/lyx-xforms
> 
> size src/lyx-xforms src/lyx-qt
>    text    data     bss     dec     hex filename
> 3907253   18948   53660 3979861  3cba55 src/lyx-xforms
> 4088833   17180   28732 4134745  3f1759 src/lyx-qt
> 
> So a lot better, but still a bit disappointing, since the compilation
> time is still increasing.

Can you, please, try to test it with --param max-inline-insns-single=100
It is quite interesting that code size has increased so much.  I
recently increased the limit form 100 to 500 and we probably should go
for something in between (as SPEC scores has regressed somewhat too at
least in combination of loop unrolling)

Honza
> 
> PS. Gcc configured like this:
> 
> ../configure --prefix=/opt/gcc34 --enable-languages=c,c++ --disable-checking
> 
> -- 
> 	Lgb

Re: Speed issues taken seriously?

[Lars Gullik Bjønnes]

I hope this will be useful for someone.


Updated numbers:

a = g++ (GCC) 3.3.2 20031015 (prerelease)
b = g++ (GCC) 3.4 20031014 (experimental)

            | user time  | file size | size dec | size text | size data |
------------+------------+-----------+----------+-----------+-----------|
-O      | a | 19m44.730s |  4788930  | 3759350  |  3684558  |   19884   |
        | b | 22m45.080s |  5582322  | 4025379  |  3950263  |   19980   |
--------+---+------------+-----------+----------+-----------+-----------|
-O2     | a | 22m39.550s |  5052961  | 4024174  |  3949410  |   19884   |
        | b | 24m57.820s |  5084835  | 4160489  |  4085829  |   19860   |
--------+---+------------+-----------+----------+-----------+-----------|
-O3     | a | 23m07.770s |  5037109  | 3992125  |  3917397  |   19880   |
        | b | 26m30.820s |  5332135  | 4408351  |  4333699  |   19856   |
--------+---+------------+-----------+----------+-----------+-----------|
-O2 (1) | b | 24m25.590s |  4860852  | 3884278  |  3809882  |   19888   |
--------+---+------------+-----------+----------+-----------+-----------|
-O3 (2) | b | 25m45.220s |  5025193  | 4047663  |  3973255  |   19896   |
-------------------------------------------------------------------------

(1) -O2 --param max-inline-insns-single=100
(2) -O3 --param max-inline-insns-single=100 --param max-inline-insns-auto=100


The "raw" numbers:


g++ (GCC) 3.3.2 20031015 (prerelease)

with -O

real    21m44.439s
user    19m44.730s
sys     1m47.030s

ls -l
-rwxrwxr-x    1 larsbj   larsbj    5081976 Oct 15 13:33 src/lyx-qt
-rwxrwxr-x    1 larsbj   larsbj    4788930 Oct 15 13:33 src/lyx-xforms

size src/lyx-xforms src/lyx-qt
   text    data     bss     dec     hex filename
3684558   19884   54908 3759350  395cf6 src/lyx-xforms
3897368   18292   29948 3945608  3c3488 src/lyx-qt

with -O2

real    24m38.958s
user    22m39.550s
sys     1m48.460s

ls -l
-rwxrwxr-x    1 larsbj   larsbj    5302353 Oct 15 15:33 src/lyx-qt
-rwxrwxr-x    1 larsbj   larsbj    5052961 Oct 15 15:33 src/lyx-xforms

size src/lyx-xforms src/lyx-qt
   text    data     bss     dec     hex filename
3949410   19884   54880 4024174  3d676e src/lyx-xforms
4117692   18292   29952 4165936  3f9130 src/lyx-qt

with -O3

real    25m11.832s
user    23m7.770s
sys     1m50.550s

ls -l
-rwxrwxr-x    1 larsbj   larsbj    5288073 Oct 15 16:07 src/lyx-qt
-rwxrwxr-x    1 larsbj   larsbj    5037109 Oct 15 16:07 src/lyx-xforms

   text    data     bss     dec     hex filename
3917397   19880   54848 3992125  3cea3d src/lyx-xforms
4089953   18292   29920 4138165  3f24b5 src/lyx-qt


g++ (GCC) 3.4 20031014 (experimental)

with -O

real    24m44.100s
user    22m45.080s
sys     1m47.200s

ls -l
-rwxrwxr-x    1 larsbj   larsbj    5908781 Oct 15 14:17 src/lyx-qt
-rwxrwxr-x    1 larsbj   larsbj    5582322 Oct 15 14:17 src/lyx-xforms

size src/lyx-xforms src/lyx-qt
   text    data     bss     dec     hex filename
3950263   19980   55136 4025379  3d6c23 src/lyx-xforms
4132675   18372   30144 4181191  3fccc7 src/lyx-qt

with -O2

real    27m6.047s
user    24m57.820s
sys     1m51.480s

ls -l
-rwxrwxr-x    1 larsbj   larsbj    5390050 Oct 15 16:45 src/lyx-qt
-rwxrwxr-x    1 larsbj   larsbj    5084835 Oct 15 16:44 src/lyx-xforms

size src/lyx-xforms src/lyx-qt
   text    data     bss     dec     hex filename
4085829   19860   54800 4160489  3f7be9 src/lyx-xforms
4223050   18312   29904 4271266  412ca2 src/lyx-qt

with -O3

real    28m41.760s
user    26m30.820s
sys     1m53.170s

[larsbj@tyfon lyx34]$ ls -l src/lyx-xforms src/lyx-qt
-rwxrwxr-x    1 larsbj   larsbj    5681254 Oct 15 17:22 src/lyx-qt
-rwxrwxr-x    1 larsbj   larsbj    5332135 Oct 15 17:22 src/lyx-xforms

size src/lyx-xforms src/lyx-qt
   text    data     bss     dec     hex filename
4333699   19856   54796 4408351  43441f src/lyx-xforms
4510584   18308   29900 4558792  458fc8 src/lyx-qt

with -O2 --param max-inline-insns-single=100

real    26m31.552s
user    24m25.590s
sys     1m51.450s

ls -l
-rwxrwxr-x    1 larsbj   larsbj    5164703 Oct 15 18:29 src/lyx-qt
-rwxrwxr-x    1 larsbj   larsbj    4860852 Oct 15 18:29 src/lyx-xforms

size src/lyx-xforms src/lyx-qt
   text    data     bss     dec     hex filename
3809882   19888   54508 3884278  3b44f6 src/lyx-xforms
3942533   18348   29580 3990461  3ce3bd src/lyx-qt

with -O3 --param max-inline-insns-single=100 --param max-inline-insns-auto=100

real    27m56.120s
user    25m45.220s
sys     1m57.550s

ls -l
-rwxrwxr-x    1 larsbj   larsbj    5368972 Oct 15 19:08 src/lyx-qt
-rwxrwxr-x    1 larsbj   larsbj    5025193 Oct 15 19:07 src/lyx-xforms

size src/lyx-xforms src/lyx-qt
   text    data     bss     dec     hex filename
3973255   19896   54512 4047663  3dc32f src/lyx-xforms
4147050   18352   29584 4194986  4002aa src/lyx-qt


-- 
	Lgb

Re: Speed issues taken seriously?

[Gerald Pfeifer]

On Wed, 15 Oct 2003, Lars Gullik Bjønnes wrote:
> a = g++ (GCC) 3.3.2 20031015 (prerelease)
> b = g++ (GCC) 3.4 20031014 (experimental)
>
>             | user time  | file size | size dec | size text | size data |
> ------------+------------+-----------+----------+-----------+-----------|
> -O      | a | 19m44.730s |  4788930  | 3759350  |  3684558  |   19884   |
>         | b | 22m45.080s |  5582322  | 4025379  |  3950263  |   19980   |
> --------+---+------------+-----------+----------+-----------+-----------|
> -O2     | a | 22m39.550s |  5052961  | 4024174  |  3949410  |   19884   |
>         | b | 24m57.820s |  5084835  | 4160489  |  4085829  |   19860   |
> --------+---+------------+-----------+----------+-----------+-----------|
> -O3     | a | 23m07.770s |  5037109  | 3992125  |  3917397  |   19880   |
>         | b | 26m30.820s |  5332135  | 4408351  |  4333699  |   19856   |
> --------+---+------------+-----------+----------+-----------+-----------|

Hmm, that's bad.  C++ in GCC 3.4 shouldn't be slower, but faster, at least
at -O2 and -O3 according to previous tests. Jan?

Or file a PR with one preprocessed file that shows the slowdown? (Something
like PR8361.)

> -O2 (1) | b | 24m25.590s |  4860852  | 3884278  |  3809882  |   19888   |
> --------+---+------------+-----------+----------+-----------+-----------|
> -O3 (2) | b | 25m45.220s |  5025193  | 4047663  |  3973255  |   19896   |
> -------------------------------------------------------------------------
>
> (1) -O2 --param max-inline-insns-single=100
> (2) -O3 --param max-inline-insns-single=100 --param max-inline-insns-auto=100

Gerald
-- 
Gerald Pfeifer (Jerry)   gerald@pfeifer.com   http://www.pfeifer.com/gerald/

Re: Speed issues taken seriously?

[Jan Hubicka]

> On Wed, 15 Oct 2003, Lars Gullik Bj?nnes wrote:
> > a = g++ (GCC) 3.3.2 20031015 (prerelease)
> > b = g++ (GCC) 3.4 20031014 (experimental)
> >
> >             | user time  | file size | size dec | size text | size data |
> > ------------+------------+-----------+----------+-----------+-----------|
> > -O      | a | 19m44.730s |  4788930  | 3759350  |  3684558  |   19884   |
> >         | b | 22m45.080s |  5582322  | 4025379  |  3950263  |   19980   |
> > --------+---+------------+-----------+----------+-----------+-----------|
> > -O2     | a | 22m39.550s |  5052961  | 4024174  |  3949410  |   19884   |
> >         | b | 24m57.820s |  5084835  | 4160489  |  4085829  |   19860   |
> > --------+---+------------+-----------+----------+-----------+-----------|
> > -O3     | a | 23m07.770s |  5037109  | 3992125  |  3917397  |   19880   |
> >         | b | 26m30.820s |  5332135  | 4408351  |  4333699  |   19856   |
> > --------+---+------------+-----------+----------+-----------+-----------|
> 
> Hmm, that's bad.  C++ in GCC 3.4 shouldn't be slower, but faster, at least
> at -O2 and -O3 according to previous tests. Jan?

It seems to me that in this particular (i've missed the begining of
thread) case we are not hitting inlining heuristics problems in 3.3.2
and thus 3.4 is not better (the produced code sizes are mostly
comparable when taking into the account the increase of inlining
limits). What is the runtime performance?

3.4 seems to be overall somewhat slower than 3.3 again when optimizing
for sources where inlining does not make important difference.
I plan to do some oprofiling and analysis of SPEC data we have
accumulated during stage3 to figure out what are the main reasons for
this.  (I am experimenting with a tool automatically looking for changes
of performance in Andrea's SPEC testers and logging changelogs so we at
least can identify offending patches more easilly.  Very first results
are at http://www.ucw.cz/~hubicka/analysis.html
I don't have currently much time to improve my tool but I will try to do
so after hunting all the bugs in my TODO list.

Honza
> 
> Or file a PR with one preprocessed file that shows the slowdown? (Something
> like PR8361.)
> 
> > -O2 (1) | b | 24m25.590s |  4860852  | 3884278  |  3809882  |   19888   |
> > --------+---+------------+-----------+----------+-----------+-----------|
> > -O3 (2) | b | 25m45.220s |  5025193  | 4047663  |  3973255  |   19896   |
> > -------------------------------------------------------------------------
> >
> > (1) -O2 --param max-inline-insns-single=100
> > (2) -O3 --param max-inline-insns-single=100 --param max-inline-insns-auto=100
> 
> Gerald
> -- 
> Gerald Pfeifer (Jerry)   gerald@pfeifer.com   http://www.pfeifer.com/gerald/

Re: Speed issues taken seriously?

[Joel Sherrill]

Jan Hubicka wrote:

>3.4 seems to be overall somewhat slower than 3.3 again when optimizing
>for sources where inlining does not make important difference.
>I plan to do some oprofiling and analysis of SPEC data we have
>accumulated during stage3 to figure out what are the main reasons for
>this.  (I am experimenting with a tool automatically looking for changes
>of performance in Andrea's SPEC testers and logging changelogs so we at
>least can identify offending patches more easilly.  Very first results
>are at http://www.ucw.cz/~hubicka/analysis.html
>I don't have currently much time to improve my tool but I will try to do
>so after hunting all the bugs in my TODO list.
>  
>
Sounds potentially helpful but the links are broken on the top table.

The size tester that someone has posted information about is also a 
useful metric.

--joel sherrill

Re: Speed issues taken seriously?

[Jan Hubicka]

> Jan Hubicka wrote:
> 
> >3.4 seems to be overall somewhat slower than 3.3 again when optimizing
> >for sources where inlining does not make important difference.
> >I plan to do some oprofiling and analysis of SPEC data we have
> >accumulated during stage3 to figure out what are the main reasons for
> >this.  (I am experimenting with a tool automatically looking for changes
> >of performance in Andrea's SPEC testers and logging changelogs so we at
> >least can identify offending patches more easilly.  Very first results
> >are at http://www.ucw.cz/~hubicka/analysis.html
> >I don't have currently much time to improve my tool but I will try to do
> >so after hunting all the bugs in my TODO list.
> > 
> >
> Sounds potentially helpful but the links are broken on the top table.

The page is supposed to sit in the same directory as others pages
produced by Andreas' scripts.  I hope that we will merge our scripts
soonish.
> 
> The size tester that someone has posted information about is also a 
> useful metric.

I also included size differences in the script, it will be interesting
to compare these.  Additionally we should have some checking of speed,
but the data of SPEC compilations appears to be way too noisy to be
usefull to be processed automatically (even the speed metrics are in
many cases)

Honza
> 
> --joel sherrill
> 
> 

Re: Speed issues taken seriously?

[Daniel Berlin]

On Oct 10, 2003, at 10:49 AM, Lars Gullik Bjønnes wrote:

>
> I just thought that I'd see how gcc 3.3.2 (CVS) and 3.4(CVS) compared,
> I was hoping (expecting) the recent months long threads on compile
> performance to have had an effect... the result was a bit
> disappointing...

Did you specifically disable checking in the 3.4 compiler?