performance regression with trunk's gengtype on ARM?

performance regression with trunk's gengtype on ARM?

[Mikael Pettersson]

I'm seeing what appears to be a recent massive performance regression
with trunk's gengtype, as compiled and run in stage 2, on ARM V5TE.

Right now 4.7-20110827's stage2 gengtype has been running for almost
10 hours on my ARM build machine, but the process is tiny and no swapping
occurs.  To put those 10 hours in perspective, on this machine (1.6 GHz
ARM V5TE uniprocessor running Linux) I regularly do full bootstraps and
regression test suite runs for c,c++,ada,fortran in about 18 hours for
gcc 4.4, about 20 hours for gcc 4.5, about 24 hours for gcc 4.6, and
about 27 hours for trunk until recently.  So 10 hours or more just in
stage 2 gengtype is suspicious.

I believe 4.7-20110820 also was unusually slow to build, but I didn't
monitor that build very carefully so can't say if gengtype was involved
then too.

/Mikael

Re: performance regression with trunk's gengtype on ARM?

[Michael Hope]

On Mon, Aug 29, 2011 at 8:57 AM, Mikael Pettersson <mikpe@it.uu.se> wrote:
> I'm seeing what appears to be a recent massive performance regression
> with trunk's gengtype, as compiled and run in stage 2, on ARM V5TE.
>
> Right now 4.7-20110827's stage2 gengtype has been running for almost
> 10 hours on my ARM build machine, but the process is tiny and no swapping
> occurs.  To put those 10 hours in perspective, on this machine (1.6 GHz
> ARM V5TE uniprocessor running Linux) I regularly do full bootstraps and
> regression test suite runs for c,c++,ada,fortran in about 18 hours for
> gcc 4.4, about 20 hours for gcc 4.5, about 24 hours for gcc 4.6, and
> about 27 hours for trunk until recently.  So 10 hours or more just in
> stage 2 gengtype is suspicious.
>
> I believe 4.7-20110820 also was unusually slow to build, but I didn't
> monitor that build very carefully so can't say if gengtype was involved
> then too.

FWIW, I build trunk once a week on a PandaBoard.  r178096 took 10
hours to bootstrap C, C++, and Fortran and 9 hours to test.  The 4.5
release branch at r177893 takes 3:50 to bootstrap and 6:15 to test.

I've put the user time in seconds below.  4.5 is ~20000 s, 4.6 is
~23000 s, and current trunk ~46000 (2.3 x slower).

See http://builds.linaro.org/toolchain/ for more.

-- Michael

gcc-4.5+svn175369 20602.07
gcc-4.5+svn175745 19768.21
gcc-4.5+svn176026 19739.35
gcc-4.5+svn176306 19711.70
gcc-4.5+svn176615 19668.38
gcc-4.5+svn176915 19728.38
gcc-4.5+svn177422 19713.14
gcc-4.5+svn177688 19746.67
gcc-4.5+svn177893 19744.96
gcc-4.6+svn175136 22979.51
gcc-4.6+svn175369 23092.50
gcc-4.6+svn175745 22958.21
gcc-4.6+svn176026 23009.37
gcc-4.6+svn176306 22952.93
gcc-4.6+svn176615 22952.11
gcc-4.6+svn177422 22946.22
gcc-4.6+svn177688 22847.87
gcc-4.6+svn177894 22964.09
gcc-4.6+svn178096 22934.61
gcc-4.7~svn175284 34518.10
gcc-4.7~svn175368 34887.17
gcc-4.7~svn175422 34975.48
gcc-4.7~svn175617 34908.60
gcc-4.7~svn175745 35040.42
gcc-4.7~svn175795 35110.84
gcc-4.7~svn175904 34893.29
gcc-4.7~svn176026 34972.99
gcc-4.7~svn176133 35171.65
gcc-4.7~svn176224 35247.44
gcc-4.7~svn176306 35038.07
gcc-4.7~svn176494 26151.21
gcc-4.7~svn176615 26257.04
gcc-4.7~svn176733 40401.18
gcc-4.7~svn176816 40048.32
gcc-4.7~svn176915 40102.52
gcc-4.7~svn176998 40161.10
gcc-4.7~svn177229 28604.27
gcc-4.7~svn177422 44991.89
gcc-4.7~svn177554 45199.05
gcc-4.7~svn177610 45173.94
gcc-4.7~svn177688 45469.00
gcc-4.7~svn177823 45391.00
gcc-4.7~svn177949 28769.64
gcc-4.7~svn178025 45605.43
gcc-4.7~svn178096 45599.59

Re: performance regression with trunk's gengtype on ARM?

[Basile Starynkevitch]

On Sun, 28 Aug 2011 22:57:54 +0200
Mikael Pettersson <mikpe@it.uu.se> wrote:

> I'm seeing what appears to be a recent massive performance regression
> with trunk's gengtype, as compiled and run in stage 2, on ARM V5TE.
> 
> Right now 4.7-20110827's stage2 gengtype has been running for almost
> 10 hours on my ARM build machine, but the process is tiny and no swapping
> occurs.  [...]

This is indeed very suspicious (especially since it happens in stage2, not in stage1,
IIUY). Can you ltrace or strace the gengtype process?

You could set GENGTYPE_FLAGS= -v in gcc/Makefile.in to understand more what gengtype is
doing. And gengtype accepts several -v flags.

Cheers.

-- 
Basile STARYNKEVITCH         http://starynkevitch.net/Basile/
email: basile<at>starynkevitch<dot>net mobile: +33 6 8501 2359
8, rue de la Faiencerie, 92340 Bourg La Reine, France
*** opinions {are only mine, sont seulement les miennes} ***

Re: performance regression with trunk's gengtype on ARM?

[Mikael Pettersson]

Mikael Pettersson writes:
 > I'm seeing what appears to be a recent massive performance regression
 > with trunk's gengtype, as compiled and run in stage 2, on ARM V5TE.
 > 
 > Right now 4.7-20110827's stage2 gengtype has been running for almost
 > 10 hours on my ARM build machine, but the process is tiny and no swapping
 > occurs.  To put those 10 hours in perspective, on this machine (1.6 GHz
 > ARM V5TE uniprocessor running Linux) I regularly do full bootstraps and
 > regression test suite runs for c,c++,ada,fortran in about 18 hours for
 > gcc 4.4, about 20 hours for gcc 4.5, about 24 hours for gcc 4.6, and
 > about 27 hours for trunk until recently.  So 10 hours or more just in
 > stage 2 gengtype is suspicious.
 > 
 > I believe 4.7-20110820 also was unusually slow to build, but I didn't
 > monitor that build very carefully so can't say if gengtype was involved
 > then too.

It's now been running for almost 21 hours, with no output produced at all.
According to strace it's currently not doing any syscalls, and according
to gdb it's looping in yylex().  I suspect a miscompilation.

Anyway I'm killing that build now.  I have a pile of stable branch
backports to test, after that I might try a regression hunt on trunk.

Re: performance regression with trunk's gengtype on ARM?

[Mikael Pettersson]

Mikael Pettersson writes:
 > Mikael Pettersson writes:
 >  > I'm seeing what appears to be a recent massive performance regression
 >  > with trunk's gengtype, as compiled and run in stage 2, on ARM V5TE.
 >  > 
 >  > Right now 4.7-20110827's stage2 gengtype has been running for almost
 >  > 10 hours on my ARM build machine, but the process is tiny and no swapping
 >  > occurs.  To put those 10 hours in perspective, on this machine (1.6 GHz
 >  > ARM V5TE uniprocessor running Linux) I regularly do full bootstraps and
 >  > regression test suite runs for c,c++,ada,fortran in about 18 hours for
 >  > gcc 4.4, about 20 hours for gcc 4.5, about 24 hours for gcc 4.6, and
 >  > about 27 hours for trunk until recently.  So 10 hours or more just in
 >  > stage 2 gengtype is suspicious.
 >  > 
 >  > I believe 4.7-20110820 also was unusually slow to build, but I didn't
 >  > monitor that build very carefully so can't say if gengtype was involved
 >  > then too.
 > 
 > It's now been running for almost 21 hours, with no output produced at all.
 > According to strace it's currently not doing any syscalls, and according
 > to gdb it's looping in yylex().  I suspect a miscompilation.
 > 
 > Anyway I'm killing that build now.  I have a pile of stable branch
 > backports to test, after that I might try a regression hunt on trunk.

The regression

a) was repeatable, and
b) got fixed in 4.7-20110903.

I'm not going to investigate this any further, unless I'm forced
to do a regression hunt in this date range in the future.