Extremely bad performance compiling bzip2

Extremely bad performance compiling bzip2

[Osku Salerma]

I upgraded my system from libc5 and gcc 2.7.2.1 to glibc2 2.0.6 and
egcs 1.1.1 over the weekend. Everything seems to be working, but the
first thing that I did to test to see if egcs was any better at
optimizing than gcc was to recompile bzip2, version 0.1pl2 to be
exact.

I use the latest netscape distribution as the testfile, which is a 12M
.bz2 file. Using the command "time bzip2 -dc nav.tar.bz2 > /dev/null"
as the benchmark command, my old bzip2 binary, compiled with gcc
2.7.2.1 and linked to libc5, does it in 33-35 seconds.

The flags used to compile with the old gcc were "-O3
-fomit-frame-pointer -funroll-loops", which are the default ones in the
bzip2 distribution.

To make a long story short, compiling with egcs 1.1.1 and those same
flags and linking against glibc2, I get a running time of 58-63
seconds. Adding "-mcpu=pentiumpro" to the flags I get down to 52-53
seconds. (My machine is a K6/233 with 96M of memory). That's still 50%
slower than plain old gcc.

Is there something I'm doing wrong, or should I just accept that bzip2
compiled with egcs is dog slow? I did some tests with gzip too
afterwards, and gzip compiled with egcs is in the +-1% range with my
old gzip.

--
Osku Salerma - osku@iki.fi - http://www.iki.fi/osku/
I'd give my right arm to be ambidextrous.

Re: Extremely bad performance compiling bzip2

[Jeffrey A Law]

  In message < Pine.LNX.3.91.981207180935.1228A-100000@in58.in.helsinki.fi >you w
rite:
  > I upgraded my system from libc5 and gcc 2.7.2.1 to glibc2 2.0.6 and
  > egcs 1.1.1 over the weekend. Everything seems to be working, but the
  > first thing that I did to test to see if egcs was any better at
  > optimizing than gcc was to recompile bzip2, version 0.1pl2 to be
  > exact.
  > 
  > I use the latest netscape distribution as the testfile, which is a 12M
  > .bz2 file. Using the command "time bzip2 -dc nav.tar.bz2 > /dev/null"
  > as the benchmark command, my old bzip2 binary, compiled with gcc
  > 2.7.2.1 and linked to libc5, does it in 33-35 seconds.
  > 
  > The flags used to compile with the old gcc were "-O3
  > -fomit-frame-pointer -funroll-loops", which are the default ones in the
  > bzip2 distribution.
  > 
  > To make a long story short, compiling with egcs 1.1.1 and those same
  > flags and linking against glibc2, I get a running time of 58-63
  > seconds. Adding "-mcpu=pentiumpro" to the flags I get down to 52-53
  > seconds. (My machine is a K6/233 with 96M of memory). That's still 50%
  > slower than plain old gcc.
  > 
  > Is there something I'm doing wrong, or should I just accept that bzip2
  > compiled with egcs is dog slow? I did some tests with gzip too
  > afterwards, and gzip compiled with egcs is in the +-1% range with my
  > old gzip.
You'd have to look at the hot loops.  It may also be the case that these
problems are fixed by current snapshots.  We've done a lot of work to improve
spill code and register allocation, which can have a big impact on x86
performance.

In general, I don't recommend loop unrolling on the x86 -- unrolling loops
takes more registers, and the x86 is painfully short of registers.

jeff

Re: Extremely bad performance compiling bzip2

[H.J. Lu]

> I use the latest netscape distribution as the testfile, which is a 12M
> .bz2 file. Using the command "time bzip2 -dc nav.tar.bz2 > /dev/null"
> as the benchmark command, my old bzip2 binary, compiled with gcc
> 2.7.2.1 and linked to libc5, does it in 33-35 seconds.
> 
> The flags used to compile with the old gcc were "-O3
> -fomit-frame-pointer -funroll-loops", which are the default ones in the
> bzip2 distribution.
> 
> To make a long story short, compiling with egcs 1.1.1 and those same
> flags and linking against glibc2, I get a running time of 58-63
> seconds. Adding "-mcpu=pentiumpro" to the flags I get down to 52-53
> seconds. (My machine is a K6/233 with 96M of memory). That's still 50%
> slower than plain old gcc.
> 

Have you tried bzip2 compiled with gcc 2.7.2.3 and linked to glibc 2?
I know if there are many getc/putc's in the code, glibc 2 can much
slower than libc 5.

-- 
H.J. Lu (hjl@gnu.org)

Re: Extremely bad performance compiling bzip2

[Osku Salerma]

> You'd have to look at the hot loops. It may also be the case that
> these problems are fixed by current snapshots. We've done a lot of
> work to improve spill code and register allocation, which can have a
> big impact on x86 performance.

Egcs 1.1.1 was released five days ago. The newest .c file in the
sources is six days old. Am I misunderstanding something about the
egcs development policy or what do you mean by a "current snapshot"?

--
Osku Salerma - osku@iki.fi - http://www.iki.fi/osku/
I'd give my right arm to be ambidextrous.

Re: Extremely bad performance compiling bzip2

[Osku Salerma]

> Have you tried bzip2 compiled with gcc 2.7.2.3 and linked to glibc 2?
> I know if there are many getc/putc's in the code, glibc 2 can much
> slower than libc 5.

I would, but I didn't keep my old gcc around. Could someone out there
test bzip2 with some compiler with both libc5 and glibc2 and post some
figures?

BTW, why is it slower? Is glibc 2.1 any faster?

--
Osku Salerma - osku@iki.fi - http://www.iki.fi/osku/
I'd give my right arm to be ambidextrous.

Re: Extremely bad performance compiling bzip2

[Jeffrey A Law]

  In message < Pine.LNX.3.91.981208223730.3436A-100000@oskula >you write:
  > > You'd have to look at the hot loops. It may also be the case that
  > > these problems are fixed by current snapshots. We've done a lot of
  > > work to improve spill code and register allocation, which can have a
  > > big impact on x86 performance.
  > 
  > Egcs 1.1.1 was released five days ago. The newest .c file in the
  > sources is six days old. Am I misunderstanding something about the
  > egcs development policy or what do you mean by a "current snapshot"?
The easiest way to look at it is there's a stable release tree and an
unstable development tree.

Snapshots happen from the development tree.

jeff

Re: Extremely bad performance compiling bzip2

[Ulrich Drepper]

Osku Salerma <osku@iki.fi> writes:

> BTW, why is it slower? Is glibc 2.1 any faster?

If a program uses putc on a stream which has no problems with multiple
threads it should use putc_unlocked.  Similar functions variants exist
for fputc, fputs etc.

-- 
---------------.      drepper at gnu.org  ,-.   1325 Chesapeake Terrace
Ulrich Drepper  \    ,-------------------'   \  Sunnyvale, CA 94089 USA
Cygnus Solutions `--' drepper at cygnus.com   `------------------------

Re: Extremely bad performance compiling bzip2

[Andi Kleen]

In muc.lists.egcs.misc, you wrote:
>Have you tried bzip2 compiled with gcc 2.7.2.3 and linked to glibc 2?
>I know if there are many getc/putc's in the code, glibc 2 can much
>slower than libc 5.

I am curious why. Does checking an uncongested lock really involve that
much overhead? 

-Andi

Re: Extremely bad performance compiling bzip2

[Tobias Ringstrom]

On 09-Dec-98 Andi Kleen wrote:
> In muc.lists.egcs.misc, you wrote:
>>Have you tried bzip2 compiled with gcc 2.7.2.3 and linked to glibc 2?
>>I know if there are many getc/putc's in the code, glibc 2 can much
>>slower than libc 5.
> 
> I am curious why. Does checking an uncongested lock really involve that
> much overhead? 

I ran a quick test on my i686-pc-linux-gnu with glibc-2.0.7-13.rpm and got:

> time ./putchar_unlocked 100000000 > /dev/null
3.830u 0.010s 0:03.83 100.2%    0+0k 0+0io 65pf+0w

> time ./putchar 100000000 > /dev/null
16.160u 0.040s 0:16.20 100.0%   0+0k 0+0io 66pf+0w

The test program is running in a tight loop, printing spaces. Notice the
impressive CPU utilization of 100.2% :-)

/ Tobias

Re: Extremely bad performance compiling bzip2

[Andreas Schwab]

Andi Kleen <ak@muc.de> writes:

|> In muc.lists.egcs.misc, you wrote:
|> >Have you tried bzip2 compiled with gcc 2.7.2.3 and linked to glibc 2?
|> >I know if there are many getc/putc's in the code, glibc 2 can much
|> >slower than libc 5.
|> 
|> I am curious why. Does checking an uncongested lock really involve that
|> much overhead? 

The overhead is most likely in the function call.  getc/putc in glibc 2
are a real functions, whereas in libc 5 they are macros that only call a
real function on under/overflow.

-- 
Andreas Schwab                                      "And now for something
schwab@issan.cs.uni-dortmund.de                      completely different"
schwab@gnu.org

Re: Extremely bad performance compiling bzip2

[Osku Salerma]

> > BTW, why is it slower? Is glibc 2.1 any faster?
> 
> If a program uses putc on a stream which has no problems with multiple
> threads it should use putc_unlocked.  Similar functions variants exist
> for fputc, fputs etc.

I put these at the start of bzip2.c, but the running time only went
down about 5 seconds, to 47 seconds. (There weren't any places where
putc/getc instead of fputc/fgetc seemed likely to cause troubles.)
     
#define ferror(fp) ferror_unlocked(fp)
#define getc(fp) getc_unlocked(fp)
#define putc(a,fp) putc_unlocked(a,fp)
#define fputc(a,fp) putc_unlocked(a,fp)
#define fgetc(fp) getc_unlocked(fp)

--
Osku Salerma - osku@iki.fi - http://www.iki.fi/osku/
I'd give my right arm to be ambidextrous.