public inbox for gcc@gcc.gnu.org
 help / color / mirror / Atom feed
* RE: Precompiled headers (or other speed ups)
@ 2001-01-10  9:56 David Korn
  2001-01-10 11:14 ` Zack Weinberg
  0 siblings, 1 reply; 30+ messages in thread
From: David Korn @ 2001-01-10  9:56 UTC (permalink / raw)
  To: 'dewar@gnat.com'; +Cc: gcc

><<  Well, only really surprising if you assume the parsing code is well
>written :-) After all, the mid/backend is mostly doing memory-memory
>transforms on the trees, and that's going to run as fast as your cache
>and cpu can handle.  Parsers, OTOH, have to I/O.  Badly written parsers
>might even use c stdio functions to fgetc the entire file one char at a
>time....
>>>
>
>Sure, but that presumably is an easy thing to fix. In GNAT, the entire
>file is read with a single read, and then all access to the file is to
>the in memory copy, which stays around throughout the compilation.

  Oh yeah; I was just suggesting it wouldn't be *surprising*, not that it
would be 'right'!  Last time I looked at bison (or was it lex or yacc? I
forget) was around '95 and it did tend to turn out fairly sucky code (in
terms of f-stdio) if you didn't know what you were doing and how to hook
it into a memory buffer.  I haven't looked at the gcc parsers yet but I
would hope that they don't have such obvious design flaws.....

  Another thing that could eat time in a parser would be if there were
numerous shift-reduce conflicts that caused a lot of backtracking on fairly
common constructs.  That's just a for-example also :)

    DaveK

-- 
The Boulder Pledge: "Under no circumstances will I ever purchase anything 
offered to me as the result of an unsolicited email message. Nor will I 
forward chain letters, petitions, mass mailings, or virus warnings to large 
numbers of others. This is my contribution to the survival of the online
community." 


**********************************************************************
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. If you have received this email in error please notify
the system manager.

This footnote also confirms that this email message has been swept by
MIMEsweeper for the presence of computer viruses.

www.mimesweeper.com
**********************************************************************

^ permalink raw reply	[flat|nested] 30+ messages in thread

* Re: Precompiled headers (or other speed ups)
  2001-01-10  9:56 Precompiled headers (or other speed ups) David Korn
@ 2001-01-10 11:14 ` Zack Weinberg
  2001-01-10 13:54   ` Neil Booth
  2001-01-10 15:26   ` Joe Buck
  0 siblings, 2 replies; 30+ messages in thread
From: Zack Weinberg @ 2001-01-10 11:14 UTC (permalink / raw)
  To: David Korn; +Cc: 'dewar@gnat.com', gcc

On Wed, Jan 10, 2001 at 05:59:36PM -0000, David Korn wrote:
> 
> ><<  Well, only really surprising if you assume the parsing code is well
> >written :-) After all, the mid/backend is mostly doing memory-memory
> >transforms on the trees, and that's going to run as fast as your cache
> >and cpu can handle.  Parsers, OTOH, have to I/O.  Badly written parsers
> >might even use c stdio functions to fgetc the entire file one char at a
> >time....
> >>>
> >
> >Sure, but that presumably is an easy thing to fix. In GNAT, the entire
> >file is read with a single read, and then all access to the file is to
> >the in memory copy, which stays around throughout the compilation.
> 
>   Oh yeah; I was just suggesting it wouldn't be *surprising*, not that it
> would be 'right'!  Last time I looked at bison (or was it lex or yacc? I
> forget) was around '95 and it did tend to turn out fairly sucky code (in
> terms of f-stdio) if you didn't know what you were doing and how to hook
> it into a memory buffer.  I haven't looked at the gcc parsers yet but I
> would hope that they don't have such obvious design flaws.....
> 
>   Another thing that could eat time in a parser would be if there were
> numerous shift-reduce conflicts that caused a lot of backtracking on fairly
> common constructs.  That's just a for-example also :)

C++ reads files through cpplib, which is naturally as fast as I and
Neil Booth have been able to make it.  It does indeed read the entire
file at once and process it in memory.  The tokenizer is, in my tests,
consistently faster than the analogue in GCC 2.95.  Macro expansion
can still be a bottleneck under some conditions.

The parser, on the other hand, is not efficient at all.  By the parser
I mean specifically the code which takes in the token stream from
cpplib and generates a tree representation.  We spend a great deal of
time in yyparse itself, which I suspect means there is indeed a lot of
backtracking involved.  Its immediate subroutines are expensive also.

I don't know enough about C++ or its implementation in GCC to say for
sure what the ultimate cause is.  I've been told that C++ has an
extremely ambiguous grammar and the parser has to accumulate entire
statements on the stack before it can begin to match anything.
There's an ad-hoc layer in between the lexer and the parser
(cp/spew.c) whose purpose is to rearrange the order in which certain
constructs appear so they are easier to parse; you can imagine the
mess that would require that.

It isn't merely a matter of efficiency; there are some parser bugs in
the current implementation that have been pushed aside as too
difficult to fix.  A rewrite has been discussed numerous times, e.g. 
http://gcc.gnu.org/ml/gcc/2000-10/msg00573.html .

zw

^ permalink raw reply	[flat|nested] 30+ messages in thread

* Re: Precompiled headers (or other speed ups)
  2001-01-10 11:14 ` Zack Weinberg
@ 2001-01-10 13:54   ` Neil Booth
  2001-01-10 15:45     ` Zack Weinberg
  2001-01-10 15:26   ` Joe Buck
  1 sibling, 1 reply; 30+ messages in thread
From: Neil Booth @ 2001-01-10 13:54 UTC (permalink / raw)
  To: Zack Weinberg; +Cc: gcc

Zack Weinberg wrote:-

> The tokenizer is, in my tests, consistently faster than the analogue
> in GCC 2.95.  Macro expansion can still be a bottleneck under some
> conditions.

That's interesting.  Are these tests you've done recently?  i.e. is it
since I moved us to the forwards-looking, almost-no-step-back lexer?
It's good news if it is.

I think our object-like macro expansion is near optimal, particularly
as integrated CPP.  We simply dish out tokens straight from the
replacement list, with just a quick test that the new token isn't a
macro.

Stand-alone CPP is dog slow on token output (50%-70% of run-time!),
because of GLIBC.  But we know that, and decided we don't care too
much for the moment since for the common case we're now integrated.

If you like, I can work on the function-like macro expansion as a
"performance enhancement" during the GCC 3.0 branch for release.  I
think we can speed up nested function-like macros a lot (and have an
old patch on my drive to do so, but it would need dusting off a
little).

Neil.

^ permalink raw reply	[flat|nested] 30+ messages in thread

* Re: Precompiled headers (or other speed ups)
  2001-01-10 11:14 ` Zack Weinberg
  2001-01-10 13:54   ` Neil Booth
@ 2001-01-10 15:26   ` Joe Buck
  1 sibling, 0 replies; 30+ messages in thread
From: Joe Buck @ 2001-01-10 15:26 UTC (permalink / raw)
  To: Zack Weinberg; +Cc: David Korn, 'dewar@gnat.com', gcc

Zack Weinberg writes:

> C++ reads files through cpplib, which is naturally as fast as I and
> Neil Booth have been able to make it...
> 
> The parser, on the other hand, is not efficient at all...
>
> I don't know enough about C++ or its implementation in GCC to say for
> sure what the ultimate cause is.  I've been told that C++ has an
> extremely ambiguous grammar and the parser has to accumulate entire
> statements on the stack before it can begin to match anything.

It is true that the C++ grammar is ugly, but it is not nearly as ugly as
gcc/cp/parse.y is.  The bison grammar has had about 10 years worth of
changes applied to track a rapidly evolving language, and it contains many
expensive kludges that no one dares touch.  Mark Mitchell's been talking
about pitching the whole thing and redoing it (after 3.0).

^ permalink raw reply	[flat|nested] 30+ messages in thread

* Re: Precompiled headers (or other speed ups)
  2001-01-10 13:54   ` Neil Booth
@ 2001-01-10 15:45     ` Zack Weinberg
  0 siblings, 0 replies; 30+ messages in thread
From: Zack Weinberg @ 2001-01-10 15:45 UTC (permalink / raw)
  To: Neil Booth; +Cc: gcc

On Wed, Jan 10, 2001 at 09:53:18PM +0000, Neil Booth wrote:
> Zack Weinberg wrote:-
> 
> > The tokenizer is, in my tests, consistently faster than the analogue
> > in GCC 2.95.  Macro expansion can still be a bottleneck under some
> > conditions.
> 
> That's interesting.  Are these tests you've done recently?  i.e. is it
> since I moved us to the forwards-looking, almost-no-step-back lexer?
> It's good news if it is.

I haven't done a rigorous head-to-head test recently, but we are
measurably faster than we were in July, and that wasn't much slower
than 2.95.  This is C at -O0, so no major backend or parser
regressions to confuse the issue.

Most of the win is probably from not tokenizing twice.

> Stand-alone CPP is dog slow on token output (50%-70% of run-time!),
> because of GLIBC.  But we know that, and decided we don't care too
> much for the moment since for the common case we're now integrated.

*nod*

> If you like, I can work on the function-like macro expansion as a
> "performance enhancement" during the GCC 3.0 branch for release.  I
> think we can speed up nested function-like macros a lot (and have an
> old patch on my drive to do so, but it would need dusting off a
> little).

I think that'd be a good idea.

zw

^ permalink raw reply	[flat|nested] 30+ messages in thread

* Re: Precompiled headers (or other speed ups)
  2001-01-09 15:43 ` Stan Shebs
@ 2001-01-10 22:55   ` Toon Moene
  0 siblings, 0 replies; 30+ messages in thread
From: Toon Moene @ 2001-01-10 22:55 UTC (permalink / raw)
  To: Stan Shebs; +Cc: dewar, amep, gcc

Stan Shebs wrote:

> Unlike the Linux situation, where GCC is effectively a "monopoly"
> since there aren't really any other compiler choices,

This is only true for the C/C++/Java part.  *We* have plenty of
competition, both on the ix86 platform as well as others.

I have *never* received complaints about the compile speed of g77.

-- 
Toon Moene - mailto:toon@moene.indiv.nluug.nl - phoneto: +31 346 214290
Saturnushof 14, 3738 XG  Maartensdijk, The Netherlands
Maintainer, GNU Fortran 77: http://gcc.gnu.org/onlinedocs/g77_news.html
Join GNU Fortran 95: http://g95.sourceforge.net/ (under construction)

^ permalink raw reply	[flat|nested] 30+ messages in thread

* Re: Precompiled headers (or other speed ups)
  2001-01-10  9:35 dewar
@ 2001-01-10 18:53 ` Yu Xuanwei
  0 siblings, 0 replies; 30+ messages in thread
From: Yu Xuanwei @ 2001-01-10 18:53 UTC (permalink / raw)
  To: dewar, gcc, pedwards

----- Original Message -----
From: <dewar@gnat.com>
To: <gcc@gcc.gnu.org>; <pedwards@disaster.jaj.com>
Sent: Thursday, January 11, 2001 1:35 AM
Subject: Re: Precompiled headers (or other speed ups)


> <<Perhaps in Ada parsing only takes microseconds.  In most of the C-family
> languages with which I am familiar, parsing takes the majority of the
time.
> >>
>
> If by parsing you mean lexical analysis and construction of the syntax
> tree (NOT including static semantic analysis), then this should be very
> rapid. There is nothing especially difficult about C or C++ with respect
> to lexical analysis and parsing.
>
> If the static semantic analysis is taking a lot of time, perhaps there
> are some non-optimal algorithms that need looking at. I do not see why
> C++ should be significantly harder to handle than Ada 95 in this regard.
>
>
I agree with you. Lex and Yacc generate pretty rapid C code.

^ permalink raw reply	[flat|nested] 30+ messages in thread

* Re: Precompiled headers (or other speed ups)
  2001-01-10  5:29 ` Andi Kleen
  2001-01-10  8:32   ` Phil Edwards
@ 2001-01-10 15:12   ` Stan Shebs
  1 sibling, 0 replies; 30+ messages in thread
From: Stan Shebs @ 2001-01-10 15:12 UTC (permalink / raw)
  To: Andi Kleen; +Cc: dewar, gcc

Andi Kleen wrote:
> 
> I guess it would be useful if Stan could post a few a profile logs of
> g++ compiling PowerPlant code, just to see where the hot spots are.

I trying to get there as fast as I can!  The only numbers I could post
right now are for Apple's hacked 2.95 compiler, which would be misleading
at best.  I had a pretty good test run with patched CVS code last night,
but I'll have to add some other extensions (-fpascal-strings, woo hoo)
before PowerPlant will even compile...

Stan

^ permalink raw reply	[flat|nested] 30+ messages in thread

* Re: Precompiled headers (or other speed ups)
@ 2001-01-10 11:45 dewar
  0 siblings, 0 replies; 30+ messages in thread
From: dewar @ 2001-01-10 11:45 UTC (permalink / raw)
  To: dkorn, zackw; +Cc: dewar, gcc

<<I don't know enough about C++ or its implementation in GCC to say for
sure what the ultimate cause is.  I've been told that C++ has an
extremely ambiguous grammar and the parser has to accumulate entire
statements on the stack before it can begin to match anything.
There's an ad-hoc layer in between the lexer and the parser
(cp/spew.c) whose purpose is to rearrange the order in which certain
constructs appear so they are easier to parse; you can imagine the
mess that would require that.
>>

Sounds to me like the new Code Sourcery project to redo the parser has
the potential for good gains here. 

^ permalink raw reply	[flat|nested] 30+ messages in thread

* RE: Precompiled headers (or other speed ups)
@ 2001-01-10 10:21 dewar
  0 siblings, 0 replies; 30+ messages in thread
From: dewar @ 2001-01-10 10:21 UTC (permalink / raw)
  To: dewar, dkorn; +Cc: gcc

Well I guess neither of us knows enough to contribute further, but I
remain susprised that the issue should be front end speed rather than
backend speed.

^ permalink raw reply	[flat|nested] 30+ messages in thread

* Re: Precompiled headers (or other speed ups)
@ 2001-01-10  9:35 dewar
  2001-01-10 18:53 ` Yu Xuanwei
  0 siblings, 1 reply; 30+ messages in thread
From: dewar @ 2001-01-10  9:35 UTC (permalink / raw)
  To: gcc, pedwards

<<Perhaps in Ada parsing only takes microseconds.  In most of the C-family
languages with which I am familiar, parsing takes the majority of the time.
>>

If by parsing you mean lexical analysis and construction of the syntax
tree (NOT including static semantic analysis), then this should be very
rapid. There is nothing especially difficult about C or C++ with respect
to lexical analysis and parsing.

If the static semantic analysis is taking a lot of time, perhaps there
are some non-optimal algorithms that need looking at. I do not see why
C++ should be significantly harder to handle than Ada 95 in this regard.

^ permalink raw reply	[flat|nested] 30+ messages in thread

* RE: Precompiled headers (or other speed ups)
@ 2001-01-10  9:29 dewar
  0 siblings, 0 replies; 30+ messages in thread
From: dewar @ 2001-01-10  9:29 UTC (permalink / raw)
  To: dewar, dkorn; +Cc: gcc

<<  Well, only really surprising if you assume the parsing code is well
written :-) After all, the mid/backend is mostly doing memory-memory
transforms on the trees, and that's going to run as fast as your cache
and cpu can handle.  Parsers, OTOH, have to I/O.  Badly written parsers
might even use c stdio functions to fgetc the entire file one char at a
time....
>>

Sure, but that presumably is an easy thing to fix. In GNAT, the entire
file is read with a single read, and then all access to the file is to
the in memory copy, which stays around throughout the compilation.

^ permalink raw reply	[flat|nested] 30+ messages in thread

* RE: Precompiled headers (or other speed ups)
@ 2001-01-10  9:11 David Korn
  0 siblings, 0 replies; 30+ messages in thread
From: David Korn @ 2001-01-10  9:11 UTC (permalink / raw)
  To: 'dewar@gnat.com'; +Cc: gcc

>-----Original Message-----
>From: dewar@gnat.com [ mailto:dewar@gnat.com ]

><<was posted in last August, but there is a good example of how the parser
>can dominate just about everything.  That message kicked off a very good
>discussion on precompiled headers and parsing performance in C++.
>>.
>
>This is quite surprising, depending on what parsing means, if parsing
>means just building a syntax tree from the grammar, it is amazing that
>this should take any time at all compared to other phases.

  Well, only really surprising if you assume the parsing code is well
written :-) After all, the mid/backend is mostly doing memory-memory
transforms on the trees, and that's going to run as fast as your cache
and cpu can handle.  Parsers, OTOH, have to I/O.  Badly written parsers 
might even use c stdio functions to fgetc the entire file one char at a 
time....

    cheers, 
       DaveK
-- 
The Boulder Pledge: "Under no circumstances will I ever purchase anything 
offered to me as the result of an unsolicited email message. Nor will I 
forward chain letters, petitions, mass mailings, or virus warnings to large 
numbers of others. This is my contribution to the survival of the online
community."


**********************************************************************
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. If you have received this email in error please notify
the system manager.

This footnote also confirms that this email message has been swept by
MIMEsweeper for the presence of computer viruses.

www.mimesweeper.com
**********************************************************************

^ permalink raw reply	[flat|nested] 30+ messages in thread

* Re: Precompiled headers (or other speed ups)
@ 2001-01-10  9:02 Phil Edwards
  0 siblings, 0 replies; 30+ messages in thread
From: Phil Edwards @ 2001-01-10  9:02 UTC (permalink / raw)
  To: gcc

> This is quite surprising, depending on what parsing means, if parsing
> means just building a syntax tree from the grammar, it is amazing that
> this should take any time at all compared to other phases.

Perhaps in Ada parsing only takes microseconds.  In most of the C-family
languages with which I am familiar, parsing takes the majority of the time.

My only point here is that precompiled headers /do/ make a difference and
/will/ be a very big win once they are implemented, especially for C++.
The current 10x-20x lossage on PPC is painful to hear about, and I don't
even use that platform.  :-)


Phil

-- 
pedwards at disaster dot jaj dot com  |  pme at sources dot redhat dot com
devphil at several other less interesting addresses in various dot domains
The gods do not protect fools.  Fools are protected by more capable fools.

^ permalink raw reply	[flat|nested] 30+ messages in thread

* Re: Precompiled headers (or other speed ups)
@ 2001-01-10  8:50 dewar
  0 siblings, 0 replies; 30+ messages in thread
From: dewar @ 2001-01-10  8:50 UTC (permalink / raw)
  To: ak, pedwards; +Cc: dewar, gcc, shebs

<<was posted in last August, but there is a good example of how the parser
can dominate just about everything.  That message kicked off a very good
discussion on precompiled headers and parsing performance in C++.
>.

This is quite surprising, depending on what parsing means, if parsing
means just building a syntax tree from the grammar, it is amazing that
this should take any time at all compared to other phases.

^ permalink raw reply	[flat|nested] 30+ messages in thread

* Re: Precompiled headers (or other speed ups)
  2001-01-10  5:29 ` Andi Kleen
@ 2001-01-10  8:32   ` Phil Edwards
  2001-01-10 15:12   ` Stan Shebs
  1 sibling, 0 replies; 30+ messages in thread
From: Phil Edwards @ 2001-01-10  8:32 UTC (permalink / raw)
  To: Andi Kleen; +Cc: dewar, gcc, shebs

On Wed, Jan 10, 2001 at 02:29:03PM +0100, Andi Kleen wrote:
> 
> I don't know why that's the case, but I suspect all the operator overloading
> resolving and handling of the non regular C++ grammar etc. has a cost over C.
> [It actually quite surprised me, I always assumed that the backend would
> dominate compile cost]

There have been some changes since

    http://gcc.gnu.org/ml/gcc/2000-08/msg00515.html

was posted in last August, but there is a good example of how the parser
can dominate just about everything.  That message kicked off a very good
discussion on precompiled headers and parsing performance in C++.


Phil

-- 
pedwards at disaster dot jaj dot com  |  pme at sources dot redhat dot com
devphil at several other less interesting addresses in various dot domains
The gods do not protect fools.  Fools are protected by more capable fools.

^ permalink raw reply	[flat|nested] 30+ messages in thread

* Re: Precompiled headers (or other speed ups)
  2001-01-09 14:55 dewar
  2001-01-09 15:43 ` Stan Shebs
@ 2001-01-10  5:29 ` Andi Kleen
  2001-01-10  8:32   ` Phil Edwards
  2001-01-10 15:12   ` Stan Shebs
  1 sibling, 2 replies; 30+ messages in thread
From: Andi Kleen @ 2001-01-10  5:29 UTC (permalink / raw)
  To: dewar; +Cc: gcc, shebs

dewar@gnat.com writes:

> 
> Of course the speeds that Stan is reporting are horribly slow in absolute
> terms. The real question is *why* slo slow. 150K lines of decls
> to be crunched should not be taking anywhere NEAR that long, since it
> is not necessary to generate full code for these decls, or at least 
> should not be.

In my experience g++ is much slower than the gcc C compiler, even without
excessive include files -- the include heavy style common for C++ makes
things just worse. The difference is visible when you just watch Makefile
runs. 

I don't know why that's the case, but I suspect all the operator overloading
resolving and handling of the non regular C++ grammar etc. has a cost over C.
[It actually quite surprised me, I always assumed that the backend would
dominate compile cost]

I guess it would be useful if Stan could post a few a profile logs of 
g++ compiling PowerPlant code, just to see where the hot spots are.

-Andi 

^ permalink raw reply	[flat|nested] 30+ messages in thread

* Re: Precompiled headers (or other speed ups)
@ 2001-01-09 19:52 dewar
  0 siblings, 0 replies; 30+ messages in thread
From: dewar @ 2001-01-09 19:52 UTC (permalink / raw)
  To: dewar, shebs; +Cc: amep, gcc

<<> By comparison, if you use GNAT to compile the DEC starlet library, which
> is 64,000 lines of very dense declarations, that is with'ed by typical
> applications, much as a C++ compiler eats headers with #include, it takes
> a few seconds on a fast PC.

Watch out for those "few seconds"! - if you're compiling 360 source
files, 10 seconds/compile adds up to a full hour.  To be as fast as MW,
you have to get done with each file in under a second.
>>

r
few seconds actually is about 2.3 seconds ....


my point here is that there is a big discprepancy between the speeds, and
that is strange. As I say, our impression for the GNAT case is that we
would not save much by the equivalent of precompiling headers ...

^ permalink raw reply	[flat|nested] 30+ messages in thread

* Re: Precompiled headers (or other speed ups)
  2001-01-09 14:55 dewar
@ 2001-01-09 15:43 ` Stan Shebs
  2001-01-10 22:55   ` Toon Moene
  2001-01-10  5:29 ` Andi Kleen
  1 sibling, 1 reply; 30+ messages in thread
From: Stan Shebs @ 2001-01-09 15:43 UTC (permalink / raw)
  To: dewar; +Cc: amep, gcc

dewar@gnat.com wrote:
> 
> [...]
> These days, it is probably 1 million lines/minute vs 100,000 lines/minute
> or something like that and it does not matter at all.
> 
> So absolute speeds are important.

OK, there is probably some level at which the absolute speed
difference is unnoticeable, but we're nowhere close to that.
Unlike the Linux situation, where GCC is effectively a "monopoly"
since there aren't really any other compiler choices, on Macs
developers can choose between Metrowerks and GCC, and they're
astonished that anybody would use a compiler as slow as GCC.

> By comparison, if you use GNAT to compile the DEC starlet library, which
> is 64,000 lines of very dense declarations, that is with'ed by typical
> applications, much as a C++ compiler eats headers with #include, it takes
> a few seconds on a fast PC.

Watch out for those "few seconds"! - if you're compiling 360 source
files, 10 seconds/compile adds up to a full hour.  To be as fast as MW,
you have to get done with each file in under a second.

> When I said that I did not consider fast compilation to be a major goal
> of gcc, I was talking about the back end and code generator, but if I
> understand things clearly, it is the C++ *front end* that is eating up
> time here, and that seems surprising.

C gets hit too.  I have to stay focussed on getting CVS GCC to bootstrap
on Darwin systems right now, but once that's in, I'll be able to go
back for another round of profiling.  A year ago, most of the wall
clock time in the hour-long builds I was measuring disappeared into
yyparse and friends, but that was 2.95, and I'm more interested in the
current code now.

Stan

^ permalink raw reply	[flat|nested] 30+ messages in thread

* Re: Precompiled headers (or other speed ups)
@ 2001-01-09 14:55 dewar
  2001-01-09 15:43 ` Stan Shebs
  2001-01-10  5:29 ` Andi Kleen
  0 siblings, 2 replies; 30+ messages in thread
From: dewar @ 2001-01-09 14:55 UTC (permalink / raw)
  To: dewar, shebs; +Cc: amep, gcc

<<They would just make MW builds go faster too, so GCC still looks
just as lame.
>>

Well I am not so sure about that. Early on, Realia COBOL compiled 10,000
lines/minute on a 4.77MHz PC1. The major competitor was about ten times
slower. That made a real difference.

But after a while, by the time we moved to 25MHz 386's, the difference
was 100,000 lines/minute vs 10,000 lines/minute, and that was not so
critical in practice.

These days, it is probably 1 million lines/minute vs 100,000 lines/minute
or something like that and it does not matter at all.

So absolute speeds are important.

Of course the speeds that Stan is reporting are horribly slow in absolute
terms. The real question is *why* slo slow. 150K lines of decls
to be crunched should not be taking anywhere NEAR that long, since it
is not necessary to generate full code for these decls, or at least 
should not be.

By comparison, if you use GNAT to compile the DEC starlet library, which
is 64,000 lines of very dense declarations, that is with'ed by typical
applications, much as a C++ compiler eats headers with #include, it takes
a few seconds on a fast PC. 

That's too big of a difference in performance to be easily explainable.

In the case of GNAT, we are not at all convinced that the equivalent of
compiled headers would be a win, since the intermediate compiled output
of something like starlet is pretty big (the compressed tree is 10 megs
compared to the source size of 2.2 megs, so you have a lot more I/O).

When I said that I did not consider fast compilation to be a major goal
of gcc, I was talking about the back end and code generator, but if I
understand things clearly, it is the C++ *front end* that is eating up
time here, and that seems surprising.

^ permalink raw reply	[flat|nested] 30+ messages in thread

* Re: Precompiled headers (or other speed ups)
  2001-01-08 20:41 dewar
  2001-01-09  3:01 ` Rob Taylor
  2001-01-09 10:04 ` amep
@ 2001-01-09 14:36 ` Stan Shebs
  2 siblings, 0 replies; 30+ messages in thread
From: Stan Shebs @ 2001-01-09 14:36 UTC (permalink / raw)
  To: dewar; +Cc: amep, gcc

dewar@gnat.com wrote:
> 
> However, super fast compilation speed has never been a gcc goal. Early on
> this was problematic, but our experience, at least in the GNAT world, is
> that machines are getting fast enough that compilation time is not a
> bottleneck for most folks, even though gcc may have got slower in absolute
> terms.

Just to make sure there are no misapprehensions, compile time is the
*most* serious issue for GCC acceptance by Mac developers working on
OS X.  GCC is 10x - 20x slower than Metrowerks compiling the same code
on the same system, which means that frequently-recompiled programs
like the Finder take a couple minutes in MW, and nearly an hour with
GCC.  And just to make things worse, much of the interesting code is
C++ headers, so full recompiles are quite common.  Faster machines?
They would just make MW builds go faster too, so GCC still looks
just as lame.

The problem is basically headers - the Mac toolbox is about 100K lines
of definitions and declarations, and PowerPlant, which is a popular
C++ high-level toolkit, adds another 50K.  So we crunch 150K lines
of decls before getting to the 500 lines of actual source code,
over and over.  So yes, precompiled headers would win big, and indeed
that's where MW gets a big part of its advantage.  In the absence
of precompiled headers in GCC, I've been having to hack on Apple's
funky PCH-using preprocessor, just because it gets us 3x-4x speedup,
which is better than nothing.

Stan

^ permalink raw reply	[flat|nested] 30+ messages in thread

* Re: Precompiled headers (or other speed ups)
@ 2001-01-09 11:26 dewar
  0 siblings, 0 replies; 30+ messages in thread
From: dewar @ 2001-01-09 11:26 UTC (permalink / raw)
  To: amep, gcc; +Cc: dewar

<<I'm sorry if my message came across as a complaint. It was only meant
as an inquiry to see if there was something I could play with and test
for people. I know my machine is "archaic", but I'm a pour 16-year
old. ;) I can't a afford even a $1000 dollar machine (and that
wouldn't be a particularly great machine). Anyway, sorry if I
offended. :-)
>>

I don't think you offended anyone!

The real issue is how much effort to spend on compilation time issues,
and also whether this particular case is anomolous or not.

^ permalink raw reply	[flat|nested] 30+ messages in thread

* Re: Precompiled headers (or other speed ups)
  2001-01-08 20:41 dewar
  2001-01-09  3:01 ` Rob Taylor
@ 2001-01-09 10:04 ` amep
  2001-01-09 14:36 ` Stan Shebs
  2 siblings, 0 replies; 30+ messages in thread
From: amep @ 2001-01-09 10:04 UTC (permalink / raw)
  To: gcc; +Cc: amep, dewar

I'm sorry if my message came across as a complaint. It was only meant
as an inquiry to see if there was something I could play with and test
for people. I know my machine is "archaic", but I'm a pour 16-year
old. ;) I can't a afford even a $1000 dollar machine (and that
wouldn't be a particularly great machine). Anyway, sorry if I
offended. :-)

Also, Geoff, Is there any PCH patch or anything that I could test? I'm
willing to deal with a system that is still under heavy testing and
I'll send in bug reports. Sorry, I'm getting pushy. Keep up the good
works, guys. :-)

-Arthur

From: dewar@gnat.com
Subject: Re: Precompiled headers (or other speed ups)
Date: Mon,  8 Jan 2001 23:41:08 -0500 (EST)

> <<<Skip this paragraph if your not interested in the details of my
> situation.> I have a single source file that takes 5-7 min to compile
> on my Pentium 133 64M RAM. The file uses the following libraries:
> GTK--, MySQL++ and STL. Pre-processed the file is 1.2M and it
> contains 423 classes. That's really big I know, but there ought to be
> some way to at least bring down the compile time a bit.
> >>
> 
> But why use such a slow obsolete machine for such a large project? That
> seems a real mismatch to me. What would be interesting is to see how
> fast your program compiles on a modern inexpensive machine (for under
> $2K, you can get a gigahertz Athlon machine with 256 meg of memory).
> 
-- 
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.4 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iD8DBQA6W1KWZPQa4yHFlTARAvIFAJ4qVi5tQFsx/NSxe3H/oILEqvEmKgCeOyyJ
ZpOzMXXos7B2E26LrFfd4Ow=
=5+/u
-----END PGP SIGNATURE-----

^ permalink raw reply	[flat|nested] 30+ messages in thread

* RE: Precompiled headers (or other speed ups)
  2001-01-08 20:41 dewar
@ 2001-01-09  3:01 ` Rob Taylor
  2001-01-09 10:04 ` amep
  2001-01-09 14:36 ` Stan Shebs
  2 siblings, 0 replies; 30+ messages in thread
From: Rob Taylor @ 2001-01-09  3:01 UTC (permalink / raw)
  To: dewar, Gcc@Gcc. Gnu. Org

> Of course it is always worth smoking out bottlenecks, but I find the
> compilation performance you report not unreasonable. Why on EARTH do
> you have a single source file this long -- sounds like simply awful
> programming style???
>

preprocessed source this size is a pretty common occurence with c++ code of any
size, in my experience. just include a couple of libraries and you're there. I
know our code takes donkeys to compile with gcc -- on MSVC a complete recompile
of our tree takes 15-20 mins. On gcc it takes at least 4 hours! (these times
both in 128Mb 500 Mhz PIII's) A major problem I see is that compilation time for
gcc is non-linear to the size of the source. precompiled header would probably
help these problems sizably.

Rob

^ permalink raw reply	[flat|nested] 30+ messages in thread

* Re: Precompiled headers (or other speed ups)
  2001-01-08 21:00   ` Geoff Keating
@ 2001-01-09  0:41     ` Adrien Hernot
  0 siblings, 0 replies; 30+ messages in thread
From: Adrien Hernot @ 2001-01-09  0:41 UTC (permalink / raw)
  To: Geoff Keating; +Cc: gcc

On Mon, Jan 08, 2001 at 08:59:58PM -0800, Geoff Keating wrote:
> 
> Yes, I'm actively working on precompiled headers.  They've been
> delayed a bit to allow for the GCC 3.0 release to get out the door,
> but they should be in 3.1.

I am very interested in looking at what you did so far.
Is there any pointer to documentation, code ?

Adrien

^ permalink raw reply	[flat|nested] 30+ messages in thread

* Re: Precompiled headers (or other speed ups)
  2001-01-08 21:09 dewar
@ 2001-01-09  0:33 ` Geoff Keating
  0 siblings, 0 replies; 30+ messages in thread
From: Geoff Keating @ 2001-01-09  0:33 UTC (permalink / raw)
  To: dewar; +Cc: amep, gcc

> From: dewar@gnat.com
> Cc: gcc@gcc.gnu.org
> Date: Tue,  9 Jan 2001 00:09:25 -0500 (EST)
> 
> <<Yes, I'm actively working on precompiled headers.  They've been
> delayed a bit to allow for the GCC 3.0 release to get out the door,
> but they should be in 3.1.
> >>
> 
> It's always interesting to know whether this will be a significant 
> speed up, or not. Do you have data yet?

I have measured a 2x speedup for one C code testcase.  No C++ data,
but I expect that'll be better (since more processing is done on C++
header files).

-- 
- Geoffrey Keating <geoffk@geoffk.org>

^ permalink raw reply	[flat|nested] 30+ messages in thread

* Re: Precompiled headers (or other speed ups)
@ 2001-01-08 21:09 dewar
  2001-01-09  0:33 ` Geoff Keating
  0 siblings, 1 reply; 30+ messages in thread
From: dewar @ 2001-01-08 21:09 UTC (permalink / raw)
  To: amep, geoffk; +Cc: gcc

<<Yes, I'm actively working on precompiled headers.  They've been
delayed a bit to allow for the GCC 3.0 release to get out the door,
but they should be in 3.1.
>>

It's always interesting to know whether this will be a significant 
speed up, or not. Do you have data yet?

^ permalink raw reply	[flat|nested] 30+ messages in thread

* Re: Precompiled headers (or other speed ups)
  2001-01-08 19:45 ` amep
@ 2001-01-08 21:00   ` Geoff Keating
  2001-01-09  0:41     ` Adrien Hernot
  0 siblings, 1 reply; 30+ messages in thread
From: Geoff Keating @ 2001-01-08 21:00 UTC (permalink / raw)
  To: amep; +Cc: gcc

amep@softhome.net writes:

> I was wondering if there is anyone actively working on precompiled
> headers (my program is a good candidate for this) or something else to
> speed up the compilation? If so, I'd love help out by testing or doing
> anything else I can?

Yes, I'm actively working on precompiled headers.  They've been
delayed a bit to allow for the GCC 3.0 release to get out the door,
but they should be in 3.1.

-- 
- Geoffrey Keating <geoffk@geoffk.org>

^ permalink raw reply	[flat|nested] 30+ messages in thread

* Re: Precompiled headers (or other speed ups)
@ 2001-01-08 20:41 dewar
  2001-01-09  3:01 ` Rob Taylor
                   ` (2 more replies)
  0 siblings, 3 replies; 30+ messages in thread
From: dewar @ 2001-01-08 20:41 UTC (permalink / raw)
  To: amep, gcc

<<<Skip this paragraph if your not interested in the details of my
situation.> I have a single source file that takes 5-7 min to compile
on my Pentium 133 64M RAM. The file uses the following libraries:
GTK--, MySQL++ and STL. Pre-processed the file is 1.2M and it
contains 423 classes. That's really big I know, but there ought to be
some way to at least bring down the compile time a bit.
>>

But why use such a slow obsolete machine for such a large project? That
seems a real mismatch to me. What would be interesting is to see how
fast your program compiles on a modern inexpensive machine (for under
$2K, you can get a gigahertz Athlon machine with 256 meg of memory).

Now, of course it is nice if things compile fast on your relatively slow
machine, and from a conceptual point of view, there is no reason why this
should not be the case (Realia COBOL would compile probably close to 
half a million lines a minute on your machine, and COBOL is quite a
complex language!). 

However, super fast compilation speed has never been a gcc goal. Early on
this was problematic, but our experience, at least in the GNAT world, is
that machines are getting fast enough that compilation time is not a
bottleneck for most folks, even though gcc may have got slower in absolute
terms.

Of course it is always worth smoking out bottlenecks, but I find the
compilation performance you report not unreasonable. Why on EARTH do
you have a single source file this long -- sounds like simply awful
programming style???

^ permalink raw reply	[flat|nested] 30+ messages in thread

* Precompiled headers (or other speed ups)
       [not found] <978997896.4164.ezmlm@gcc.gnu.org>
@ 2001-01-08 19:45 ` amep
  2001-01-08 21:00   ` Geoff Keating
  0 siblings, 1 reply; 30+ messages in thread
From: amep @ 2001-01-08 19:45 UTC (permalink / raw)
  To: gcc; +Cc: amep

Hi all,

I'm a fairly long time GCC (C/C++) user. Compile times have been fine
for most everything I've done. But, recently I've started doing some
heavy duty C++ stuff (STL, GTK--, etc.) and the compile time has shot
through the roof.

<Skip this paragraph if your not interested in the details of my
situation.> I have a single source file that takes 5-7 min to compile
on my Pentium 133 64M RAM. The file uses the following libraries:
GTK--, MySQL++ and STL. Pre-processed the file is 1.2M and it
contains 423 classes. That's really big I know, but there ought to be
some way to at least bring down the compile time a bit.

I was wondering if there is anyone actively working on precompiled
headers (my program is a good candidate for this) or something else to
speed up the compilation? If so, I'd love help out by testing or doing
anything else I can?

I don't know anything about writing compilers or the innards of GCC,
but I can do stack traces and other debugging stuff. Thanx.

Your impatient programmer, ;-)
-Arthur Peters
-- 
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.4 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iD8DBQA6WokeZPQa4yHFlTARAuRSAJ45TZpDASMk9MRWEbYk3HBa0GcfUwCeNspE
U2aGeq6naa6HN03hrNnKAF0=
=MTmt
-----END PGP SIGNATURE-----

^ permalink raw reply	[flat|nested] 30+ messages in thread

end of thread, other threads:[~2001-01-10 22:55 UTC | newest]

Thread overview: 30+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2001-01-10  9:56 Precompiled headers (or other speed ups) David Korn
2001-01-10 11:14 ` Zack Weinberg
2001-01-10 13:54   ` Neil Booth
2001-01-10 15:45     ` Zack Weinberg
2001-01-10 15:26   ` Joe Buck
  -- strict thread matches above, loose matches on Subject: below --
2001-01-10 11:45 dewar
2001-01-10 10:21 dewar
2001-01-10  9:35 dewar
2001-01-10 18:53 ` Yu Xuanwei
2001-01-10  9:29 dewar
2001-01-10  9:11 David Korn
2001-01-10  9:02 Phil Edwards
2001-01-10  8:50 dewar
2001-01-09 19:52 dewar
2001-01-09 14:55 dewar
2001-01-09 15:43 ` Stan Shebs
2001-01-10 22:55   ` Toon Moene
2001-01-10  5:29 ` Andi Kleen
2001-01-10  8:32   ` Phil Edwards
2001-01-10 15:12   ` Stan Shebs
2001-01-09 11:26 dewar
2001-01-08 21:09 dewar
2001-01-09  0:33 ` Geoff Keating
2001-01-08 20:41 dewar
2001-01-09  3:01 ` Rob Taylor
2001-01-09 10:04 ` amep
2001-01-09 14:36 ` Stan Shebs
     [not found] <978997896.4164.ezmlm@gcc.gnu.org>
2001-01-08 19:45 ` amep
2001-01-08 21:00   ` Geoff Keating
2001-01-09  0:41     ` Adrien Hernot

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).