GCC 4.1: Buildable on GHz machines only?

GCC 4.1: Buildable on GHz machines only?

[Matt Thomas]

Over the past month I've been making sure that GCC 4.1 works on NetBSD.
I've completed bootstraps on sparc, sparc64, arm, x86_64, i386, alpha,
mipsel, mipseb, and powerpc.  I've done cross-build targets for vax.
Results have been sent to gcc-testsuite.

The times to complete bootstraps on older machines has been bothering me.
It took nearly 72 hours for 233MHz StrongArm with 64MB to complete a
bootstrap (with libjava).  It took over 48 hours for a 120MHz MIPS R4400
(little endian) with 128MB to finish (without libjava) and a bit over 24
hours for a 250MHz MIPS R4400 (big endian) with 256MB to finish (again,
no libjava).  That doesn't even include the time to run the testsuites.

I have a 50MHz 68060 with 96MB of memory (MVME177) approaching 100 hours
(48 hours just to exit stage3 and start on the libraries) doing a bootstrap
knowing that it's going to die when doing the ranlib of libjava.  The kernel
for the 060 isn't configured with a large enough dataspace to complete the
ranlib.

Most of the machines I've listed above are relatively powerful machines
near the apex of performance of their target architecture.  And yet GCC4.1
can barely be bootstrapped on them.

I do most of my GCC work on a 2GHz x86_64 because it's so fast.  I'm afraid
the widespread availability of such fast machines hides the fast that the
current performance of GCC on older architectures is appalling.

I'm going to run some bootstraps with --disable-checking just to see how
much faster they are.  I hope I'm going to pleasantly surprised but I'm
not counting on it.
-- 
Matt Thomas                     email: matt@3am-software.com
3am Software Foundry              www: http://3am-software.com/bio/matt/
Cupertino, CA              disclaimer: I avow all knowledge of this message.

Re: GCC 4.1: Buildable on GHz machines only?

[Daniel Jacobowitz]

On Tue, Apr 26, 2005 at 07:50:40PM -0700, Matt Thomas wrote:
> Over the past month I've been making sure that GCC 4.1 works on NetBSD.
> I've completed bootstraps on sparc, sparc64, arm, x86_64, i386, alpha,
> mipsel, mipseb, and powerpc.  I've done cross-build targets for vax.
> Results have been sent to gcc-testsuite.
> 
> The times to complete bootstraps on older machines has been bothering me.
> It took nearly 72 hours for 233MHz StrongArm with 64MB to complete a
> bootstrap (with libjava).  It took over 48 hours for a 120MHz MIPS R4400
> (little endian) with 128MB to finish (without libjava) and a bit over 24
> hours for a 250MHz MIPS R4400 (big endian) with 256MB to finish (again,
> no libjava).  That doesn't even include the time to run the testsuites.
> 
> I have a 50MHz 68060 with 96MB of memory (MVME177) approaching 100 hours
> (48 hours just to exit stage3 and start on the libraries) doing a bootstrap
> knowing that it's going to die when doing the ranlib of libjava.  The kernel
> for the 060 isn't configured with a large enough dataspace to complete the
> ranlib.
> 
> Most of the machines I've listed above are relatively powerful machines
> near the apex of performance of their target architecture.  And yet GCC4.1
> can barely be bootstrapped on them.

Note that the MIPSen are not near the top of modern MIPS performance. 
The ARM isn't quite there either, but the higher-powered ARMs are a bit
scarcer than the MIPS.

None of this detracts from your point, though.

> I'm going to run some bootstraps with --disable-checking just to see how
> much faster they are.  I hope I'm going to pleasantly surprised but I'm
> not counting on it.

I would expect it to be drastically faster.  However this won't show up
clearly in the bootstrap.  The, bar none, longest bit of the bootstrap
is building stage2; and stage1 is always built with optimization off and
(IIRC) checking on.

-- 
Daniel Jacobowitz
CodeSourcery, LLC

Re: GCC 4.1: Buildable on GHz machines only?

[Richard Henderson]

On Tue, Apr 26, 2005 at 10:57:07PM -0400, Daniel Jacobowitz wrote:
> I would expect it to be drastically faster.  However this won't show up
> clearly in the bootstrap.  The, bar none, longest bit of the bootstrap
> is building stage2; and stage1 is always built with optimization off and
> (IIRC) checking on.

Which is why I essentially always supply STAGE1_CFLAGS='-O -g' when
building on risc machines.


r~

Re: GCC 4.1: Buildable on GHz machines only?

[Matt Thomas]

Richard Henderson wrote:
> On Tue, Apr 26, 2005 at 10:57:07PM -0400, Daniel Jacobowitz wrote:
> 
>>I would expect it to be drastically faster.  However this won't show up
>>clearly in the bootstrap.  The, bar none, longest bit of the bootstrap
>>is building stage2; and stage1 is always built with optimization off and
>>(IIRC) checking on.
> 
> 
> Which is why I essentially always supply STAGE1_CFLAGS='-O -g' when
> building on risc machines.

Alas, the --disable-checking and STAGE1_CFLAGS="-O2 -g" (which I was
already doing) only decreased the bootstrap time by 10%.  By far, the
longest bit of the bootstrap is building libjava.

-- 
Matt Thomas                     email: matt@3am-software.com
3am Software Foundry              www: http://3am-software.com/bio/matt/
Cupertino, CA              disclaimer: I avow all knowledge of this message.

RE: GCC 4.1: Buildable on GHz machines only?

[Gary Funck]

> -----Original Message-----
> From: Matt Thomas
> Sent: Tuesday, April 26, 2005 10:42 PM
[...]
> 
> Alas, the --disable-checking and STAGE1_CFLAGS="-O2 -g" (which I was
> already doing) only decreased the bootstrap time by 10%.  By far, the
> longest bit of the bootstrap is building libjava.
> 

Is it fair to compare current build times, with libjava included,
against past build times when it didn't exist?  Would a closer
apples-to-apples comparison be to bootstrap GCC Core only on
the older sub Ghz platforms?

Re: GCC 4.1: Buildable on GHz machines only?

[Ian Lance Taylor]

Matt Thomas <matt@3am-software.com> writes:

> I have a 50MHz 68060 with 96MB of memory (MVME177) approaching 100 hours
> (48 hours just to exit stage3 and start on the libraries) doing a bootstrap
> knowing that it's going to die when doing the ranlib of libjava.  The kernel
> for the 060 isn't configured with a large enough dataspace to complete the
> ranlib.

If so, that is particularly irritating since the ranlib is completely
unnecessary.  GNU ar always builds an archive symbol table by default.

Ian

Re: GCC 4.1: Buildable on GHz machines only?

[Matt Thomas]

Gary Funck wrote:
> 
>>-----Original Message-----
>>From: Matt Thomas
>>Sent: Tuesday, April 26, 2005 10:42 PM
> 
> [...]
> 
>>Alas, the --disable-checking and STAGE1_CFLAGS="-O2 -g" (which I was
>>already doing) only decreased the bootstrap time by 10%.  By far, the
>>longest bit of the bootstrap is building libjava.
>>
> 
> 
> Is it fair to compare current build times, with libjava included,
> against past build times when it didn't exist?  Would a closer
> apples-to-apples comparison be to bootstrap GCC Core only on
> the older sub Ghz platforms?

libjava is built on everything but vax and mips.  Bootstrapping core
might be better but do the configure on the fly it's not as easy as
it used to be.

It would be nice if bootstrap emitted timestamps when it was started
and when it completed a stage so one could just look at the make output.

Regardless, GCC4.1 is a computational pig.
-- 
Matt Thomas                     email: matt@3am-software.com
3am Software Foundry              www: http://3am-software.com/bio/matt/
Cupertino, CA              disclaimer: I avow all knowledge of this message.

Re: GCC 4.1: Buildable on GHz machines only?

[Zack Weinberg]

Matt Thomas <matt@3am-software.com> writes:

> libjava is built on everything but vax and mips.  Bootstrapping core
> might be better but do the configure on the fly it's not as easy as
> it used to be.

--enable-languages=c,c++ (or even perhaps --enable-languages=c)
doesn't work for you?

Also, I believe "make all-gcc TARGET-gcc=bootstrap" will bootstrap the
compiler normally but not build the runtime libraries.

> It would be nice if bootstrap emitted timestamps when it was started
> and when it completed a stage so one could just look at the make output.

Patches are welcome.

zw

Re: GCC 4.1: Buildable on GHz machines only?

[Dan Nicolaescu]

Matt Thomas <matt@3am-software.com> writes:

  > Richard Henderson wrote:
  > > On Tue, Apr 26, 2005 at 10:57:07PM -0400, Daniel Jacobowitz wrote:
  > > 
  > >>I would expect it to be drastically faster.  However this won't show up
  > >>clearly in the bootstrap.  The, bar none, longest bit of the bootstrap
  > >>is building stage2; and stage1 is always built with optimization off and
  > >>(IIRC) checking on.
  > > 
  > > 
  > > Which is why I essentially always supply STAGE1_CFLAGS='-O -g' when
  > > building on risc machines.
  > 
  > Alas, the --disable-checking and STAGE1_CFLAGS="-O2 -g" (which I was

I don't think that is enough,  also edit gcc/Makefile.in and change the line:
STAGE1_CHECKING = -DENABLE_CHECKING -DENABLE_ASSERT_CHECKING
to be
STAGE1_CHECKING =

Is there a better way to do this? STAGE1_CHECKING is not passed from
the toplevel make, so one cannot put it on the make bootstrap command
line...

Re: GCC 4.1: Buildable on GHz machines only?

[David Edelsohn]

>>>>> Matt Thomas writes:

Matt> Regardless, GCC4.1 is a computational pig.

	If you are referring to the compiler itself, this has no basis in
reality.  If you are referring to the entire compiler collection,
including runtimes, you are not using a fair comparison or are making
extreme statements without considering the cause.

	GCC now supports C++, Fortran 90 and Java.  Those languages have
extensive, complicated runtimes.  The GCC Java environment is becoming
much more complete and standards compliant, which means adding more and
more features.

	If your point is that fully supporting modern, richly featured
languages results in a longer build process, that is correct.  Using
disparaging terms like "pig" is missing the point.  As others have pointed
out, if you do not want to build some languages and runtimes, you can
disable them.  GCC is providing features that users want and that has a
cost.

David

Re: GCC 4.1: Buildable on GHz machines only?

[Matt Thomas]

David Edelsohn wrote:
>>>>>>Matt Thomas writes:
> 
> 
> Matt> Regardless, GCC4.1 is a computational pig.
> 
> 	If you are referring to the compiler itself, this has no basis in
> reality.  If you are referring to the entire compiler collection,
> including runtimes, you are not using a fair comparison or are making
> extreme statements without considering the cause.

When I see the native stage2 m68k compiler spend 30+ minutes compute bound
with no paging activity compiling a single source file, I believe
that is an accurate term.  Compiling stage3 on a 50MHz 68060 took 18 hours.
(That 30 minutes was for fold-const.c if you care to know).

At some points, I had no idea whether GCC had just gone into an infinite
loop due a bug or was actually doing what it was supposed to.

> 	GCC now supports C++, Fortran 90 and Java.  Those languages have
> extensive, complicated runtimes.  The GCC Java environment is becoming
> much more complete and standards compliant, which means adding more and
> more features.

That's all positive but if GCC also becomes too expensive to build then
all those extra features become worthless.  What is the slowest system
that GCC has been recently bootstrapped on?

> 	If your point is that fully supporting modern, richly featured
> languages results in a longer build process, that is correct.  Using
> disparaging terms like "pig" is missing the point.  As others have pointed
> out, if you do not want to build some languages and runtimes, you can
> disable them.  GCC is providing features that users want and that has a
> cost.

Yes they have a cost, but the cost is mitigated by running fast processors.
They are just so fast they can hide ineffiences and bloat.  We have seen
that for NetBSD and it's just as true for GCC or any other software.
These slower processor perform usefull feedback but only if a GCC bootstrap
is attempted on them on a semi-regular basis.

Am I the only person who has attempted to do a native bootstrap on a system
as slow as a M68k?  I thought about doing a bootstrap on a MicroSparc based
system but instead I decided to use a UltraSparcIIi system running with a
32bit kernel.
-- 
Matt Thomas                     email: matt@3am-software.com
3am Software Foundry              www: http://3am-software.com/bio/matt/
Cupertino, CA              disclaimer: I avow all knowledge of this message.

Re: GCC 4.1: Buildable on GHz machines only?

[Jonathan Wakely]

On Wed, Apr 27, 2005 at 08:05:39AM -0700, Matt Thomas wrote:

> David Edelsohn wrote:
> 
> > 	GCC now supports C++, Fortran 90 and Java.  Those languages have
> > extensive, complicated runtimes.  The GCC Java environment is becoming
> > much more complete and standards compliant, which means adding more and
> > more features.
> 
> That's all positive but if GCC also becomes too expensive to build then
> all those extra features become worthless.

Worthless to whom?

The features under discussion are new, they didn't exist before.

If you survived without them previously you can do so now.
(i.e. don't build libjava if your machine isn't capable of it)

But claiming it's "worthless" when plenty of people are using it is
just, well ... worthless.

jon

-- 
"Speed, it seems to me, provides the one genuinely modern pleasure."
	- Aldous Huxley

Re: GCC 4.1: Buildable on GHz machines only?

[David Edelsohn]

>>>>> Matt Thomas writes:

Matt> That's all positive but if GCC also becomes too expensive to build then
Matt> all those extra features become worthless.  What is the slowest system
Matt> that GCC has been recently bootstrapped on?

	GCC recently was bootstrapped on a VAX.

	The GCC build times are not unreasonable compared to other,
commercial compilers with similar functionality.  And the GCC developers
ave plans to address inefficiencies -- GCC 4.0 often is faster than GCC
3.4.

David

Re: GCC 4.1: Buildable on GHz machines only?

[Matt Thomas]

Jonathan Wakely wrote:
> On Wed, Apr 27, 2005 at 08:05:39AM -0700, Matt Thomas wrote:
> 
> 
>>David Edelsohn wrote:
>>
>>
>>>	GCC now supports C++, Fortran 90 and Java.  Those languages have
>>>extensive, complicated runtimes.  The GCC Java environment is becoming
>>>much more complete and standards compliant, which means adding more and
>>>more features.
>>
>>That's all positive but if GCC also becomes too expensive to build then
>>all those extra features become worthless.
> 
> 
> Worthless to whom?

To users of that platform that can no longer afford to build GCC.

> The features under discussion are new, they didn't exist before.

And because they never existed before, their cost for older platforms
may not have been correctly assessed.  If no one builds natively on
older platforms, the recognition that the new features maybe a problem
for older platforms will never be made.

> If you survived without them previously you can do so now.
> (i.e. don't build libjava if your machine isn't capable of it)

Yes, you can skip building libjava.  But can you skip building GCC?
Will GCC 3.x be supported forever?  If not, your compiler may have
to rely being cross-built.  Being able to do a bootstrap is useful
and is part of the expected GCC testing but when it can only be
done one or two a week, it becomes a less practical test method.

> But claiming it's "worthless" when plenty of people are using it is
> just, well ... worthless.

Depends on your point of view.
-- 
Matt Thomas                     email: matt@3am-software.com
3am Software Foundry              www: http://3am-software.com/bio/matt/
Cupertino, CA              disclaimer: I avow all knowledge of this message.

Re: GCC 4.1: Buildable on GHz machines only?

[Richard Earnshaw]

On Wed, 2005-04-27 at 16:31, David Edelsohn wrote:

> 	The GCC build times are not unreasonable compared to other,
> commercial compilers with similar functionality.  And the GCC developers
> ave plans to address inefficiencies -- GCC 4.0 often is faster than GCC
> 3.4.

If you are going to make sweeping statements like this you need to back
them up with hard data.

For C code, on average (obviously there are cases which are exceptions)
GCC 4.0 is about 7% slower on ARM than gcc 3.4.

http://www.inf.u-szeged.hu/csibe/ocomp.php?branchid_a=gcc_3_4_0_release&branchid_b=gcc_4_0_0_release&targetid_a=arm-elf&targetid_b=arm-elf&timestamp_a=2003-01-01%2012:00:00&timestamp_b=2003-01-01%2012:00:00&flags_a=-O2&flags_b=-O2&csibever_a=2.x.x&csibever_b=2.x.x&dataview=Compilation%20time&viewmode=Summarized%20bar%20chart&finish_button=Finish

And it is >10% slower than 3.3.

And ...

R.

RE: GCC 4.1: Buildable on GHz machines only?

[Dave Korn]

----Original Message----
>From: David Edelsohn
>Sent: 27 April 2005 16:32

>>>>>> Matt Thomas writes:
> 
> Matt> That's all positive but if GCC also becomes too expensive to build
> then Matt> all those extra features become worthless.  What is the
> slowest system Matt> that GCC has been recently bootstrapped on?
> 
> 	GCC recently was bootstrapped on a VAX.
> 
> 	The GCC build times are not unreasonable compared to other,
> commercial compilers with similar functionality.  And the GCC developers
> ave plans to address inefficiencies -- GCC 4.0 often is faster than GCC
> 3.4.
> 
> David


  I'm a bit concerned about the increasing bootstrap times as well.

  I wanted to test a small patch at the weekend, so I checked out a couple
of copies of HEAD, modified one slightly, then set two full "make bootstrap;
make check" runs going.  This is on a 850Mhz Athlon.  It took over 24 hours.
In 2.95.x days it would have taken maybe three or three-and-a-bit hours on
the very same machine.

  This makes it very much slower and more difficult to submit properly
tested patches.

  I don't have any good ideas how to deal with this really, without cutting
down on the overall amount of testing (either explicitly, by not running the
full testsuite, or implicitly, by disabling most of the languages at
configure time).  I would like to be thorough, but find it quite a handicap.
Suggestions are welcome!

    cheers,
      DaveK
-- 
Can't think of a witty .sigline today....

Re: GCC 4.1: Buildable on GHz machines only?

[David Edelsohn]

>>>>> Richard Earnshaw writes:

Richard> On Wed, 2005-04-27 at 16:31, David Edelsohn wrote:
>> The GCC build times are not unreasonable compared to other,
>> commercial compilers with similar functionality.  And the GCC developers
>> ave plans to address inefficiencies -- GCC 4.0 often is faster than GCC
>> 3.4.

Richard> If you are going to make sweeping statements like this you need to back
Richard> them up with hard data.

	What does gcc_4_0_0_release branch at time 2003-01-01 mean?

David

Re: GCC 4.1: Buildable on GHz machines only?

[Richard Earnshaw]

On Wed, 2005-04-27 at 17:19, David Edelsohn wrote:
> >>>>> Richard Earnshaw writes:
> 
> Richard> On Wed, 2005-04-27 at 16:31, David Edelsohn wrote:
> >> The GCC build times are not unreasonable compared to other,
> >> commercial compilers with similar functionality.  And the GCC developers
> >> ave plans to address inefficiencies -- GCC 4.0 often is faster than GCC
> >> 3.4.
> 
> Richard> If you are going to make sweeping statements like this you need to back
> Richard> them up with hard data.
> 
> 	What does gcc_4_0_0_release branch at time 2003-01-01 mean?

Hmm, you'll have to ask the CSiBE folks that one...

R.

Re: GCC 4.1: Buildable on GHz machines only?

[Richard Earnshaw]

On Wed, 2005-04-27 at 17:29, Richard Earnshaw wrote:
> On Wed, 2005-04-27 at 17:19, David Edelsohn wrote:
> > >>>>> Richard Earnshaw writes:
> > 
> > Richard> On Wed, 2005-04-27 at 16:31, David Edelsohn wrote:
> > >> The GCC build times are not unreasonable compared to other,
> > >> commercial compilers with similar functionality.  And the GCC developers
> > >> ave plans to address inefficiencies -- GCC 4.0 often is faster than GCC
> > >> 3.4.
> > 
> > Richard> If you are going to make sweeping statements like this you need to back
> > Richard> them up with hard data.
> > 
> > 	What does gcc_4_0_0_release branch at time 2003-01-01 mean?
> 
> Hmm, you'll have to ask the CSiBE folks that one...

In fact the time is irrelevant in this case, since it's not a branch but
a specific tag on a single version.

R.

Re: GCC 4.1: Buildable on GHz machines only?

[Daniel Jacobowitz]

On Wed, Apr 27, 2005 at 08:45:01AM -0700, Matt Thomas wrote:
> > If you survived without them previously you can do so now.
> > (i.e. don't build libjava if your machine isn't capable of it)
> 
> Yes, you can skip building libjava.  But can you skip building GCC?
> Will GCC 3.x be supported forever?  If not, your compiler may have
> to rely being cross-built.  Being able to do a bootstrap is useful
> and is part of the expected GCC testing but when it can only be
> done one or two a week, it becomes a less practical test method.

I have no idea what you're trying to say in this paragraph.  Not only
can you skip building libjava, you can skip building the compilers for
any languages that you do not want to test.

-- 
Daniel Jacobowitz
CodeSourcery, LLC

Re: GCC 4.1: Buildable on GHz machines only?

[Mark Mitchell]

Matt Thomas wrote:

> Alas, the --disable-checking and STAGE1_CFLAGS="-O2 -g" (which I was
> already doing) only decreased the bootstrap time by 10%.  By far, the
> longest bit of the bootstrap is building libjava.

Building libjava takes forever on any platform, relative to the rest of 
the compiler build.  That's why I often disable it, especially since I 
don't use Java very much, and gcj even less often than Java in general. 
  Do you really *need* gcj on these embedded platforms?

Also, I don't really think that bootstrapping GCC directly on target 
hardware is very useful -- except for testing GCC.  I'd build GCC on 
some fast workstation, even if I eventually wanted it to run native on 
the target.

-- 
Mark Mitchell
CodeSourcery, LLC
mark@codesourcery.com
(916) 791-8304

Re: GCC 4.1: Buildable on GHz machines only?

[Tom Tromey]

>>>>> "Ian" == Ian Lance Taylor <ian@airs.com> writes:

>> Matt Thomas <matt@3am-software.com> writes:
>> I have a 50MHz 68060 with 96MB of memory (MVME177) approaching 100 hours
>> (48 hours just to exit stage3 and start on the libraries) doing a bootstrap
>> knowing that it's going to die when doing the ranlib of libjava.  The kernel
>> for the 060 isn't configured with a large enough dataspace to complete the
>> ranlib.

Ian> If so, that is particularly irritating since the ranlib is completely
Ian> unnecessary.  GNU ar always builds an archive symbol table by default.

Matt, could you file a PR for this?  I agree with Ian, we really
shouldn't be running ranlib here.  Perhaps we can get the libtool
maintainers to fix this.

Tom

Re: GCC 4.1: Buildable on GHz machines only?

[Steven Bosscher]

On Wednesday 27 April 2005 17:45, Matt Thomas wrote:
> > The features under discussion are new, they didn't exist before.
>
> And because they never existed before, their cost for older platforms
> may not have been correctly assessed.

If someone had cared about them, it would have been noticed earlier.
But since _nobody_ has complained before you, I guess we can conclude
that by far the majority if GCC users are quite happy with the cost
assesments that were made.

> If no one builds natively on 
> older platforms, the recognition that the new features maybe a problem
> for older platforms will never be made.

Maybe the older platform should stick to the older compiler then,
if it is too slow to support the kind of compiler that modern
systems need.

Gr.
Steven

Re: GCC 4.1: Buildable on GHz machines only?

[Paul Koning]

>>>>> "Steven" == Steven Bosscher <s.bosscher@student.tudelft.nl> writes:

 Steven> On Wednesday 27 April 2005 17:45, Matt Thomas wrote:

 >> If no one builds natively on older platforms, the recognition that
 >> the new features maybe a problem for older platforms will never be
 >> made.

 Steven> Maybe the older platform should stick to the older compiler
 Steven> then, if it is too slow to support the kind of compiler that
 Steven> modern systems need.

Maybe I'm missing something, but...

Isn't a full bootstrap (all languages) part of the required test
procedure for changes?  That's what the website says right now. Since
Matt is the Vax port maintainer, he therefore has good reasons for
needing to run bootstraps on slow machines.

Your comment seems to translate to: "when GCC grows to the point that
you can't reasonably run a full bootstrap on platform X anymore, then
platform X is obsolete".  That seems like a strange new obsoletion
criterion.

The alternative of course is to do only crossbuilds.  Is it reasonable
to say that, for platforms where a bootstrap is no longer feasible, a
successful crossbuild is an acceptable test procedure to use instead?

	   paul

Re: GCC 4.1: Buildable on GHz machines only?

[sjhill]

> The alternative of course is to do only crossbuilds.  Is it reasonable
> to say that, for platforms where a bootstrap is no longer feasible, a
> successful crossbuild is an acceptable test procedure to use instead?
> 
Sure, and get flamed and trounced by Uli on glibc when you talk
about problems with crossbuilding.

-Steve

Re: GCC 4.1: Buildable on GHz machines only?

[Steven Bosscher]

On Wednesday 27 April 2005 22:06, Paul Koning wrote:
> >>>>> "Steven" == Steven Bosscher <s.bosscher@student.tudelft.nl> writes:
>
>  Steven> On Wednesday 27 April 2005 17:45, Matt Thomas wrote:
>  >> If no one builds natively on older platforms, the recognition that
>  >> the new features maybe a problem for older platforms will never be
>  >> made.
>
>  Steven> Maybe the older platform should stick to the older compiler
>  Steven> then, if it is too slow to support the kind of compiler that
>  Steven> modern systems need.
>
> Maybe I'm missing something, but...
>
> Isn't a full bootstrap (all languages) part of the required test
> procedure for changes?  That's what the website says right now.

Isn't there a special text about port changes?  Some ports don't even
support all languages.

> Since 
> Matt is the Vax port maintainer, he therefore has good reasons for
> needing to run bootstraps on slow machines.

Yes, he has.  Is that a valid reason to call GCC4 a pig? No.

> Your comment seems to translate to: "when GCC grows to the point that
> you can't reasonably run a full bootstrap on platform X anymore, then
> platform X is obsolete".  That seems like a strange new obsoletion
> criterion.

Interesting way of arguing.  First you twist my words and put something
in my mouth that I did not say, then you comment on that...

What I'm saying is that if you really want/need for some reason to do
full bootstraps of the latest and greatest GCC on something as old and
slow as m68k (the old kind), VAX, or PDP-11, you should not complain
that other people have moved on to recent targets where a bootstrap is
not such a big deal and the new features in GCC make the difference
between a good or poor compiler for that target.

I don't think there's any disagreement that GCC is not as fast as it
should be.  But bootstrapping <insert SLOC count here>[*] lines of code
on a real VAX is just never going to be very fast.  There is no reason
to blame GCC for that.

Gr.
Steven


[*] Does anyone have an idea of how large GCC really is?

Re: GCC 4.1: Buildable on GHz machines only?

[Diego Novillo]

On Wed, Apr 27, 2005 at 10:36:08PM +0200, Steven Bosscher wrote:

> [*] Does anyone have an idea of how large GCC really is?
>
~1.8 MLOC.

Courtesy of David Wheeler's SLOCCount (testsuites excluded):

SLOC    Directory       SLOC-by-Language (Sorted)
1179994 gcc             ansic=745370,ada=395409,asm=21934,yacc=13865,sh=1841,
                        lex=857,awk=439,perl=193,pascal=86
398689  libjava         java=282613,cpp=69449,ansic=39675,sh=5417,perl=858,
                        yacc=653,awk=24
50724   libstdc++-v3    cpp=34477,ansic=15657,sh=421,perl=130,awk=39
31583   libgfortran     ansic=31166,f90=365,sh=52
30400   boehm-gc        ansic=28510,cpp=1146,asm=443,sh=301
23021   libiberty       ansic=22722,perl=299
19533   zlib            ansic=13100,asm=2710,ada=1637,pascal=1085,cpp=1001
13772   libffi          ansic=8135,asm=5637
12324   libcpp          ansic=12324
12102   top_dir         sh=12102
8565    libobjc         ansic=7752,objc=427,cpp=386
6955    intl            ansic=6639,yacc=316
4861    libmudflap      ansic=4861
3958    contrib         cpp=2306,sh=1122,perl=374,awk=67,lisp=59,ansic=30
2758    fastjar         ansic=2758
2697    fixincludes     ansic=2254,sh=443
1734    include         ansic=1706,cpp=28
837     maintainer-scripts sh=828,perl=9


Totals grouped by language (dominant language first):
ansic:       942659 (52.24%)
ada:         397046 (22.00%)
java:        282613 (15.66%)
cpp:         108793 (6.03%)
asm:          30724 (1.70%)
sh:           22527 (1.25%)
yacc:         14834 (0.82%)
perl:          1863 (0.10%)
pascal:        1171 (0.06%)
lex:            857 (0.05%)
awk:            569 (0.03%)
objc:           427 (0.02%)
f90:            365 (0.02%)
lisp:            59 (0.00%)




Total Physical Source Lines of Code (SLOC)                = 1,804,507
Development Effort Estimate, Person-Years (Person-Months) = 525.06 (6,300.68)
 (Basic COCOMO model, Person-Months = 2.4 * (KSLOC**1.05))
Schedule Estimate, Years (Months)                         = 5.79 (69.46)
 (Basic COCOMO model, Months = 2.5 * (person-months**0.38))
Estimated Average Number of Developers (Effort/Schedule)  = 90.72
Total Estimated Cost to Develop                           = $ 70,928,067
 (average salary = $56,286/year, overhead = 2.40).
SLOCCount, Copyright (C) 2001-2004 David A. Wheeler
SLOCCount is Open Source Software/Free Software, licensed under the GNU GPL.
SLOCCount comes with ABSOLUTELY NO WARRANTY, and you are welcome to
redistribute it under certain conditions as specified by the GNU GPL license;
see the documentation for details.
Please credit this data as "generated using David A. Wheeler's 'SLOCCount'."

Re: GCC 4.1: Buildable on GHz machines only?

[Paul Koning]

>>>>> "Steven" == Steven Bosscher <s.bosscher@student.tudelft.nl> writes:

 Steven> On Wednesday 27 April 2005 22:06, Paul Koning wrote:
 >> Isn't a full bootstrap (all languages) part of the required test
 >> procedure for changes?  That's what the website says right now.

 Steven> Isn't there a special text about port changes?  Some ports
 Steven> don't even support all languages.

http://gcc.gnu.org/contribute.html#testing does not list any special
text that affects the testing rules, at least not that I can find.

Yes, I suppose not all ports don't support all languages.  I don't
know if that's true for VAX.  If so, then I would assume the rule
becomes "all supported languages".

 Steven> What I'm saying is that if you really want/need for some
 Steven> reason to do full bootstraps of the latest and greatest GCC
 Steven> on something as old and slow as m68k (the old kind), VAX, or
 Steven> PDP-11, you should not complain that other people have moved
 Steven> on to recent targets where a bootstrap is not such a big deal
 Steven> and the new features in GCC make the difference between a
 Steven> good or poor compiler for that target.

So long as the testing rules are what they are, I can't agree with
that.  If the testing rules are changed to allow less testing when the
platform in question is slow, that's a different matter.

Note that the PDP-11 case is different, that's not a native target.

Also, when platform obsoletion is discussed, lack of test results is
often mentioned as a metric.  It seems unfair to complain about lack
of test results when the software being tested is the cause (or part
of the cause), not necessarily the lack of tester interest.

 Steven> I don't think there's any disagreement that GCC is not as
 Steven> fast as it should be.  But bootstrapping <insert SLOC count
 Steven> here>[*] lines of code on a real VAX is just never going to
 Steven> be very fast.  There is no reason to blame GCC for that.

Maybe.  Then again, maybe there are real problems here.  The ranlib
one was already mentioned.  And I wonder if libjava really needs to
bring the host to its knees, as it does.

I normally work on a 2 GHz Linux 2.6 system.  It does lots of things
quite fast.  I can do large system builds and do other stuff at the
same time while immense makefiles are crunching away.

However, I can always tell when a GCC build has hit the libjava build
-- that's when the *whole system* suddenly slows to a crawl.  Maybe
it comes from doing some processing on 5000 foo.o files all at
once... :-(

	paul

Re: GCC 4.1: Buildable on GHz machines only?

[Karel Gardas]

On Wed, 27 Apr 2005, Steven Bosscher wrote:

> [*] Does anyone have an idea of how large GCC really is?

Using sloccount, 4.0.0 release looks like:

Totals grouped by language (dominant language first):
ansic:      1076327 (43.81%)
ada:         541135 (22.03%)
java:        276544 (11.26%)
cpp:         272101 (11.08%)
sh:          222630 (9.06%)
asm:          31194 (1.27%)
yacc:         14900 (0.61%)
exp:           8127 (0.33%)
objc:          5919 (0.24%)
fortran:       4507 (0.18%)
perl:          1954 (0.08%)
lex:            768 (0.03%)
awk:            542 (0.02%)
lisp:            59 (0.00%)
sed:             20 (0.00%)




Total Physical Source Lines of Code (SLOC)                = 2,456,727
Development Effort Estimate, Person-Years (Person-Months) = 725.95 (8,711.36)
 (Basic COCOMO model, Person-Months = 2.4 * (KSLOC**1.05))
Schedule Estimate, Years (Months)                         = 6.55 (78.55)
 (Basic COCOMO model, Months = 2.5 * (person-months**0.38))
Estimated Average Number of Developers (Effort/Schedule)  = 110.90
Total Estimated Cost to Develop                           = $ 98,065,527
 (average salary = $56,286/year, overhead = 2.40).
Please credit this data as "generated using 'SLOCCount' by David A. Wheeler."


Cheers,
Karel
--
Karel Gardas                  kgardas@objectsecurity.com
ObjectSecurity Ltd.           http://www.objectsecurity.com

Re: GCC 4.1: Buildable on GHz machines only?

[Andrew Pinski]

> However, I can always tell when a GCC build has hit the libjava build
> -- that's when the *whole system* suddenly slows to a crawl.  Maybe
> it comes from doing some processing on 5000 foo.o files all at
> once... :-(

But that is not GCC fault that binutils cannot handle that load.

-- Pinski

Re: GCC 4.1: Buildable on GHz machines only?

[Mike Stump]

On Apr 26, 2005, at 11:12 PM, Matt Thomas wrote:
> It would be nice if bootstrap emitted timestamps when it was started
> and when it completed a stage so one could just look at the make  
> output.

You can get them differenced for free by using:

     time make boostrap

and written to a log file with

     nohup time make boostrap

!

Re: GCC 4.1: Buildable on GHz machines only?

[Matt Thomas]

Mike Stump wrote:
> On Apr 26, 2005, at 11:12 PM, Matt Thomas wrote:
> 
>> It would be nice if bootstrap emitted timestamps when it was started
>> and when it completed a stage so one could just look at the make  output.
> 
> 
> You can get them differenced for free by using:
> 
>     time make boostrap

I know that.  But it's only works overall.  I want the per-stage
times.  Here's a sparc64--netbsd full bootstrap including libjava
(the machine has 640MB and was doing nothing but building gcc):

     25406.01 real     21249.17 user      6283.15 sys
          0  maximum resident set size
          0  average shared memory size
          0  average unshared data size
          0  average unshared stack size
   54689526  page reclaims
       5349  page faults
        110  swaps
        723  block input operations
     377302  block output operations
         52  messages sent
         52  messages received
     285329  signals received
    1037478  voluntary context switches
     253151  involuntary context switches


-- 
Matt Thomas                     email: matt@3am-software.com
3am Software Foundry              www: http://3am-software.com/bio/matt/
Cupertino, CA              disclaimer: I avow all knowledge of this message.

Re: GCC 4.1: Buildable on GHz machines only?

[Paul Koning]

>>>>> "Andrew" == Andrew Pinski <pinskia@physics.uc.edu> writes:

 >> However, I can always tell when a GCC build has hit the libjava
 >> build -- that's when the *whole system* suddenly slows to a crawl.
 >> Maybe it comes from doing some processing on 5000 foo.o files all
 >> at once... :-(

 Andrew> But that is not GCC fault that binutils cannot handle that
 Andrew> load.

Perhaps not.  But perhaps there are workarounds that allow the gcc
build to do its job without using binutils in a way that stresses it
beyond its ability to cope.

Matt's time output suggests that massive pagefaulting is a big issue
-- and it wouldn't surprise me if the libjava build procedure were a
major contributor there.

      paul

Re: GCC 4.1: Buildable on GHz machines only?

[Andrew Pinski]

> 
> >>>>> "Andrew" == Andrew Pinski <pinskia@physics.uc.edu> writes:
> 
>  >> However, I can always tell when a GCC build has hit the libjava
>  >> build -- that's when the *whole system* suddenly slows to a crawl.
>  >> Maybe it comes from doing some processing on 5000 foo.o files all
>  >> at once... :-(
> 
>  Andrew> But that is not GCC fault that binutils cannot handle that
>  Andrew> load.
> 
> Perhaps not.  But perhaps there are workarounds that allow the gcc
> build to do its job without using binutils in a way that stresses it
> beyond its ability to cope.

Well on ppc-darwin, linking libjava takes almost no time with only 256MB
of memory.  What times the time on ppc-darwin with libjava is libtool
generating the list of files to link.  So again this is a binutils problem
as it works just fine with a good linker.

-- Pinski

Re: GCC 4.1: Buildable on GHz machines only?

[Stan Shebs]

Steven Bosscher wrote:

>On Wednesday 27 April 2005 17:45, Matt Thomas wrote:
>
>>>The features under discussion are new, they didn't exist before.
>>>
>>And because they never existed before, their cost for older platforms
>>may not have been correctly assessed.
>>
>
>If someone had cared about them, it would have been noticed earlier.
>But since _nobody_ has complained before you, I guess we can conclude
>that by far the majority if GCC users are quite happy with the cost
>assesments that were made.
>
No, there have been plenty of complaints, but the GCC mailing
lists have, shall we say, a "reputation", and a great many
users will not post to them, either for fear of being ridiculed,
or in the expection that they will not be heard. (Everything is
archived, and they can see what happens to others.) If you want
to see what users are *really* saying, look at the mailing lists
of other projects, and see what is said when GCC comes up for
discussion.

Rightly or wrongly, the reward structure for GCC values new
features on the latest hardware over speedy bootstrapping on
old, so I don't expect things to change anytime soon.

Stan

Re: GCC 4.1: Buildable on GHz machines only?

[Daniel Berlin]

On Wed, 2005-04-27 at 15:13 -0700, Stan Shebs wrote:
> Steven Bosscher wrote:
> 
> >On Wednesday 27 April 2005 17:45, Matt Thomas wrote:
> >
> >>>The features under discussion are new, they didn't exist before.
> >>>
> >>And because they never existed before, their cost for older platforms
> >>may not have been correctly assessed.
> >>
> >
> >If someone had cared about them, it would have been noticed earlier.
> >But since _nobody_ has complained before you, I guess we can conclude
> >that by far the majority if GCC users are quite happy with the cost
> >assesments that were made.
> >
> No, there have been plenty of complaints, but the GCC mailing
> lists have, shall we say, a "reputation", and a great many
> users will not post to them, 

I've never in my life heard this from another mailing list, and i
contribute to a *great* many open source projects.

> either for fear of being ridiculed,
> or in the expection that they will not be heard. (Everything is
> archived, and they can see what happens to others.)

The only person i see ridiculing people frequently happens to be from
@apple.com.

>  If you want
> to see what users are *really* saying, look at the mailing lists
> of other projects, and see what is said when GCC comes up for
> discussion.

I do, and most appreciate the work we do.

> 
> Rightly or wrongly, the reward structure for GCC values new
> features on the latest hardware over speedy bootstrapping on
> old, so I don't expect things to change anytime soon.

<rolleyes>

Re: GCC 4.1: Buildable on GHz machines only?

[Joe Buck]

On Wed, Apr 27, 2005 at 03:13:21PM -0700, Stan Shebs wrote:
> No, there have been plenty of complaints, but the GCC mailing
> lists have, shall we say, a "reputation", and a great many
> users will not post to them, either for fear of being ridiculed,
> or in the expection that they will not be heard. (Everything is
> archived, and they can see what happens to others.)

Oh, come on, Stan.  Compared to what?  The GCC lists are quite civil
compared to many free software developer lists.  I'd say criticism
of proposals is harsher on the Linux kernel list (though most of that
criticism is fair).  And if you *really* want to see a hostile
environment, try debian-devel.

That said, the gcc list is not intended to be a user support list, and
users who come asking basic questions are (properly) asked to go
elsewhere.  We should make sure that we do this in a polite way, though.

> Rightly or wrongly, the reward structure for GCC values new
> features on the latest hardware over speedy bootstrapping on
> old, so I don't expect things to change anytime soon.

I will agree with you on this point, but more than half of the time
to bootstrap consists of the time to build the Java library, and speeding
that up is a losing battle, as Sun keeps adding new stuff that
libgjc/classpath is then expected to clone, and the addition rate
seems to exceed Moore's Law.

Re: GCC 4.1: Buildable on GHz machines only?

[Zack Weinberg]

Daniel Berlin <dberlin@dberlin.org> writes:

> On Wed, 2005-04-27 at 15:13 -0700, Stan Shebs wrote:
>> Steven Bosscher wrote:
>> >If someone had cared about them, it would have been noticed
>> >earlier.  But since _nobody_ has complained before you, I guess we
>> >can conclude that by far the majority if GCC users are quite happy
>> >with the cost assesments that were made.
>> >
>> No, there have been plenty of complaints, but the GCC mailing lists
>> have, shall we say, a "reputation", and a great many users will not
>> post to them,
>
> I've never in my life heard this from another mailing list, and i
> contribute to a *great* many open source projects.

I have seen such complaints.  Not about bootstrap times, no, that only
affects people who compile the compiler; but the more general case of
'gcc takes forever to compile this program' does appear on a regular
basis.

I do also think that the amount of ridicule heaped on people who come
to the gcc lists is, in general, too high.  People should not be
ridiculed for complaining that the compiler is slow, even if they are
insisting on using vintage hardware.  It is slow, even on fast
hardware; it's just easier to see that on slow hardware.  Rather more
importantly, people should not be ridiculed for submitting bug
reports, even if they are wrong.  I suspect the bad public image that
Stan refers to, has more to do with this than anything else.  To be
fair, it can be hard to explain that someone is wrong without
belittling them; but it is important to make the effort, particularly
when the reason they are wrong is esoteric (e.g. any of the legion of
counterintuitive things in the C standard).

Some people are worthy of ridicule; those who are trolling, or those
who are persistently and wilfully clueless.  However, ridicule is
*not* an effective way of making these people shut up and go away,
which is the appropriate goal when dealing with them.  It can be fun
to argue with them, but long argument threads are a drag on the
mailing list, so we should not make a habit of it.  This is not
alt.flame.

> The only person i see ridiculing people frequently happens to be
> from @apple.com.

I can think of four people who regularly ridicule posters.  Only two
of them are Apple employees.

zw

Re: GCC 4.1: Buildable on GHz machines only?

[Zack Weinberg]

Having seen Joe's comment, I should say that I agree with him that a
lot of other projects' mailing lists are worse.  However, that isn't
an excuse in my book.

zw

Re: GCC 4.1: Buildable on GHz machines only?

[Daniel Berlin]

On Wed, 2005-04-27 at 16:40 -0700, Zack Weinberg wrote:
> Daniel Berlin <dberlin@dberlin.org> writes:
> 
> > On Wed, 2005-04-27 at 15:13 -0700, Stan Shebs wrote:
> >> Steven Bosscher wrote:
> >> >If someone had cared about them, it would have been noticed
> >> >earlier.  But since _nobody_ has complained before you, I guess we
> >> >can conclude that by far the majority if GCC users are quite happy
> >> >with the cost assesments that were made.
> >> >
> >> No, there have been plenty of complaints, but the GCC mailing lists
> >> have, shall we say, a "reputation", and a great many users will not
> >> post to them,
> >
> > I've never in my life heard this from another mailing list, and i
> > contribute to a *great* many open source projects.
> 
> I have seen such complaints.  Not about bootstrap times, no, that only
> affects people who compile the compiler; but the more general case of
> 'gcc takes forever to compile this program' does appear on a regular
> basis.

People would complain even if the compiler took 1 second on every file,
regardless of size or optimization level.

If you want a faster compiler, it's hard work.  It means not adding
features because the design isn't a good one, *even if the user would
still find it useful*. People aren't willing to do this.  It means lots
and lots of profiling, and taking care of little inefficiencies.  All I
ever see people suggest is magic bullets.

We also have some deep datastructure problems in terms of IL, but those
aren't going to give us a 5000% speedup or be a magic bullet either.
--Dan

Re: GCC 4.1: Buildable on GHz machines only?

[Zack Weinberg]

Daniel Berlin <dberlin@dberlin.org> writes:
> On Wed, 2005-04-27 at 16:40 -0700, Zack Weinberg wrote:
>> I have seen such complaints.  Not about bootstrap times, no, that only
>> affects people who compile the compiler; but the more general case of
>> 'gcc takes forever to compile this program' does appear on a regular
>> basis.
>
> People would complain even if the compiler took 1 second on every file,
> regardless of size or optimization level.

Well, yes.  1 second/file is still slow!  I want "make" to complete
instantaneously!  Don't you?

> If you want a faster compiler, it's hard work.  It means not adding
> features because the design isn't a good one, *even if the user
> would still find it useful*. People aren't willing to do this.  It
> means lots and lots of profiling, and taking care of little
> inefficiencies.  All I ever see people suggest is magic bullets.
>
> We also have some deep datastructure problems in terms of IL, but
> those aren't going to give us a 5000% speedup or be a magic bullet
> either.

What you say is true.  Does that mean we shouldn't try?

zw

Re: GCC 4.1: Buildable on GHz machines only?

[Daniel Berlin]

On Wed, 2005-04-27 at 17:10 -0700, Zack Weinberg wrote:
> Daniel Berlin <dberlin@dberlin.org> writes:
> > On Wed, 2005-04-27 at 16:40 -0700, Zack Weinberg wrote:
> >> I have seen such complaints.  Not about bootstrap times, no, that only
> >> affects people who compile the compiler; but the more general case of
> >> 'gcc takes forever to compile this program' does appear on a regular
> >> basis.
> >
> > People would complain even if the compiler took 1 second on every file,
> > regardless of size or optimization level.
> 
> Well, yes.  1 second/file is still slow!  I want "make" to complete
> instantaneously!  Don't you?
> 
> > If you want a faster compiler, it's hard work.  It means not adding
> > features because the design isn't a good one, *even if the user
> > would still find it useful*. People aren't willing to do this.  It
> > means lots and lots of profiling, and taking care of little
> > inefficiencies.  All I ever see people suggest is magic bullets.
> >
> > We also have some deep datastructure problems in terms of IL, but
> > those aren't going to give us a 5000% speedup or be a magic bullet
> > either.
> 
> What you say is true.  Does that mean we shouldn't try?

Let me point out the important part again:
All I ever see people suggest is magic bullets.

We should try, but by doing the hard work.
Not by expecting magic.

> 
> zw

Re: GCC 4.1: Buildable on GHz machines only?

[Zack Weinberg]

Daniel Berlin <dberlin@dberlin.org> writes:

>> What you say is true.  Does that mean we shouldn't try?
>
> Let me point out the important part again: All I ever see people
> suggest is magic bullets.
>
> We should try, but by doing the hard work.  Not by expecting magic.

Sure.  CodeSourcery did have a contract last year to do some of that
hard work, and there might be another one in the pipe (tho it probably
won't come through until after I leave).  Each individual piece isn't
that hard, either, although they also tend not to produce easurable
gains (but if you do lots of them, they add up...)

Sadly, I have no time these days for any GCC work other than what I'm
being paid to do.

zw

Re: GCC 4.1: Buildable on GHz machines only?

[Peter Barada]

>Well, yes.  1 second/file is still slow!  I want "make" to complete
>instantaneously!  Don't you?

Actually I want it to complete before I even start, but I don't want
to get too greedy. :)

What's really sad is that for cross-compilation of the toolchain, we
have to repeat a few steps (build gcc twice, build glibc twice)
because glibc and gcc assume that a near-complete environment is
available(such as gcc needing headers, and glibc needing -lgcc-eh), so
even really fast machines(2.4Ghz P4) take an hour to do a cross-build
from scratch. 

-- 
Peter Barada
peter@the-baradas.com

Re: GCC 4.1: Buildable on GHz machines only?

[Marcin Dalecki]

On 2005-04-27, at 21:57, Steven Bosscher wrote:

> On Wednesday 27 April 2005 17:45, Matt Thomas wrote:
>>> The features under discussion are new, they didn't exist before.
>>
>> And because they never existed before, their cost for older platforms
>> may not have been correctly assessed.
>
> If someone had cared about them, it would have been noticed earlier.
> But since _nobody_ has complained before you, I guess we can conclude
> that by far the majority if GCC users are quite happy with the cost
> assesments that were made.

That is simply not true. I don't feel happy about the whole gcj thing.
Already libstdc++ provided alongside with the compiler is pain enough.
But the runtime support for Java is a disaster in the making since it's
getting to be a truly huge behemoth which is constantly changing and 
"eating"
new APIs. Worst thing about it is it's grief for external libraries 
like GTK+ for
example.

Re: GCC 4.1: Buildable on GHz machines only?

[Marcin Dalecki]

On 2005-04-28, at 03:06, Peter Barada wrote:

>
>> Well, yes.  1 second/file is still slow!  I want "make" to complete
>> instantaneously!  Don't you?
>
> Actually I want it to complete before I even start, but I don't want
> to get too greedy. :)
>
> What's really sad is that for cross-compilation of the toolchain, we
> have to repeat a few steps (build gcc twice, build glibc twice)
> because glibc and gcc assume that a near-complete environment is
> available(such as gcc needing headers, and glibc needing -lgcc-eh), so
> even really fast machines(2.4Ghz P4) take an hour to do a cross-build
> from scratch.

Actually what GCC needs to know are the following:

1. What will the signal strucutre look alike on the target system.
    In esp.: What does the kernel think? What does the glibc think?
2. Do we do TLS?
3. Do we do linuxthreads or nptl?

glibc just wants:

1. Say hello to libgcc_s
2. Does the compiler support TLS?

And then don't forget that libgcc_s wants:

1. Say hello to C++, in a way requiring libc functions for exception
handling.

With a "tad bit" of work the double compilation can be avoided for the 
glibc.
You will have to build a GCC with static libgcc first and you will only 
need
the second gcc build cycle to get a dynamic libgcc_s as well as C++, 
since
C++ makes a shared libgcc mandatory.

The whole double build could be avoided if:

1. It would be possible to build libgcc for itself without rebuilding 
the
whole compiler.

2. It would be possible to build first the C compiler and then just the 
C++ compiler.

2. The information required by glibc could be provided statically to it.

All of the above are basically problems of the "configure" system - 
which isn't pretty.

Re: GCC 4.1: Buildable on GHz machines only?

[Tom Tromey]

>>>>> "Paul" == Paul Koning <pkoning@equallogic.com> writes:

Paul> Maybe.  Then again, maybe there are real problems here.  The ranlib
Paul> one was already mentioned.  And I wonder if libjava really needs to
Paul> bring the host to its knees, as it does.

Killing machines is only a secondary goal, if that's what you mean ;-)

The bad news is that libjava is only going to grow.

On the other hand, while I haven't measured it myself, I've heard that
a lot of the time in the libjava build is spent in libtool (versus
plain old ld).  Perhaps that can be alleviated somehow.

It is getting to be a major problem, to the point where it is simpler
to do one's development in Classpath and avoid libgcj just because of
the terrible turnaround time.  Unfortunately it seems to be an easier
problem to diagnose than to cure.

Tom

Re: GCC 4.1: Buildable on GHz machines only?

[Marcin Dalecki]

On 2005-04-28, at 01:35, Joe Buck wrote:
>
> I will agree with you on this point, but more than half of the time
> to bootstrap consists of the time to build the Java library, and 
> speeding
> that up is a losing battle, as Sun keeps adding new stuff that
> libgjc/classpath is then expected to clone, and the addition rate
> seems to exceed Moore's Law.

Those are just symptoms of the fact that Java is an
emulated machine with his own OS, which just happened to forget
about any lesson in this area learned during the last 30 years.
Other then this:

please.please.bePolite.toThe.niceFine.javaApi(nowAndAlways);

Re: GCC 4.1: Buildable on GHz machines only?

[Marcin Dalecki]

On 2005-04-27, at 22:54, Karel Gardas wrote:
>
> Total Physical Source Lines of Code (SLOC)                = 2,456,727
> Development Effort Estimate, Person-Years (Person-Months) = 725.95 
> (8,711.36)
>  (Basic COCOMO model, Person-Months = 2.4 * (KSLOC**1.05))
> Schedule Estimate, Years (Months)                         = 6.55 
> (78.55)
>  (Basic COCOMO model, Months = 2.5 * (person-months**0.38))
> Estimated Average Number of Developers (Effort/Schedule)  = 110.90
> Total Estimated Cost to Develop                           = $ 
> 98,065,527
>  (average salary = $56,286/year, overhead = 2.40).
> Please credit this data as "generated using 'SLOCCount' by David A. 
> Wheeler."

One question remains open: Who is Mr. Person?

Re: GCC 4.1: Buildable on GHz machines only?

[Zack Weinberg]

Peter Barada <peter@the-baradas.com> writes:

> What's really sad is that for cross-compilation of the toolchain, we
> have to repeat a few steps (build gcc twice, build glibc twice)
> because glibc and gcc assume that a near-complete environment is
> available(such as gcc needing headers, and glibc needing -lgcc-eh), so
> even really fast machines(2.4Ghz P4) take an hour to do a cross-build
> from scratch. 

This could be made substantially easier if libgcc moved to the top
level.  You wanna help out with that?

zw

Re: GCC 4.1: Buildable on GHz machines only?

[Peter Barada]

>> What's really sad is that for cross-compilation of the toolchain, we
>> have to repeat a few steps (build gcc twice, build glibc twice)
>> because glibc and gcc assume that a near-complete environment is
>> available(such as gcc needing headers, and glibc needing -lgcc-eh), so
>> even really fast machines(2.4Ghz P4) take an hour to do a cross-build
>> from scratch. 
>
>This could be made substantially easier if libgcc moved to the top
>level.  You wanna help out with that?

Uh, ok.  What do you mean by "move to the top level"?

-- 
Peter Barada
peter@the-baradas.com

Re: GCC 4.1: Buildable on GHz machines only?

[Zack Weinberg]

Peter Barada <peter@the-baradas.com> writes:

>>> What's really sad is that for cross-compilation of the toolchain, we
>>> have to repeat a few steps (build gcc twice, build glibc twice)
>>> because glibc and gcc assume that a near-complete environment is
>>> available(such as gcc needing headers, and glibc needing -lgcc-eh), so
>>> even really fast machines(2.4Ghz P4) take an hour to do a cross-build
>>> from scratch. 
>>
>>This could be made substantially easier if libgcc moved to the top
>>level.  You wanna help out with that?
>
> Uh, ok.  What do you mean by "move to the top level"?

libgcc should have its own top-level directory like all the rest of
the runtime libraries, instead of being nested inside the gcc
directory.  This helps because it's libgcc, not the compiler itself,
that needs the libc headers.  You could build your cross-gcc, use it
to configure libc to the point where the headers are available, build
libgcc, then go back and finish building libc, and not have to repeat
anything.  At least that's the theory.  I'm sure there are all kinds
of practical problems that will crop up when we get to the point we
can actually try this, but it's a long step in the right direction.

I'm not sure who is working on this now, but Paolo Bonzini should
know.

zw

Re: GCC 4.1: Buildable on GHz machines only?

[Thorsten Glaser]

Zack Weinberg dixit:

>This could be made substantially easier if libgcc moved to the top
>level.  You wanna help out with that?

What about crtstuff?

//mirabile

Re: GCC 4.1: Buildable on GHz machines only?

[Karel Gardas]

On Wed, 27 Apr 2005, Daniel Berlin wrote:

> If you want a faster compiler, it's hard work.  It means not adding
> features because the design isn't a good one, *even if the user would
> still find it useful*. People aren't willing to do this.  It means lots
> and lots of profiling, and taking care of little inefficiencies.  All I
> ever see people suggest is magic bullets.

Daniel,

I can not agree with it at all! At least for our C++ code, I've seen
compiler speedup from GCC 3.2, which was the most slowest compiler, 3.3
was faster because of better memory heuristics, 3.4 was faster probably
because of better parser and 4.0 is faster because of a work of many
developers here whom I would like to say Thank You!

Imagine that G++ is now faster at least about 50% (at least!), when
comparing 3.2 and 4.0.0, not just talking about great 25% speedup between
3.4.x and 4.0.0!

Conclusion: people are willing to investigate compiler slowness and even
they add new features to the compiler itself.

Thank you all for writing GCC!
Karel
--
Karel Gardas                  kgardas@objectsecurity.com
ObjectSecurity Ltd.           http://www.objectsecurity.com

Re: GCC 4.1: Buildable on GHz machines only?

[Andrew Haley]

Paul Koning writes:
 > >>>>> "Andrew" == Andrew Pinski <pinskia@physics.uc.edu> writes:
 > 
 >  >> However, I can always tell when a GCC build has hit the libjava
 >  >> build -- that's when the *whole system* suddenly slows to a crawl.
 >  >> Maybe it comes from doing some processing on 5000 foo.o files all
 >  >> at once... :-(
 > 
 >  Andrew> But that is not GCC fault that binutils cannot handle that
 >  Andrew> load.
 > 
 > Perhaps not.  But perhaps there are workarounds that allow the gcc
 > build to do its job without using binutils in a way that stresses it
 > beyond its ability to cope.
 > 
 > Matt's time output suggests that massive pagefaulting is a big issue

I'm sure that's true.  The working set exceeds the amount of RAM in
the system, leading to near worst-case behaviour.

 > -- and it wouldn't surprise me if the libjava build procedure were a
 > major contributor there.

Yes.  This is a profile of the libgcj build.  The single biggest user
of CPU is the libtool shell script, which uses more than a quarter of
the total (non-kernel) CPU time.  However, it's important not to be
misled -- I'm sure the linker causes a huge amount of disk activity,
so it's not just CPU time that is important.

Having said that, I suspect that the single biggest improvement to the
libgcj build time would be either to remove the libtool shell script
altogether or find some way to reduce its use or make it faster.


CPU: CPU with timer interrupt, speed 0 MHz (estimated)
Profiling through timer interrupt
          TIMER:0|
  samples|      %|
------------------
  1770547 63.0596 no-vmlinux
   415708 14.8058 libc-2.3.4.so
   259889  9.2562 ltsh
   257355  9.1659 jc1
    22111  0.7875 cc1plus
    20260  0.7216 as
    19289  0.6870 ld-2.3.4.so
    10502  0.3740 make
     5921  0.2109 sed
     5163  0.1839 libbfd-2.15.92.0.2.so
     2855  0.1017 gcj
     2724  0.0970 cc1
     2218  0.0790 libz.so.1.2.1.2
     2154  0.0767 grep
     2019  0.0719 xterm
     1864  0.0664 ld

Andrew.

RE: GCC 4.1: Buildable on GHz machines only?

[Dave Korn]

----Original Message----
>From: Marcin Dalecki
>Sent: 28 April 2005 02:58

> On 2005-04-27, at 22:54, Karel Gardas wrote:
>> 
>> Total Physical Source Lines of Code (SLOC)                = 2,456,727
>> Development Effort Estimate, Person-Years (Person-Months) = 725.95
>>  (8,711.36) (Basic COCOMO model, Person-Months = 2.4 * (KSLOC**1.05))
>> Schedule Estimate, Years (Months)                         = 6.55 (78.55)
>>  (Basic COCOMO model, Months = 2.5 * (person-months**0.38))
>> Estimated Average Number of Developers (Effort/Schedule)  = 110.90
>> Total Estimated Cost to Develop                           = $ 98,065,527
>>  (average salary = $56,286/year, overhead = 2.40).
>> Please credit this data as "generated using 'SLOCCount' by David A.
>> Wheeler."
> 
> One question remains open: Who is Mr. Person?


  What makes you think it's not Ms. Person?  Chauvinist!

;)


    cheers,
      DaveK
-- 
Can't think of a witty .sigline today....

Re: GCC 4.1: Buildable on GHz machines only?

[Marcin Dalecki]

On 2005-04-28, at 12:03, Dave Korn wrote:

> ----Original Message----
>> From: Marcin Dalecki
>> Sent: 28 April 2005 02:58
>
>> On 2005-04-27, at 22:54, Karel Gardas wrote:
>>>
>>> Total Physical Source Lines of Code (SLOC)                = 2,456,727
>>> Development Effort Estimate, Person-Years (Person-Months) = 725.95
>>>  (8,711.36) (Basic COCOMO model, Person-Months = 2.4 * (KSLOC**1.05))
>>> Schedule Estimate, Years (Months)                         = 6.55 
>>> (78.55)
>>>  (Basic COCOMO model, Months = 2.5 * (person-months**0.38))
>>> Estimated Average Number of Developers (Effort/Schedule)  = 110.90
>>> Total Estimated Cost to Develop                           = $ 
>>> 98,065,527
>>>  (average salary = $56,286/year, overhead = 2.40).
>>> Please credit this data as "generated using 'SLOCCount' by David A.
>>> Wheeler."
>>
>> One question remains open: Who is Mr. Person?
>
>
>   What makes you think it's not Ms. Person?  Chauvinist!

Oh indeed... Yes I missed it: It's Lady Wheeler!

Re: GCC 4.1: Buildable on GHz machines only?

[Richard Earnshaw]

On Wed, 2005-04-27 at 20:57, Steven Bosscher wrote:
> On Wednesday 27 April 2005 17:45, Matt Thomas wrote:
> > > The features under discussion are new, they didn't exist before.
> >
> > And because they never existed before, their cost for older platforms
> > may not have been correctly assessed.
> 
> If someone had cared about them, it would have been noticed earlier.
> But since _nobody_ has complained before you, I guess we can conclude
> that by far the majority if GCC users are quite happy with the cost
> assesments that were made.

This is false.  I've been complaining (at various levels of volume) ever
since we switched to using the garbage collector.[1]

R.

[1] I don't think the Garbage Collector is the only source of slowdown. 
It was just the change that tipped the balance from not-very good to
insane.  Unfortunately, things have continued to go down-hill from
there.

Re: GCC 4.1: Buildable on GHz machines only?

[Daniel Jacobowitz]

On Thu, Apr 28, 2005 at 07:31:06AM +0000, Thorsten Glaser wrote:
> Zack Weinberg dixit:
> 
> >This could be made substantially easier if libgcc moved to the top
> >level.  You wanna help out with that?
> 
> What about crtstuff?

Yes, they should be moved at the same time; I consider them closer to
part of libgcc than to part of gcc proper.

-- 
Daniel Jacobowitz
CodeSourcery, LLC

Re: GCC 4.1: Buildable on GHz machines only?

[Richard Earnshaw]

On Wed, 2005-04-27 at 21:55, Andrew Pinski wrote:
> > However, I can always tell when a GCC build has hit the libjava build
> > -- that's when the *whole system* suddenly slows to a crawl.  Maybe
> > it comes from doing some processing on 5000 foo.o files all at
> > once... :-(
> 
> But that is not GCC fault that binutils cannot handle that load.
> 
> -- Pinski

It's not as simple as just saying "binutils should be able to cope with
any number of objects thrown at it".  Part of the problem is that 5000
object files exceeds the system limits of the host machine (eg command
line length, etc).

R.

Re: GCC 4.1: Buildable on GHz machines only?

[Richard Earnshaw]

On Thu, 2005-04-28 at 02:40, Tom Tromey wrote:
> >>>>> "Paul" == Paul Koning <pkoning@equallogic.com> writes:
> 
> Paul> Maybe.  Then again, maybe there are real problems here.  The ranlib
> Paul> one was already mentioned.  And I wonder if libjava really needs to
> Paul> bring the host to its knees, as it does.
> 
> Killing machines is only a secondary goal, if that's what you mean ;-)
> 
> The bad news is that libjava is only going to grow.
> 
> On the other hand, while I haven't measured it myself, I've heard that
> a lot of the time in the libjava build is spent in libtool (versus
> plain old ld).  Perhaps that can be alleviated somehow.
> 
Splitting libjava into multiple libraries would be a good start.

It would probably be a good thing for application start-up time to when
using shared libs.

R.

Re: GCC 4.1: Buildable on GHz machines only?

[Andrew Haley]

Richard Earnshaw writes:
 > On Wed, 2005-04-27 at 21:55, Andrew Pinski wrote:
 > > > However, I can always tell when a GCC build has hit the libjava build
 > > > -- that's when the *whole system* suddenly slows to a crawl.  Maybe
 > > > it comes from doing some processing on 5000 foo.o files all at
 > > > once... :-(
 > > 
 > > But that is not GCC fault that binutils cannot handle that load.
 > > 
 > > -- Pinski
 > 
 > It's not as simple as just saying "binutils should be able to cope with
 > any number of objects thrown at it".  Part of the problem is that 5000
 > object files exceeds the system limits of the host machine (eg command
 > line length, etc).

If ld can't accept a list of files from a stream but is instead
limited by command line length, then that *is* the fault of ld.

Andrew.

Re: GCC 4.1: Buildable on GHz machines only?

[Richard Earnshaw]

On Thu, 2005-04-28 at 14:35, Andrew Haley wrote:
> Richard Earnshaw writes:
>  > On Wed, 2005-04-27 at 21:55, Andrew Pinski wrote:
>  > > > However, I can always tell when a GCC build has hit the libjava build
>  > > > -- that's when the *whole system* suddenly slows to a crawl.  Maybe
>  > > > it comes from doing some processing on 5000 foo.o files all at
>  > > > once... :-(
>  > > 
>  > > But that is not GCC fault that binutils cannot handle that load.
>  > > 
>  > > -- Pinski
>  > 
>  > It's not as simple as just saying "binutils should be able to cope with
>  > any number of objects thrown at it".  Part of the problem is that 5000
>  > object files exceeds the system limits of the host machine (eg command
>  > line length, etc).
> 
> If ld can't accept a list of files from a stream but is instead
> limited by command line length, then that *is* the fault of ld.

A certain proverb relating to bad workmen and tools springs to mind
here.  

I think we really need to consider whether requiring ld to support
linking of 5000 object files might mean poor library design.

R.

Re: GCC 4.1: Buildable on GHz machines only?

[Andreas Schwab]

Andrew Haley <aph@redhat.com> writes:

> If ld can't accept a list of files from a stream but is instead
> limited by command line length, then that *is* the fault of ld.

You can always use a linker script.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: GCC 4.1: Buildable on GHz machines only?

[Lars Segerlund]

 I have to agree with Richard's assessment, gcc is currently on the verge of being
 unusable in many instances.
 If you have a lot of software to build and have to do complete rebuilds it's painful,
 the binutils guys have a 3x speedup patch coming up, but every time there is a speedup
 it gets eaten up.

 gcc seem's to be moving more and more data around and does more and more temporary
 allocations of huge chunks of memory. Sometimes a concearn pop's up and some 5 to 
 10 percent is gained only to be lost again.

 My opinion on the point is that if there was a 100% speedup of gcc it would be very good
 but not enough since it has slowed down so much mainly from memory usage.
 I used to do kernel compiles in an hour or two on a pentium laptop, and now I can
 forget it since it's memory is soon exhausted and it goes into swap.

 I have never done any 'memory profiling' but I think it might be time to give it a
 shot, does anybody have any hints on how to go about something like this ?

 It should be enough to have a look at the places where were searching for information
 that we already have had, or to make any kind of search more efficient if possible.
 Perhaps it's possible to graft some kind of indexing scheme on the worst offenders,
 so that some O(n^2) can get to O(1 ) or something.

 The main problem seem's to be that almost any efforts to point out a single scapegoat
 have not been very successful.

 / regards, Lars Segerlund.


On Thu, 28 Apr 2005 14:24:11 +0100
Richard Earnshaw <rearnsha@gcc.gnu.org> wrote:

> On Wed, 2005-04-27 at 20:57, Steven Bosscher wrote:
> > On Wednesday 27 April 2005 17:45, Matt Thomas wrote:
> > > > The features under discussion are new, they didn't exist before.
> > >
> > > And because they never existed before, their cost for older platforms
> > > may not have been correctly assessed.
> > 
> > If someone had cared about them, it would have been noticed earlier.
> > But since _nobody_ has complained before you, I guess we can conclude
> > that by far the majority if GCC users are quite happy with the cost
> > assesments that were made.
> 
> This is false.  I've been complaining (at various levels of volume) ever
> since we switched to using the garbage collector.[1]
> 
> R.
> 
> [1] I don't think the Garbage Collector is the only source of slowdown. 
> It was just the change that tipped the balance from not-very good to
> insane.  Unfortunately, things have continued to go down-hill from
> there.

Re: GCC 4.1: Buildable on GHz machines only?

[Gunther Nikl]

On Wed, Apr 27, 2005 at 08:05:39AM -0700, Matt Thomas wrote:
> Am I the only person who has attempted to do a native bootstrap on a
> system as slow as a M68k?

  I am using an Amiga with 68060@50Mhz myself. My last GCC bootstrap on
  that machine was done in 1999 for GCC 2.95.2 and it took several hours.
  I never tried to boostrap newer GCC version because I fear that it would
  take much too long. I only did a non-optimized non-bootstrap build of
  3.2.2 with 2.95.2 as build compiler on that machine. I have to cross-build
  GCC3 and newer to use them on this machine. Anything else wouldn't work.
  FWIW, I experienced a constant slowdown of the C compiler with every
  release since 2.7. While C is still usable, C++ with templates is not.

  Gunther

Re: GCC 4.1: Buildable on GHz machines only?

[Joel Sherrill <joel@OARcorp.com>]

Peter Barada wrote:
>>Well, yes.  1 second/file is still slow!  I want "make" to complete
>>instantaneously!  Don't you?
> 
> 
> Actually I want it to complete before I even start, but I don't want
> to get too greedy. :)
> 
> What's really sad is that for cross-compilation of the toolchain, we
> have to repeat a few steps (build gcc twice, build glibc twice)
> because glibc and gcc assume that a near-complete environment is
> available(such as gcc needing headers, and glibc needing -lgcc-eh), so
> even really fast machines(2.4Ghz P4) take an hour to do a cross-build
> from scratch. 

That sounds comparable to the time required to build RTEMS toolsets.  I 
just looked at the timestamp on the build logs for a gcc 4.0.0 CVS build 
with newlib 1.13.0 and it is on the order of 60-90 minutes per target on 
a 2.4 Ghz P4 w/512 MB RAM.  This is just C and C++ and the variance is 
probably mostly due to the number of multilibs.

I checked build logs of gcc 3.3.5 with newlib 1.13 from December on the 
same machine and times were comparable so it isn't a recent slowdown.

When I built my native 4.0.0 on this machine, I did notice that 
compiling the Java libraries took long enough to notice and when I did a 
top, I noticed that gjc was running for 30+ seconds of CPU time with RSS 
on the order of 150 MB.  Another time I noticed that sh had taken 90 
seconds and that was an invocation of libtool with a very long command line.

Here is the configure command and output of time on this machine for a 
gcc 4.0.0 build:

../gcc-4.0.0/configure --prefix=/opt/gcc-4.0.0 
--enable-languages=c,c++,ada,java,objc

4271.94user 685.49system 1:31:18elapsed 90%CPU (0avgtext+0avgdata 
0maxresident)k0inputs+0outputs (33176183major+40970199minor)pagefaults 
0swaps

A 2.4 Ghz P4 isn't what I would consider an obsolete machine and it took 
90 minutes for "make" -- not a full bootstrap.

--joel

Re: GCC 4.1: Buildable on GHz machines only?

[Gunther Nikl]

On Wed, Apr 27, 2005 at 04:40:29PM -0700, Zack Weinberg wrote:
> Daniel Berlin <dberlin@dberlin.org> writes:
> 
> > On Wed, 2005-04-27 at 15:13 -0700, Stan Shebs wrote:
> >> Steven Bosscher wrote:
> >> >If someone had cared about them, it would have been noticed
> >> >earlier.  But since _nobody_ has complained before you, I guess we
> >> >can conclude that by far the majority if GCC users are quite happy
> >> >with the cost assesments that were made.
> >> >
> >> No, there have been plenty of complaints, but the GCC mailing lists
> >> have, shall we say, a "reputation", and a great many users will not
> >> post to them,
> >
> > I've never in my life heard this from another mailing list, and i
> > contribute to a *great* many open source projects.
> 
> I have seen such complaints.  Not about bootstrap times, no, that only
> affects people who compile the compiler; but the more general case of
> 'gcc takes forever to compile this program' does appear on a regular
> basis.

  Maybe there are less/no complaints about bootstrap times, because people
  that are able to make a bootstrap know that complaining doesn't help. My
  primary target is m68k and I never attempted a bootstrap of GCC3 there
  because it would take much to much time. Now I got used to cross-build
  GCC (only C and C++) for m68k-amigaos. And since this target isn't in
  the official tree its even more painful to inquire the list.

> I do also think that the amount of ridicule heaped on people who come
> to the gcc lists is, in general, too high.  People should not be
> ridiculed for complaining that the compiler is slow, even if they are
> insisting on using vintage hardware.

  IMHO GCC3 is better than GCC2 and thus its worthwile to use it even
  on "vintage" hardware.

> It is slow, even on fast hardware; it's just easier to see that on
> slow hardware.

  I mainly use C and thats acceptable with GCC3 but eg. C++ with templates
  is really slow on my m68k Amiga.

> Rather more importantly, people should not be ridiculed for submitting
> bug reports, even if they are wrong.  I suspect the bad public image
> that Stan refers to, has more to do with this than anything else.

  FWIW, I fully agree.

  Gunther

Re: GCC 4.1: Buildable on GHz machines only?

[Peter Barada]

>> What's really sad is that for cross-compilation of the toolchain, we
>> have to repeat a few steps (build gcc twice, build glibc twice)
>> because glibc and gcc assume that a near-complete environment is
>> available(such as gcc needing headers, and glibc needing -lgcc-eh), so
>> even really fast machines(2.4Ghz P4) take an hour to do a cross-build
>> from scratch. 
>
>That sounds comparable to the time required to build RTEMS toolsets.  I 
>just looked at the timestamp on the build logs for a gcc 4.0.0 CVS build 
>with newlib 1.13.0 and it is on the order of 60-90 minutes per target on 
>a 2.4 Ghz P4 w/512 MB RAM.  This is just C and C++ and the variance is 
>probably mostly due to the number of multilibs.

This is for a m68k-linux build (with coldfire-linux config for glibc),
and its only the C compiler, so adding C++ will obvioulsy make it take
longer.

>A 2.4 Ghz P4 isn't what I would consider an obsolete machine and it took 
>90 minutes for "make" -- not a full bootstrap.

Even on a 3.0Ghz P4 with HT, 1Gb DDR and a hardware RAID with SATA
drives it takes about 30 minutes so there's a *lot* of work going on,
and I'd call that near cutting-edge.

-- 
Peter Barada
peter@the-baradas.com

Re: GCC 4.1: Buildable on GHz machines only?

[Andrew Haley]

Andreas Schwab writes:
 > Andrew Haley <aph@redhat.com> writes:
 > 
 > > If ld can't accept a list of files from a stream but is instead
 > > limited by command line length, then that *is* the fault of ld.
 > 
 > You can always use a linker script.

Yeah, good point.  libtool seems to go to extraordinary lengths to
avoid doing so, I presume because it isn't portable.

Andrew.

Re: GCC 4.1: Buildable on GHz machines only?

[David Edelsohn]

>>>>> Andrew Haley writes:

Andrew> Yeah, good point.  libtool seems to go to extraordinary lengths to
Andrew> avoid doing so, I presume because it isn't portable.

	Current libtool does allow a list of files, but the version used
by GCC is not recent.

David

Re: GCC 4.1: Buildable on GHz machines only?

[Daniel Berlin]

On Thu, 2005-04-28 at 11:58 -0400, Peter Barada wrote:

> This is for a m68k-linux build (with coldfire-linux config for glibc),
> and its only the C compiler, so adding C++ will obvioulsy make it take
> longer.
> 
> >A 2.4 Ghz P4 isn't what I would consider an obsolete machine and it took 
> >90 minutes for "make" -- not a full bootstrap.
> 
> Even on a 3.0Ghz P4 with HT, 1Gb DDR and a hardware RAID with SATA
> drives it takes about 30 minutes so there's a *lot* of work going on,
> and I'd call that near cutting-edge.
> 


1. make bootstrap on a 2.4ghz p4 takes 90 minutes for me as of
yesterday.
2. Building XLC with (C,C++,Fortran) and a single backend takes roughly
the same time as building GCC.  And they aren't three staging, AFAIK.

--Dan

Re: GCC 4.1: Buildable on GHz machines only?

[Daniel Berlin]

On Wed, 2005-04-27 at 16:40 -0700, Zack Weinberg wrote:
> Daniel Berlin <dberlin@dberlin.org> writes:
> 
> > On Wed, 2005-04-27 at 15:13 -0700, Stan Shebs wrote:
> >> Steven Bosscher wrote:
> >> >If someone had cared about them, it would have been noticed
> >> >earlier.  But since _nobody_ has complained before you, I guess we
> >> >can conclude that by far the majority if GCC users are quite happy
> >> >with the cost assesments that were made.
> >> >
> >> No, there have been plenty of complaints, but the GCC mailing lists
> >> have, shall we say, a "reputation", and a great many users will not
> >> post to them,
> >
> > I've never in my life heard this from another mailing list, and i
> > contribute to a *great* many open source projects.
> 
> I have seen such complaints.  Not about bootstrap times, no, that only
> affects people who compile the compiler; but the more general case of
> 'gcc takes forever to compile this program' does appear on a regular
> basis.
> 
> I do also think that the amount of ridicule heaped on people who come
> to the gcc lists is, in general, too high.  People should not be
> ridiculed for complaining that the compiler is slow, even if they are
> insisting on using vintage hardware.  It is slow, even on fast
> hardware; it's just easier to see that on slow hardware.  

For people who use hardware most developers don't have access to, we
need preprocessed source and profiling data, or else there is no chance
of fixing it.  
It seems users of these platforms are not willing to provide this, even
when asked.  

Those people who have provided preprocessed source and profiling data
(8361, omniorb, etc) have had their compilation times sped up.
So whenever i hear that "we don't care about compilation time", i wonder
if the user has even put code in bugzilla that demonstrates the problem.
Most often, the answer is no, AFAICT.

--Dan

Re: GCC 4.1: Buildable on GHz machines only?

[Ian Lance Taylor]

Andrew Haley <aph@redhat.com> writes:

> If ld can't accept a list of files from a stream but is instead
> limited by command line length, then that *is* the fault of ld.

GNU ld won't currently read a list of files from stdin, but it will
read a list of files from a file.

For example, look at /usr/lib/libc.so on a typical GNU/Linux system.

Ian

Re: GCC 4.1: Buildable on GHz machines only?

[Andrew Haley]

Ian Lance Taylor writes:
 > Andrew Haley <aph@redhat.com> writes:
 > 
 > > If ld can't accept a list of files from a stream but is instead
 > > limited by command line length, then that *is* the fault of ld.
 > 
 > GNU ld won't currently read a list of files from stdin, but it will
 > read a list of files from a file.
 > 
 > For example, look at /usr/lib/libc.so on a typical GNU/Linux system.

Yes thanks, I've had that pointed out to me.  Apparently the real
issue here is that we have an older version of libtool in the gcc
tree.

Andrew.

Re: GCC 4.1: Buildable on GHz machines only?

[Joe Buck]

On Wed, Apr 27, 2005 at 07:40:37PM -0600, Tom Tromey wrote:
> >>>>> "Paul" == Paul Koning <pkoning@equallogic.com> writes:
> 
> Paul> Maybe.  Then again, maybe there are real problems here.  The ranlib
> Paul> one was already mentioned.  And I wonder if libjava really needs to
> Paul> bring the host to its knees, as it does.
> 
> Killing machines is only a secondary goal, if that's what you mean ;-)
> 
> The bad news is that libjava is only going to grow.
> 
> On the other hand, while I haven't measured it myself, I've heard that
> a lot of the time in the libjava build is spent in libtool (versus
> plain old ld).  Perhaps that can be alleviated somehow.

Has anyone looked at oprofile data for the libjava build?

Re: GCC 4.1: Buildable on GHz machines only?

[Joe Buck]

On Thu, Apr 28, 2005 at 12:09:35PM -0400, David Edelsohn wrote:
> >>>>> Andrew Haley writes:
> 
> Andrew> Yeah, good point.  libtool seems to go to extraordinary lengths to
> Andrew> avoid doing so, I presume because it isn't portable.
> 
> 	Current libtool does allow a list of files, but the version used
> by GCC is not recent.

Is there a reason why we aren't using a recent libtool?

Re: GCC 4.1: Buildable on GHz machines only?

[David Edelsohn]

>>>>> Andrew Haley writes:

Andrew> Yes thanks, I've had that pointed out to me.  Apparently the real
Andrew> issue here is that we have an older version of libtool in the gcc
Andrew> tree.

	Any feature in libtool CVS is fair game to be backported to
libtool in GCC.  I am planning to backport a subset of the feature for
AIX.  I can backport it all, including the GNU ld support, if you want.

David

Re: GCC 4.1: Buildable on GHz machines only?

[David Edelsohn]

>>>>> Joe Buck writes:

Joe> Is there a reason why we aren't using a recent libtool?

	Porting and testing effort to upgrade. 

David

Re: GCC 4.1: Buildable on GHz machines only?

[Andrew Haley]

Joe Buck writes:
 > On Wed, Apr 27, 2005 at 07:40:37PM -0600, Tom Tromey wrote:
 > > >>>>> "Paul" == Paul Koning <pkoning@equallogic.com> writes:
 > > 
 > > Paul> Maybe.  Then again, maybe there are real problems here.  The ranlib
 > > Paul> one was already mentioned.  And I wonder if libjava really needs to
 > > Paul> bring the host to its knees, as it does.
 > > 
 > > Killing machines is only a secondary goal, if that's what you mean ;-)
 > > 
 > > The bad news is that libjava is only going to grow.
 > > 
 > > On the other hand, while I haven't measured it myself, I've heard that
 > > a lot of the time in the libjava build is spent in libtool (versus
 > > plain old ld).  Perhaps that can be alleviated somehow.
 > 
 > Has anyone looked at oprofile data for the libjava build?

Again:

CPU: CPU with timer interrupt, speed 0 MHz (estimated)
Profiling through timer interrupt
          TIMER:0|
  samples|      %|
------------------
  1770547 63.0596 no-vmlinux
   415708 14.8058 libc-2.3.4.so
   259889  9.2562 ltsh
   257355  9.1659 jc1
    22111  0.7875 cc1plus
    20260  0.7216 as
    19289  0.6870 ld-2.3.4.so
    10502  0.3740 make
     5921  0.2109 sed
     5163  0.1839 libbfd-2.15.92.0.2.so
     2855  0.1017 gcj
     2724  0.0970 cc1
     2218  0.0790 libz.so.1.2.1.2
     2154  0.0767 grep
     2019  0.0719 xterm
     1864  0.0664 ld

Andrew.

Re: GCC 4.1: Buildable on GHz machines only?

[Marcin Dalecki]

On 2005-04-28, at 16:26, Lars Segerlund wrote:
>
>  I have never done any 'memory profiling' but I think it might be time 
> to give it a
>  shot, does anybody have any hints on how to go about something like 
> this ?

The main performance problem for GCC as I see it is structural. GCC is 
emulating
the concept of high polymorphism of data structures in plain C. Well 
actually it's
not even plain C any longer... GTY().

Using C++ with simple inheritance could help *a lot* here. (Duck...)

There is too much of pointer indirection in the data structures too.
Look throughout the code and wonder how very few iterations go around 
plan,
simple, fast, spare on register set, allowing cache prefetch, to work C 
arrays.
GCC is using linked lists to test the "efficiency" of TLB and other 
mechanisms
of the CPU.

Re: GCC 4.1: Buildable on GHz machines only?

[Devang Patel]

On Apr 28, 2005, at 9:10 AM, Daniel Berlin wrote:

> 1. make bootstrap on a 2.4ghz p4 takes 90 minutes for me as of
> yesterday.
> 2. Building XLC with (C,C++,Fortran) and a single backend takes  
> roughly
> the same time as building GCC.  And they aren't three staging, AFAIK.

"..ain't the same ballpark, it ain't the same league,
it ain't even the same..." :-)

-
Devang

Re: GCC 4.1: Buildable on GHz machines only?

[David Carlton]

On Wed, 27 Apr 2005 16:52:25 -0400, Paul Koning <pkoning@equallogic.com> said:

> However, I can always tell when a GCC build has hit the libjava build
> -- that's when the *whole system* suddenly slows to a crawl.  Maybe
> it comes from doing some processing on 5000 foo.o files all at
> once... :-(

It's also too bad the final steps of the libjava build aren't more
parallelizable, though I can't say I have any productive suggestions
to add there.  I just tried a C/C++ bootstrap and a C/C++/Java
bootstrap on a four-processor machine; the latter took 2.6 times as
long as the former, and for most of the non-compilation part of the
libjava build, three of the processors were sitting around being
bored.  (Not that I really know exactly what was causing the delays;
maybe the disk and memory were being stressed enough by the single
processor that was active.)

This also showed up a little in the C build: while make found other
stuff to do while gen-attrtab was going on, shortly after it started
compiling insn-attrtab.c, make ran out of other files to compile.

Not that I'm really complaining: you can get quite a lot of mileage
out of multiple CPUs as it is, more than enough (in my opinion) to
justify purchasing some nice build servers by software shops that do a
lot of GCC work.  (I won't post the actual bootstrap times out of fear
of being lynched.)  This might show up more as people start moving
towards dual-core and/or multiple CPU systems even on the low end.

David Carlton
david.carlton@sun.com

Re: GCC 4.1: Buildable on GHz machines only?

[Daniel Berlin]

On Thu, 2005-04-28 at 10:23 -0700, Devang Patel wrote:
> On Apr 28, 2005, at 9:10 AM, Daniel Berlin wrote:
> 
> > 1. make bootstrap on a 2.4ghz p4 takes 90 minutes for me as of
> > yesterday.
> > 2. Building XLC with (C,C++,Fortran) and a single backend takes  
> > roughly
> > the same time as building GCC.  And they aren't three staging, AFAIK.
> 
> "..ain't the same ballpark, it ain't the same league,
> it ain't even the same..." :-)

Bullshit.
They are both production quality compilers.

People complain gcc bootstrap is too long. XLC at O2 (which is what they
compile at, last i looked) isn't running the IPA middle end, etc.

--Dan

Re: GCC 4.1: Buildable on GHz machines only?

[Peter Barada]

>Not that I'm really complaining: you can get quite a lot of mileage
>out of multiple CPUs as it is, more than enough (in my opinion) to
>justify purchasing some nice build servers by software shops that do a
>lot of GCC work.  (I won't post the actual bootstrap times out of fear
>of being lynched.)  This might show up more as people start moving
>towards dual-core and/or multiple CPU systems even on the low end.

<minor-rant>

That's great if the software can be cross-built.  As it is,
cross-building a toolchain requires a lot of extra work, and if it
weren't for Dan Kegel's commitment, I'd dare say near impossible.
I've watched the sometimes near-indifference to the problems we have
trying to put together toolchains for non-hosted environments.

Even when I have a cross-toolchain, its still a *long* uphill battle
since there are too many OSS packages out there that can't
cross-configure/compile (openssh, perl as examples off the top of my
head) without a *lot* of work.

Its just that it takes a lot of time and work to cross-build a non-x86
linux environment to verify any changes in the toolchain.

And comments like "get a faster machine" are a non-starter.

</minor-rant>

-- 
Peter Barada
peter@the-baradas.com

Re: GCC 4.1: Buildable on GHz machines only?

[Matt Thomas]

Someone complained I was unfair in my gcc bootstrap times since
some builds included libjava/gfortran and some did not.

So in the past day, I've done bootstrap with just c,c++,objc on
both 3.4 and gcc4.1.  I've put the results in a web page at
http://3am-software.com/gcc-speed.html.  The initial bootstrap
compiler was gcc3.3 and they are all running off the same base
of NetBSD 3.99.3.

While taking out fortran and java reduced the disparity, there
is still a large increase in bootstrap times from 3.4 to 4.1.
-- 
Matt Thomas                     email: matt@3am-software.com
3am Software Foundry              www: http://3am-software.com/bio/matt/
Cupertino, CA              disclaimer: I avow all knowledge of this message.

Re: GCC 4.1: Buildable on GHz machines only?

[Devang Patel]

On Apr 28, 2005, at 10:54 AM, Daniel Berlin wrote:

> On Thu, 2005-04-28 at 10:23 -0700, Devang Patel wrote:
>> On Apr 28, 2005, at 9:10 AM, Daniel Berlin wrote:
>>
>>> 1. make bootstrap on a 2.4ghz p4 takes 90 minutes for me as of
>>> yesterday.
>>> 2. Building XLC with (C,C++,Fortran) and a single backend takes
>>> roughly
>>> the same time as building GCC.  And they aren't three staging,  
>>> AFAIK.
>>
>> "..ain't the same ballpark, it ain't the same league,
>> it ain't even the same..." :-)
>
> Bullshit.
> They are both production quality compilers.
>
> People complain gcc bootstrap is too long. XLC at O2 (which is what  
> they
> compile at, last i looked) isn't running the IPA middle end, etc.

Again,

"..ain't the same ballpark, it ain't the same league,
it ain't even the same ******* sport..."

Software A provides X1 functionality.
Software B provides X2 functionality.

BTW, you know more, about XLC's SPEC numbers and GCC's SPEC
numbers, than me. So in these X1 is not same as X2 but let's
ignore that for moment and say X1 is almost same as X2.

Software A and B is developed independently.
They do not share source code.

Software G compiles A in T1 time.
Software X compiles B in T2 time.

T1 is almost same as T2, so G is as fast as X.
That's what you're trying to say?


-
Devang

Re: GCC 4.1: Buildable on GHz machines only?

[Joe Buck]

On Thu, Apr 28, 2005 at 11:03:51AM -0700, Matt Thomas wrote:
> 
> Someone complained I was unfair in my gcc bootstrap times since
> some builds included libjava/gfortran and some did not.
> 
> So in the past day, I've done bootstrap with just c,c++,objc on
> both 3.4 and gcc4.1.  I've put the results in a web page at
> http://3am-software.com/gcc-speed.html.  The initial bootstrap
> compiler was gcc3.3 and they are all running off the same base
> of NetBSD 3.99.3.
> 
> While taking out fortran and java reduced the disparity, there
> is still a large increase in bootstrap times from 3.4 to 4.1.

There's some new code in libstdc++ now (the TR1 stuff) that (last time I
looked) takes a long time to build, and wasn't in 3.4.  Could that be a
factor?

A comparison with just --enable-languages=c would eliminate any
issues with libraries.

Re: GCC 4.1: Buildable on GHz machines only?

[Ian Lance Taylor]

Matt Thomas <matt@3am-software.com> writes:

> When I see the native stage2 m68k compiler spend 30+ minutes compute bound
> with no paging activity compiling a single source file, I believe
> that is an accurate term.  Compiling stage3 on a 50MHz 68060 took 18 hours.
> (That 30 minutes was for fold-const.c if you care to know).

The fold-const.c compilation seems like a good specific candidate for
a bug report at
    http://gcc.gnu.org/bugzilla/

If you can include the preprocessed file and profiler output from cc1
running on your system, there is a chance that this can be addressed.

In general the gcc developers do a reasonable job of keeping the
compiler from being too slow, but they do not use m68k systems, and
they can only work on problems which people bring to their attention.

And, yes, we clearly need to do something about the libjava build.

Thanks.

Ian

Re: GCC 4.1: Buildable on GHz machines only?

[Daniel Berlin]

On Thu, 2005-04-28 at 11:08 -0700, Devang Patel wrote:
> On Apr 28, 2005, at 10:54 AM, Daniel Berlin wrote:
> 
> > On Thu, 2005-04-28 at 10:23 -0700, Devang Patel wrote:
> >> On Apr 28, 2005, at 9:10 AM, Daniel Berlin wrote:
> >>
> >>> 1. make bootstrap on a 2.4ghz p4 takes 90 minutes for me as of
> >>> yesterday.
> >>> 2. Building XLC with (C,C++,Fortran) and a single backend takes
> >>> roughly
> >>> the same time as building GCC.  And they aren't three staging,  
> >>> AFAIK.
> >>
> >> "..ain't the same ballpark, it ain't the same league,
> >> it ain't even the same..." :-)
> >
> > Bullshit.
> > They are both production quality compilers.
> >
> > People complain gcc bootstrap is too long. XLC at O2 (which is what  
> > they
> > compile at, last i looked) isn't running the IPA middle end, etc.
> 
> Again,
> 
> "..ain't the same ballpark, it ain't the same league,
> it ain't even the same ******* sport..."
> 
> Software A provides X1 functionality.
> Software B provides X2 functionality.
> 

This has nothing to do with it.
We are talking about bootstrap times of compilers with roughly
equivalent functionality.

They both compile C, C++, and fortran.

You can assume our backends and frontend are comparable in terms of
functionality (they actually are, though not equivalent in performance
generated by that functionality, but in time taken by each functional
part they are).

As i've said, we've already excluded the running time for the
functionality we don't have (interprocedural middle end).

--Dan

Re: GCC 4.1: Buildable on GHz machines only?

[David Edelsohn]

>>>>> Matt Thomas writes:

Matt> So in the past day, I've done bootstrap with just c,c++,objc on
Matt> both 3.4 and gcc4.1.
Matt> While taking out fortran and java reduced the disparity, there
Matt> is still a large increase in bootstrap times from 3.4 to 4.1.

	libstdc++ contains a lot more features in GCC 4.1, especially
TR1.

	We understand that a lot of people use modern GCC on older
hardware and the compilation speed can be frustrating.  GCC supports a
very diverse group of processors with greatly varying performance.  Users
also want new features and want to utilize newer hardware performance for
more optimization in the same wall clock time.  It is difficult, if not
impossible, to satisfy everyone.

	The GCC developers are trying to improve compile time, but there
are no magic bullets.  If you provide testcases, developers do investigate
ways to improve compile time when they have examples to test.

	It would be helpful if this discussion thread distinguished
between compile time and bootstrap time.  The two are related but not
identical.  While GCC compile time needs to improve, the amount of time
that it takes to build GCC itself, relative to other commercial compilers
with similar features and runtimes, is similar.

David

Re: GCC 4.1: Buildable on GHz machines only?

[Hugh Sasse]

On Thu, 28 Apr 2005, Ian Lance Taylor wrote:

> If you can include the preprocessed file and profiler output from cc1
> running on your system, there is a chance that this can be addressed.

GCC comes with a test suite and a means for submitting results.  May
I suggest that it might be useful to have a timing suite built
similarly, so people can do this kind of thing really easily?  Maybe
someone can tell us if it is too big a job to create such a tool
portably?  I think having it running during stage3 would yield
useful results for comparisons.  A tool that came with GCC would
ensure the data was in a useful format, and that tests were
performed consistently.

         Hugh

Re: GCC 4.1: Buildable on GHz machines only?

[Daniel Berlin]

On Thu, 2005-04-28 at 11:03 -0700, Matt Thomas wrote:
> Someone complained I was unfair in my gcc bootstrap times since
> some builds included libjava/gfortran and some did not.
> 
> So in the past day, I've done bootstrap with just c,c++,objc on
> both 3.4 and gcc4.1.  I've put the results in a web page at
> http://3am-software.com/gcc-speed.html.  The initial bootstrap
> compiler was gcc3.3 and they are all running off the same base
> of NetBSD 3.99.3.
> 
> While taking out fortran and java reduced the disparity, there
> is still a large increase in bootstrap times from 3.4 to 4.1.

I'm sure you used disable-checking on the 4.1 test, right?
Since otherwise, it's a worthless comparison

Re: GCC 4.1: Buildable on GHz machines only?

[Daniel Berlin]

On Thu, 28 Apr 2005, Daniel Berlin wrote:

> On Thu, 2005-04-28 at 11:03 -0700, Matt Thomas wrote:
>> Someone complained I was unfair in my gcc bootstrap times since
>> some builds included libjava/gfortran and some did not.
>>
>> So in the past day, I've done bootstrap with just c,c++,objc on
>> both 3.4 and gcc4.1.  I've put the results in a web page at
>> http://3am-software.com/gcc-speed.html.  The initial bootstrap
>> compiler was gcc3.3 and they are all running off the same base
>> of NetBSD 3.99.3.
>>
>> While taking out fortran and java reduced the disparity, there
>> is still a large increase in bootstrap times from 3.4 to 4.1.
>
> I'm sure you used disable-checking on the 4.1 test, right?
> Since otherwise, it's a worthless comparison

Forget it, i missed it in the light green on black text
:)

>

Re: GCC 4.1: Buildable on GHz machines only?

[Dan Nicolaescu]

Dan Nicolaescu <dann@ics.uci.edu> writes:

  > Matt Thomas <matt@3am-software.com> writes:
  > 
  >   > Richard Henderson wrote:
  >   > > On Tue, Apr 26, 2005 at 10:57:07PM -0400, Daniel Jacobowitz wrote:
  >   > > 
  >   > >>I would expect it to be drastically faster.  However this won't show up
  >   > >>clearly in the bootstrap.  The, bar none, longest bit of the bootstrap
  >   > >>is building stage2; and stage1 is always built with optimization off and
  >   > >>(IIRC) checking on.
  >   > > 
  >   > > 
  >   > > Which is why I essentially always supply STAGE1_CFLAGS='-O -g' when
  >   > > building on risc machines.
  >   > 
  >   > Alas, the --disable-checking and STAGE1_CFLAGS="-O2 -g" (which I was
  > 
  > I don't think that is enough,  also edit gcc/Makefile.in and change the line:
  > STAGE1_CHECKING = -DENABLE_CHECKING -DENABLE_ASSERT_CHECKING
  > to be
  > STAGE1_CHECKING =   
  > 
  > Is there a better way to do this? STAGE1_CHECKING is not passed from
  > the toplevel make, so one cannot put it on the make bootstrap command
  > line...

Following up on this to show some numbers. 
On a 2.8GHz P4, 1MB cache, 512MB RAM, Fedora Core 3
gcc HEAD configured with --enable-languages=c --disable-checking --disable-nls 

time make bootstrap > & out
1227.861u 53.623s 21:26.53 99.6%        0+0k 0+0io 18pf+0w

If gcc/Makefile.in is first edited as shown above, then the bootstrap
time is: 
983.769u 53.241s 17:33.12 98.4% 0+0k 0+0io 3573pf+0w

A significant difference!

Given that with --enable-languages=c the amount of stuff built with
the slow stage1 compiler is minimal, the impact might be even higher
when more languages are enabled (I haven't tried). 

It would be nice to have some way to disable STAGE1_CHECKING other
than by editing gcc/Makefile.in (Sorry, I can't help with this, I
don't know much about how the gcc configuration process). 

      

'make bootstrap' oprofile (13% on bash?)

[Scott A Crosby]

On Thu, 28 Apr 2005 10:29:32 +0100, Andrew Haley <aph@redhat.com> writes:

>  > -- and it wouldn't surprise me if the libjava build procedure were a
>  > major contributor there.
>
> Yes.  This is a profile of the libgcj build.  The single biggest user
> of CPU is the libtool shell script, which uses more than a quarter of
> the total (non-kernel) CPU time.  However, it's important not to be
> misled -- I'm sure the linker causes a huge amount of disk activity,
> so it's not just CPU time that is important.
>
> Having said that, I suspect that the single biggest improvement to the
> libgcj build time would be either to remove the libtool shell script
> altogether or find some way to reduce its use or make it faster.
>

bash is the third-most prominent program in the result of a full 'make
bootstrap' oprofile on gcc-4.1-20050424 configured with
--disable-checking. A few other breakdowns, including for jc1 and cc1
exist on http://www.cs.rice.edu/~scrosby/tmp/GCC/

This was done on machine with 1.5gb of RAM. As the build+src
directories add to 1.1gb, there shouldn't have been any disk IO.

(Excerpted out of GCC-overall)

CPU: Athlon, speed 1830.27 MHz (estimated)
Counted CPU_CLK_UNHALTED events (Cycles outside of halt state) with a unit mask of 0x00 (No unit mask) count 1800000
CPU_CLK_UNHALT...|
  samples|      %|
------------------
  1571655 35.9269 /tmp/gccbuild/gcc/cc1
  1108036 25.3289 /tmp/gccbuild/gcc/jc1
   573302 13.1053 /bin/bash
   178865  4.0887 /tmp/gccbuild/gcc/cc1plus
   150641  3.4435 /bin/sed
   111300  2.5442 /usr/lib/gcc-lib/i486-linux/3.3.5/cc1
   110216  2.5424 /usr/bin/as
    60841  1.4034 /tmp/gccbuild/gcc/build/genattrtab
    44633  1.0296 /usr/local/src/linux-2.6.10/vmlinux
    38924  0.8979 /usr/bin/ld
    23191  0.5350 /usr/bin/expr

Scott

Re: GCC 4.1: Buildable on GHz machines only?

[Jason Thorpe]

On Apr 27, 2005, at 12:57 PM, Steven Bosscher wrote:

> Maybe the older platform should stick to the older compiler then,
> if it is too slow to support the kind of compiler that modern
> systems need.

This is an unreasonable request.  Consider NetBSD, which runs on new  
and old hardware.  The OS continues to evolve, and that often  
requires adopting newer compilers (so e.g. other language features  
can be used in the base OS).

The GCC performance issue is not new.  It seems to come up every so  
often... last time I recall a discussion on the topic, it was thought  
that the new memory allocator (needed for pch) was cause cache-thrash  
(what was the resolution of that discussion, anyway?)

-- thorpej

Re: GCC 4.1: Buildable on GHz machines only?

[Jason Thorpe]

On Apr 27, 2005, at 7:41 AM, David Edelsohn wrote:

>     GCC now supports C++, Fortran 90 and Java.  Those languages have
> extensive, complicated runtimes.  The GCC Java environment is becoming
> much more complete and standards compliant, which means adding more  
> and
> more features.

Except it's not just bootstrapping GCC.  It's everything.  When the  
NetBSD Project switched from 2.95.3 to 3.3, we had a noticeably  
increase in time to do the "daily" builds because the 3.3 compiler  
was so much slower at compiling the same OS source code.  And we're  
talking almost entirely C code, here.

-- thorpej

Re: 'make bootstrap' oprofile (13% on bash?)

[Andrew Haley]

Scott A Crosby writes:
 > On Thu, 28 Apr 2005 10:29:32 +0100, Andrew Haley <aph@redhat.com> writes:
 > 
 > >  > -- and it wouldn't surprise me if the libjava build procedure were a
 > >  > major contributor there.
 > >
 > > Yes.  This is a profile of the libgcj build.  The single biggest user
 > > of CPU is the libtool shell script, which uses more than a quarter of
 > > the total (non-kernel) CPU time.  However, it's important not to be
 > > misled -- I'm sure the linker causes a huge amount of disk activity,
 > > so it's not just CPU time that is important.
 > >
 > > Having said that, I suspect that the single biggest improvement to the
 > > libgcj build time would be either to remove the libtool shell script
 > > altogether or find some way to reduce its use or make it faster.
 > >
 > 
 > bash is the third-most prominent program in the result of a full 'make
 > bootstrap' oprofile on gcc-4.1-20050424 configured with
 > --disable-checking. A few other breakdowns, including for jc1 and cc1
 > exist on http://www.cs.rice.edu/~scrosby/tmp/GCC/
 > 
 > This was done on machine with 1.5gb of RAM. As the build+src
 > directories add to 1.1gb, there shouldn't have been any disk IO.
 > 
 > (Excerpted out of GCC-overall)
 > 
 > CPU: Athlon, speed 1830.27 MHz (estimated)
 > Counted CPU_CLK_UNHALTED events (Cycles outside of halt state) with a unit mask of 0x00 (No unit mask) count 1800000
 > CPU_CLK_UNHALT...|
 >   samples|      %|
 > ------------------
 >   1571655 35.9269 /tmp/gccbuild/gcc/cc1
 >   1108036 25.3289 /tmp/gccbuild/gcc/jc1
 >    573302 13.1053 /bin/bash
 >    178865  4.0887 /tmp/gccbuild/gcc/cc1plus

There was a difference in the way that I did the measurements, in that
I separated out the shell used by libtool and the shell used for all
other shell scripts.

However, there is another major disparity here, in that on your box
jc1 uses much more cpu than bash.  I don't know why that might be.

Andrew.

Re: GCC 4.1: Buildable on GHz machines only?

[Andrew Haley]

Ian Lance Taylor writes:
 > 
 > And, yes, we clearly need to do something about the libjava build.

OK, I know nothing about libtool so this might not be possible, but
IMO the easiest way of making a dramatic difference is to cease to
compile every file twice, once with PIC and once without.  There would
be a small performance regression for statically linked Java apps, but
in practice Java is very hard to use with static linkage.

Andrew.

Re: GCC 4.1: Buildable on GHz machines only?

[Jakub Jelinek]

On Fri, Apr 29, 2005 at 10:47:06AM +0100, Andrew Haley wrote:
> Ian Lance Taylor writes:
>  > 
>  > And, yes, we clearly need to do something about the libjava build.
> 
> OK, I know nothing about libtool so this might not be possible, but
> IMO the easiest way of making a dramatic difference is to cease to
> compile every file twice, once with PIC and once without.  There would
> be a small performance regression for statically linked Java apps, but
> in practice Java is very hard to use with static linkage.

Definitely.  For -static you either have the choice of linking the
binary with -Wl,--whole-archive for libgcj.a (and likely other Java libs),
or spend a lot of time adding more and more objects that are really
needed, but linker doesn't pick them up.

For the distribution, we simply remove all Java *.a libraries, but it would
be a build time win if we don't have to compile them altogether.

	Jakub

Re: GCC 4.1: Buildable on GHz machines only?

[Lars Segerlund]

 I think Zack summarises very well here, the general consensus is that gcc is slow,
 now if gcc was faster, it would perhaps not be so bad to build java ?

 If we do a reasonable comparison of compile times against the intel compiler or
 the portland group or something similar we consistenly find that gcc is slower
 by a couple of times 1x - 3x, ( this is only my impression, not backed up by
 hard data but should be in the ballpark ).

 The real killer seems to be large memory usage, and I have a hard time believing that
 if you compile fx. 1 meg of source the compiler 'have' to use some 800 megs or 
 something as working memory. ( When speaking of the real killer here I mean for
 old systems ). With all the discussions on cache hit rate and similar criterions
 lately we can't forget that less data higher means hit rate.

 So I am of the opinion that we have to be aware of the problem, and the gcc list has
 a reputation for really ticking of people complaining on the slowness, look at the
 archives, this is something that comes up every two months and the it gets even
 slower. The regular arguments are:

 1. No it's not , ( slow that is ).
 2. It's your hardware/program
 3. All modern computers have a gig or RAM and then it's fast.
 4. I have a fast computer and am not interested.
 5. gcc does so much and and is so good that it's not strange that it takes time.
 6 .

  and so on ... please correlate to the whining on the mailing list archives.

 Anyhow, gcc need's to get faster, and it needs to use less memory, perhaps it
 shouldn't be a goal in itself, but it must be desirable, and 'on' the radar.

  The above are my opinions, feel free to disagree,

 / regards, Lars Segerlund.

On Wed, 27 Apr 2005 16:40:29 -0700
Zack Weinberg <zack@codesourcery.com> wrote:

> Daniel Berlin <dberlin@dberlin.org> writes:
> 
> > On Wed, 2005-04-27 at 15:13 -0700, Stan Shebs wrote:
> >> Steven Bosscher wrote:
> >> >If someone had cared about them, it would have been noticed
> >> >earlier.  But since _nobody_ has complained before you, I guess we
> >> >can conclude that by far the majority if GCC users are quite happy
> >> >with the cost assesments that were made.
> >> >
> >> No, there have been plenty of complaints, but the GCC mailing lists
> >> have, shall we say, a "reputation", and a great many users will not
> >> post to them,
> >
> > I've never in my life heard this from another mailing list, and i
> > contribute to a *great* many open source projects.
> 
> I have seen such complaints.  Not about bootstrap times, no, that only
> affects people who compile the compiler; but the more general case of
> 'gcc takes forever to compile this program' does appear on a regular
> basis.
> 
> I do also think that the amount of ridicule heaped on people who come
> to the gcc lists is, in general, too high.  People should not be
> ridiculed for complaining that the compiler is slow, even if they are
> insisting on using vintage hardware.  It is slow, even on fast
> hardware; it's just easier to see that on slow hardware.  Rather more
> importantly, people should not be ridiculed for submitting bug
> reports, even if they are wrong.  I suspect the bad public image that
> Stan refers to, has more to do with this than anything else.  To be
> fair, it can be hard to explain that someone is wrong without
> belittling them; but it is important to make the effort, particularly
> when the reason they are wrong is esoteric (e.g. any of the legion of
> counterintuitive things in the C standard).
> 
> Some people are worthy of ridicule; those who are trolling, or those
> who are persistently and wilfully clueless.  However, ridicule is
> *not* an effective way of making these people shut up and go away,
> which is the appropriate goal when dealing with them.  It can be fun
> to argue with them, but long argument threads are a drag on the
> mailing list, so we should not make a habit of it.  This is not
> alt.flame.
> 
> > The only person i see ridiculing people frequently happens to be
> > from @apple.com.
> 
> I can think of four people who regularly ridicule posters.  Only two
> of them are Apple employees.
> 
> zw

Re: 'make bootstrap' oprofile (13% on bash?)

[Scott A Crosby]

On Fri, 29 Apr 2005 10:43:57 +0100, Andrew Haley <aph@redhat.com> writes:

> Scott A Crosby writes:
>  > On Thu, 28 Apr 2005 10:29:32 +0100, Andrew Haley <aph@redhat.com> writes:
>  > 
>  > > Having said that, I suspect that the single biggest improvement to the
>  > > libgcj build time would be either to remove the libtool shell script
>  > > altogether or find some way to reduce its use or make it faster.
>  > >
>  > 
>  > bash is the third-most prominent program in the result of a full 'make
>  > bootstrap' oprofile on gcc-4.1-20050424 configured with
>  > --disable-checking. A few other breakdowns, including for jc1 and cc1
>  > exist on http://www.cs.rice.edu/~scrosby/tmp/GCC/

> There was a difference in the way that I did the measurements, in that
> I separated out the shell used by libtool and the shell used for all
> other shell scripts.

If there is interest and you tell me how, I can do this.

> However, there is another major disparity here, in that on your box
> jc1 uses much more cpu than bash.  I don't know why that might be.

I oprofiled a full bootstrap and you oprofiled just building libgcj?

Scott

Re: 'make bootstrap' oprofile (13% on bash?)

[Andrew Haley]

Scott A Crosby writes:
 > On Fri, 29 Apr 2005 10:43:57 +0100, Andrew Haley <aph@redhat.com> writes:
 > 
 > > However, there is another major disparity here, in that on your box
 > > jc1 uses much more cpu than bash.  I don't know why that might be.
 > 
 > I oprofiled a full bootstrap and you oprofiled just building libgcj?

That should be the opposite way round.  libgcj is the only place that
runs jc1, so jc1 should be a *smaller* part of a fill bootstrap.

Andrew.

Re: GCC 4.1: Buildable on GHz machines only?

[Daniel Jacobowitz]

On Fri, Apr 29, 2005 at 12:49:37PM +0200, Lars Segerlund wrote:
>  If we do a reasonable comparison of compile times against the intel compiler or
>  the portland group or something similar we consistenly find that gcc is slower
>  by a couple of times 1x - 3x, ( this is only my impression, not backed up by
>  hard data but should be in the ballpark ).

Please don't add additional speculation to this already messy subject. 
Feel free to come back with data.

>  The real killer seems to be large memory usage, and I have a hard time believing that
>  if you compile fx. 1 meg of source the compiler 'have' to use some 800 megs or 
>  something as working memory. ( When speaking of the real killer here I mean for
>  old systems ). With all the discussions on cache hit rate and similar criterions
>  lately we can't forget that less data higher means hit rate.

Same here.  You've shown pretty clearly that you haven't looked at what
GCC does with its memory usage.  Yes, a lot of it is wasted, but a lot
of that is constant factors (e.g. structures that are wastefully
large), not things which would affect a non-linear blowup.  Do you
think it adds any value to GCC development to shout "please think about
this problem" without any concrete suggestions?

-- 
Daniel Jacobowitz
CodeSourcery, LLC

Re: GCC 4.1: Buildable on GHz machines only?

[Ian Lance Taylor]

Andrew Haley <aph@redhat.com> writes:

> Ian Lance Taylor writes:
>  > 
>  > And, yes, we clearly need to do something about the libjava build.
> 
> OK, I know nothing about libtool so this might not be possible, but
> IMO the easiest way of making a dramatic difference is to cease to
> compile every file twice, once with PIC and once without.  There would
> be a small performance regression for statically linked Java apps, but
> in practice Java is very hard to use with static linkage.

Try putting

enable_shared=no

in configure.ac somewhere before AC_PROG_LIBTOOL.

Ian

Re: GCC 4.1: Buildable on GHz machines only?

[Ian Lance Taylor]

Andreas Schwab <schwab@suse.de> writes:

> > Try putting
> >
> > enable_shared=no
> >
> > in configure.ac somewhere before AC_PROG_LIBTOOL.
> 
> I think you rather want AC_DISABLE_STATIC.

Even better.  Thanks.

Ian

Re: GCC 4.1: Buildable on GHz machines only?

[Andreas Schwab]

Ian Lance Taylor <ian@airs.com> writes:

> Andrew Haley <aph@redhat.com> writes:
>
>> Ian Lance Taylor writes:
>>  > 
>>  > And, yes, we clearly need to do something about the libjava build.
>> 
>> OK, I know nothing about libtool so this might not be possible, but
>> IMO the easiest way of making a dramatic difference is to cease to
>> compile every file twice, once with PIC and once without.  There would
>> be a small performance regression for statically linked Java apps, but
>> in practice Java is very hard to use with static linkage.
>
> Try putting
>
> enable_shared=no
>
> in configure.ac somewhere before AC_PROG_LIBTOOL.

I think you rather want AC_DISABLE_STATIC.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: GCC 4.1: Buildable on GHz machines only?

[Ian Lance Taylor]

Jason Thorpe <thorpej@shagadelic.org> writes:

> Except it's not just bootstrapping GCC.  It's everything.  When the
> NetBSD Project switched from 2.95.3 to 3.3, we had a noticeably
> increase in time to do the "daily" builds because the 3.3 compiler
> was so much slower at compiling the same OS source code.  And we're
> talking almost entirely C code, here.

Well, there are two different issues.  Matt was originally talking
about bootstrap time, at least that is how I took it.  You are talking
about speed of compilation.  The issues are not unrelated, but they
are not the same.

The gcc developers have done a lot of work on speeding up the compiler
for 3.4 and 4.0, with some success.  On many specific test cases, 4.0
is faster than 3.3 and even 2.95.  The way to help this process along
is to report bugs at http://gcc.gnu.org/bugzilla.

In particular, if you provide a set of preprocessed .i files, from,
say, sys, libc, or libcrypto, whichever seems worst, and open a gcc PR
about them, that would be a great baseline for measuring speed of
compilation, in a way that particularly matters to NetBSD developers.

Ian

Re: GCC 4.1: Buildable on GHz machines only?

[Diego Novillo]

On Fri, Apr 29, 2005 at 12:05:03PM -0400, Ian Lance Taylor wrote:

> The way to help this process along is to report bugs at
> http://gcc.gnu.org/bugzilla.
> 
> In particular, if you provide a set of preprocessed .i files,
> from, say, sys, libc, or libcrypto, whichever seems worst, and
> open a gcc PR about them, that would be a great baseline for
> measuring speed of compilation, in a way that particularly
> matters to NetBSD developers.
> 
Yes, .i/.ii files in bugzilla help enormously.  What I and other
have done is harvest these pre-processed files from bugzilla and
stick them in our local test directories.

When I add new features or fix bugs, I will run the clean and
patched compiler through those files to make sure nothing has
regressed significantly (or even better, it's progressed).

That's at least a good smoke screen test.  But having
preprocessed code attached to a bugzilla PR is of absolutely
essential.  Without that, we won't be able to fix the problem.

This is essentially how we fixed all the memory and performance
problems in DLV (PR 8361).  Gerald is also kind enough to poke us
every time we make 8361 rear its ugly head again.


Diego.

Re: GCC 4.1: Buildable on GHz machines only?

[Joe Buck]

On Thu, Apr 28, 2005 at 10:05:13PM -0700, Jason Thorpe wrote:
> Except it's not just bootstrapping GCC.  It's everything.  When the  
> NetBSD Project switched from 2.95.3 to 3.3, we had a noticeably  
> increase in time to do the "daily" builds because the 3.3 compiler  
> was so much slower at compiling the same OS source code.  And we're  
> talking almost entirely C code, here.

We are well aware that 3.3 was slower, but we believe that 4.0 is in
many cases significantly faster than 3.3.  Try it and see what you think.

Re: GCC 4.1: Buildable on GHz machines only?

[Andrew Haley]

Andreas Schwab writes:
 > Ian Lance Taylor <ian@airs.com> writes:
 > 
 > > Andrew Haley <aph@redhat.com> writes:
 > >
 > >> Ian Lance Taylor writes:
 > >>  > 
 > >>  > And, yes, we clearly need to do something about the libjava build.
 > >> 
 > >> OK, I know nothing about libtool so this might not be possible, but
 > >> IMO the easiest way of making a dramatic difference is to cease to
 > >> compile every file twice, once with PIC and once without.  There would
 > >> be a small performance regression for statically linked Java apps, but
 > >> in practice Java is very hard to use with static linkage.
 > >
 > > Try putting
 > >
 > > enable_shared=no
 > >
 > > in configure.ac somewhere before AC_PROG_LIBTOOL.
 > 
 > I think you rather want AC_DISABLE_STATIC.

Won't that disable static libraries?  I don't want to do that.

Andrew.

Re: GCC 4.1: Buildable on GHz machines only?

[Ian Lance Taylor]

Andrew Haley <aph@redhat.com> writes:

>  > >> OK, I know nothing about libtool so this might not be possible, but
>  > >> IMO the easiest way of making a dramatic difference is to cease to
>  > >> compile every file twice, once with PIC and once without.  There would
>  > >> be a small performance regression for statically linked Java apps, but
>  > >> in practice Java is very hard to use with static linkage.
>  > >
>  > > Try putting
>  > >
>  > > enable_shared=no
>  > >
>  > > in configure.ac somewhere before AC_PROG_LIBTOOL.
>  > 
>  > I think you rather want AC_DISABLE_STATIC.
> 
> Won't that disable static libraries?  I don't want to do that.

Sorry, my misunderstanding.

I don't know of a way to tell libtool to not do duplicate compiles.
You can use -prefer-pic, but at least from looking at the script it
will still compile twice, albeit with -fPIC both times.

Incidentally, at least on my system I don't think this will make a
dramatic difference.  The incredibly slow parts of building libjava
all seem to have to do with building the .a and .so files, both of
which tend to cause my system to start swapping.  While it would
obviously help to not build the objects twice, that doesn't seem to be
the major time sink, at least not for me.

Ian

Re: GCC 4.1: Buildable on GHz machines only?

[Joe Buck]

On Fri, Apr 29, 2005 at 01:30:13PM -0400, Ian Lance Taylor wrote:
> I don't know of a way to tell libtool to not do duplicate compiles.
> You can use -prefer-pic, but at least from looking at the script it
> will still compile twice, albeit with -fPIC both times.
> 
> Incidentally, at least on my system I don't think this will make a
> dramatic difference.  The incredibly slow parts of building libjava
> all seem to have to do with building the .a and .so files, both of
> which tend to cause my system to start swapping.

It seems that this is a binutils issue.  Particularly in the case of .a
files, it is puzzling why creating them is so resource-intensive.

Re: GCC 4.1: Buildable on GHz machines only?

[Andrew Haley]

Ian Lance Taylor writes:
 > Andrew Haley <aph@redhat.com> writes:
 > 
 > >  > >> OK, I know nothing about libtool so this might not be possible, but
 > >  > >> IMO the easiest way of making a dramatic difference is to cease to
 > >  > >> compile every file twice, once with PIC and once without.  There would
 > >  > >> be a small performance regression for statically linked Java apps, but
 > >  > >> in practice Java is very hard to use with static linkage.
 > >  > >
 > >  > > Try putting
 > >  > >
 > >  > > enable_shared=no
 > >  > >
 > >  > > in configure.ac somewhere before AC_PROG_LIBTOOL.
 > >  > 
 > >  > I think you rather want AC_DISABLE_STATIC.
 > > 
 > > Won't that disable static libraries?  I don't want to do that.
 > 
 > Sorry, my misunderstanding.
 > 
 > I don't know of a way to tell libtool to not do duplicate compiles.
 > You can use -prefer-pic, but at least from looking at the script it
 > will still compile twice, albeit with -fPIC both times.
 > 
 > Incidentally, at least on my system I don't think this will make a
 > dramatic difference.  The incredibly slow parts of building libjava
 > all seem to have to do with building the .a and .so files, both of
 > which tend to cause my system to start swapping.

IME the biggest thing there is the libtool shell script.  

 > While it would obviously help to not build the objects twice, that
 > doesn't seem to be the major time sink, at least not for me.

That's not the problem I have: the link is slow, but it's not the
dominant thing.  Maybe it depends how much RAM you have?  I have 1 Gbyte.

Andrew.

Re: GCC 4.1: Buildable on GHz machines only?

[Richard Henderson]

On Fri, Apr 29, 2005 at 01:30:13PM -0400, Ian Lance Taylor wrote:
> I don't know of a way to tell libtool to not do duplicate compiles.
> You can use -prefer-pic, but at least from looking at the script it
> will still compile twice, albeit with -fPIC both times.

Incidentally, libtool does not compile things twice when you use
convenience libraries.  And the entirity of libgcj is built via
a convenience library these days.

So we are not, in fact, building everything twice.



r~

Re: GCC 4.1: Buildable on GHz machines only?

[Stan Shebs]

Joe Buck wrote:

>On Wed, Apr 27, 2005 at 03:13:21PM -0700, Stan Shebs wrote:
>
>>No, there have been plenty of complaints, but the GCC mailing
>>lists have, shall we say, a "reputation", and a great many
>>users will not post to them, either for fear of being ridiculed,
>>or in the expection that they will not be heard. (Everything is
>>archived, and they can see what happens to others.)
>>
>
>Oh, come on, Stan.  Compared to what?  The GCC lists are quite civil
>compared to many free software developer lists.  I'd say criticism
>of proposals is harsher on the Linux kernel list (though most of that
>criticism is fair).  And if you *really* want to see a hostile
>environment, try debian-devel.
>
Note that I said "fear of", which may very well be unwarranted,
and "ridiculed" is not quite the word I wanted. Sometimes the
critiquing of a user's message is factually accurate, but piled
on, where GCCers point out all the faults in the poster's message,
presumably on the offchance that one of those is relevant. But
even though this may be the logical thing to do, the effect can
be withering; more than a few times an angry user has bent my
ear on how they felt belittled in the process. (My response
was of course to make soothing sounds and sell them a Cygnus
support contract. :-) )

Mind you, I'm not saying this is a wrong way to answer. I was
just reacting to the fallacy that the lack of complaints
means that everybody is happy; it can also mean that the
unhappy people are not willing to speak up for some reason.
(And I note that all of the hostile posts in this thread
are now permanently archived and soon to be Google-indexed,
for the future convenience of people considering whether they
want to run that gauntlet themselves.)

Stan

Re: GCC 4.1: Buildable on GHz machines only?

[Ian Lance Taylor]

Andrew Haley <aph@redhat.com> writes:

> That's not the problem I have: the link is slow, but it's not the
> dominant thing.  Maybe it depends how much RAM you have?  I have 1 Gbyte.

Probably.  I pay for my own machines, and it's three years old, and I
have 256M.

Ian

Re: GCC 4.1: Buildable on GHz machines only?

[Joe Buck]

On Fri, Apr 29, 2005 at 05:41:30PM -0400, Ian Lance Taylor wrote:
> Andrew Haley <aph@redhat.com> writes:
> 
> > That's not the problem I have: the link is slow, but it's not the
> > dominant thing.  Maybe it depends how much RAM you have?  I have 1 Gbyte.
> 
> Probably.  I pay for my own machines, and it's three years old, and I
> have 256M.

I think you need to talk to the binutils people.  It should be possible
to make ar and ld more memory-efficient.

Re: GCC 4.1: Buildable on GHz machines only?

[Richard Henderson]

On Fri, Apr 29, 2005 at 03:22:59PM -0700, Joe Buck wrote:
> I think you need to talk to the binutils people.  It should be possible
> to make ar and ld more memory-efficient.

For ld, at least, --no-keep-memory.  Normally it makes things run
slower, but that may not actually be the case for libjava.


r~

Re: GCC 4.1: Buildable on GHz machines only?

[Hugh Sasse]

On Fri, 29 Apr 2005, Richard Henderson wrote:

> For ld, at least, --no-keep-memory.  Normally it makes things run
> slower, but that may not actually be the case for libjava.

Another suggestion, even though my last one sank without trace :-)
There is a lot to read through to find out things like this, so I
think it would be helpful if there were options for configure to
support older machines.  I'm imagining things like
--slow-cpu-heuristics, --small-memory-heuristics,
--slow-disk-heuristics.  They could then bring options on linkers
and so forth into play, which could tune the build performance.
They'd be imperfect, but they could help.  I see this as being in
the spirit of `gmake bootstrap-lean`

I've just spent the best part of a week trying to get a recent-ish
compiler on an old Solaris-2.5.1 system.  It takes time to explore
the options.

         Thank you,
         Hugh

Re: GCC 4.1: Buildable on GHz machines only?

[Matt Thomas]

Joe Buck wrote:
> I think you need to talk to the binutils people.  It should be possible
> to make ar and ld more memory-efficient.

Even though systems maybe demand paged, having super large libraries
that consume lots of address space can be a problem.

I'd like to libjava be split into multiple shared libraries.
In C, we have libc, libm, libpthread, etc.  In X11, there's X11, Xt, etc.
So why does java have everything in one shared library?  Could
the swing stuff be moved to its own?  Are there other logical
divisions?

Unlike other modern systems with a two level page table structure,
the VAX uses a single page table of indirection.  This greatly reduces
the amount of address space a process can efficiently use.  If there
are components that will not be needed by some java programs, it would
nice if they could be separated into their shared libraries.
-- 
Matt Thomas                     email: matt@3am-software.com
3am Software Foundry              www: http://3am-software.com/bio/matt/
Cupertino, CA              disclaimer: I avow all knowledge of this message.

Re: GCC 4.1: Buildable on GHz machines only?

[David Daney]

Matt Thomas wrote:
> I'd like to libjava be split into multiple shared libraries.
> In C, we have libc, libm, libpthread, etc.  In X11, there's X11, Xt, etc.
> So why does java have everything in one shared library?  Could
> the swing stuff be moved to its own?  Are there other logical
> divisions?
> 
> Unlike other modern systems with a two level page table structure,
> the VAX uses a single page table of indirection.  This greatly reduces
> the amount of address space a process can efficiently use.  If there
> are components that will not be needed by some java programs, it would
> nice if they could be separated into their shared libraries.

This is oft requested.  Perhaps we can do something like this for 4.1. 
Patches are of course welcome.

David Daney.

Re: GCC 4.1: Buildable on GHz machines only?

[Ian Lance Taylor]

Joe Buck <Joe.Buck@synopsys.COM> writes:

> I think you need to talk to the binutils people.  It should be possible
> to make ar and ld more memory-efficient.

And here I thought I had dug myself out of that swamp several years
ago.  I once spent a month making the linker more memory efficient,
including adding the --no-keep-memory option, so that it would work on
the FSF's clunky old HP300 systems.

Ian

Re: GCC 4.1: Buildable on GHz machines only?

[Joe Buck]

On Fri, Apr 29, 2005 at 08:54:00PM -0400, Ian Lance Taylor wrote:
> Joe Buck <Joe.Buck@synopsys.COM> writes:
> 
> > I think you need to talk to the binutils people.  It should be possible
> > to make ar and ld more memory-efficient.
> 
> And here I thought I had dug myself out of that swamp several years
> ago.  I once spent a month making the linker more memory efficient,
> including adding the --no-keep-memory option, so that it would work on
> the FSF's clunky old HP300 systems.

Yes, Richard mentioned that option.  Next step is to use it in gcc builds
and see if it helps.

I've seen claims that Darwin's linker is much more efficient than the
GNU linker, though I haven't confirmed this.

Re: GCC 4.1: Buildable on GHz machines only?

[Andrew Haley]

Richard Henderson writes:
 > On Fri, Apr 29, 2005 at 01:30:13PM -0400, Ian Lance Taylor wrote:
 > > I don't know of a way to tell libtool to not do duplicate compiles.
 > > You can use -prefer-pic, but at least from looking at the script it
 > > will still compile twice, albeit with -fPIC both times.
 > 
 > Incidentally, libtool does not compile things twice when you use
 > convenience libraries.  And the entirity of libgcj is built via
 > a convenience library these days.
 > 
 > So we are not, in fact, building everything twice.

You know, I didn't even think to check, and you're absolutely right.

OK, so the low-hanging fruit that remains is the libtools script and
the linker.  In the latter case, it seems that the big link causes
severe problems with small memory systems.

Andrew.

Re: GCC 4.1: Buildable on GHz machines only?

[Andrew Haley]

Matt Thomas writes:
 > Joe Buck wrote:
 > > I think you need to talk to the binutils people.  It should be possible
 > > to make ar and ld more memory-efficient.
 > 
 > Even though systems maybe demand paged, having super large
 > libraries that consume lots of address space can be a problem.
 > 
 > I'd like to libjava be split into multiple shared libraries.  In C,
 > we have libc, libm, libpthread, etc.  In X11, there's X11, Xt, etc.
 > So why does java have everything in one shared library?  Could the
 > swing stuff be moved to its own?  Are there other logical
 > divisions?

It might be nice, certainly.  However, there are some surprising
dependencies between parts of the Java library, and these would cause
separate shared libraries to depend on each other, negating most of
the advantage of separation.

We are in the process of rewriting the Java ABI so that sumbol
resolution in libraries is done lazily rather than eagerly.  This will
help.  Even so, I would prefer to divide libjava -- if it is to be
divided -- on a logical basis rather than simply in order to make
libraries smaller.

Andrew.

Re: GCC 4.1: Buildable on GHz machines only?

[Giovanni Bajo]

Lars Segerlund <lars.segerlund@comsys.se> wrote:

>  I have to agree with Richard's assessment, gcc is currently on the
>  verge of being unusable in many instances.
>  If you have a lot of software to build and have to do complete
>  rebuilds it's painful, the binutils guys have a 3x speedup patch
>  coming up, but every time there is a speedup it gets eaten up.

This is simply not true. Most of the benchmarks we have seen posted to the gcc
mailing lists show that GCC4 can be much faster than GCC3 (especially on C++
code). There are of course also regressions, and we are not trying to hide
them.

Did you *ever* provide a preprocessed source code which shows a compile-time
regression? If not, please do NOT hesitate! Post it in Bugzilla. People that
did it in the past are mostly satisfied by how GCC developers improved things
for them.

Otherwise, I do not want to sound rude, but your posts seem more like trolling
to me. I am *ready* to admit that GCC4 is much slower than GCC3 or GCC2, but I
would like to do this in front of real measurable data, not just random
complaints and told legends. Thus, I am really awaiting your preprocessed
testcases which prove your points.

Please.

Giovanni Bajo

Re: GCC 4.1: Buildable on GHz machines only?

[Giovanni Bajo]

Jason Thorpe <thorpej@shagadelic.org> wrote:

>> Maybe the older platform should stick to the older compiler then,
>> if it is too slow to support the kind of compiler that modern
>> systems need.
>
> This is an unreasonable request.  Consider NetBSD, which runs on new
> and old hardware.  The OS continues to evolve, and that often
> requires adopting newer compilers (so e.g. other language features
> can be used in the base OS).
>
> The GCC performance issue is not new.  It seems to come up every so
> often... last time I recall a discussion on the topic, it was thought
> that the new memory allocator (needed for pch) was cause cache-thrash
> (what was the resolution of that discussion, anyway?)


There is no outcome, because it is just the Nth legend. Like people say "I
believe GCC is slow because of pointer indirection" or stuff like that.

Please, provide preprocessed sources and we *will* analyze them. Just file a
bugreport in Bugzilla, it took 10 minutes of your time.

Giovanni Bajo

Re: GCC 4.1: Buildable on GHz machines only?

[Giovanni Bajo]

Richard Earnshaw <rearnsha@gcc.gnu.org> wrote:

>> The GCC build times are not unreasonable compared to other,
>> commercial compilers with similar functionality.  And the GCC
>> developers
>> ave plans to address inefficiencies -- GCC 4.0 often is faster than
>> GCC
>> 3.4.
>
> If you are going to make sweeping statements like this you need to
> back them up with hard data.


It is surely faster for C++ code thanks to the work done by Codesourcery on the
lexing upfront and the name lookup. "Faster" here means 20-30% faster, so it's
not 1% or something like that.

There are many posts on gcc@ that show this, I can dig them up in the archive
for you if you want, but I'm sure you can use google as well as I do. Two of
them are very recente (see Karel Gardas' post on this thread, and the recent
benchmark posted by Rene Rebe).

Giovanni Bajo

Re: GCC 4.1: Buildable on GHz machines only?

[Giovanni Bajo]

Ian Lance Taylor <ian@airs.com> wrote:

>> Except it's not just bootstrapping GCC.  It's everything.  When the
>> NetBSD Project switched from 2.95.3 to 3.3, we had a noticeably
>> increase in time to do the "daily" builds because the 3.3 compiler
>> was so much slower at compiling the same OS source code.  And we're
>> talking almost entirely C code, here.
>
> Well, there are two different issues.  Matt was originally talking
> about bootstrap time, at least that is how I took it.  You are talking
> about speed of compilation.  The issues are not unrelated, but they
> are not the same.
>
> The gcc developers have done a lot of work on speeding up the compiler
> for 3.4 and 4.0, with some success.  On many specific test cases, 4.0
> is faster than 3.3 and even 2.95.  The way to help this process along
> is to report bugs at http://gcc.gnu.org/bugzilla.
>
> In particular, if you provide a set of preprocessed .i files, from,
> say, sys, libc, or libcrypto, whichever seems worst, and open a gcc PR
> about them, that would be a great baseline for measuring speed of
> compilation, in a way that particularly matters to NetBSD developers.

I would also like to note that I *myself* requested preprocessed source code to
NetBSD developers at least 6 times in the past 2 years. I am sure Andrew Pinski
did too, a comparable amound of times. These requests, as far as I can
understand, were never answered. This also helped building up a stereotype of
the average NetBSD developer being "just a GCC whine troll".

I am sure this is *far* from true, but I would love to see NetBSD developers
*collaborating* with us, especially since what we are asking (filing bug
reports with preprocessed sources) cannot take more than 1-2 hours of their
time.

Giovanni Bajo

Re: GCC 4.1: Buildable on GHz machines only?

[Nix]

On 30 Apr 2005, Giovanni Bajo uttered the following:
> There is no outcome, because it is just the Nth legend. Like people say "I
> believe GCC is slow because of pointer indirection" or stuff like that.

Don't we have actual *evidence* that it's slow because of cache
thrashing?

-- 
`End users are just test loads for verifying that the system works, kind of
 like resistors in an electrical circuit.' - Kaz Kylheku in c.o.l.d.s

Re: GCC 4.1: Buildable on GHz machines only?

[Jason Thorpe]

On Apr 30, 2005, at 12:33 PM, Giovanni Bajo wrote:

> I would also like to note that I *myself* requested preprocessed  
> source code to
> NetBSD developers at least 6 times in the past 2 years. I am sure  
> Andrew Pinski
> did too, a comparable amound of times. These requests, as far as I can
> understand, were never answered. This also helped building up a  
> stereotype of
> the average NetBSD developer being "just a GCC whine troll".

While I have not had much time for a quite a while to work on GCC  
myself, I am listed as NetBSD maintainer... you can always drop me a  
note directly when this sort of thing happens.

-- thorpej

libjava build times

[Richard Henderson]

[-- Attachment #1: Type: text/plain, Size: 1389 bytes --]

On Sat, Apr 30, 2005 at 10:33:43AM +0100, Andrew Haley wrote:
> OK, so the low-hanging fruit that remains is the libtools script and
> the linker.  In the latter case, it seems that the big link causes
> severe problems with small memory systems.

I did some experiments today just to see what kind of time it actually
takes to compile the actual objects, and thus how much time is on the
table to be retrieved from libtool.

The following was performed on a 2.3Ghz G5, with 2G of ram.  So I'm
not swapping and in fact everything can reside in cache.  I.e. just
about the ideal setup.  There are in fact two cpus, but I'm not using
the -j option to make at all.

I began by building the whole of libjava, and then using find to 
delete all of *.o *.lo *.a *.la.  I then timed rebuilding the library:

  2248.43user 661.42system 47:46.01elapsed 101%CPU (0major+47501310minor)

Next, I canibalized the makefile in order to bypass libtool, and invoke
gcc directly.  My solution does assume gnu ld and ar, but this is just
a test after all.

  -O2 -fPIC compile
  824.80user 86.88system 15:11.69elapsed 99%CPU (0major+7102491minor)

  .so link + .a create
  10.45user 9.59system 0:19.97elapsed 100%CPU (0major+851815minor)

Now, unless I've done something drastically wrong, it appears as if we
are spending 2/3 of our time in the libtool script.

Test makefiles attached for the record.


r~

[-- Attachment #2: test-make.tar.gz --]
[-- Type: application/x-gzip, Size: 23678 bytes --]

Re: libjava build times

[Andrew Haley]

Richard Henderson writes:
 > 
 > Now, unless I've done something drastically wrong, it appears as if we
 > are spending 2/3 of our time in the libtool script.

Yes, that's right.  That's similar to what my oprofile experiments suggest.

Andrew.

Re: libjava build times

[Thorsten Glaser]

Andrew Haley dixit:

>Richard Henderson writes:
> > 
> > Now, unless I've done something drastically wrong, it appears as if we
> > are spending 2/3 of our time in the libtool script.
>
>Yes, that's right.  That's similar to what my oprofile experiments suggest.

Could you please go to http://wiki.mirbsd.de/MirbsdKsh, get the source,
compile it and try with it instead of your usual /bin/sh (I suppose GNU
bash) again?

I'd be interested if that warrants a noticeable speedup. I've done only
a few comparisions regarding string handling, and ksh both used less
memory and was by times faster. Also, GNU bash started getting MUCH
slower at 16 KiB strings, while ksh was linear until 256 KiB strings.

Thanks,
//mirabile
-- 
> Hi, does anyone sell openbsd stickers by themselves and not packaged
> with other products?
No, the only way I've seen them sold is for $40 with a free OpenBSD CD.
	-- Haroon Khalid and Steve Shockley in gmane.os.openbsd.misc

Re: GCC 4.1: Buildable on GHz machines only?

[Kai Henningsen]

aph@redhat.com (Andrew Haley)  wrote on 30.04.05 in <17011.21540.530634.763156@cuddles.cambridge.redhat.com>:

> Matt Thomas writes:
>  > Joe Buck wrote:
>  > > I think you need to talk to the binutils people.  It should be possible
>  > > to make ar and ld more memory-efficient.
>  >
>  > Even though systems maybe demand paged, having super large
>  > libraries that consume lots of address space can be a problem.
>  >
>  > I'd like to libjava be split into multiple shared libraries.  In C,
>  > we have libc, libm, libpthread, etc.  In X11, there's X11, Xt, etc.
>  > So why does java have everything in one shared library?  Could the
>  > swing stuff be moved to its own?  Are there other logical
>  > divisions?
>
> It might be nice, certainly.  However, there are some surprising
> dependencies between parts of the Java library, and these would cause
> separate shared libraries to depend on each other, negating most of
> the advantage of separation.
>
> We are in the process of rewriting the Java ABI so that sumbol
> resolution in libraries is done lazily rather than eagerly.  This will
> help.  Even so, I would prefer to divide libjava -- if it is to be
> divided -- on a logical basis rather than simply in order to make
> libraries smaller.

Surely the other mentioned library divisions (libc, X) were *also* done on  
a logical basis?!

MfG Kai

Re: GCC 4.1: Buildable on GHz machines only?

[Giovanni Bajo]

Jason Thorpe <thorpej@shagadelic.org> wrote:

>> I would also like to note that I *myself* requested preprocessed
>> source code to
>> NetBSD developers at least 6 times in the past 2 years. I am sure
>> Andrew Pinski
>> did too, a comparable amound of times. These requests, as far as I
>> can understand, were never answered. This also helped building up a
>> stereotype of
>> the average NetBSD developer being "just a GCC whine troll".
>
> While I have not had much time for a quite a while to work on GCC
> myself, I am listed as NetBSD maintainer... you can always drop me a
> note directly when this sort of thing happens.


Thanks! Are you then going to file in Bugzilla some preprocessed sources that
show the 2.95 -> 3.3 slowdown experimented by NetBSD folks?

Giovanni Bajo

Re: libjava build times

[Richard Henderson]

On Sun, May 01, 2005 at 10:31:05PM +0000, Thorsten Glaser wrote:
> Could you please go to http://wiki.mirbsd.de/MirbsdKsh, get the source,
> compile it and try with it instead of your usual /bin/sh (I suppose GNU
> bash) again?
> 
> I'd be interested if that warrants a noticeable speedup.

No visible change.

2105.44user 1003.09system 49:58.59elapsed 103%CPU (3major+54738718minor)


r~

Re: GCC 4.1: Buildable on GHz machines only?

[Marc Espie]

In article <17007.61667.295386.348123@gargle.gargle.HOWL> you write:
>The alternative of course is to do only crossbuilds.  Is it reasonable
>to say that, for platforms where a bootstrap is no longer feasible, a
>successful crossbuild is an acceptable test procedure to use instead?

No.

I've been playing enough with crossbuilds to know that a crossbuild will
show you bugs that do not exist in native builds, and VICE-VERSA.

Building a full system natively, compiler-included, is still one of the
best stress-test for an operating system.

This mind frame, that because the compiler is too slow, it's acceptable
to do cross-builds, is killing older systems.  Very quickly, you end up
with fast, robust systems that are heavily tested through the build of
lots of software, and with slow, untested systems that never see a
build, and are only tested casually by people running a handful of
specialized applications on them.

I'm speaking from experience: you wouldn't believe how many bugs we
tracked and fixed in OpenBSD on fringe platforms (arm, sparc64) simply
because we do native builds and see stuff people doing cross-builds
don't see.

This is not even the first time I talk about this on this list.
Except for embedded systems where memory and disk space don't make it
practical to compile anything natively, having a compiler so slow that
it makes it impossible to compile stuff natively kills old platforms.

Do you know why GCC4 is deprecated on sparc-openbsd ? It's simply
because no-one so far has been able to dedicate the CPU time to track
down the few bugs that prevented us from switching to gcc 3.x from 2.95.

That's right, I said CPU-time. It takes too long to bootstrap the compiler,
it takes too long to rebuild the whole system. And thus, it rots.

Re: GCC 4.1: Buildable on GHz machines only?

[Marc Espie]

How about replacing that piece of junk that is called libtool with
something else ?

Preferably something that works.

Between it's really poor quoting capabitilities, and the fact that
half the tests are done at configure time, and half the tests are done
at run-time, libtool is really poor engineering.   It's really atrocious
when you see operating systems tests all over the place *in the libtool
script* and not in the configure process in the first place.

Heck, last time I even tried to figure out some specs for libtool
options from  the script, I nearly went mad.

It won't be any use for GCC, but I ought to tell you that the OpenBSD
folks are seriously considering replacing libtool entirely with a
home-made perl script that would ONLY handle libtool stuff on OpenBSD
and nowhere else.  Between the fact that the description is too
low-level (very hard to move libraries around, or -L stuff that pops up
in the wrong order all the time and gets you to link with the wrong
version of the library), and that some of the assertions it makes are
downright bogus (hardcoding -R even when it's not needed, or being real
nosy about the C compiler in the wrong way and assuming that the default
set of libraries without options will be the same one as the set with
-fpic), it's getting to the point where it would be a real gain to just
reverse-engineer its features and rewrite it from scratch.

Re: GCC 4.1: Buildable on GHz machines only?

[Peter O'Gorman]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Marc Espie wrote:
| How about replacing that piece of junk that is called libtool with
| something else ?
|
| Preferably something that works.

<bug-libtool@gnu.org>
<libtool-patches@gnu.org>

I will be happy to see your bug reports and/or patches. Whining on the gcc
list has never been known to fix a libtool bug.

Peter
- --
Peter O'Gorman - http://www.pogma.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.0 (Darwin)

iQCVAwUBQnY4v7iDAg3OZTLPAQI5wQP8Dwy/k5Oqx0LdWwrz1Yc+CafsVhlZ20sR
QzETUpZPIMyEWniL37/Sw0eu6PIKjOMZaOaHryvORRD9gparbc9fGtKxBWWmWmXX
AM3xUSqT6Uz2g//yzv/Kcly06q+T5n3i3D0esRNTlTrDV7baRp1BzylJx3GZP9Hl
RifpBmEMtko=
=O/yp
-----END PGP SIGNATURE-----

Re: GCC 4.1: Buildable on GHz machines only?

[Daniel Berlin]

> Do you know why GCC4 is deprecated on sparc-openbsd ? It's simply
> because no-one so far has been able to dedicate the CPU time to track
> down the few bugs that prevented us from switching to gcc 3.x from 2.95.
> 
> That's right, I said CPU-time. It takes too long to bootstrap the compiler,
> it takes too long to rebuild the whole system. And thus, it rots.
> 
-bash-2.05b$ perl userlookup.pl "%espie%"
Name:espie@schutzenberger.liafa.jussieu.fr      ID:569

mysql> select COUNT(*) from bugs where reporter = 569;
+----------+
| COUNT(*) |
+----------+
|        0 |
+----------+
1 row in set (0.06 sec)

mysql> select COUNT(*) from longdescs where who = 569;
+----------+
| COUNT(*) |
+----------+
|        1 |
+----------+
1 row in set (0.00 sec)

mysql> select COUNT(*) from attachments where submitter_id = 569;
+----------+
| COUNT(*) |
+----------+
|        0 |
+----------+
1 row in set (3.59 sec)

Re: GCC 4.1: Buildable on GHz machines only?

[Joe Buck]

On Mon, May 02, 2005 at 11:27:12PM +0900, Peter O'Gorman wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Marc Espie wrote:
> | How about replacing that piece of junk that is called libtool with
> | something else ?
> |
> | Preferably something that works.
> 
> <bug-libtool@gnu.org>
> <libtool-patches@gnu.org>
> 
> I will be happy to see your bug reports and/or patches. Whining on the gcc
> list has never been known to fix a libtool bug.

I've just submitted Richard Henderson's recent data, showing that 2/3
of the time spent in building libjava is wasted in libtool, along with
detailed logs to back this up, to bug-libtool.  We'll all welcome any
improvements the libtool developers can make.

Re: GCC 4.1: Buildable on GHz machines only?

[Mike Stump]

On Apr 29, 2005, at 6:03 PM, Joe Buck wrote:
> I've seen claims that Darwin's linker is much more efficient than the
> GNU linker, though I haven't confirmed this.

:-)  I have a vague recollection this is true (32=bit only).

If someone wants to post linux numbers and the command, I'll redo on  
my box.

make all && rm */libjava/? && time make all-target-libjava

type of thing.  Better would be someone that has both Yellow dog and  
Darwin on the same hardware, but...  don't know if we'll find that  
person.

[ pause ]  Ok, here is what I get for mainline 20050428:

$ rm powerpc-apple-darwin8.0.0/libjava/.libs/libgcj.la
$ rm powerpc-apple-darwin8.0.0/libjava/libgcj.la
$ time make -j2 all-target-libjava
real    3m44.677s
user    1m15.000s
sys     1m19.868s

G5 DP 2.5 GHz 512MB

Re: GCC 4.1: Buildable on GHz machines only?

[Mike Stump]

On Apr 29, 2005, at 7:41 AM, Daniel Jacobowitz wrote:
> On Fri, Apr 29, 2005 at 12:49:37PM +0200, Lars Segerlund wrote:
>>  If we do a reasonable comparison of compile times against the  
>> intel compiler or
>>  the portland group or something similar we consistenly find that  
>> gcc is slower
>>  by a couple of times 1x - 3x, ( this is only my impression, not  
>> backed up by
>>  hard data but should be in the ballpark ).
>>
>
> Please don't add additional speculation to this already messy subject.
> Feel free to come back with data.

Finder_FE_v3 on 1GHz G4 PowerBook, 512MB RAM:

10.4 Tiger 8A428 using gcc4.0 (build results window closed), PCH,  
native target, debug on, no optimization, indexing off.
            full build: 574 seconds = 2.25 WarMarks

10.3.4 with Metroworks Code Warrior 9.0 with debug on, no optimization
            full build: 255 seconds

but please, don't let our data get in the way...  mind you, this is  
better than 36x slower, which is where it used to be, but, still  
horrible.  To get a feel for it, drive 30 on the freeway in the fast  
lane.  :-)

For source, try QT, we believe it is representative.

Re: GCC 4.1: Buildable on GHz machines only?

[Alexandre Oliva]

On Apr 28, 2005, David Edelsohn <dje@watson.ibm.com> wrote:

>>>>>> Joe Buck writes:
Joe> Is there a reason why we aren't using a recent libtool?

> 	Porting and testing effort to upgrade. 

FWIW, I'd love to switch to a newer version of libtool, but I don't
have easy access to as many OSs as I used to several years ago, so
whatever testing I could offer would be quite limited.

The other issue is that I'm aware of some changes that we've adopted
in GCC libtool that are in libtool CVS mainline (very unstable), but
not in the libtool 1.5 branch (stable releases come out of it) nor in
the 2.0 branch (where the next major stable release is hopefully soon
coming from).

As much as I'd rather avoid switching from one random CVS snapshot of
libtool, now heavily patched, to another random CVS snapshot, it's
either that or waiting a long time until 2.0 is released, then
backport whatever features from libtool mainline we happen to be
relying on.  Or even wait for 2.2.

At this point, it doesn't feel like switching to 1.5.16 is worth the
effort.  2.0 should be far more maintainable, and hopefully
significantly more efficient on hosts where the use of shell functions
optimized for properties of the build machine and/or the host
machine can bring us such improvement.

Thoughts?

-- 
Alexandre Oliva             http://www.ic.unicamp.br/~oliva/
Red Hat Compiler Engineer   aoliva@{redhat.com, gcc.gnu.org}
Free Software Evangelist  oliva@{lsd.ic.unicamp.br, gnu.org}

Re: GCC 4.1: Buildable on GHz machines only?

[Alexandre Oliva]

On Apr 29, 2005, Jakub Jelinek <jakub@redhat.com> wrote:

> On Fri, Apr 29, 2005 at 10:47:06AM +0100, Andrew Haley wrote:
>> Ian Lance Taylor writes:
>> > 
>> > And, yes, we clearly need to do something about the libjava build.
>> 
>> OK, I know nothing about libtool so this might not be possible, but
>> IMO the easiest way of making a dramatic difference is to cease to
>> compile every file twice, once with PIC and once without.  There would
>> be a small performance regression for statically linked Java apps, but
>> in practice Java is very hard to use with static linkage.

> Definitely.  For -static you either have the choice of linking the
> binary with -Wl,--whole-archive for libgcj.a (and likely other Java libs),
> or spend a lot of time adding more and more objects that are really
> needed, but linker doesn't pick them up.

> For the distribution, we simply remove all Java *.a libraries, but it would
> be a build time win if we don't have to compile them altogether.

We had a patch that did exactly this at some point, but RTH said it
broke GNU/Linux/alpha and never gave me the details on what the
failure mode was, and I wasn't able to trigger the error myself.  I
still have the patch in my tree, and it does indeed save lots of
cycles.

-- 
Alexandre Oliva             http://www.ic.unicamp.br/~oliva/
Red Hat Compiler Engineer   aoliva@{redhat.com, gcc.gnu.org}
Free Software Evangelist  oliva@{lsd.ic.unicamp.br, gnu.org}

Re: GCC 4.1: Buildable on GHz machines only?

[Alexandre Oliva]

On Apr 29, 2005, Richard Henderson <rth@redhat.com> wrote:

> On Fri, Apr 29, 2005 at 01:30:13PM -0400, Ian Lance Taylor wrote:
>> I don't know of a way to tell libtool to not do duplicate compiles.
>> You can use -prefer-pic, but at least from looking at the script it
>> will still compile twice, albeit with -fPIC both times.

> Incidentally, libtool does not compile things twice when you use
> convenience libraries.

Yes, it does, because when you compile libtool still doesn't know
you're going to only use the object file for convenience libraries.
Besides, the fact that only the PIC version of object files is used
for convenience libraries is effectively a limitation of libtool, that
should eventually be addressed.

We should try to reinstate that --tag disable-static patch and get
detailed information on what broke for you, and fix that.

-- 
Alexandre Oliva             http://www.ic.unicamp.br/~oliva/
Red Hat Compiler Engineer   aoliva@{redhat.com, gcc.gnu.org}
Free Software Evangelist  oliva@{lsd.ic.unicamp.br, gnu.org}

Re: GCC 4.1: Buildable on GHz machines only?

[Joe Buck]

On Tue, May 03, 2005 at 04:57:10PM -0300, Alexandre Oliva wrote:
> At this point, it doesn't feel like switching to 1.5.16 is worth the
> effort.  2.0 should be far more maintainable, and hopefully
> significantly more efficient on hosts where the use of shell functions
> optimized for properties of the build machine and/or the host
> machine can bring us such improvement.

> Thoughts?

Richard Henderson showed that the libjava build spends 2/3 of its time
in libtool, and that his hand-hacked (but not portable) modification to
invoke the appropriate binutils commands directly gave a huge speedup.
To me, 300% overhead means major breakage, so we need a better libtool.
However, this better libtool might not yet exist.

RE: GCC 4.1: Buildable on GHz machines only?

[Dave Korn]

----Original Message----
>From: Joe Buck
>Sent: 03 May 2005 23:04

> On Tue, May 03, 2005 at 04:57:10PM -0300, Alexandre Oliva wrote:
>> At this point, it doesn't feel like switching to 1.5.16 is worth the
>> effort.  2.0 should be far more maintainable, and hopefully
>> significantly more efficient on hosts where the use of shell functions
>> optimized for properties of the build machine and/or the host
>> machine can bring us such improvement.
> 
>> Thoughts?
> 
> Richard Henderson showed that the libjava build spends 2/3 of its time
> in libtool, and that his hand-hacked (but not portable) modification to
> invoke the appropriate binutils commands directly gave a huge speedup.
> To me, 300% overhead means major breakage, so we need a better libtool.
> However, this better libtool might not yet exist.

  Ok, here's a really *nasty* kludge:  libtool is basically a big script
that generates command lines for the other tools based on passed-in args and
local configure settings, yeh?  And a lot of the time it's used for lots and
lots and lots of library files one after another in exactly the same way,
yes?

  So couldn't quite a lot of its uses be replaced in the makefile machinery
with something that invokes it just once (for a given batch of library
source files in a single build object subdir), and records the command line
that it generates, and just uses sed to duplicate all the different source
and object file names into the right places in many repetitions of it?

  I did a little experiment.  I took a recent build directory of HEAD.  I
keep all the output from the build process in a script file, so I fetched
all the lines that invoke libtool with

grep -B0 -A1 libtool build.log | grep -v -- ^-- > libt.txt

  This hopefully gets the libtool invocation and the command line that
libtool generates.  It also gets a few false positives, such as mentions in
configure output, but they're a small number.  The output contains 761
lines, 744 of which are unique. 

DK@ubik /gnu/obj-HEAD> grep -B0 -A1 libtool build.log | grep -v -- ^-- >
libt.txt
DK@ubik /gnu/obj-HEAD> wc -l libt.txt
761 libt.txt
DK@ubik /gnu/obj-HEAD> sort < libt.txt | uniq | wc -l
744
DK@ubik /gnu/obj-HEAD>

  Then I use sed to replace all the names of source and object files I can
find with dummy text:

DK@ubik /gnu/obj-HEAD> sed < libt.txt -e 's/[-a-zA-Z0-9_]*\.cc/SRC/g' -e '
s/[-a-zA-Z0-9_]*\.o/OBJ/g' -e 's/[-a-zA-Z0-9_]*\..o/OBJ/g' -e
's/[-a-zA-Z0-9_]*
\.c/SRC/g' -e 's/[-a-zA-Z0-9_]*\.f90/SRC/g' -e 's/[-a-zA-Z0-9_]*\.m/SRC/g'
>lib
t2.txt

  Now look what that does!

DK@ubik /gnu/obj-HEAD> wc -l libt2.txt
761 libt2.txt
DK@ubik /gnu/obj-HEAD> sort < libt2.txt | uniq | wc -l
63
DK@ubik /gnu/obj-HEAD>

  If I manually edit out the false positives from that, there are only 41
lines.

  Now, libjava wasn't included in this build because it seems to be disabled
on cygwin target, but it's probably much the same.  So ISTM that vast
swathes of libtool invocations could be replaced by a far simpler generation
of command lines from templates, with a bit of help from libtool to generate
the templates.  Give libtool an option where it only generates the command
line instead of invoking it, pass it args that look like "-c SRC" and "-o
OBJ", and then postprocess it to substitute real names in.

  Is there some terrible gotcha that I don't understand libtool well enough
to see here?

    cheers,
      DaveK
-- 
Can't think of a witty .sigline today....

Re: GCC 4.1: Buildable on GHz machines only?

[Peter O'Gorman]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Dave Korn wrote:
|   Ok, here's a really *nasty* kludge:  libtool is basically a big script
| that generates command lines for the other tools based on passed-in args and
| local configure settings, yeh?  And a lot of the time it's used for lots and
| lots and lots of library files one after another in exactly the same way,
| yes?

<http://libtool-cache.sourceforge.net/>

Peter
- --
Peter O'Gorman - http://www.pogma.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.0 (Darwin)

iQCVAwUBQngOnriDAg3OZTLPAQItXAP+NTK3ye0bzQOAWMYlctBfADpVvvTa+cB8
4Ft4AkUwdIQ1hdUjq5aNWF2btksxD33rd3Idse5esLzeTY3zXajkqkFTJWc2HKlM
h9ZM/1lc6Q2mQq/bbzj2kAH+TRfck3jFQlkoOwo7gJB8xrDROMX0LdvpAKeN9DP0
n1hTLfwVweE=
=ZQUv
-----END PGP SIGNATURE-----

RE: GCC 4.1: Buildable on GHz machines only?

[Dave Korn]

----Original Message----
>From: Peter O'Gorman
>Sent: 04 May 2005 00:52

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Dave Korn wrote:
>>   Ok, here's a really *nasty* kludge:  libtool is basically a big script
>> that generates command lines for the other tools based on passed-in args
>> and local configure settings, yeh?  And a lot of the time it's used for
>> lots and lots and lots of library files one after another in exactly the
>> same way, yes?
> 
> <http://libtool-cache.sourceforge.net/>
> 

"  Caching compile commands
gives nice speedups even the first run as the source and target file name
are
replaced by a generic placeholder before caching (the 2.6 branch of GTK+
requires only 9 different compile commands), ...  "

  Cool!  I'll give it a try and see if I can get some figures!

    cheers,
      DaveK
-- 
Can't think of a witty .sigline today....

Re: GCC 4.1: Buildable on GHz machines only?

[Peter O'Gorman]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Joe Buck wrote:

| Richard Henderson showed that the libjava build spends 2/3 of its time
| in libtool, and that his hand-hacked (but not portable) modification to
| invoke the appropriate binutils commands directly gave a huge speedup.
| To me, 300% overhead means major breakage, so we need a better libtool.
| However, this better libtool might not yet exist.

Probably doesn't. Ralf has done lots of work on libtool HEAD, making it 20%
faster, but that will not be in a libtool release anytime soon.

Part of the problem here is the use of a convenienve library to hold several
thousand object files and then making a shared library with the convenience
library. On many platforms, those without a --whole-archive flag, libtool
will extract the convenience archive all over again. Linking the shared
library all in one go would be faster.

Peter
- --
Peter O'Gorman - http://www.pogma.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.0 (Darwin)

iQCVAwUBQngR9riDAg3OZTLPAQLUwwP+I+xq38TklAgu/YSi81QJn4UzbOCOrRro
5SWfj7QM9Os66QxpKp6Ds0l0lREr3p/ytj4OlHtZ4NeAMt33rD4j5KGaK3K83jbj
Qcij/uJHHoSe3KJftnoJg/9/RWAWlxhFTS5oJhgBOSpcdYtrdAdj9m2k1qV+BQum
q2ZuThhgd2c=
=lYSE
-----END PGP SIGNATURE-----

Re: GCC 4.1: Buildable on GHz machines only?

[Richard Henderson]

On Wed, May 04, 2005 at 09:06:14AM +0900, Peter O'Gorman wrote:
> Part of the problem here is the use of a convenienve library to hold several
> thousand object files and then making a shared library with the convenience
> library. On many platforms, those without a --whole-archive flag, libtool
> will extract the convenience archive all over again. Linking the shared
> library all in one go would be faster.

Frankly, this is only a very small part of the problem.  MOST of the
time is wasted in the thousands of libtool invokations that preceed
the final link.


r~

Re: GCC 4.1: Buildable on GHz machines only?

[Andrew Haley]

Joe Buck writes:
 > On Tue, May 03, 2005 at 04:57:10PM -0300, Alexandre Oliva wrote:
 > > At this point, it doesn't feel like switching to 1.5.16 is worth the
 > > effort.  2.0 should be far more maintainable, and hopefully
 > > significantly more efficient on hosts where the use of shell functions
 > > optimized for properties of the build machine and/or the host
 > > machine can bring us such improvement.
 > 
 > > Thoughts?
 > 
 > Richard Henderson showed that the libjava build spends 2/3 of its time
 > in libtool, and that his hand-hacked (but not portable) modification to
 > invoke the appropriate binutils commands directly gave a huge speedup.

Yes, but please bear in mind that this *only* happens when you have a
machine with huge RAM.  For other people with small RAM, the link
itself is an important factor.  Also, other people have found that the
libtool script consumes a smaller part of total execution time: rth's
measurements are at one extreme of the scale.

Andrew.

Re: GCC 4.1: Buildable on GHz machines only?

[H. J. Lu]

On Wed, May 04, 2005 at 11:23:20AM +0100, Andrew Haley wrote:
> Joe Buck writes:
>  > On Tue, May 03, 2005 at 04:57:10PM -0300, Alexandre Oliva wrote:
>  > > At this point, it doesn't feel like switching to 1.5.16 is worth the
>  > > effort.  2.0 should be far more maintainable, and hopefully
>  > > significantly more efficient on hosts where the use of shell functions
>  > > optimized for properties of the build machine and/or the host
>  > > machine can bring us such improvement.
>  > 
>  > > Thoughts?
>  > 
>  > Richard Henderson showed that the libjava build spends 2/3 of its time
>  > in libtool, and that his hand-hacked (but not portable) modification to
>  > invoke the appropriate binutils commands directly gave a huge speedup.
> 
> Yes, but please bear in mind that this *only* happens when you have a
> machine with huge RAM.  For other people with small RAM, the link
> itself is an important factor.  Also, other people have found that the
> libtool script consumes a smaller part of total execution time: rth's
> measurements are at one extreme of the scale.
> 

We have been working on linker speed. If you have a number to show
that the GNU linker is very slow on certain things, I will take a
look.


H.J.

Re: GCC 4.1: Buildable on GHz machines only?

[Andrew Haley]

H. J. Lu writes:
 > On Wed, May 04, 2005 at 11:23:20AM +0100, Andrew Haley wrote:
 > > Joe Buck writes:
 > >  > On Tue, May 03, 2005 at 04:57:10PM -0300, Alexandre Oliva wrote:
 > >  > > At this point, it doesn't feel like switching to 1.5.16 is worth the
 > >  > > effort.  2.0 should be far more maintainable, and hopefully
 > >  > > significantly more efficient on hosts where the use of shell functions
 > >  > > optimized for properties of the build machine and/or the host
 > >  > > machine can bring us such improvement.
 > >  > 
 > >  > > Thoughts?
 > >  > 
 > >  > Richard Henderson showed that the libjava build spends 2/3 of its time
 > >  > in libtool, and that his hand-hacked (but not portable) modification to
 > >  > invoke the appropriate binutils commands directly gave a huge speedup.
 > > 
 > > Yes, but please bear in mind that this *only* happens when you have a
 > > machine with huge RAM.  For other people with small RAM, the link
 > > itself is an important factor.  Also, other people have found that the
 > > libtool script consumes a smaller part of total execution time: rth's
 > > measurements are at one extreme of the scale.
 > 
 > We have been working on linker speed. If you have a number to show
 > that the GNU linker is very slow on certain things, I will take a
 > look.

I haven't, no.  Ian Taylor reported the problem.

Andrew.

Re: GCC 4.1: Buildable on GHz machines only?

[Joe Buck]

On Wed, May 04, 2005 at 06:41:59AM -0700, H. J. Lu wrote:
> On Wed, May 04, 2005 at 11:23:20AM +0100, Andrew Haley wrote:
> > Joe Buck writes:
> >  > Richard Henderson showed that the libjava build spends 2/3 of its time
> >  > in libtool, and that his hand-hacked (but not portable) modification to
> >  > invoke the appropriate binutils commands directly gave a huge speedup.
> > 
> > Yes, but please bear in mind that this *only* happens when you have a
> > machine with huge RAM.  For other people with small RAM, the link
> > itself is an important factor.  Also, other people have found that the
> > libtool script consumes a smaller part of total execution time: rth's
> > measurements are at one extreme of the scale.
> 
> We have been working on linker speed. If you have a number to show
> that the GNU linker is very slow on certain things, I will take a
> look.

I'm glad you're looking at speeding up the linker.  Please make sure to
look at memory consumption as well, since performance falls off a cliff
once the working set exceeds physical memory.  A good test would be to
bootstrap gcc on a machine with 256M, or that is artificially limited to
256M (I seem to recall that you can tell a Linux kernel you have only a
given amount of memory).

Re: GCC 4.1: Buildable on GHz machines only?

[H. J. Lu]

On Wed, May 04, 2005 at 09:00:05AM -0700, Joe Buck wrote:
> On Wed, May 04, 2005 at 06:41:59AM -0700, H. J. Lu wrote:
> > On Wed, May 04, 2005 at 11:23:20AM +0100, Andrew Haley wrote:
> > > Joe Buck writes:
> > >  > Richard Henderson showed that the libjava build spends 2/3 of its time
> > >  > in libtool, and that his hand-hacked (but not portable) modification to
> > >  > invoke the appropriate binutils commands directly gave a huge speedup.
> > > 
> > > Yes, but please bear in mind that this *only* happens when you have a
> > > machine with huge RAM.  For other people with small RAM, the link
> > > itself is an important factor.  Also, other people have found that the
> > > libtool script consumes a smaller part of total execution time: rth's
> > > measurements are at one extreme of the scale.
> > 
> > We have been working on linker speed. If you have a number to show
> > that the GNU linker is very slow on certain things, I will take a
> > look.
> 
> I'm glad you're looking at speeding up the linker.  Please make sure to
> look at memory consumption as well, since performance falls off a cliff
> once the working set exceeds physical memory.  A good test would be to
> bootstrap gcc on a machine with 256M, or that is artificially limited to
> 256M (I seem to recall that you can tell a Linux kernel you have only a
> given amount of memory).

Given the current hardware, I would say 512MB is a reasonable number.
If I need to more than 256M to get 5% speed up, I will take it.

BTW, for gcc 4.0, I got

11687.92user 2089.89system 2:11:43elapsed 174%CPU (0avgtext+0avgdata
0maxresident)k

for bootstrap

9710.54user 2628.79system 1:52:03elapsed 183%CPU (0avgtext+0avgdata
0maxresident)k

for "make check" on a dual P3 550MHz with 512MB with everything except
for Ada.


H.J.

Re: GCC 4.1: Buildable on GHz machines only?

[Joe Buck]

I wrote:
> > I'm glad you're looking at speeding up the linker.  Please make sure to
> > look at memory consumption as well, since performance falls off a cliff
> > once the working set exceeds physical memory.  A good test would be to
> > bootstrap gcc on a machine with 256M, or that is artificially limited to
> > 256M (I seem to recall that you can tell a Linux kernel you have only a
> > given amount of memory).

On Wed, May 04, 2005 at 09:17:19AM -0700, H. J. Lu wrote:
> Given the current hardware, I would say 512MB is a reasonable number.
> If I need to more than 256M to get 5% speed up, I will take it.

We're not talking about 5% speedup; if the linker starts thrashing because
of insufficient memory you pay far more than that.  And certainly anyone
with an older computer who is dissatified with its performance, but
doesn't have a lot of money, should look into getting more memory before
anything else.  Still, the GNU project shouldn't be telling people in the
third world with cast-off machines that they are out of luck; to many of
them, 256M is more than they have.

So the basic issue is this: why the hell does the linker need so much
memory?  Sure, if you have tons available, it pays to trade memory for
time, mmap everything, then build all the hashes you want to look up
relationships in every direction.  But if it doesn't really fit, it's
a big lose.  Ideally ld, ar and the like could detect and adapt if there
isn't enough physical memory to hold everything.

Re: GCC 4.1: Buildable on GHz machines only?

[Ian Lance Taylor]

Joe Buck <Joe.Buck@synopsys.COM> writes:

> So the basic issue is this: why the hell does the linker need so much
> memory?  Sure, if you have tons available, it pays to trade memory for
> time, mmap everything, then build all the hashes you want to look up
> relationships in every direction.  But if it doesn't really fit, it's
> a big lose.  Ideally ld, ar and the like could detect and adapt if there
> isn't enough physical memory to hold everything.

At present the linker provides command line options --no-keep-memory
and --reduce-memory-overheads to significantly reduce the amount of
memory required during the link.

It should be possible in principle to partially adapt to available
memory based on, e.g., physmem_total.  The linker could keep track of
how much memory it has allocated via bfd_alloc and bfd_malloc.  If
that total gets to be 75% of physmem_total, or something like that,
the linker could switch to --no-keep-memory.

Unfortunately the decisions made by --reduce-memory-overhead apply at
the start of the link.  At that time it is difficult to tell how much
memory will be needed.

Ian

Re: GCC 4.1: Buildable on GHz machines only?

[Joe Buck]

On Wed, May 04, 2005 at 12:43:46PM -0400, Ian Lance Taylor wrote:
> At present the linker provides command line options --no-keep-memory
> and --reduce-memory-overheads to significantly reduce the amount of
> memory required during the link.
> 
> It should be possible in principle to partially adapt to available
> memory based on, e.g., physmem_total.  The linker could keep track of
> how much memory it has allocated via bfd_alloc and bfd_malloc.  If
> that total gets to be 75% of physmem_total, or something like that,
> the linker could switch to --no-keep-memory.
> 
> Unfortunately the decisions made by --reduce-memory-overhead apply at
> the start of the link.  At that time it is difficult to tell how much
> memory will be needed.

If the number gets much above 100%, it would probably be faster for the linker
to quit and start over than to proceed, and such an approach wouldn't be
hard to implement.

Re: GCC 4.1: Buildable on GHz machines only?

[Per Bothner]

Mark Mitchell wrote:
> Building libjava takes forever on any platform, relative to the rest of 
> the compiler build.

In addition to fixing/replacing libtool (could it be rewritten as a C
program?) there are a number of other known gcj performance problems.

When compiling A.java, gcj needs to read a lot of other classes that A
depends on.  It seems to be reason a lot more classes than it needs to:
it is a lot less agressive about loading a class than it should be.

On the other hand, much time spend improving gcj performance might
be wasteful it it will be replaced by Tom's new gcjx.

One way to speed up libcgj compilation by quite a bit would be to
compile more than one .java file at a time.  For example:
   gcj -c java/util/*.java -o java-util.o
This reduces libtool overhead, reduces the duplication in reading
dependencies, and probably reduces link overheads.  It can also
produce better code, since intermodule references get trurned into
intramodule references.  This just requires Makefile-hacking.
Disadvantages:
- Static linking (which we don't really support very well anyway)
might causes some classes to be needlessly linked in.
- Compiling multiple classes at once might increase virtual memory use.

I think the net benefit could be large - an experiment would be
quite worthwhile.  Perhaps somebody could write a post-configure
script to munge the Makefile and give us some numbers.

Ideally, there'd be a configure flag to control "chunking".
-- 
	--Per Bothner
per@bothner.com   http://per.bothner.com/

Re: GCC 4.1: Buildable on GHz machines only?

[David Daney]

Per Bothner wrote:
> 
> One way to speed up libcgj compilation by quite a bit would be to
> compile more than one .java file at a time.  For example:
>   gcj -c java/util/*.java -o java-util.o
> This reduces libtool overhead, reduces the duplication in reading
> dependencies, and probably reduces link overheads.  It can also
> produce better code, since intermodule references get trurned into
> intramodule references.  This just requires Makefile-hacking.

I was going to recommend the same thing.

We have taken this approach with a fairly large program (about 1000 
classes) and it does speed things up considerably.  We see a 75% 
reduction in total build times (12 minutes vs 48).

David Daney

Re: GCC 4.1: Buildable on GHz machines only?

[Alan Modra]

On Wed, May 04, 2005 at 09:29:44AM -0700, Joe Buck wrote:
> So the basic issue is this: why the hell does the linker need so much
> memory?

- long symbol names and lots of symbols
- lots of sections
- optimizations that edit section contents, requiring the contents to
  be kept in memory.  eg. string merging, relaxation.

-- 
Alan Modra
IBM OzLabs - Linux Technology Centre

Re: GCC 4.1: Buildable on GHz machines only?

[Ranjit Mathew]

Per Bothner wrote:
> Mark Mitchell wrote:
> 
>>Building libjava takes forever on any platform, relative to the rest of 
>>the compiler build.

[...]

> One way to speed up libcgj compilation by quite a bit would be to
> compile more than one .java file at a time.  For example:
>    gcj -c java/util/*.java -o java-util.o
> This reduces libtool overhead, reduces the duplication in reading
> dependencies, and probably reduces link overheads.  It can also
> produce better code, since intermodule references get trurned into
> intramodule references.  This just requires Makefile-hacking.

[...]

> Ideally, there'd be a configure flag to control "chunking".

Note that libgcj already supports an "--enable-libgcj-multifile"
configuration option that coarsely attempts to do the above.

See:

  http://gcc.gnu.org/ml/java-patches/2003-q3/msg00658.html

Ranjit.

-- 
Ranjit Mathew      Email: rmathew AT gmail DOT com

Bangalore, INDIA.    Web: http://ranjitmathew.hostingzero.com/

Re: GCC 4.1: Buildable on GHz machines only?

[Ranjit Mathew]

Ranjit Mathew wrote:
>>Ideally, there'd be a configure flag to control "chunking".
> 
> 
> Note that libgcj already supports an "--enable-libgcj-multifile"
> configuration option that coarsely attempts to do the above.
> 
> See:
> 
>   http://gcc.gnu.org/ml/java-patches/2003-q3/msg00658.html

I tried this out by bootstrapping mainline on i686-pc-linux-gnu
(RHEL 3AS U4, P4 2.4GHz, 1GB RAM, 512MB Swap) and found
*no difference* in the total bootstrap time. :-/

Either this option has bitrotted or something else undoes
its effect.

Ranjit.

-- 
Ranjit Mathew      Email: rmathew AT gmail DOT com

Bangalore, INDIA.    Web: http://ranjitmathew.hostingzero.com/

Re: GCC 4.1: Buildable on GHz machines only?

[Tom Tromey]

>>>>> "Ranjit" == Ranjit Mathew <rmathew@gmail.com> writes:

Ranjit> Note that libgcj already supports an "--enable-libgcj-multifile"
Ranjit> configuration option that coarsely attempts to do the above.
Ranjit> See:
Ranjit>   http://gcc.gnu.org/ml/java-patches/2003-q3/msg00658.html

--enable-libgcj-multifile controls how .class files are built; if
used they are all built in a single gcj invocation, otherwise they
are built one at a time.

We could allow different amounts of aggregation other than 0% or
100%; that might help some builds.

But what Per is talking about is how .o files are built.  This change
would probably not be very difficult fwiw; we already have done this
in a place or two where we've needed BC ABI support.

Tom

Re: GCC 4.1: Buildable on GHz machines only?

[Per Bothner]

Tom Tromey wrote:
> --enable-libgcj-multifile controls how .class files are built;  ... 
> But what Per is talking about is how .o files are built. 

Both, actually.

We could save some time by building .class and .o files at the same
time, though that requires jc1 changes.

We could also save time by making --disable-static the default.
Building static libraries is not very useful on other than
embedded-class systems.
-- 
	--Per Bothner
per@bothner.com   http://per.bothner.com/

Re: GCC 4.1: Buildable on GHz machines only?

[Andrew Haley]

Per Bothner writes:
 > 
 > We could also save time by making --disable-static the default.
 > Building static libraries is not very useful on other than
 > embedded-class systems.

I <empahsis>strongly</emphasis> agree.

Andrew.

Re: GCC 4.1: Buildable on GHz machines only?

[Alexandre Oliva]

On May  5, 2005, Andrew Haley <aph@redhat.com> wrote:

> Per Bothner writes:
>> 
>> We could also save time by making --disable-static the default.
>> Building static libraries is not very useful on other than
>> embedded-class systems.

> I <empahsis>strongly</emphasis> agree.

The savings of creating static libraries would be small if we
refrained from building non-PIC object files.  This is exactly what
--tag disable-static for compilations accomplishes, and we had a patch
to use that in our tree for some time, but RTH ran into a (probably
libtool) bug on alpha that led him to revert the change, and then
didn't provide enough info for anyone else without access to an alpha
box to figure out what the problem was and then try to fix it :-(

-- 
Alexandre Oliva             http://www.ic.unicamp.br/~oliva/
Red Hat Compiler Engineer   aoliva@{redhat.com, gcc.gnu.org}
Free Software Evangelist  oliva@{lsd.ic.unicamp.br, gnu.org}

Re: GCC 4.1: Buildable on GHz machines only?

[Richard Henderson]

On Thu, May 05, 2005 at 04:57:48PM -0300, Alexandre Oliva wrote:
> The savings of creating static libraries would be small if we
> refrained from building non-PIC object files.

But still largely useless.  Who in their right mind is going to
use an 83MB static library when a shared library is available.

> This is exactly what
> --tag disable-static for compilations accomplishes, and we had a patch
> to use that in our tree for some time, but RTH ran into a (probably
> libtool) bug on alpha that led him to revert the change, and then
> didn't provide enough info for anyone else without access to an alpha
> box to figure out what the problem was and then try to fix it :-(

I don't recall the failure mode anymore, nor do I have the patch.  If
you feel like working on --tag disable-static, don't let me stop you.
I'll whine if it breaks again, but not before.


r~

Re: GCC 4.1: Buildable on GHz machines only?

[David Daney]

Richard Henderson wrote:
> On Thu, May 05, 2005 at 04:57:48PM -0300, Alexandre Oliva wrote:
> 
>>The savings of creating static libraries would be small if we
>>refrained from building non-PIC object files.
> 
> 
> But still largely useless.  Who in their right mind is going to
> use an 83MB static library when a shared library is available.
> 

Perhaps the crazy person that only needs 2MB worth of the files from 
said static library when the corresponding shared library is 8MB. 
Especially if this lunatic is trying to make the program, OS kernel etc 
fit in an 8MB flash memory device.

David Daney.

Re: GCC 4.1: Buildable on GHz machines only?

[Rutger Ovidius]

Thursday, May 5, 2005, 1:16:05 PM, you wrote:

RH> On Thu, May 05, 2005 at 04:57:48PM -0300, Alexandre Oliva wrote:
>> The savings of creating static libraries would be small if we
>> refrained from building non-PIC object files.

RH> But still largely useless.  Who in their right mind is going to
RH> use an 83MB static library when a shared library is available.

Everyone on win32 builds libgcj static, and probably wants to keep it
that way if they plan to distribute their apps.

What would be the point of distributing a giant shared library with
every application compiled with gcj, especially when awt/swing only
hook into gtk?

I don't think libgcj.dll will ever be standard on any distribution of
win32, yet the many advantages of compiling java code to fast, native
executables apply on this platform.

Re: GCC 4.1: Buildable on GHz machines only?

[Andi Vajda]

Well, if you don't dynamically load classes, statically linking this 83Mb 
behemoth enables you to get rid of most of it. On Windows, with MinGW, where 
this is possible, I build a shared library (a python extension) that is 
statically linked with libgcj.a and the resulting .dll is only 4.6MB in size.

I wish the same were possible on Linux and Mac OS X but I have not been able 
to create a shared library that is statically linked against libgcj.a

Andi..

On Thu, 5 May 2005, Richard Henderson wrote:

> On Thu, May 05, 2005 at 04:57:48PM -0300, Alexandre Oliva wrote:
>> The savings of creating static libraries would be small if we
>> refrained from building non-PIC object files.
>
> But still largely useless.  Who in their right mind is going to
> use an 83MB static library when a shared library is available.
>
>> This is exactly what
>> --tag disable-static for compilations accomplishes, and we had a patch
>> to use that in our tree for some time, but RTH ran into a (probably
>> libtool) bug on alpha that led him to revert the change, and then
>> didn't provide enough info for anyone else without access to an alpha
>> box to figure out what the problem was and then try to fix it :-(
>
> I don't recall the failure mode anymore, nor do I have the patch.  If
> you feel like working on --tag disable-static, don't let me stop you.
> I'll whine if it breaks again, but not before.
>
>
> r~
>

Re: GCC 4.1: Buildable on GHz machines only?

[Tom Tromey]

>>>>> "Rutger" == Rutger Ovidius <r_ovidius@eml.cc> writes:

RH> But still largely useless.  Who in their right mind is going to
RH> use an 83MB static library when a shared library is available.

Rutger> Everyone on win32 builds libgcj static, and probably wants to keep it
Rutger> that way if they plan to distribute their apps.

I thought the reason everybody on win32 built static was that there
was no choice.  Is it really preferable?

I'm sympathetic to folks who want static builds for various reasons.
I wouldn't mind an option to allow libgcj to be built in some reduced
form, either; though it would be up to the folks using this mode to
maintain it.

Tom

Re: GCC 4.1: Buildable on GHz machines only?

[Mike Stump]

On Thursday, May 5, 2005, at 02:53  PM, Andi Vajda wrote:
> I wish the same were possible on Linux and Mac OS X but I have not 
> been able to create a shared library that is statically linked against 
> libgcj.a

Should just work, though, you don't want to link -static built objects 
into a .dylib, you merely want to link in -fPIC built objects from a .a 
into the dylib.  Can you manage to do this with a small example?

Re: GCC 4.1: Buildable on GHz machines only?

[Ranjit Mathew]

On 05 May 2005 17:01:05 -0600, Tom Tromey <tromey@redhat.com> wrote:
> >>>>> "Rutger" == Rutger Ovidius <r_ovidius@eml.cc> writes:
> 
> RH> But still largely useless.  Who in their right mind is going to
> RH> use an 83MB static library when a shared library is available.
> 
> Rutger> Everyone on win32 builds libgcj static, and probably wants to keep it
> Rutger> that way if they plan to distribute their apps.
> 
> I thought the reason everybody on win32 built static was that there
> was no choice.  Is it really preferable?

The native libgcj library on Win32 is around 32MB and the JAR
is around 8MB (as of 4.0 and judging from Mohan's distribution
found in http://www.thisiscool.com/gcc_mingw.htm).

If you are building a small and nifty application in Java,
it would be unreasonable for you to expect your users 
to either download the whole Java runtime (either Sun's
JRE or GCJ) separately or to have you bundle it with 
your application.

It also doesn't help that the Java runtime is a moving
target (in Sun's JRE and more so in GCJ) so you need
to keep updating it for new apps or newer versions of
existing apps. It keeps bloating at an alarming rate
too.

It's a similar case for MS .NET. Is it small wonder
then that these have not penetrated much outside
of enterprises? Even there, you find that many an
enterprise software product implemented in Java
comes with its own JRE (or even JDK). It's not
uncommon to find *many* JREs on the machine
of a single enterprise software developer or on a
server.

Static linking in GCJ and SWT offer a chance
to get out of this vicious cycle, though it does
lead to (a relatively small) duplication between 
various such applications. Many a time, it is
the only way people are going to even bother
trying out your application.

One day, Java/GCJ would have stabilised 
and matured enough for this requirement to
go away, but till then it's a crucial intermediate
step for platforms like Win32.

Ranjit.


> I'm sympathetic to folks who want static builds for various reasons.
> I wouldn't mind an option to allow libgcj to be built in some reduced
> form, either; though it would be up to the folks using this mode to
> maintain it.
> 
> Tom
> 


-- 
Ranjit Mathew      Email: rmathew AT gmail DOT com

Bangalore, INDIA.    Web: http://ranjitmathew.hostingzero.com/

Re: GCC 4.1: Buildable on GHz machines only?

[Paolo Bonzini]

> We could allow different amounts of aggregation other than 0% or
> 100%; that might help some builds.

Per-directory could be useful to the guys using the static library, too.

> But what Per is talking about is how .o files are built.  This change
> would probably not be very difficult fwiw; we already have done this
> in a place or two where we've needed BC ABI support.

As long as libtool supports it, it should not be very difficult indeed.

Paolo

Re: GCC 4.1: Buildable on GHz machines only?

[Paolo Bonzini]

> We could allow different amounts of aggregation other than 0% or
> 100%; that might help some builds.

Per-directory could be useful to the guys using the static library, too.

> But what Per is talking about is how .o files are built.  This change
> would probably not be very difficult fwiw; we already have done this
> in a place or two where we've needed BC ABI support.

As long as libtool supports it, it should not be very difficult indeed.

Paolo

Re: GCC 4.1: Buildable on GHz machines only?

[Andrew Haley]

Rutger Ovidius writes:
 > Thursday, May 5, 2005, 1:16:05 PM, you wrote:
 > 
 > RH> On Thu, May 05, 2005 at 04:57:48PM -0300, Alexandre Oliva wrote:
 > >> The savings of creating static libraries would be small if we
 > >> refrained from building non-PIC object files.
 > 
 > RH> But still largely useless.  Who in their right mind is going to
 > RH> use an 83MB static library when a shared library is available.
 > 
 > Everyone on win32 builds libgcj static, and probably wants to keep it
 > that way if they plan to distribute their apps.
 > 
 > What would be the point of distributing a giant shared library with
 > every application compiled with gcj, especially when awt/swing only
 > hook into gtk?
 > 
 > I don't think libgcj.dll will ever be standard on any distribution of
 > win32, yet the many advantages of compiling java code to fast, native
 > executables apply on this platform.

I don't think that anyone is proposing to drop static libraries on
Win32.  Win32 systems have their own requirements that make static
libs preferable in some cases.  On GNU systems, however, static libs
make no sense at all for the Java language.

Andrew.

Re: GCC 4.1: Buildable on GHz machines only?

[Paolo Bonzini]

> We could allow different amounts of aggregation other than 0% or
> 100%; that might help some builds.

Per-directory could be useful to the guys using the static library, too.

> But what Per is talking about is how .o files are built.  This change
> would probably not be very difficult fwiw; we already have done this
> in a place or two where we've needed BC ABI support.

As long as libtool supports it, it should not be very difficult indeed.

Paolo

Re: GCC 4.1: Buildable on GHz machines only?

[Rutger Ovidius]

Friday, May 6, 2005, 1:33:32 AM, you wrote:

AH> Rutger Ovidius writes:
 >> Thursday, May 5, 2005, 1:16:05 PM, you wrote:
 >> 
 >> RH> On Thu, May 05, 2005 at 04:57:48PM -0300, Alexandre Oliva wrote:
 >> >> The savings of creating static libraries would be small if we
 >> >> refrained from building non-PIC object files.
 >> 
 >> RH> But still largely useless.  Who in their right mind is going to
 >> RH> use an 83MB static library when a shared library is available.
 >> 
 >> Everyone on win32 builds libgcj static, and probably wants to keep it
 >> that way if they plan to distribute their apps.
 >> 
 >> What would be the point of distributing a giant shared library with
 >> every application compiled with gcj, especially when awt/swing only
 >> hook into gtk?
 >> 
 >> I don't think libgcj.dll will ever be standard on any distribution of
 >> win32, yet the many advantages of compiling java code to fast, native
 >> executables apply on this platform.

AH> I don't think that anyone is proposing to drop static libraries on
AH> Win32.  Win32 systems have their own requirements that make static
AH> libs preferable in some cases.  On GNU systems, however, static libs
AH> make no sense at all for the Java language.

One of the first things I had hoped for from gcj was static linking
(except for libc) on GNU systems.

There is new era of shared library hell and it seems to only apply to
libgcj. Having to manually pare down the libgcj .so and distribute it
with apps seems necessary; expecting target users of a new gcj
compiled app to have an absolutely up-to-date and compatible libgcj.so
(probably compiled with small patches along the way to make this
specific app work) is not reasonable. Plus, the release cycle of gcc
will never match the development speed of libgcj. There are die hard
followers of gcc that do have up to date systems, but the vast
majority do not and never will.

Java is a simple language, used as the intro learning language in most
universities that I know of. Not having to plan memory management like
c++ motivates very fast development. Compiling small utils with it and
having them be portable, even on GNU systems, is an incredible selling
point. This isn't really possible without static linking.

Sometimes I see a great divide between the developers of gcj, and the
actual users of it. It seems to only target the server setting, or the
user who wishes to compile existing apps and only run them on their
own system. This target could be much wider in scope with static
linking.

Re: GCC 4.1: Buildable on GHz machines only?

[Andrew Haley]

Rutger Ovidius writes:
 > Friday, May 6, 2005, 1:33:32 AM, you wrote:
 > 
 > AH> I don't think that anyone is proposing to drop static libraries on
 > AH> Win32.  Win32 systems have their own requirements that make static
 > AH> libs preferable in some cases.  On GNU systems, however, static libs
 > AH> make no sense at all for the Java language.
 > 
 > One of the first things I had hoped for from gcj was static linking
 > (except for libc) on GNU systems.
 > 
 > There is new era of shared library hell and it seems to only apply to
 > libgcj. Having to manually pare down the libgcj .so and distribute it
 > with apps seems necessary; expecting target users of a new gcj
 > compiled app to have an absolutely up-to-date and compatible libgcj.so
 > (probably compiled with small patches along the way to make this
 > specific app work) is not reasonable.

Yes, which is why we're redesigning the ABI so that compiled apps
won't depend on a specific release of the library.  We're fixing the
real problem rather than depending on nasty kludges like static
linking.

 > Plus, the release cycle of gcc will never match the development
 > speed of libgcj. There are die hard followers of gcc that do have
 > up to date systems, but the vast majority do not and never will.

That too.

 > Java is a simple language, used as the intro learning language in most
 > universities that I know of. Not having to plan memory management like
 > c++ motivates very fast development. Compiling small utils with it and
 > having them be portable, even on GNU systems, is an incredible selling
 > point.

Is it really?  There are users out there insane enough to be passing
around precompiled binaries of small utils?

 > This isn't really possible without static linking.

But Java isn't compatible with static linking.  Java is, by its very
nature, a dynamic language, where classes invoke and even generate
other classes on the fly.  There is no way when linking to determine
what set of libraries is required.  This is a simple fact, and no
amount of declaring " this is what users want!"  is going to change
it.

 > Sometimes I see a great divide between the developers of gcj, and
 > the actual users of it. 

Spare us the ad homs, please.

Andrew.

Re: GCC 4.1: Buildable on GHz machines only?

[Mark Wielaard]

[-- Attachment #1: Type: text/plain, Size: 2066 bytes --]

Hi Rutger,

On Fri, 2005-05-06 at 07:24 -0700, Rutger Ovidius wrote:
> AH> I don't think that anyone is proposing to drop static libraries on
> AH> Win32.  Win32 systems have their own requirements that make static
> AH> libs preferable in some cases.  On GNU systems, however, static libs
> AH> make no sense at all for the Java language.
> 
> One of the first things I had hoped for from gcj was static linking
> (except for libc) on GNU systems.
> 
> There is new era of shared library hell and it seems to only apply to
> libgcj.

I might be to young to have experienced this last era of share library
hell. Do you have a pointer to what you are referring to and how it was
solved back then on GNU systems?

> Plus, the release cycle of gcc
> will never match the development speed of libgcj. There are die hard
> followers of gcc that do have up to date systems, but the vast
> majority do not and never will.

This is indeed a problem. But it has a happy cause, GNU Classpath and
libgcj development is going really strong and fast. We are slowly
working towards both working closer with the rest of GCC (sharing more
infrastructure) and decoupling some of the development a bit more (the
new bc-abi indirect-dispatch introduced in gcj 4). But it might take a
couple of gcc releases till we have the right balance.

> Sometimes I see a great divide between the developers of gcj, and the
> actual users of it. It seems to only target the server setting, or the
> user who wishes to compile existing apps and only run them on their
> own system. This target could be much wider in scope with static
> linking.

Where/How do these actual users communicate about the problems they are
experiencing and the disconnect between us developers and them? I would
like to better understand the actual problems these users see that we
developers might be missing.

Thanks,

Mark

-- 
Escape the Java Trap with GNU Classpath!
http://www.gnu.org/philosophy/java-trap.html

Join the community at http://planet.classpath.org/

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: GCC 4.1: Buildable on GHz machines only?

[Per Bothner]

Paolo Bonzini wrote: >> But what Per is talking about is how .o files 
are built.  This change
>> would probably not be very difficult fwiw; we already have done this
>> in a place or two where we've needed BC ABI support.
> 
> 
> As long as libtool supports it, it should not be very difficult indeed.

It semi-supports it.  I build Kawa one directory at a time, using
libtool.  (Albeit the CVS version of libtool.)
-- 
	--Per Bothner
per@bothner.com   http://per.bothner.com/

Re: GCC 4.1: Buildable on GHz machines only?

[Rutger Ovidius]

Friday, May 6, 2005, 8:06:49 AM, you wrote:

AH> Rutger Ovidius writes:
 >> Friday, May 6, 2005, 1:33:32 AM, you wrote:
 >> 
 >> AH> I don't think that anyone is proposing to drop static libraries on
 >> AH> Win32.  Win32 systems have their own requirements that make static
 >> AH> libs preferable in some cases.  On GNU systems, however, static libs
 >> AH> make no sense at all for the Java language.
 >> 
 >> One of the first things I had hoped for from gcj was static linking
 >> (except for libc) on GNU systems.
 >> 
 >> There is new era of shared library hell and it seems to only apply to
 >> libgcj. Having to manually pare down the libgcj .so and distribute it
 >> with apps seems necessary; expecting target users of a new gcj
 >> compiled app to have an absolutely up-to-date and compatible libgcj.so
 >> (probably compiled with small patches along the way to make this
 >> specific app work) is not reasonable.

AH> Yes, which is why we're redesigning the ABI so that compiled apps
AH> won't depend on a specific release of the library.  We're fixing the
AH> real problem rather than depending on nasty kludges like static
AH> linking.

It would be a feature; a kludge seems to be your opinion, and no one
would depend upon it. I don't think having the option to link static
would affect the natural java way(?) of linking dynamically.

 >> Java is a simple language, used as the intro learning language in most
 >> universities that I know of. Not having to plan memory management like
 >> c++ motivates very fast development. Compiling small utils with it and
 >> having them be portable, even on GNU systems, is an incredible selling
 >> point.

AH> Is it really?  There are users out there insane enough to be passing
AH> around precompiled binaries of small utils?

I don't need to pass it around to anyone else to experiene the
limitation. The utilities can be moved on to systems on which I do not
have access to install the latest gcc globally. I must also
include a large libgcj each time.

 >> This isn't really possible without static linking.

AH> But Java isn't compatible with static linking.  Java is, by its very
AH> nature, a dynamic language, where classes invoke and even generate
AH> other classes on the fly.  There is no way when linking to determine
AH> what set of libraries is required.  This is a simple fact, and no
AH> amount of declaring " this is what users want!"  is going to change
AH> it.

I didn't know that java had a nature.

It has features. Some features will work when it is implemented in a
certain way and some won't. Compiling natively with GCJ adds the
opportunity to add features. Static linking would be a feature.

I don't think "natural java" supports CNI, but this is a nice feature
that compiling with gcj adds.

 >> Sometimes I see a great divide between the developers of gcj, and
 >> the actual users of it. 

AH> Spare us the ad homs, please.

I think I can freely express a personal viewpoint just like you have
above.  Thank you.

Re: GCC 4.1: Buildable on GHz machines only?

[wfor]

Hi Mark,

> Hi Rutger,
> 
> On Fri, 2005-05-06 at 07:24 -0700, Rutger Ovidius wrote:
> > AH> I don't think that anyone is proposing to drop static libraries on
> > AH> Win32.  Win32 systems have their own requirements that make static
> > AH> libs preferable in some cases.  On GNU systems, however, static libs
> > AH> make no sense at all for the Java language.
> > 
> > One of the first things I had hoped for from gcj was static linking
> > (except for libc) on GNU systems.
> > 
> > There is new era of shared library hell and it seems to only apply to
> > libgcj.
> 
> I might be to young to have experienced this last era of share library
> hell. Do you have a pointer to what you are referring to and how it was
> solved back then on GNU systems?

Example:
Compile a C++ program on x86 Linux with gcc 2.9x, install it on a system
with gcc 3.x

As long as your program does not throw an exception, everything is fine.
If your program throws an exception, the application crashes.

Reason: /usr/lib/libstdc++.so

You have to fiddle around with LD_PRELOAD, I found no other way.
An unexperianced user is lost in space :-(

Wolfgang


Machen Sie aus 14 Cent spielend bis zu 100 Euro!
Die neue Gaming-Area von Arcor - über 50 Onlinespiele im Angebot.
http://www.arcor.de/rd/emf-gaming-1

Re: GCC 4.1: Buildable on GHz machines only?

[Andrew Haley]

Rutger Ovidius writes:
 > Friday, May 6, 2005, 8:06:49 AM, you wrote:
 > 
 > AH> But Java isn't compatible with static linking.  Java is, by its very
 > AH> nature, a dynamic language, where classes invoke and even generate
 > AH> other classes on the fly.  There is no way when linking to determine
 > AH> what set of libraries is required.  This is a simple fact, and no
 > AH> amount of declaring " this is what users want!"  is going to change
 > AH> it.
 > 
 > I didn't know that java had a nature.

Now you do.

 > It has features. Some features will work when it is implemented in a
 > certain way and some won't.

The set of features that work when linking statically is unspecified
and changes over time, depending on implementation details within the
library.

If we wanted to come up with a new language subset compatible with
static linkage we could do that, but it would be a substantial design
effort and we'd need someone to do the work.  Personally speaking, I
don't think it's a very good idea, as a lot of the Java language as
specified depends on dynamic linking, but I wouldn't obstruct someone
who really wanted to do it.

Andrew.

Re: GCC 4.1: Buildable on GHz machines only?

[Rutger Ovidius]

Friday, May 6, 2005, 9:14:31 AM, you wrote:

AH> Rutger Ovidius writes:
 >> Friday, May 6, 2005, 8:06:49 AM, you wrote:
 >> 
 >> AH> But Java isn't compatible with static linking.  Java is, by its very
 >> AH> nature, a dynamic language, where classes invoke and even generate
 >> AH> other classes on the fly.  There is no way when linking to determine
 >> AH> what set of libraries is required.  This is a simple fact, and no
 >> AH> amount of declaring " this is what users want!"  is going to change
 >> AH> it.
 >> 
 >> I didn't know that java had a nature.

AH> Now you do.

Hallelujah! God is great.

 >> It has features. Some features will work when it is implemented in a
 >> certain way and some won't.

AH> The set of features that work when linking statically is unspecified
AH> and changes over time, depending on implementation details within the
AH> library.

The implementation details within the current library already allow
the unspecified set of features to work when statically linked on
win32. Chalk another one up to the big G.


AH> If we wanted to come up with a new language subset compatible with
AH> static linkage we could do that, but it would be a substantial design
AH> effort and we'd need someone to do the work.  Personally speaking, I
AH> don't think it's a very good idea, as a lot of the Java language as
AH> specified depends on dynamic linking, but I wouldn't obstruct someone
AH> who really wanted to do it.

I wasn't asking for a new language subset. I can understand that
bestowing yet another nature on java would be a superhuman task. I
wasn't asking for anything really. I was just expressing an opinion.
Sorry for that.

Bug in multifile operation?

[Paolo Bonzini]

Per Bothner wrote:

> Paolo Bonzini wrote: >> But what Per is talking about is how .o files 
> are built.  This change
>
>>> would probably not be very difficult fwiw; we already have done this
>>> in a place or two where we've needed BC ABI support.
>>
>>
>>
>> As long as libtool supports it, it should not be very difficult indeed.
>
> It semi-supports it.

With @aaa.list it supports it, it seems.  I made a patch, but it fails 
because of a bug.  It is enough to supply these two files together to jc1

    package java.awt;
    import java.util.EventObject;
    public abstract class AWTEvent extends EventObject {}

    package java.awt;
    import java.awt.event.KeyEvent;
    public class B {
      Object[] o = KeyEvent.class.getFields();
    }

I'm tracking the failure, so far it seems that, in find_in_imports for 
java.awt.event.KeyEvent, there's already something different in the 
IDENTIFIER_CLASS_VALUE for KeyEvent.  But I have not understood where it 
is set. :-(

Paolo

Re: GCC 4.1: Buildable on GHz machines only?

[Marcin Dalecki]

On 2005-05-06, at 18:14, Andrew Haley wrote:

> Rutger Ovidius writes:
>
>> Friday, May 6, 2005, 8:06:49 AM, you wrote:
>>
>> AH> But Java isn't compatible with static linking.  Java is, by  
>> its very
>> AH> nature, a dynamic language, where classes invoke and even  
>> generate
>> AH> other classes on the fly.  There is no way when linking to  
>> determine
>> AH> what set of libraries is required.  This is a simple fact, and no
>> AH> amount of declaring " this is what users want!"  is going to  
>> change
>> AH> it.
>>
>> I didn't know that java had a nature.
>>
>
> Now you do.
>
>
>> It has features. Some features will work when it is implemented in a
>> certain way and some won't.
>>
>
> The set of features that work when linking statically is unspecified
> and changes over time, depending on implementation details within the
> library.
>
> If we wanted to come up with a new language subset compatible with
> static linkage we could do that, but it would be a substantial design
> effort and we'd need someone to do the work.  Personally speaking, I
> don't think it's a very good idea, as a lot of the Java language as
> specified depends on dynamic linking, but I wouldn't obstruct someone
> who really wanted to do it.


You don't understand that it's perfectly valid to put PIC symbols  
inside an .a file.
 From the users perspective this may very well appear to be like  
static linkage.
There are no principal reasons to shatter every single application in  
to a heap
of .so files.

Re: GCC 4.1: Buildable on GHz machines only?

[Joe Buck]

On Fri, May 06, 2005 at 04:06:49PM +0100, Andrew Haley wrote:
> Rutger Ovidius writes:
>  > Java is a simple language, used as the intro learning language in most
>  > universities that I know of. Not having to plan memory management like
>  > c++ motivates very fast development. Compiling small utils with it and
>  > having them be portable, even on GNU systems, is an incredible selling
>  > point.
> 
> Is it really?  There are users out there insane enough to be passing
> around precompiled binaries of small utils?

Yes.  They're called "applets".  They run in your web browser.

Re: GCC 4.1: Buildable on GHz machines only?

[Tom Tromey]

>>>>> "Matt" == Matt Thomas <matt@3am-software.com> writes:

Matt> I'd like to libjava be split into multiple shared libraries.
Matt> In C, we have libc, libm, libpthread, etc.  In X11, there's X11, Xt, etc.
Matt> So why does java have everything in one shared library?  Could
Matt> the swing stuff be moved to its own?  Are there other logical
Matt> divisions?

This is possible, though it isn't completely trivial.

To get the best effect at runtime, you want completely separate shared
libraries, which aren't linked in by default.  So then you need a
mechanism to load these libraries dynamically.  There are a few ways
to implement this, but you have to be careful to let the test suite
continue to work even if the user hasn't run "make install".


I would suggest separating out AWT and Swing as a first test.  Tom
Fitzsimmons' patch to BC-compile these libraries will make this a lot
simpler, as it already handles the weird special cases, and already
breaks up the build along the required lines.

Tom

Re: GCC 4.1: Buildable on GHz machines only?

[Tom Tromey]

>>>>> "David" == David Daney <ddaney@avtrex.com> writes:

David> Perhaps the crazy person that only needs 2MB worth of the files from
David> said static library when the corresponding shared library is
David> 8MB. Especially if this lunatic is trying to make the program, OS
David> kernel etc fit in an 8MB flash memory device.

In general I think this thread is merging several issues that, while
related, also require separate consideration:

* Build times are too slow
* Many programs that one might run on a Linux distro don't require all
  of libgcj.so; therefore it is too big
* In some environments (including, surprisingly to me, Linux) it is
  convenient to ship a statically-linked-to-libgcj app
* The humongosity of libgcj crushes embedded boxes

In particular I think build times should ideally be addressed as a
separate issue from deployment approaches, as the latter represent
real user requirements, whereas the former can be turned into a
collection of ad hoc hacks if need be.

Splitting up libgcj.so probably makes sense even for the Linux distro
case (the one I am most concerned with at the moment), just so that
apps that don't use AWT or Swing don't really pay for it.  The
overhead of, say, BC-compiling AWT and putting it in a separate .so
doesn't seem unreasonable, based on our experiences with BC compiling
large apps.  (Even this situation has its losers, for instance CNI
users accessing AWT, like the xlib peers.  It looks impossible to
please everybody 100%.)

This split would help Windows users, too, I think, as at least AWT and
Swing are fairly useless on that platform (no native peers; you can
run the Gtk peers but that is weird).

For embedded platforms, I would not mind seeing patches to allow
completely disabling parts of the build based on configure switches.
E.g., a switch to drop AWT entirely would be fine by me.  Someone
would probably have to tend to this to prevent it breaking; I'd be
reluctant to accept it as "fully supported" simply because I know I
would never be building that way.  (There is obviously some limit to
how far this program could be carried; and at the extrema of
customizability we would either need a radically different approach,
or you would just have to accept that you need local changes.)

FWIW, as far as splitting goes, I think it isn't just AWT/Swing,
though those are the big ones.  RMI and java.sql could probably be
meaningfully broken off into separate libraries as well.  It isn't
clear how fine we should grind this.


One actual conflict I foresee is that, as we compile more of the core
library with -findirect-dispatch, it will get harder to use static
linking sensibly.  This is still a ways away, as the CNI problem has
not yet been solved; and in any case exactly where the BC-ification
will stop is still unknown.  But, we have already started this and I
think more will be showing up in the near future.

Perhaps this can be solved by letting folks optionally compile libgcj
with the C++ ABI.  -whole-archive also "solves" this, though it
eliminates the useful ability of static linking to drop the bits you
aren't using (I think this is irreconcilable with how BC works).


Our thinking has been that we would solve a lot of gcj problems with
the BC ABI.  And, in particular, once it is solid and complete, we
would make it the default and then feel free to break C++ ABI
compatibility even in minor releases; the idea being that this would
be a way around the fact that GCC's release schedule is too slow for
libgcj.

I'm not sure what Plan B would be.  Maybe separate libgcj releases
somehow.

Tom

Re: GCC 4.1: Buildable on GHz machines only?

[Andi Kleen]

Tom Tromey <tromey@redhat.com> writes:
>
> Splitting up libgcj.so probably makes sense even for the Linux distro
> case (the one I am most concerned with at the moment), just so that
> apps that don't use AWT or Swing don't really pay for it.  The

Hmm? Unless you initialize AWT/swing in all programs that code
should never be paged in for non GUI programs. Ok in theory
if you use a random build order then a lot of pages could
contain GUI and non GUI code together, but that is probably
unlikely.

The only reason to split it out would be to allow a libgcj
installation that is not dependent on the X11 libraries on the
RPM/dpkg/etc. level for small setups, but I am not sure how useful
that is anymore for most distributions.

And perhaps to make the linking steps in the gcc build a bit
faster, but just for that it seems like a lot of work.

> overhead of, say, BC-compiling AWT and putting it in a separate .so
> doesn't seem unreasonable, based on our experiences with BC compiling
> large apps.  (Even this situation has its losers, for instance CNI
> users accessing AWT, like the xlib peers.  It looks impossible to
> please everybody 100%.)
>
> This split would help Windows users, too, I think, as at least AWT and
> Swing are fairly useless on that platform (no native peers; you can
> run the Gtk peers but that is weird).
>
> For embedded platforms, I would not mind seeing patches to allow
> completely disabling parts of the build based on configure switches.
> E.g., a switch to drop AWT entirely would be fine by me.  Someone

Shouldnt the linker do that already at runtime? Ok you pay the cost
of building it when gcc is built and storing the libraries on disk,
but in the final executable all should be only linked in when actually
referenced.

I know it can take quite some effort to make libraries really static
linking friendly so that libraries dont pull in other code intentionally
(take a look at nm of a a static glibc program to see how it should be not
done), but I would hope at least that the GUI and non GUI parts 
of libgcj are clearly separated.

-Andi

Re: GCC 4.1: Buildable on GHz machines only?

[Steinar Bang]

>>>>> Mike Stump <mrs@apple.com>:

> On Thursday, May 5, 2005, at 02:53  PM, Andi Vajda wrote:
>> I wish the same were possible on Linux and Mac OS X but I have not
>> been able to create a shared library that is statically linked
>> against libgcj.a

> Should just work, though, you don't want to link -static built objects 
> into a .dylib, you merely want to link in -fPIC built objects from a .a 
> into the dylib.  Can you manage to do this with a small example?

It's possible to do this.  I've done it on linux.  But if pieces of
the .a appear in several .so files, and if these pieces comes from
different versions, or different build configurations of the .a, there
may be trouble.

If you use -Bsymbolic when linking the .so files, you _may_ be able to
make each .so file see the symbols from its own .a file.  I wasn't
able to make it do this.  I was only able to use it to make an .so
look inside itself for symbols, before looking at the globally
available symbols.

More here:
	http://thread.gmane.org/gmane.text.xml.expat.general/920

Re: GCC 4.1: Buildable on GHz machines only?

[Fernando Lozano]

Hi Andi,

>>Splitting up libgcj.so probably makes sense even for the Linux distro
>>case (the one I am most concerned with at the moment), just so that
>>apps that don't use AWT or Swing don't really pay for it.  The
>>    
>>
>The only reason to split it out would be to allow a libgcj
>installation that is not dependent on the X11 libraries on the
>RPM/dpkg/etc. level for small setups, but I am not sure how useful
>that is anymore for most distributions.
>  
>
If I develop for an embebed Linux distro (like Familiar for iPAQs) and 
use JavaGnome (so I can get the customized widgets for small screen 
sizes) it would save a lot of flash do not have awt and swing on 
libgcj.  Same if you opt for SWT.

[]s, Fernando Lozano

Re: GCC 4.1: Buildable on GHz machines only?

[Richard Earnshaw]

On Sat, 2005-05-07 at 13:58, Andi Kleen wrote:
> Tom Tromey <tromey@redhat.com> writes:
> >
> > Splitting up libgcj.so probably makes sense even for the Linux distro
> > case (the one I am most concerned with at the moment), just so that
> > apps that don't use AWT or Swing don't really pay for it.  The
> 
> Hmm? Unless you initialize AWT/swing in all programs that code
> should never be paged in for non GUI programs. Ok in theory
> if you use a random build order then a lot of pages could
> contain GUI and non GUI code together, but that is probably
> unlikely.
> 
> The only reason to split it out would be to allow a libgcj
> installation that is not dependent on the X11 libraries on the
> RPM/dpkg/etc. level for small setups, but I am not sure how useful
> that is anymore for most distributions.
> 
> And perhaps to make the linking steps in the gcc build a bit
> faster, but just for that it seems like a lot of work.

Don't forget that the dynamic linker still has to do some relocation at
load time.  Not all platforms support pre-linking to mitigate this, and
even those that do don't necessarily have it done on all shared
libraries.

The size of libgcj is such that it has non-trivial amounts of start up
linking to do.  If you then add in redundant X libraries that it pulls
in for apps that don't need the graphics the overall effect can be
significant.

R.

Re: GCC 4.1: Buildable on GHz machines only?

[Peter Barada]

>We're not talking about 5% speedup; if the linker starts thrashing because
>of insufficient memory you pay far more than that.  And certainly anyone
>with an older computer who is dissatified with its performance, but
>doesn't have a lot of money, should look into getting more memory before
>anything else.  Still, the GNU project shouldn't be telling people in the
>third world with cast-off machines that they are out of luck; to many of
>them, 256M is more than they have.

Also don't forget us embedded people that are *desperately* trying to
do native compilations using an NFSroot with limited main memory and
don't have a disk in the hardware design to swap to.

-- 
Peter Barada
peter@the-baradas.com

Re: GCC 4.1: Buildable on GHz machines only?

[Richard Earnshaw]

On Mon, 2005-05-16 at 13:19, Robert Dewar wrote:

> > 
> > Also don't forget us embedded people that are *desperately* trying to
> > do native compilations using an NFSroot with limited main memory and
> > don't have a disk in the hardware design to swap to.
> 
> Why would you work in such a crippled environment?
> 

One reason is that many people write open-source packages that can't be
cross compiled (there are far too many autoconf scripts around that try
to build executable test programs).

Robert, please stop trying to shoot the messenger.  The problems are
real, and users often cannot 'fix' these problems themselves.  Just like
they can't 'fix' the compiler bloat themselves.

R.

Re: GCC 4.1: Buildable on GHz machines only?

[Richard Earnshaw]

On Mon, 2005-05-16 at 13:44, Robert Dewar wrote:
> Richard Earnshaw wrote:
> 
> > Robert, please stop trying to shoot the messenger.  The problems are
> > real, and users often cannot 'fix' these problems themselves.  Just like
> > they can't 'fix' the compiler bloat themselves.
> 
> Right, but again, if developers make bad decisions about
> development environments, it is not clear that gcc should
> be trying to bail them out.

I didn't say it was the developers of the package, I said it was the
folks trying to *build* the package.  These are very often different
people in the open-source community.  The folks building the package are
restricted in whether or not they can do cross compilation by whether or
not the *developers* have built that facility into their package. 
Sadly, 90%[1] of the time they have not.


>  Personally, I would rather have
> a gcc generating better code and using more memory, than
> the other way round. Of course, there are limits, and the
> places that gcc uses unreasonable amounts of memory should
> be fixed. But making it a goal to compile in less than 256 megs
> seems dubious in days when any decently configured notebook
> should have at least a gig or memory (mine has 2 gigs).

I find this laughable.  Most desktop machines around here have between
512M and 1G.  And if I were lucky enough to own a laptop with 2G of
memory I'd want it to be used to avoid having to spin up the disk on a
regular basis, not as squander-resource for compiler developers who were
being too lazy to think through the consequences of their decisions.  
Anyway, with all that extra memory I'd rather be running multiple
parallel compilations than one single one that consumed all the RAM,
especially if I had a multi-core processor.

R.

[1] Random number, I've not measured it.  But I'd be very surprised if
it were less than this.

Re: GCC 4.1: Buildable on GHz machines only?

[Paul Koning]

>>>>> "Robert" == Robert Dewar <dewar@adacore.com> writes:

 Robert> Peter Barada wrote:
 >>> We're not talking about 5% speedup; if the linker starts
 >>> thrashing because of insufficient memory you pay far more than
 >>> that.  And certainly anyone with an older computer who is
 >>> dissatified with its performance, but doesn't have a lot of
 >>> money, should look into getting more memory before anything else.
 >>> Still, the GNU project shouldn't be telling people in the third
 >>> world with cast-off machines that they are out of luck; to many
 >>> of them, 256M is more than they have.

 Robert> Sure it would be nice if GCC could operate well on obsolete
 Robert> machines but you can't expect the mainline development to pay
 Robert> too much attention to extreme cases like this. After all, you
 Robert> can buy from Dell today a 2.4GHz machine with a 17" monitor,
 Robert> DVD drive, and 256Meg memory for $299 complete. 

Certainly.  But GCC doesn't build well at all on such a machine.  It
seems to me that the expectation right now is at least a gig or two of
RAM.  My machine roughly fits your description except for having 512
meg of RAM, and builds on it are ok only if you avoid Java.

 Robert> Perhaps a separate project dedicated to old tiny machines is
 Robert> the way to go.

I.e., all non-GHz platforms should be on the obsolete list?

      paul

Re: GCC 4.1: Buildable on GHz machines only?

[Peter Barada]

>> Also don't forget us embedded people that are *desperately* trying to
>> do native compilations using an NFSroot with limited main memory and
>> don't have a disk in the hardware design to swap to.
>
>Why would you work in such a crippled environment?

Arrrrgh!

Believe me, I do as much work on a 3+Ghz 2GbDDR x86 box, but then
I'm literally screwed by the plethora of Linux packages that just
can't cross build because their configure thinks it can build/run test
programs to figure out things like byte ordering, etc.  Take perl, zlib,
openssh, as an example.  Also there are so many interdependencies
between packages that we have to build a pile of libraries and support
stuff that is never used on the target just so we can get a package
that we do need to configure/build(like sed and perl).

Until package maintainers take cross-compilation *seriously*, I have
no choice but to do native compilation of a large hunk of the packages
on eval boards that can literally takes *DAYS* to build.

We embedded linux developers have been harping on this for the past
couple of years, but no one really takes our problem seriously.
Instead we keep getting the "get faster hardware" as the patent
cure-all to execution speed problems, but in my case, there is no
other hardware I can use.

-- 
Peter Barada
peter@the-baradas.com

Re: GCC 4.1: Buildable on GHz machines only?

[Steven Bosscher]

On Monday 16 May 2005 16:53, Scott Robert Ladd wrote:
> The problem is, a bloated GCC has no consequences for the majority of
> GCC developers -- their employers have other (and valid) concerns. It's
> less a matter of laziness than it is of not caring outside one's own
> backyard.

And to second your point in an awkward way: I don't see this as a
problem.  If all those people who think this is a problem would
also fund GCC development (with hard cash or with developers), who
knows, probably things would look different.

But AFAICT even the developers who work on embedded targets focus
on code quality and new features, instead of on the compile time
and memory footprint issues that you would expect their group of
users to complain about.

Gr.
Steven

Re: GCC 4.1: Buildable on GHz machines only?

[Peter Barada]

>But AFAICT even the developers who work on embedded targets focus
>on code quality and new features, instead of on the compile time
>and memory footprint issues that you would expect their group of
>users to complain about.

I think that most of us embedded developers are trying to keep up with
where GCC is going.  Personally I spend most of my time in
gcc/config/m68k instead of the optimizers since its the target
description that I know, not the optimizers.

Also the mainline developers for x86 don't have the constraints that
we have, so its a case of "out of sight, out of mind" and a batch of
them have those glitzy workstations that they build native code for
instead of the hardware us embedded developers have.

Since I don't have any choice but to build natively on what to GCC
developers is "crippled hardware" (only 263 BogoMips) then it takes
somwhere 20 times as long to build the packages, and a "minor" 3%
slowdown means it takes a *lot* longer to go through a build cycle.
This also means that I can't track snapshots since they show up
quicker than the amount of raw compute time to just build everything
while hoping that the build doesn't blow its brains out due to a "minor"
increase in memory consumption.  

-- 
Peter Barada
peter@the-baradas.com

Re: GCC 4.1: Buildable on GHz machines only?

[Richard Earnshaw]

On Mon, 2005-05-16 at 16:17, Steven Bosscher wrote:
> On Monday 16 May 2005 16:53, Scott Robert Ladd wrote:
> > The problem is, a bloated GCC has no consequences for the majority of
> > GCC developers -- their employers have other (and valid) concerns. It's
> > less a matter of laziness than it is of not caring outside one's own
> > backyard.
> 
> And to second your point in an awkward way: I don't see this as a
> problem.  If all those people who think this is a problem would
> also fund GCC development (with hard cash or with developers), who
> knows, probably things would look different.

if only it were that simple[1].  However, even if the money does get
spent it's unlikely to help because there are too many developers that
just DON'T CARE about (or worse, seem to be openly hostile to) making
the compiler more efficient.

No company is going to spend money on fixing this until we adjust our
(collective) attitude and take this seriously.  If one person can
continue to undo the good work of a dozen others with one lousy commit
we'll never get anywhere here.

R.

[1] Spending money is never simple ;-)

Re: GCC 4.1: Buildable on GHz machines only?

[Daniel Berlin]

On Mon, 2005-05-16 at 16:53 +0100, Richard Earnshaw wrote:
> On Mon, 2005-05-16 at 16:17, Steven Bosscher wrote:
> > On Monday 16 May 2005 16:53, Scott Robert Ladd wrote:
> > > The problem is, a bloated GCC has no consequences for the majority of
> > > GCC developers -- their employers have other (and valid) concerns. It's
> > > less a matter of laziness than it is of not caring outside one's own
> > > backyard.
> > 
> > And to second your point in an awkward way: I don't see this as a
> > problem.  If all those people who think this is a problem would
> > also fund GCC development (with hard cash or with developers), who
> > knows, probably things would look different.
> 
> if only it were that simple[1].  However, even if the money does get
> spent it's unlikely to help because there are too many developers that
> just DON'T CARE about (or worse, seem to be openly hostile to) making
> the compiler more efficient.

They don't care because nobody pays them to care (IE you've got it
backwards), and they have other higher priority spare time projects that
they like to work on.

If you want to change the priorities of paid developers, you will have
to do so by affecting the work they are paid to do, not by trying to
convince them that speeding up the compiler is better than whatever
hobby projects they enjoy working on.  This is because speeding up the
compiler is almost never an enjoyable hobby project :).

Re: GCC 4.1: Buildable on GHz machines only?

[Steven Bosscher]

On Monday 16 May 2005 17:53, Richard Earnshaw wrote:
> On Mon, 2005-05-16 at 16:17, Steven Bosscher wrote:
> > On Monday 16 May 2005 16:53, Scott Robert Ladd wrote:
> > > The problem is, a bloated GCC has no consequences for the majority of
> > > GCC developers -- their employers have other (and valid) concerns. It's
> > > less a matter of laziness than it is of not caring outside one's own
> > > backyard.
> >
> > And to second your point in an awkward way: I don't see this as a
> > problem.  If all those people who think this is a problem would
> > also fund GCC development (with hard cash or with developers), who
> > knows, probably things would look different.
>
> if only it were that simple[1].  However, even if the money does get
> spent it's unlikely to help because there are too many developers that
> just DON'T CARE about (or worse, seem to be openly hostile to) making
> the compiler more efficient.

I've not seen anyone who is hostile to the idea of making the compiler
more efficient.  But it is difficult to expect people to care about a
thing that is not a problem for them.  I think it _would_ help if there
were people working specifically on reducing e.g. the memory footprint.
It is not like we don't know where the problems are, there is just not
a soul who cares enough to fix these problems.  Most souls start caring
when there is a monetary compensation in it for them ;-)

> No company is going to spend money on fixing this until we adjust our
> (collective) attitude and take this seriously.

Agreed.

A lot of work has already been done on speeding up the compiler, but
we are not really going anywhere if we add 80+ tree passes and remove
nothing.  There is also a SUSE bot that posts to gcc-regressions if
the memory footprint has increased, but people are completely ignoring
it.

>  If one person can
> continue to undo the good work of a dozen others with one lousy commit
> we'll never get anywhere here.

I don't think people are deliberately sabotaging efforts to make GCC
faster/smaller/etc.  And there are rules for what is considered a
regression, and for what to do with patches that cause them.

Gr.
Steven

Re: GCC 4.1: Buildable on GHz machines only?

[Richard Earnshaw]

On Mon, 2005-05-16 at 17:03, Daniel Berlin wrote:

> > if only it were that simple[1].  However, even if the money does get
> > spent it's unlikely to help because there are too many developers that
> > just DON'T CARE about (or worse, seem to be openly hostile to) making
> > the compiler more efficient.
> 
> They don't care because nobody pays them to care (IE you've got it
> backwards), and they have other higher priority spare time projects that
> they like to work on.
> 

It shouldn't be necessary to pay every developer to care.  We need to
buy into the fact that if some of the developer community cares enough
to pay for the work to be done, then doing things that undo that work
are going to be unpopular.  That is, we should treat increases in memory
usage/slow-downs in the compiler as regressions in the same way as we
treat worse code as regressions.  That's the only way we'll ever get
serious about this.  Unless and until we can accept this then nobody is
going to put money into it, because it'll just be wasted money.

> If you want to change the priorities of paid developers, you will have
> to do so by affecting the work they are paid to do, not by trying to
> convince them that speeding up the compiler is better than whatever
> hobby projects they enjoy working on.  This is because speeding up the
> compiler is almost never an enjoyable hobby project :).

I'm fully aware of this fact.  It doesn't change things though.  If we
are serious about engineering a good compiler, then we need to be just
as serious about these issues.

R.

Re: GCC 4.1: Buildable on GHz machines only?

[Paolo Bonzini]

 > Also there are so many interdependencies
> between packages that we have to build a pile of libraries and support
> stuff that is never used on the target just so we can get a package
> that we do need to configure/build(like sed and perl).

Please give me as much information as possible on sed.  AFAIK, 
configuring with --disable-nls should be enough to skip libiconv, 
libintl, etc. and cross-build.

Paolo

Re: GCC 4.1: Buildable on GHz machines only?

[Peter Barada]

>> Also there are so many interdependencies
>> between packages that we have to build a pile of libraries and support
>> stuff that is never used on the target just so we can get a package
>> that we do need to configure/build(like sed and perl).
>
>Please give me as much information as possible on sed.  AFAIK, 
>configuring with --disable-nls should be enough to skip libiconv, 
>libintl, etc. and cross-build.

I don't think sed has a problem cross-building, its just all the junk
that each package uses in its configure that if it *has* to be
natively built that compounds the problem.

-- 
Peter Barada
peter@the-baradas.com

Re: GCC 4.1: Buildable on GHz machines only?

[Steven Bosscher]

[-- Attachment #1: Type: text/plain, Size: 1705 bytes --]

On Monday 16 May 2005 18:15, Richard Earnshaw wrote:
> On Mon, 2005-05-16 at 17:03, Daniel Berlin wrote:
> > > if only it were that simple[1].  However, even if the money does get
> > > spent it's unlikely to help because there are too many developers that
> > > just DON'T CARE about (or worse, seem to be openly hostile to) making
> > > the compiler more efficient.
> >
> > They don't care because nobody pays them to care (IE you've got it
> > backwards), and they have other higher priority spare time projects that
> > they like to work on.
>
> It shouldn't be necessary to pay every developer to care.  We need to
> buy into the fact that if some of the developer community cares enough
> to pay for the work to be done, then doing things that undo that work
> are going to be unpopular.  That is, we should treat increases in memory
> usage/slow-downs in the compiler as regressions in the same way as we
> treat worse code as regressions.  That's the only way we'll ever get
> serious about this.  Unless and until we can accept this then nobody is
> going to put money into it, because it'll just be wasted money.

Just for the record, attached is gcctest's history of the overall
memory requirement at -O[0123] for combine.i, insn-attrtab.i, and
generate.ii (aka PR8361).  Honza's bot has been sending these
reports since Septemper 2004, so that's where I started.

It's not perfectly accurate, but it gives some indication of how
much worse we have made GCC since September last year wrt. memory
consumption.  Actually, the graph shows that things have improved
a lot since then.  Especially in the last two months.  I don't have
numbers from GCC3 based compilers to compare with.

Gr.
Steven


[-- Attachment #2: out.pdf --]
[-- Type: application/pdf, Size: 62816 bytes --]

Re: GCC 4.1: Buildable on GHz machines only?

[Georg Bauhaus]

Peter Barada wrote:

> Until package maintainers take cross-compilation *seriously*,

Or cross-programming in general,
or until GNU programmers write software in a way such that if
the GNU platform changes, translation of configuration tools is
still possible by design.

I've just given up running the GCC 3.4 testsuite on MacOS X 10.2,
after spending a few days getting from 3.1-incl-Ada -> 3.3-incl-Ada
(src 1495) -> system-3.3 + 3.3-incl-Ada -> 3.4.
I've got a compiler, after injection of a number of shoehorns
into Make-lang.in and into the system.
The configuration trouble appears not to be caused by the GCC
sources proper, but mostly by the assumption-based configuration
programs used and *their* associated version hel^H^H^H dependence
graph etc.. (one workaround in another thread.)

Yes, MacOS X 10.2 is old, not GNU etc., still it seems to me that
the quality of the configuration programs could be improved by
just slightly relaxing the at-least-complete-GNU-x86 or equivalent
assumption, if only the attitude towards configuration changes.
This can't hurt even within GNU. End of rant.


Georg

Re: GCC 4.1: Buildable on GHz machines only?

[Russ Allbery]

Peter Barada <peter@the-baradas.com> writes:

> Until package maintainers take cross-compilation *seriously*, I have no
> choice but to do native compilation of a large hunk of the packages on
> eval boards that can literally takes *DAYS* to build.

And package maintainers will never take cross-compilation seriously even
if they really want to because they, for the most part, can't test it.
Very few people who are not cross-compiling for specific reasons have any
sort of cross-compilation setup available or even know how to start with
one, and it's the sad fact in software development that anything that
isn't regularly tested breaks.

Most free software packages are doing well if they even have a basic test
suite for core features, let alone something perceived as obscure like
cross-compilation.

To really make cross-compilation work on a widespread basis would require
a huge amount of effort in setting up automated test environments where
package maintainers could try it out, along with a lot of help in
debugging problems and providing patches.  It seems unlikely to me that
it's going to happen outside the handful of packages that are regularly
used in cross-build environments and receive active regular testing by
people who are part of the development team (like gcc).

-- 
Russ Allbery (rra@stanford.edu)             <http://www.eyrie.org/~eagle/>

Re: GCC 4.1: Buildable on GHz machines only?

[Karel Gardas]

On Mon, 16 May 2005, Steven Bosscher wrote:

> Just for the record, attached is gcctest's history of the overall
> memory requirement at -O[0123] for combine.i, insn-attrtab.i, and
> generate.ii (aka PR8361).  Honza's bot has been sending these
> reports since Septemper 2004, so that's where I started.

Is it possible to also add -Os to your tested option set? IMHO this option 
is quite necessary for embedded developers who seems to complain in this 
thread.

Thanks,
Karel
--
Karel Gardas                  kgardas@objectsecurity.com
ObjectSecurity Ltd.           http://www.objectsecurity.com

Re: GCC 4.1: Buildable on GHz machines only?

[Alexandre Oliva]

On May 16, 2005, Robert Dewar <dewar@adacore.com> wrote:

> After all, you can buy from Dell today a 2.4GHz machine with a 17"
> monitor, DVD drive, and 256Meg memory for $299 complete. Sure, some
> people cannot even afford that, but it is not clear that the gcc
> project can regard this as a major user segment that should be taken
> into account.

Just step back for a second and consider that the most common
computation platform these days is cell phones.  Also consider that a
number of cell phone manufacturers are adopting, or considering
adopting, GNU/Linux.  Consider that at least some of them are going to
enable users to download programs into the cell phones and run them.
Also consider that not all cell phones are identical.

Now wouldn't it be nice to be able to download some useful program in
source form and build it locally on your cell phone, while on the
road?  Sure, few people might be able to accomplish that without a
nice building wizard front-end, but that's doable.  Would we want GCC
to be tool that prevents this vision from coming true?

-- 
Alexandre Oliva             http://www.ic.unicamp.br/~oliva/
Red Hat Compiler Engineer   aoliva@{redhat.com, gcc.gnu.org}
Free Software Evangelist  oliva@{lsd.ic.unicamp.br, gnu.org}

Re: GCC 4.1: Buildable on GHz machines only?

[DJ Delorie]

> No company is going to spend money on fixing this until we adjust
> our (collective) attitude and take this seriously.

We could call ulimit() to force everyone to have less available RAM.
Connect it with one of the maintainer flags, like enable-checking or
something, so it doesn't penalize distributors.

Re: GCC 4.1: Buildable on GHz machines only?

[Andrew Pinski]

On May 16, 2005, at 2:46 PM, DJ Delorie wrote:

>
>> No company is going to spend money on fixing this until we adjust
>> our (collective) attitude and take this seriously.
>
> We could call ulimit() to force everyone to have less available RAM.
> Connect it with one of the maintainer flags, like enable-checking or
> something, so it doesn't penalize distributors.

We already do that for when checking is enabled, well the GC heuristics
are tuned such that it does not change which is why
--enable-checking=release is always faster than without it.

-- Pinski

Re: GCC 4.1: Buildable on GHz machines only?

[Alexandre Oliva]

On May 16, 2005, Russ Allbery <rra@stanford.edu> wrote:

> And package maintainers will never take cross-compilation seriously even
> if they really want to because they, for the most part, can't test it.

configure --build=i686-pc-linux-gnu \
--host=i686-somethingelse-linux-gnu 

should be enough to exercise most of the cross-compilation issues, if
you're using a sufficiently recent version of autoconf, but I believe
you already knew that.

The most serious problem regarding cross compilation is that it's
regarded as hard, so many people would rather not even bother to try
to figure it out.  So it indeed becomes a hard problem, because then
you have to fix a lot of stuff in order to get it to work.

-- 
Alexandre Oliva             http://www.ic.unicamp.br/~oliva/
Red Hat Compiler Engineer   aoliva@{redhat.com, gcc.gnu.org}
Free Software Evangelist  oliva@{lsd.ic.unicamp.br, gnu.org}

Re: GCC 4.1: Buildable on GHz machines only?

[Russ Allbery]

Alexandre Oliva <aoliva@redhat.com> writes:
> On May 16, 2005, Russ Allbery <rra@stanford.edu> wrote:

>> And package maintainers will never take cross-compilation seriously
>> even if they really want to because they, for the most part, can't test
>> it.

> configure --build=i686-pc-linux-gnu \
> --host=i686-somethingelse-linux-gnu 

> should be enough to exercise most of the cross-compilation issues, if
> you're using a sufficiently recent version of autoconf, but I believe
> you already knew that.

What, you mean my lovingly hacked upon Autoconf 2.13 doesn't work?  But I
can't possibly upgrade; I rewrote all of the option handling in a macro!

Seriously, though, I think the above only tests things out to the degree
that Autoconf would already be warning about no default specified for
cross-compiling, yes?  Wouldn't you have to at least cross-compile from a
system with one endianness and int size to a system with a different
endianness and int size and then try to run the resulting binaries to
really see if the package would cross-compile?

A scary number of packages, even ones that use Autoconf, bypass Autoconf
completely when checking certain things or roll their own broken macros to
do so.

> The most serious problem regarding cross compilation is that it's
> regarded as hard, so many people would rather not even bother to try to
> figure it out.  So it indeed becomes a hard problem, because then you
> have to fix a lot of stuff in order to get it to work.

It's not just that it's perceived as hard.  It's that it's perceived as
hard *and* obscure.  Speaking as the maintainer of a package that I'm
pretty sure could be cross-compiled with some work but that I'm also
pretty sure likely wouldn't work just out of the box, I have never once
gotten a single bug report, request, or report of anyone cross-compiling
INN.  Given that, it's hard to care except in some abstract cleanliness
sense (and I already got rid of all of the Autoconf warnings as best as I
could figure out, in the abstract caring department).

-- 
Russ Allbery (rra@stanford.edu)             <http://www.eyrie.org/~eagle/>

Re: GCC 4.1: Buildable on GHz machines only?

[DJ Delorie]

> We already do that for when checking is enabled, well the GC heuristics
> are tuned such that it does not change which is why
> --enable-checking=release is always faster than without it.

Right, but it doesn't call ulimit(), so other sources of memory
leakage wouldn't be affected.  I'm thinking if the gcc driver set a
per-process limit of, say, 128M, developers would learn to care about
working set performance.

Re: GCC 4.1: Buildable on GHz machines only?

[Hugh Sasse]

On Mon, 16 May 2005, Alexandre Oliva wrote:

> On May 16, 2005, Robert Dewar <dewar@adacore.com> wrote:
>
>> After all, you can buy from Dell today a 2.4GHz machine with a 17"
>> monitor, DVD drive, and 256Meg memory for $299 complete. Sure, some
>> people cannot even afford that, but it is not clear that the gcc
>> project can regard this as a major user segment that should be taken
>> into account.
>
> Just step back for a second and consider that the most common
> computation platform these days is cell phones.  Also consider that a
> number of cell phone manufacturers are adopting, or considering
> adopting, GNU/Linux.  Consider that at least some of them are going to
         [...]

Is it pertinent to remind people of the wider spread of Free
Software, such as Bangladesh (Brave GNU World, issue 56) and Africa
(various issues of Brave GNU World Eg 53,43) where people have
considerably more difficulties keeping up with Moore's Law?

Apologies if you didn't need reminding. :-)

I approve of the spirit of the suggestion about ulimit.

         Hugh

Re: GCC 4.1: Buildable on GHz machines only?

[Florian Weimer]

* Russ Allbery:

> Seriously, though, I think the above only tests things out to the degree
> that Autoconf would already be warning about no default specified for
> cross-compiling, yes?  Wouldn't you have to at least cross-compile from a
> system with one endianness and int size to a system with a different
> endianness and int size and then try to run the resulting binaries to
> really see if the package would cross-compile?

Is this really necessary?  I would think that a LD_PRELOADed DSO which
prevents execution of freshly compiled binaries would be sufficient to
catch the most obvious errors.

If configure is broken, you can still bypass it and manually write a
config.h.  Even I can remember the days when this was a rather common
task, even when you were not cross-compiling.

> It's not just that it's perceived as hard.  It's that it's perceived as
> hard *and* obscure.

Well, it's hard to keep something working which you cannot test
reliably.  I think it would be pretty straightforward to support some
form of cross-compiling for the software I currently maintain
(especially if I go ahead and write that GCC patch for exporting
structure layout and compile-time constants), but there's no point in
doing so if it's not requested by users, I cannot test it as part of
the release procedures, and anybody who needs a binary can typically
cross-compile it without much trouble anyway ("vi config.h ; gcc
*/*.o").

Re: GCC 4.1: Buildable on GHz machines only?

[Mike Stump]

On May 16, 2005, at 12:25 PM, Hugh Sasse wrote:
> Is it pertinent to remind people of the wider spread of Free
> Software, such as Bangladesh (Brave GNU World, issue 56) and Africa
> (various issues of Brave GNU World Eg 53,43) where people have
> considerably more difficulties keeping up with Moore's Law?

?  I'd predict they are exponential too...  They just lag by, oh,  
5-40 years.

Re: GCC 4.1: Buildable on GHz machines only?

[Joseph S. Myers]

On Mon, 16 May 2005, Steven Bosscher wrote:

> Just for the record, attached is gcctest's history of the overall
> memory requirement at -O[0123] for combine.i, insn-attrtab.i, and
> generate.ii (aka PR8361).  Honza's bot has been sending these
> reports since Septemper 2004, so that's where I started.

When memory consumption regresses, I think it's necessary to *file bugs in 
Bugzilla* reporting the issue, and quite likely to track down the 
responsible patch and add the responsible individual to the CC list of the 
PR.

Bots are useful, but they don't narrow things down to an individual 
patch and don't lead to the problem's existence being tracked beyond the 
immediate discussion.  Automatic regression testers are more effective 
when backed up by the people running them examining all the reported 
regressions and reporting them (less those which are already fixed or 
reported or are just noise from problems with the tester or tests which 
randomly pass or fail) to Bugzilla.

That's what I do with all testsuite regressions for C or C++ appearing on 
i686-linux, ia64-hpux, hppa2.0w-hpux or hppa64-hpux.  When a bug is 
reported this way there is a significant chance that there will be 
productive discussion involving the people responsible for causing or 
exposing the bug and leading to it being fixed.  (This doesn't always 
happen, especially for regressions only showing up on a subset of targets; 
so the reporter or someone else who cares about the problem may need to 
fix the problem someone else caused in the end.  Bugs 20605 and 21050 are 
examples of testsuite regressions affecting at least one secondary release 
platform which have the responsible patch identified but little attention 
shown.)

There has been discussion of regression testers automatically reporting 
failures to Bugzilla, and even a "regression" component existing for that 
purpose (presumably with the idea that people will refile those bugs into 
more specific components after analysis) - but there are only two open 
bugs in that component, both apparently manually reported.  From 
experience I think having people look at the regressions before reporting 
them in order at least to identify the distinct bugs involved and whether 
any are already known is desirable; at least it would avoid floods of 
automatic duplicate bugs for essentially the same issue.

-- 
Joseph S. Myers               http://www.srcf.ucam.org/~jsm28/gcc/
    jsm@polyomino.org.uk (personal mail)
    joseph@codesourcery.com (CodeSourcery mail)
    jsm28@gcc.gnu.org (Bugzilla assignments and CCs)

Re: GCC 4.1: Buildable on GHz machines only?

[Karel Gardas]

On Mon, 16 May 2005, DJ Delorie wrote:

>
>> We already do that for when checking is enabled, well the GC heuristics
>> are tuned such that it does not change which is why
>> --enable-checking=release is always faster than without it.
>
> Right, but it doesn't call ulimit(), so other sources of memory
> leakage wouldn't be affected.  I'm thinking if the gcc driver set a
> per-process limit of, say, 128M, developers would learn to care about
> working set performance.

I like the idea, but will it really work? While compiling MICO I hardly 
see mem usage below 128MB on 512MB/1GB RAM boxes, perhaps more on 512MB 
due to memory usage heuristic(s) -- so I assume setting hard ulimit to 
128MB will just result in build process crashing instead of slowdown and 
swapping, which would man get while using mem=128m as a linux boot param. 
Or am I completely mistaken?

Thanks,
Karel
--
Karel Gardas                  kgardas@objectsecurity.com
ObjectSecurity Ltd.           http://www.objectsecurity.com

Re: GCC 4.1: Buildable on GHz machines only?

[Steven Bosscher]

On Monday 16 May 2005 20:26, Karel Gardas wrote:
> On Mon, 16 May 2005, Steven Bosscher wrote:
> > Just for the record, attached is gcctest's history of the overall
> > memory requirement at -O[0123] for combine.i, insn-attrtab.i, and
> > generate.ii (aka PR8361).  Honza's bot has been sending these
> > reports since Septemper 2004, so that's where I started.
>
> Is it possible to also add -Os to your tested option set? IMHO this option
> is quite necessary for embedded developers who seems to complain in this
> thread.

Not interesting.  -Os is basically -O2 with some passes disabled.

Gr.
Steven

Re: GCC 4.1: Buildable on GHz machines only?

[DJ Delorie]

> so I assume setting hard ulimit to 128MB will just result in build
> process crashing instead of slowdown and swapping,

We would limit physical ram, not virtual ram.  If you do a "man
setrlimit", I'm talking about RLIMIT_RSS.  The result would be slowing
down and swapping, not crashing.

Re: GCC 4.1: Buildable on GHz machines only?

[Karel Gardas]

On Mon, 16 May 2005, DJ Delorie wrote:

>
>> so I assume setting hard ulimit to 128MB will just result in build
>> process crashing instead of slowdown and swapping,
>
> We would limit physical ram, not virtual ram.  If you do a "man
> setrlimit", I'm talking about RLIMIT_RSS.  The result would be slowing
> down and swapping, not crashing.

But will this really work? For example FreeBSD's manpage says:

``RLIMIT_RSS      The maximum size (in bytes) to which a process's resident
                   set size may grow.  This imposes a limit on the amount of
                   physical memory to be given to a process; if memory is
                   tight, the system will prefer to take memory from pro-
                   cesses that are exceeding their declared resident set
                   size.''

What I have problem understanding is the last sentence of this paragraph 
in the light of your claim that it will results in swapping especially 
when we consider developers' machines with 512MB/1GB RAM, i.e. machines 
where memory is not "tight".

Thanks,
Karel
--
Karel Gardas                  kgardas@objectsecurity.com
ObjectSecurity Ltd.           http://www.objectsecurity.com

Re: GCC 4.1: Buildable on GHz machines only?

[Peter Barada]

>What I have problem understanding is the last sentence of this paragraph 
>in the light of your claim that it will results in swapping especially 
>when we consider developers' machines with 512MB/1GB RAM, i.e. machines 
>where memory is not "tight".

Sure, and this is the point.  Pick a number for the RSS and stick to
it.  Crank it up if you don't want to be bothered, but this
"yardstick" can be used to measure trends in the compiler's footprint
by measuring the number of page swaps.  It might also be a way to
measuer/improve the locality of reference since poor locality will
cause thrashing if the RSS is set low enough.  Of course if the RSS is
set too low than *any* pattern of page access will cause thrashing.

-- 
Peter Barada
peter@the-baradas.com

Re: GCC 4.1: Buildable on GHz machines only?

[DJ Delorie]

> What I have problem understanding is the last sentence of this
> paragraph in the light of your claim that it will results in
> swapping especially when we consider developers' machines with
> 512MB/1GB RAM, i.e. machines where memory is not "tight".

Sigh, Linux works the same way.  Processes can exceed their HARD
ulimit if there happens to be memory available, making RLIMIT_RSS
basically useless.

Grrr.

Re: GCC 4.1: Buildable on GHz machines only?

[Ralf Corsepius]

On Mon, 2005-05-16 at 10:42 -0400, Peter Barada wrote:

> Until package maintainers take cross-compilation *seriously*, I have
> no choice but to do native compilation of a large hunk of the packages
> on eval boards that can literally takes *DAYS* to build.

The most amazing fact to me is: Not even GCC seems to take cross-
compilation seriously :(

Ralf

Re: GCC 4.1: Buildable on GHz machines only?

[Joe Buck]

On Mon, May 16, 2005 at 10:15:29PM +0200, Steven Bosscher wrote:
> On Monday 16 May 2005 20:26, Karel Gardas wrote:
> > On Mon, 16 May 2005, Steven Bosscher wrote:
> > > Just for the record, attached is gcctest's history of the overall
> > > memory requirement at -O[0123] for combine.i, insn-attrtab.i, and
> > > generate.ii (aka PR8361).  Honza's bot has been sending these
> > > reports since Septemper 2004, so that's where I started.
> >
> > Is it possible to also add -Os to your tested option set? IMHO this option
> > is quite necessary for embedded developers who seems to complain in this
> > thread.
> 
> Not interesting.  -Os is basically -O2 with some passes disabled.

That's not quite correct.  For example, the rule -Os uses for considering
inlining is completely different (inlining is allowed when the size of
generated code does not increase).

Re: GCC 4.1: Buildable on GHz machines only?

[Steven Bosscher]

On Monday 16 May 2005 23:43, Ralf Corsepius wrote:
> On Mon, 2005-05-16 at 10:42 -0400, Peter Barada wrote:
> > Until package maintainers take cross-compilation *seriously*, I have
> > no choice but to do native compilation of a large hunk of the packages
> > on eval boards that can literally takes *DAYS* to build.
>
> The most amazing fact to me is: Not even GCC seems to take cross-
> compilation seriously :(

BS.  Even the large disto builders do cross compilations a lot.

I am getting pretty sick of this.  Can we now start discussing
what GCC does do well, or otherwise, for further complaints
remove me from the CC: please.

I can't say all is good about GCC.  There are always ways to do
things better.  But, as Dewar already pointed out, GCC just can
not be perfect for everyone's needs.  I, for one, am very happy
that we are finally pulling GCC out of the 80s, into the 21st
century.  The compile time and memory consumption problems are
obviously there, but just complaining is not going to fix them.

Yet complaining is all some people do.  It only demotivates me
more to work on these issues that you care about, and I in all
honesty don't give a s*** about.

It is IMVHO rediculous that the list where GCC is bashed the most
is the GCC list itself.

Gr.
Steven

Re: GCC 4.1: Buildable on GHz machines only?

[Steven Bosscher]

On Tuesday 17 May 2005 00:07, Joe Buck wrote:
> On Mon, May 16, 2005 at 10:15:29PM +0200, Steven Bosscher wrote:
> > On Monday 16 May 2005 20:26, Karel Gardas wrote:
> > > On Mon, 16 May 2005, Steven Bosscher wrote:
> > > > Just for the record, attached is gcctest's history of the overall
> > > > memory requirement at -O[0123] for combine.i, insn-attrtab.i, and
> > > > generate.ii (aka PR8361).  Honza's bot has been sending these
> > > > reports since Septemper 2004, so that's where I started.
> > >
> > > Is it possible to also add -Os to your tested option set? IMHO this
> > > option is quite necessary for embedded developers who seems to complain
> > > in this thread.
> >
> > Not interesting.  -Os is basically -O2 with some passes disabled.
>
> That's not quite correct.  For example, the rule -Os uses for considering
> inlining is completely different (inlining is allowed when the size of
> generated code does not increase).

...meaning that the memory footprint at -Os is going to be smaller 
than the -O2 print for all but a few strange cases.

Gr.
Steven

Re: GCC 4.1: Buildable on GHz machines only?

[Joe Buck]

On Mon, May 16, 2005 at 05:35:43PM -0400, DJ Delorie wrote:
> 
> > What I have problem understanding is the last sentence of this
> > paragraph in the light of your claim that it will results in
> > swapping especially when we consider developers' machines with
> > 512MB/1GB RAM, i.e. machines where memory is not "tight".
> 
> Sigh, Linux works the same way.  Processes can exceed their HARD
> ulimit if there happens to be memory available, making RLIMIT_RSS
> basically useless.

The Linux kernel has a boot-time option that you can use to lie to your
kernel about how much physical memory you have.  If the number you give
is too high, you will eventually crash, but if it is too low, you can
test how well your system would behave if it had less memory.  It was
originally there because the BIOS call that Linux used to use could only
report a value up to 64m, so if you had more you had to let the kernel
know.  That problem was fixed long ago.

So you can say "mem=128m" or the like.

See

http://www.faqs.org/docs/Linux-HOWTO/BootPrompt-HOWTO.html

Re: GCC 4.1: Buildable on GHz machines only?

[DJ Delorie]

> So you can say "mem=128m" or the like.

Yes, but that doesn't help when I want to test one application on a
system that's been otherwise up and running for months, and is busy
doing other things.  The RSS limit is *supposed* to do just what we
want, but nobody seems to implement it correctly any more.

Re: GCC 4.1: Buildable on GHz machines only?

[Ralf Corsepius]

On Tue, 2005-05-17 at 00:10 +0200, Steven Bosscher wrote:
> On Monday 16 May 2005 23:43, Ralf Corsepius wrote:
> > On Mon, 2005-05-16 at 10:42 -0400, Peter Barada wrote:
> > > Until package maintainers take cross-compilation *seriously*, I have
> > > no choice but to do native compilation of a large hunk of the packages
> > > on eval boards that can literally takes *DAYS* to build.
> >
> > The most amazing fact to me is: Not even GCC seems to take cross-
> > compilation seriously :(
> 
> BS.  Even the large disto builders do cross compilations a lot.

So I suppose you have these general crossbuilding PRs fixed in your
sources:

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=21143

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=21247

Another one I haven't filed yet, is GCC-4.x not correctly propagating
CFLAGS/CFLAGS_FOR_{BUILD|HOST|TARGET} to newlib in one-tree builds (I am
still investigating).

All these to me are strong indications that GCC-4.x has been poorly
tested in cross compilation.

Ralf

Re: GCC 4.1: Buildable on GHz machines only?

[Steven Bosscher]

On Tuesday 17 May 2005 02:53, Ralf Corsepius wrote:
> On Tue, 2005-05-17 at 00:10 +0200, Steven Bosscher wrote:
> > On Monday 16 May 2005 23:43, Ralf Corsepius wrote:
> > > On Mon, 2005-05-16 at 10:42 -0400, Peter Barada wrote:
> > > > Until package maintainers take cross-compilation *seriously*, I have
> > > > no choice but to do native compilation of a large hunk of the
> > > > packages on eval boards that can literally takes *DAYS* to build.
> > >
> > > The most amazing fact to me is: Not even GCC seems to take cross-
> > > compilation seriously :(
> >
> > BS.  Even the large disto builders do cross compilations a lot.
>
> So I suppose you have these general crossbuilding PRs fixed in your
> sources:
>
> http://gcc.gnu.org/bugzilla/show_bug.cgi?id=21143

No, I just don't build gfortran as a cross.  There are many reasons
why this is a bad idea anyway.

Oh, and how helpful of you to post that patch to gcc-patches@ too...
NOT!

> http://gcc.gnu.org/bugzilla/show_bug.cgi?id=21247

I don't build Ada cross either, but AdaCore does, so you could ask
them to help you with this problem.

> Another one I haven't filed yet, is GCC-4.x not correctly propagating
> CFLAGS/CFLAGS_FOR_{BUILD|HOST|TARGET} to newlib in one-tree builds (I am
> still investigating).

I don't build with newlib either.

> All these to me are strong indications that GCC-4.x has been poorly
> tested in cross compilation.

No, just in the configurations you are using.

And since you're not posting the patches you attach to the bugzilla
PRs you open, you're not exactly helping to make things better either.

Gr.
Steven

Re: GCC 4.1: Buildable on GHz machines only?

[Steven Bosscher]

On Tuesday 17 May 2005 02:59, Steven Bosscher wrote:
> On Tuesday 17 May 2005 02:53, Ralf Corsepius wrote:
> > On Tue, 2005-05-17 at 00:10 +0200, Steven Bosscher wrote:
> > > On Monday 16 May 2005 23:43, Ralf Corsepius wrote:
> > > > On Mon, 2005-05-16 at 10:42 -0400, Peter Barada wrote:
> > > > > Until package maintainers take cross-compilation *seriously*, I
> > > > > have no choice but to do native compilation of a large hunk of the
> > > > > packages on eval boards that can literally takes *DAYS* to build.
> > > >
> > > > The most amazing fact to me is: Not even GCC seems to take cross-
> > > > compilation seriously :(
> > >
> > > BS.  Even the large disto builders do cross compilations a lot.
> >
> > So I suppose you have these general crossbuilding PRs fixed in your
> > sources:
> >
> > http://gcc.gnu.org/bugzilla/show_bug.cgi?id=21143
>
> No, I just don't build gfortran as a cross.  There are many reasons
> why this is a bad idea anyway.
>
> Oh, and how helpful of you to post that patch to gcc-patches@ too...
> NOT!

Ah, I see you did post it to gcc-patches@, but not to fortran@, which
is a requirement for gfortran patches -- and the reason why nobody
has noticed the patch.

http://gcc.gnu.org/ml/gcc-patches/2005-04/msg02287.html

The patch is OK too.

Gr.
Steven

Re: GCC 4.1: Buildable on GHz machines only?

[Joe Buck]

On Tue, May 17, 2005 at 03:11:03AM +0200, Steven Bosscher wrote:
> On Tuesday 17 May 2005 02:59, Steven Bosscher wrote:
> > Oh, and how helpful of you to post that patch to gcc-patches@ too...
> > NOT!
> 
> Ah, I see you did post it to gcc-patches@, but not to fortran@, which
> is a requirement for gfortran patches -- and the reason why nobody
> has noticed the patch.
> 
> http://gcc.gnu.org/ml/gcc-patches/2005-04/msg02287.html
> 
> The patch is OK too.

Steven, please try to be politer to someone who is trying to help.
This kind of tone will only discourage contributors.

Re: GCC 4.1: Buildable on GHz machines only?

[Steven Bosscher]

On Tuesday 17 May 2005 03:16, Joe Buck wrote:
> On Tue, May 17, 2005 at 03:11:03AM +0200, Steven Bosscher wrote:
> > On Tuesday 17 May 2005 02:59, Steven Bosscher wrote:
> > > Oh, and how helpful of you to post that patch to gcc-patches@ too...
> > > NOT!
> >
> > Ah, I see you did post it to gcc-patches@, but not to fortran@, which
> > is a requirement for gfortran patches -- and the reason why nobody
> > has noticed the patch.
> >
> > http://gcc.gnu.org/ml/gcc-patches/2005-04/msg02287.html
> >
> > The patch is OK too.
>
> Steven, please try to be politer to someone who is trying to help.

How is it helpful to not follow the rules when posting patches and
make exaggerated claims that something does not work?  All this
complaining makes me not want to contribute to GCC at all any more. 

> This kind of tone will only discourage contributors.

My tone was no different than Ralf's toward me.

This is the second time you think it necessary to "correct" me on a
public mailing list.  Don't do that.

Gr.
Steven

Re: GCC 4.1: Buildable on GHz machines only?

[Alexandre Oliva]

On May 16, 2005, Russ Allbery <rra@stanford.edu> wrote:

> Alexandre Oliva <aoliva@redhat.com> writes:
>> On May 16, 2005, Russ Allbery <rra@stanford.edu> wrote:

>>> And package maintainers will never take cross-compilation seriously
>>> even if they really want to because they, for the most part, can't test
>>> it.

>> configure --build=i686-pc-linux-gnu \
>> --host=i686-somethingelse-linux-gnu 

>> should be enough to exercise most of the cross-compilation issues, if
>> you're using a sufficiently recent version of autoconf, but I believe
>> you already knew that.

> What, you mean my lovingly hacked upon Autoconf 2.13 doesn't work?

No, just that it doesn't have the code that just compares build with
host to decide whether to enter cross-compilation mode.  Unless you
back-ported that from autoconf 2.5x, that is.

> Seriously, though, I think the above only tests things out to the degree
> that Autoconf would already be warning about no default specified for
> cross-compiling, yes?

I believe so, yes.  A configure script written with no regard to
cross-compilation may still fail to fail in catastrophic ways if
tested with native-cross.

> Wouldn't you have to at least cross-compile from a
> system with one endianness and int size to a system with a different
> endianness and int size and then try to run the resulting binaries to
> really see if the package would cross-compile?

Different endianness is indeed a harsh test on a package's
cross-compilation suitability.  Simple reliance on size of certain
types can already get you enough breakage.  Cross-building to x86 on
an x86_64 system may already catch a number of these.

> A scary number of packages, even ones that use Autoconf, bypass Autoconf
> completely when checking certain things or roll their own broken macros to
> do so.

+1

> I have never once gotten a single bug report, request, or report of
> anyone cross-compiling INN.  Given that, it's hard to care except in
> some abstract cleanliness sense

But see, you do care, and you're aware of the issues, so it just
works.  Unfortunately not all maintainers have as much knowledge or
even awareness about the subject as you do.

-- 
Alexandre Oliva             http://www.ic.unicamp.br/~oliva/
Red Hat Compiler Engineer   aoliva@{redhat.com, gcc.gnu.org}
Free Software Evangelist  oliva@{lsd.ic.unicamp.br, gnu.org}

Re: GCC 4.1: Buildable on GHz machines only?

[Alexandre Oliva]

On May 16, 2005, Florian Weimer <fw@deneb.enyo.de> wrote:

> Is this really necessary?  I would think that a LD_PRELOADed DSO which
> prevents execution of freshly compiled binaries would be sufficient to
> catch the most obvious errors.

This would break legitimate tests on the build environment, that use
e.g. CC_FOR_BUILD.

-- 
Alexandre Oliva             http://www.ic.unicamp.br/~oliva/
Red Hat Compiler Engineer   aoliva@{redhat.com, gcc.gnu.org}
Free Software Evangelist  oliva@{lsd.ic.unicamp.br, gnu.org}

Re: GCC 4.1: Buildable on GHz machines only?

[Peter Barada]

>BS.  Even the large disto builders do cross compilations a lot.

Yeah, I know.  I did consulting for a 'large disto builder'.  Do you
have a clue how long it takes to build the base packages for a PXA255
board(including X11 that won't even run on the board but is required
due to package dependecies)?  Can you think in *days*, and that was
more than a year ago.  Even then we were all concerned about the trend
in compliation speed.  Speak of what you *personally* know.

>I am getting pretty sick of this.  Can we now start discussing
>what GCC does do well, or otherwise, for further complaints
>remove me from the CC: please.

I've pulled you from the CC list, but I'm passing it on to the GCC
list in hopes that someone there cares more than you.  The RSS bloat
probelm is *not* going to go away, and *wishing* it away won't.

>I can't say all is good about GCC.  There are always ways to do
>things better.  But, as Dewar already pointed out, GCC just can
>not be perfect for everyone's needs.  I, for one, am very happy
>that we are finally pulling GCC out of the 80s, into the 21st
>century.  The compile time and memory consumption problems are
>obviously there, but just complaining is not going to fix them.

No, gcc is not perfect for all things, but the trend in resource
consumption is getting pretty serious.  As others have pointed out
before, no one complains about a resource bproblem until it gets large
enough that it made it inconvenient if not just impossible.

You don't complain to your car dealer when your car runs fine, but if
it craps out on the way to work, you'll be complaining pretty damn
loudly, expecially if its nearly brand new.

I develop GCC for ColdFire, and I have been contributing back changes
to GCC in the hopes that it will be a world-class compiler that I can
use for my work.  Unfortunately due to circumstances that have
*nothing* to do with GCC I have no choice but to build packages using
a GCC that runs natively in an Linux environment on my ColdFire V4e
embedded board where the resource constraints are *exteremely* severe,
and possibly an extra MB of RSS usage by GCC version-to-version will
be the difference between success and failure.

I have great faith in OSS and FSF code, and I don't want to demean the
valued contributions that people have made to it, but please
understand that Linux systems are built using GCC, whether its for a
workstation or an embedeed Linux device, and as such *should* consider
the problems that both encounter and not just favor the workstation end. 

-- 
Peter Barada
peter@the-baradas.com

Re: GCC 4.1: Buildable on GHz machines only?

[Michael Veksler]

Peter Barada wrote on 17/05/2005 07:12:41:

>
> >BS.  Even the large disto builders do cross compilations a lot.
>
> Yeah, I know.  I did consulting for a 'large disto builder'.  Do you
> have a clue how long it takes to build the base packages for a PXA255
> board(including X11 that won't even run on the board but is required
> due to package dependecies)?  Can you think in *days*, and that was
> more than a year ago.  Even then we were all concerned about the trend
> in compliation speed.  Speak of what you *personally* know.

If things are as bad as you say, then IMVHO you may write a small
utility for PXA255 that will impersonate a native gcc compiler. This
utility will RPC (or ssh, etc) a cross compiler on a GHz machine, and will
then pull the results back to your PXA255 (via NFS, RPC, SSH, etc).
Maybe you can even take distcc and hack it to give you what you
need. This may cut your times by a couple of orders of magnitude.

Of course, this will not help someone in Bangladesh with a Pentium.

  Michael

Re: GCC 4.1: Buildable on GHz machines only?

[Karel Gardas]

Folks,

you all are great brave men hacking on one of the most mission-critical 
free software piece ever. I'm seeing some of you are more and more 
frustrated, since this thread is turning into the flame-war.

As a long time GCC user, I would like to ask you to calm down a bit if 
this is possible, please!

Thanks,
Karel
--
Karel Gardas                  kgardas@objectsecurity.com
ObjectSecurity Ltd.           http://www.objectsecurity.com

Re: GCC 4.1: Buildable on GHz machines only?

[Ralf Corsepius]

On Tue, 2005-05-17 at 03:31 +0200, Steven Bosscher wrote:
> On Tuesday 17 May 2005 03:16, Joe Buck wrote:
> > On Tue, May 17, 2005 at 03:11:03AM +0200, Steven Bosscher wrote:
> > > On Tuesday 17 May 2005 02:59, Steven Bosscher wrote:
> > > > Oh, and how helpful of you to post that patch to gcc-patches@ too...
> > > > NOT!
> > >
> > > Ah, I see you did post it to gcc-patches@, but not to fortran@, which
> > > is a requirement for gfortran patches -- and the reason why nobody
> > > has noticed the patch.
> > >
> > > http://gcc.gnu.org/ml/gcc-patches/2005-04/msg02287.html
> > >
> > > The patch is OK too.
> >
> > Steven, please try to be politer to someone who is trying to help.
> 
> How is it helpful to not follow the rules when posting patches
Quite simple:

* I wasn't aware about this fortran specific patch posting policy. I
never have sent any gcc patch to any other list but gcc-patches for
approval before, so I also had not done so this time.

* How could I know that the responsible maintainers aren't listening to
bugzilla and gcc-patches, but are listening to a fortran specific list,
I even didn't know about until your posting?

>  and
> make exaggerated claims that something does not work?

I don't see where I exaggerated.

The fact that nobody before has hit these obvious issues, to me are
"just leaks" in GCC's QA/testing procedures/procedures, which ought to
be closed. If I weren't interested in seeing these closed, I would not
complain/file PRs on the and would not contribute/try to contribute
patches.

> > This kind of tone will only discourage contributors.
> 
> My tone was no different than Ralf's toward me.

Well, I admit I had been sarcastic/fatalistic in replying to Steven,
primarily, because I am pretty much frustrated about GCC's mainstream
developer's position/attitude on embedded targets.
Steven's answers perfectly queue-in into a long history of incidents
which had lead me to my understanding of "GCC mainstream developers'
attitude" on "embedded targets", which I already had described in former
postings.

Ralf

Re: GCC 4.1: Buildable on GHz machines only?

[Richard Earnshaw]

On Tue, 2005-05-17 at 01:59, Steven Bosscher wrote:

> No, I just don't build gfortran as a cross.  There are many reasons
> why this is a bad idea anyway.

Such as?

Re: GCC 4.1: Buildable on GHz machines only?

[Jonathan Wakely]

On Tue, May 17, 2005 at 11:14:22AM +0200, Ralf Corsepius wrote:

> * I wasn't aware about this fortran specific patch posting policy. I
> never have sent any gcc patch to any other list but gcc-patches for
> approval before, so I also had not done so this time.
> 
> * How could I know that the responsible maintainers aren't listening to
> bugzilla and gcc-patches, but are listening to a fortran specific list,
> I even didn't know about until your posting?

For future reference, where patches should be sent is explained here:
http://gcc.gnu.org/lists.html

jon

Re: GCC 4.1: Buildable on GHz machines only?

[Karel Gardas]

On Tue, 17 May 2005, Ralf Corsepius wrote:

>>> This kind of tone will only discourage contributors.
>>
>> My tone was no different than Ralf's toward me.
>
> Well, I admit I had been sarcastic/fatalistic in replying to Steven,
> primarily, because I am pretty much frustrated about GCC's mainstream
> developer's position/attitude on embedded targets.
> Steven's answers perfectly queue-in into a long history of incidents
> which had lead me to my understanding of "GCC mainstream developers'
> attitude" on "embedded targets", which I already had described in former
> postings.

Ralf,

frustration will not help anybody nor GCC itself. Please look into GCC 3.4 
and GCC 4.0 release criteria documents:

http://gcc.gnu.org/gcc-3.4/criteria.html
http://gcc.gnu.org/gcc-4.0/criteria.html

you see that 4.0 added "embedded" platforms like arm-none-elf and 
mips-none-elf to the primary platforms list. That's IMHO just a sign that 
SC takes embedded developers seriously. Anyway, if you are not satisfied 
with this list, what about to suggest to SC to add some other more real 
embedded platform to this list to at least fix some of the most obvious 
problems? What platform would you suggest? I think that if you take the
discussion into this direction then we can see very good fruits comming 
from it, at least for some future GCC releases.

Thanks and I appreciate your hard work on rtems/gcc platform!
Karel
--
Karel Gardas                  kgardas@objectsecurity.com
ObjectSecurity Ltd.           http://www.objectsecurity.com

Re: GCC 4.1: Buildable on GHz machines only?

[Steven Bosscher]

On May 17, 2005 11:29 AM, Richard Earnshaw <rearnsha@gcc.gnu.org> wrote:

> On Tue, 2005-05-17 at 01:59, Steven Bosscher wrote:
> 
> > No, I just don't build gfortran as a cross.  There are many reasons
> > why this is a bad idea anyway.
> 
> Such as?

For one thing, libgfortran requires c99 support, which is not in
newlib iiuc.

Gr.
Steven

Re: GCC 4.1: Buildable on GHz machines only?

[Richard Earnshaw]

On Tue, 2005-05-17 at 11:16, Steven Bosscher wrote:
> On May 17, 2005 11:29 AM, Richard Earnshaw <rearnsha@gcc.gnu.org> wrote:
> 
> > On Tue, 2005-05-17 at 01:59, Steven Bosscher wrote:
> > 
> > > No, I just don't build gfortran as a cross.  There are many reasons
> > > why this is a bad idea anyway.
> > 
> > Such as?
> 
> For one thing, libgfortran requires c99 support, which is not in
> newlib iiuc.

Well, technically, that's a target/deeply-embedded problem.  That
doesn't mean that it shouldn't be possible to say cross-compile fortran
for Linux targets.

R.

Re: GCC 4.1: Buildable on GHz machines only?

[Ralf Corsepius]

On Tue, 2005-05-17 at 12:16 +0200, Steven Bosscher wrote:
> On May 17, 2005 11:29 AM, Richard Earnshaw <rearnsha@gcc.gnu.org> wrote:
> 
> > On Tue, 2005-05-17 at 01:59, Steven Bosscher wrote:
> > 
> > > No, I just don't build gfortran as a cross.  There are many reasons
> > > why this is a bad idea anyway.
> > 
> > Such as?
> 
> For one thing, libgfortran requires c99 support, which is not in
> newlib iiuc.

More details please. 

Are you referring to stdint.h/inttypes.h etc.?
newlib/RTEMS has them, as well as newlib+cygwin

Ralf

Re: GCC 4.1: Buildable on GHz machines only?

[Eric Botcazou]

> For one thing, libgfortran requires c99 support, which is not in
> newlib iiuc.

In practice, no, it doesn't.  F95 works fine on Solaris 2.5.1, which is the 
typical non-C99 native platform.

-- 
Eric Botcazou

Re: GCC 4.1: Buildable on GHz machines only?

[Steven Bosscher]

On May 17, 2005 12:21 PM, Ralf Corsepius <ralf.corsepius@rtems.org> wrote:

> On Tue, 2005-05-17 at 12:16 +0200, Steven Bosscher wrote:
> > On May 17, 2005 11:29 AM, Richard Earnshaw <rearnsha@gcc.gnu.org> wrote:
> > 
> > > On Tue, 2005-05-17 at 01:59, Steven Bosscher wrote:
> > > 
> > > > No, I just don't build gfortran as a cross.  There are many reasons
> > > > why this is a bad idea anyway.
> > > 
> > > Such as?
> > 
> > For one thing, libgfortran requires c99 support, which is not in
> > newlib iiuc.
> 
> More details please. 
> 
> Are you referring to stdint.h/inttypes.h etc.?
> newlib/RTEMS has them, as well as newlib+cygwin

Some other newlib (and non-newlib) targets don't, see PR14325 and
PR16135.  There was also some issue with c99 math functions that I
have not followed closely.  Some fixes for this went in for HP-UX
and Solaris.  I don't know if newlib always provides all the math
functions libgfortran asks for.

Note that I did not mean to imply that gfortran should not be
buildable as a cross, just that I know that there used to be some
problems with this.

Gr.
Steven

Re: GCC 4.1: Buildable on GHz machines only?

[Ralf Corsepius]

On Tue, 2005-05-17 at 12:52 +0200, Steven Bosscher wrote:
> On May 17, 2005 12:21 PM, Ralf Corsepius <ralf.corsepius@rtems.org> wrote:
> 
> > On Tue, 2005-05-17 at 12:16 +0200, Steven Bosscher wrote:
> > > On May 17, 2005 11:29 AM, Richard Earnshaw <rearnsha@gcc.gnu.org> wrote:
> > > 
> > > > On Tue, 2005-05-17 at 01:59, Steven Bosscher wrote:
> > > > 
> > > > > No, I just don't build gfortran as a cross.  There are many reasons
> > > > > why this is a bad idea anyway.
> > > > 
> > > > Such as?
> > > 
> > > For one thing, libgfortran requires c99 support, which is not in
> > > newlib iiuc.
> > 
> > More details please. 
> > 
> > Are you referring to stdint.h/inttypes.h etc.?
> > newlib/RTEMS has them, as well as newlib+cygwin
> 
> Some other newlib (and non-newlib) targets don't, see PR14325 and
> PR16135. 

Well, extending the approach I chose to implement in newlib/RTEMS to
other target probably isn't too difficult, as well as it probably might
be possible to merge this approach into GCC.

>  There was also some issue with c99 math functions that I
> have not followed closely.  Some fixes for this went in for HP-UX
> and Solaris.  I don't know if newlib always provides all the math
> functions libgfortran asks for.
Neither do I know for sure, but so far, libgfortran's configure script
hasn't reported any problems.

> Note that I did not mean to imply that gfortran should not be
> buildable as a cross, just that I know that there used to be some
> problems with this.
There still are further problems on some targets (PR21203), but c99 and
math functions don't currenlty seem to be a problem with RTEMS/newlib.

Ralf

Re: GCC 4.1: Buildable on GHz machines only?

[Joel Sherrill <joel@OARcorp.com>]

This is really getting pretty far from the original topic but
I am diving in anyway.

Steven Bosscher wrote:
> On Tuesday 17 May 2005 02:53, Ralf Corsepius wrote:
> 
>>On Tue, 2005-05-17 at 00:10 +0200, Steven Bosscher wrote:
>>
>>>On Monday 16 May 2005 23:43, Ralf Corsepius wrote:
>>>
>>>>On Mon, 2005-05-16 at 10:42 -0400, Peter Barada wrote:
>>>>
>>>>>Until package maintainers take cross-compilation *seriously*, I have
>>>>>no choice but to do native compilation of a large hunk of the
>>>>>packages on eval boards that can literally takes *DAYS* to build.
>>>>
>>>>The most amazing fact to me is: Not even GCC seems to take cross-
>>>>compilation seriously :(
>>>
>>>BS.  Even the large disto builders do cross compilations a lot.
>>
>>So I suppose you have these general crossbuilding PRs fixed in your
>>sources:
>>
>>http://gcc.gnu.org/bugzilla/show_bug.cgi?id=21143
> 
> 
> No, I just don't build gfortran as a cross.  There are many reasons
> why this is a bad idea anyway.

Why is something broken in the gfortran build infrastructure?

Assuming not, then it should only be a matter of needed functionality
in the target C library and native FP types.  I know gfortran
currently makes assumptions about the FP capabilities of a CPU
and some don't meet that.  But there is no reason one should not
be able use an x86 GNU/Linux system to cross build gfortran for
a powerpc or arm GNU/Linux system.

> Oh, and how helpful of you to post that patch to gcc-patches@ too...
> NOT!

Personal gripe.. I still don't know why posting a patch with a PR isn't 
sufficient. GCC has two independent systems.  Why can't Bugzilla just 
forward attachments marked as patches?

Is there anywhere in the GCC problem reporting instructions that says
attaching a patch to a PR is insufficient?  I know you don't get that
impression from the Bugzilla page you use to add it.

>>http://gcc.gnu.org/bugzilla/show_bug.cgi?id=21247
> 
> 
> I don't build Ada cross either, but AdaCore does, so you could ask
> them to help you with this problem.

I don't really think they are the answer here.  GNAT has always been
implemented in Ada and has never been buildable without a native Ada 
compiler.  That's just the way it is.

The issue is what VERSION of GNAT is required to compile version X
of GNAT.  I don't try to track minimum versions required but in this
case, I think it moved up quite a bit.  From install.texi in gcc-4.0.0:

=================================================================
In order to build GNAT, the Ada compiler, you need a working GNAT
compiler (GNAT version 3.14 or later, or GCC version 3.1 or later),
including GNAT tools such as @command{gnatmake} and @command{gnatlink},
since the Ada front end is written in Ada (with some
GNAT-specific extensions), and GNU make.
==================================================================

So this could be viewed as only a documentation issue except that
one has to know where to get a binary for GNAT to start the build
process with.

Personally, I have always avoided this by building a native GNAT
first and using that to build the cross-GNAT.  It avoids this
issue entirely.

AdaCore has always helped avoid the problem by providing pre-built
binaries for a few platforms.  You can use these to kick-start
the process.

>>Another one I haven't filed yet, is GCC-4.x not correctly propagating
>>CFLAGS/CFLAGS_FOR_{BUILD|HOST|TARGET} to newlib in one-tree builds (I am
>>still investigating).
> 
> 
> I don't build with newlib either.

I think that's the point -- no one builds all configurations so
everyone has to be very careful to use the right build magic
to keep them all working.

>>All these to me are strong indications that GCC-4.x has been poorly
>>tested in cross compilation.
> 
> 
> No, just in the configurations you are using.

Possibly.  RTEMS may be the only non-GNU/Linux embedded cross target 
which tries to stay near the GCC head and is able to reasonably fully 
support languages other than C and C++.

The *-elf targets don't have the filesystem and threading support
required to fully support some of the other language run-times.

> And since you're not posting the patches you attach to the bugzilla
> PRs you open, you're not exactly helping to make things better either.
> 
> Gr.
> Steven


--joel

Re: GCC 4.1: Buildable on GHz machines only?

[Joel Sherrill <joel@OARcorp.com>]

Jonathan Wakely wrote:
> On Tue, May 17, 2005 at 11:14:22AM +0200, Ralf Corsepius wrote:
> 
> 
>>* I wasn't aware about this fortran specific patch posting policy. I
>>never have sent any gcc patch to any other list but gcc-patches for
>>approval before, so I also had not done so this time.
>>
>>* How could I know that the responsible maintainers aren't listening to
>>bugzilla and gcc-patches, but are listening to a fortran specific list,
>>I even didn't know about until your posting?
> 
> 
> For future reference, where patches should be sent is explained here:
> http://gcc.gnu.org/lists.html

OK .. and Bugzilla or http://gcc.gnu.org/bugs.html references that link how?

A search for "patch" in the bug reporting instructions does not mention
anything about cc'ing any patch list.

I'm not trying to be irritating here.  Just pointing out that if there 
is a procedure, it doesn't appear to be referenced in all the right 
places and isn't tied to the PR system.

Given the recent discussions of unfriendly responses, what if a new
person X found and bug and fixed it, they would file a PR with Bugzilla 
and most likely attach the patch.  And then there is a high probability 
that someone would not so kindly tell them they hadn't followed a 
procedure that doesn't appear to be mentioned anywhere they had seen
along the obvious path.

Putting on my CM hat:

   + Procedures do not exist unless they are documented.
   + Procedures that do not get assisted/enforced by tools are ignored.

> jon

--joel

Re: GCC 4.1: Buildable on GHz machines only?

[Jonathan Wakely]

On Tue, May 17, 2005 at 11:05:07AM -0500, Joel Sherrill <joel@OARcorp.com> wrote:

> Jonathan Wakely wrote:
> >On Tue, May 17, 2005 at 11:14:22AM +0200, Ralf Corsepius wrote:
> >
> >
> >>* I wasn't aware about this fortran specific patch posting policy. I
> >>never have sent any gcc patch to any other list but gcc-patches for
> >>approval before, so I also had not done so this time.
> >>
> >>* How could I know that the responsible maintainers aren't listening to
> >>bugzilla and gcc-patches, but are listening to a fortran specific list,
> >>I even didn't know about until your posting?
> >
> >
> >For future reference, where patches should be sent is explained here:
> >http://gcc.gnu.org/lists.html
> 
> OK .. and Bugzilla or http://gcc.gnu.org/bugs.html references that link how?
> 
> A search for "patch" in the bug reporting instructions does not mention
> anything about cc'ing any patch list.
> 
> I'm not trying to be irritating here.  Just pointing out that if there 
> is a procedure, it doesn't appear to be referenced in all the right 
> places and isn't tied to the PR system.

I never claimed it was easy to find!  :)

Ralf was right that posting a patch to bugzilla should get seen by the
relevant maintainers ... I was just pointing out that the procedure for
where to post patches *is* documented *somewhere*, however poorly.

jon

Re: GCC 4.1: Buildable on GHz machines only?

[Joe Buck]

On Tue, May 17, 2005 at 11:14:22AM +0200, Ralf Corsepius wrote:
> On Tue, 2005-05-17 at 03:31 +0200, Steven Bosscher wrote:
> > On Tuesday 17 May 2005 03:16, Joe Buck wrote:
> > > On Tue, May 17, 2005 at 03:11:03AM +0200, Steven Bosscher wrote:
> > > > On Tuesday 17 May 2005 02:59, Steven Bosscher wrote:
> > > > > Oh, and how helpful of you to post that patch to gcc-patches@ too...
> > > > > NOT!
> > > >
> > > > Ah, I see you did post it to gcc-patches@, but not to fortran@, which
> > > > is a requirement for gfortran patches -- and the reason why nobody
> > > > has noticed the patch.
> > > >
> > > > http://gcc.gnu.org/ml/gcc-patches/2005-04/msg02287.html
> > > >
> > > > The patch is OK too.
> > >
> > > Steven, please try to be politer to someone who is trying to help.
> > 
> > How is it helpful to not follow the rules when posting patches
> Quite simple:
> 
> * I wasn't aware about this fortran specific patch posting policy. I
> never have sent any gcc patch to any other list but gcc-patches for
> approval before, so I also had not done so this time.

There is a mention of this policy somewhere on the GCC web, but it's
in the cellar, at the bottom of a locked filing cabinet, with a sign
that says "beware of the leopard".

Or, rather, it's not mentioned on the bugs page, but on the mailing list
description page, which is not a place where it would occur to most people
to look.  Given this, we shouldn't be surprised if people make mistakes.

> > > This kind of tone will only discourage contributors.
> > 
> > My tone was no different than Ralf's toward me.

I can understand why Steven feels singled out; I jumped on him because
I really, really don't want people to be discouraged from submitting
patches, so that's why I objected publicly (rather than privately).

> Well, I admit I had been sarcastic/fatalistic in replying to Steven,
> primarily, because I am pretty much frustrated about GCC's mainstream
> developer's position/attitude on embedded targets.

To be fair, Ralf, I should ask you to be more polite as well, but you
have a perfect right to complain.

I used to be an embedded programmer myself, and while I cared very much
about the size and speed of the embedded code I wound up with, I didn't
care at all about being able to run the compiler itself on a machine that
wasn't reasonably up to date, much less trying to bootstrap the compiler
on an embedded target.  Is that really what we should be aiming for?  As
opposed to just making -Os work really well?  If I could get better embedded
code by having the compiler use a lot of memory, but I could easily afford
a machine with that amount of memory, I would have had no complaint.

It's true that there are many GCC developers who don't care much about
embedded systems, but there are others that care a lot.  But many GCC
developers lack the relevant expertise, 

It therefore seems that we have two *separate* problems: one is that
increased resource consumption makes gcc harder to use as a hosted
compiler on older systems, and the other is that embedded target support
isn't getting the attention it needs (if it weren't for the heroic efforts
of Dan Kegel, it would be far worse).  We shouldn't mix these two
together; it seems sometimes they get mixed solely because too many
free software projects don't support cross-compilation properly, but
that is a bug in those projects.

Re: GCC 4.1: Buildable on GHz machines only?

[Joel Sherrill <joel@OARcorp.com>]

Joe Buck wrote:
> 
> I used to be an embedded programmer myself, and while I cared very much
> about the size and speed of the embedded code I wound up with, I didn't
> care at all about being able to run the compiler itself on a machine that
> wasn't reasonably up to date, much less trying to bootstrap the compiler
> on an embedded target.  Is that really what we should be aiming for?  As
> opposed to just making -Os work really well?  If I could get better embedded
> code by having the compiler use a lot of memory, but I could easily afford
> a machine with that amount of memory, I would have had no complaint.

There are at least three classes of development I have noticed on this
thread:

   (1) self-hosting gcc on slow but traditional hosts (e.g. 68040 UNIX
     or old Sun's)
   (2) self-hosted embedded development (e.g. sounds like Peter Barada)
   (3) embedded development using regular cross-compilation

In general all are concerned about lower end CPUs than are used for
the mainstream GCC user on GNU/Linux and other UNIces.

(1) and (2) are similar and probably have similar requirements.  They 
would like building GCC and running it to be possible on what would
be considered low end hardware for main-stream development purposes.

(3) is the model for RTEMS, other RTOSes, no-OS targets, and could
be the model used by (2).  I won't include (1) because they want their
systems to be self supporting and users will compile their own code.

We are all concerned about the time and memory required to build GCC.
But it is a critical concern for (1) and (2) since they are on small 
hosts.  For (3) and RTEMS, it concerns us because the RTEMS Project
provides RPMs for  11 targets and tries to include every language
we can possibly support (C, C++, Ada, FORTRAN, and Java).  I know for 
the targets that it compiles on, RTEMS works well with C, C++, and Ada.
I am unsure about the precise status of RTEMS+Java and FORTRAN is
currently up for discussion.  Our tool build times are thus very long 
and when we follow up by building RTEMS and add-on libraries, it
gets longer.  We struggle to keep up which is why RTEMS reports are
sporadic and tend to cluster nearer a release point.


> It's true that there are many GCC developers who don't care much about
> embedded systems, but there are others that care a lot.  But many GCC
> developers lack the relevant expertise, 

I at least do recognize that.  And some of the problems the embedded
community reports are hard to recognize from native configurations.

> It therefore seems that we have two *separate* problems: one is that
> increased resource consumption makes gcc harder to use as a hosted
> compiler on older systems, and the other is that embedded target support
> isn't getting the attention it needs (if it weren't for the heroic efforts
> of Dan Kegel, it would be far worse).  We shouldn't mix these two
> together; it seems sometimes they get mixed solely because too many
> free software projects don't support cross-compilation properly, but
> that is a bug in those projects.

You are correct.  Many free libraries have rough edges when cross-building.

One thing that has been on my personal wish list a LONG time is
to get RTEMS configurations to properly run the GCC test suite.  [I 
normally test and report against *-elf since they are similar and 
easier.]  Many tests fail or can't run on the NO OS targets because 
there is assumption of functionality which isn't there.

RTEMS supports a RAM filesystem and POSIX threads which make it possible 
to run more tests than the NO OS targets.  For example, the complete Ada 
validation test suite can be run with near perfect results.  Similarly, 
other languages with  run-time requirements which RTEMS can meet.  It
should be possible to get better quality embedded target results across
more languages.

The problem is that the RTEMS Project does not have the expertise to do 
this.  We need some help from a DejaGNU/GCC testing expert.

Sorry for the length, this is an important issue to me.

--joel

Re: GCC 4.1: Buildable on GHz machines only?

[Joseph S. Myers]

On Tue, 17 May 2005, Joel Sherrill <joel@OARcorp.com> wrote:

> > For future reference, where patches should be sent is explained here:
> > http://gcc.gnu.org/lists.html
> 
> OK .. and Bugzilla or http://gcc.gnu.org/bugs.html references that link how?
> 
> A search for "patch" in the bug reporting instructions does not mention
> anything about cc'ing any patch list.

The instructions on contributing to GCC, 
<http://gcc.gnu.org/contribute.html>, do however link to lists.html for 
details of the appropriate lists, and give a lot of other information 
patch submitters will need to know.  (This information about mailing lists 
used to be duplicated in contribute.html and lists.html, but that way it 
got out of sync.)  I'm sure a patch to bugs.html to say "If you wish to 
submit a patch for a bug, see the <a href="contribute.html">instructions 
on contributing to GCC</a>." would be considered.  Likewise I suppose we 
could arrange for the Bugzilla pages for individual bugs to make prominent 
notices about how to contribute patches - the page for entering new bugs 
has a big notice "Before reporting a bug, please read the bug writing 
guidelines, please look at the list of most frequently reported bugs, and 
please search for the bug." so those for individual bugs could have a 
notice pointing patch contributors to contribute.html.

-- 
Joseph S. Myers               http://www.srcf.ucam.org/~jsm28/gcc/
    jsm@polyomino.org.uk (personal mail)
    joseph@codesourcery.com (CodeSourcery mail)
    jsm28@gcc.gnu.org (Bugzilla assignments and CCs)

Re: GCC 4.1: Buildable on GHz machines only?

[Alexandre Oliva]

On May 17, 2005, Karel Gardas <kgardas@objectsecurity.com> wrote:

> you see that 4.0 added "embedded" platforms like arm-none-elf and
> mips-none-elf to the primary platforms list.

These are only embedded targets.  You can't run GCC natively on them,
so they don't help embedded hosts in any way.

-- 
Alexandre Oliva             http://www.ic.unicamp.br/~oliva/
Red Hat Compiler Engineer   aoliva@{redhat.com, gcc.gnu.org}
Free Software Evangelist  oliva@{lsd.ic.unicamp.br, gnu.org}

Re: GCC 4.1: Buildable on GHz machines only?

[Joseph S. Myers]

On Tue, 17 May 2005, Joel Sherrill <joel@OARcorp.com> wrote:

> One thing that has been on my personal wish list a LONG time is
> to get RTEMS configurations to properly run the GCC test suite.  [I normally
> test and report against *-elf since they are similar and easier.]  Many tests
> fail or can't run on the NO OS targets because there is assumption of
> functionality which isn't there.

There are very few NO OS results posted to gcc-testresults and apparently 
none posted on a frequent basis; I expect this situation to change 
shortly.  All those posted (at least this month) seem to get posted with 
subject lines which do not match the normal form produced by test_summary 
and so don't get so readily found by my script which counts how many test 
results postings there are for different versions and targets.  For 
example, "Results for 4.1.020050506(experimental) testsuite on 
mips64-unknown-elf" with the components of the version number all run 
together or "Target: AVR Results for 4.1.0 200505 (experimental) in 
comparison to 4.1.0 20050416 (experimental)".  Ensuring your test results 
use the standard Subject header format makes it more likely they can 
handled properly by sites processing the gcc-testresults postings into 
databases of GCC test status on different targets (such as 
<http://www.toolchain.org/testresults/index.html> but other sites have 
done this sort of thing before and may do so in future).

There are however reasonably frequent postings of test results for 
cross-compilers to embedded targets (such as sh4-linux and m32r-linux).

-- 
Joseph S. Myers               http://www.srcf.ucam.org/~jsm28/gcc/
    jsm@polyomino.org.uk (personal mail)
    joseph@codesourcery.com (CodeSourcery mail)
    jsm28@gcc.gnu.org (Bugzilla assignments and CCs)

Re: GCC 4.1: Buildable on GHz machines only?

[Karel Gardas]

On Tue, 17 May 2005, Alexandre Oliva wrote:

> On May 17, 2005, Karel Gardas <kgardas@objectsecurity.com> wrote:
>
>> you see that 4.0 added "embedded" platforms like arm-none-elf and
>> mips-none-elf to the primary platforms list.
>
> These are only embedded targets.  You can't run GCC natively on them,
> so they don't help embedded hosts in any way.

Yes, but Ralf was complaining about embedded cross-compiling development 
for RTEMS. I have not tried to reply to Peter Barada who complains about 
GCC inablity to be run on embedded targets directly.

Karel
--
Karel Gardas                  kgardas@objectsecurity.com
ObjectSecurity Ltd.           http://www.objectsecurity.com

Re: GCC 4.1: Buildable on GHz machines only?

[Gabriel Dos Reis]

Ralf Corsepius <ralf.corsepius@rtems.org> writes:

[...]

| Well, I admit I had been sarcastic/fatalistic in replying to Steven,
| primarily, because I am pretty much frustrated about GCC's mainstream
| developer's position/attitude on embedded targets.
| Steven's answers perfectly queue-in into a long history of incidents
| which had lead me to my understanding of "GCC mainstream developers'
| attitude" on "embedded targets", which I already had described in former
| postings.

then, you must have a very selective definition of "GCC mainstream
developers" :-)
A key strenght of GCC is its supports for embedded targets -- as
imperfect and frustrating it might be sometimes.  My own definition of 
"GCC mainstream developers" do not indicate to me that they are
hostile to the "embedded targets"; quite the contrary.
I also understand your mileage varies.

-- Gaby

Re: GCC 4.1: Buildable on GHz machines only?

[Marcin Dalecki]

On 2005-05-17, at 11:14, Ralf Corsepius wrote:

> On Tue, 2005-05-17 at 03:31 +0200, Steven Bosscher wrote:
>
>> On Tuesday 17 May 2005 03:16, Joe Buck wrote:
>>
>> How is it helpful to not follow the rules when posting patches
>>
> Quite simple:
>
> * I wasn't aware about this fortran specific patch posting policy. I
> never have sent any gcc patch to any other list but gcc-patches for
> approval before, so I also had not done so this time.
>
> * How could I know that the responsible maintainers aren't  
> listening to
> bugzilla and gcc-patches, but are listening to a fortran specific  
> list,
> I even didn't know about until your posting?

If something goes in to the mainline GCC then it should be required
that the maintainers of it don't stay in their cave and read gcc and  
gcc-patches
at least. This is at least what one can reasonable expect looking
from the outside, since there are no specific gcc-c-patches or gcc-c+ 
+-patches.

Re: GCC 4.1: Buildable on GHz machines only?

[Marcin Dalecki]

On 2005-05-17, at 11:29, Richard Earnshaw wrote:

> On Tue, 2005-05-17 at 01:59, Steven Bosscher wrote:
>
>
>> No, I just don't build gfortran as a cross.  There are many reasons
>> why this is a bad idea anyway.
>>
>
> Such as?
>
The dependence on external packages which don't cross compile well  
for example.

Re: GCC 4.1: Buildable on GHz machines only?

[Gerald Pfeifer]

On Tue, 17 May 2005, Joseph S. Myers wrote:
> the page for entering new bugs has a big notice "Before reporting a bug, 
> please read the bug writing guidelines, please look at the list of most 
> frequently reported bugs, and please search for the bug." so those for 
> individual bugs could have a notice pointing patch contributors to 
> contribute.html.

Nice idea.  Daniel, would you mind adding such a link to contribute.html?

Thanks,
Gerald

Re: GCC 4.1: Buildable on GHz machines only?

[Paul Brook]

On Tuesday 17 May 2005 20:27, Marcin Dalecki wrote:
> On 2005-05-17, at 11:29, Richard Earnshaw wrote:
> > On Tue, 2005-05-17 at 01:59, Steven Bosscher wrote:
> >> No, I just don't build gfortran as a cross.  There are many reasons
> >> why this is a bad idea anyway.
> >
> > Such as?
>
> The dependence on external packages which don't cross compile well
> for example.

What external dependencies? gfortran does not require any target libraries 
other than the standard C library.

It does require gmp/mpfr on the host, but in most cases the host is native, 
and they are dead easy to cross compile anyway.

Paul

Re: GCC 4.1: Buildable on GHz machines only?

[Joel Sherrill <joel@OARcorp.com>]

Karel Gardas wrote:
> On Tue, 17 May 2005, Alexandre Oliva wrote:
> 
>> On May 17, 2005, Karel Gardas <kgardas@objectsecurity.com> wrote:
>>
>>> you see that 4.0 added "embedded" platforms like arm-none-elf and
>>> mips-none-elf to the primary platforms list.
>>
>>
>> These are only embedded targets.  You can't run GCC natively on them,
>> so they don't help embedded hosts in any way.
> 
> 
> Yes, but Ralf was complaining about embedded cross-compiling development 
> for RTEMS. I have not tried to reply to Peter Barada who complains about 
> GCC inablity to be run on embedded targets directly.

Logically Peter's situation is the same as the NetBSD issue with 
building and testing on old hosts.   He is running GNU/Linux on
ColdFire and I suspect his ColdFire target is probably faster and better 
equipped than the old UNIX boxes the BSD folks mentioned.

> Karel
> -- 
> Karel Gardas                  kgardas@objectsecurity.com
> ObjectSecurity Ltd.           http://www.objectsecurity.com


-- 
Joel Sherrill, Ph.D.             Director of Research & Development
joel@OARcorp.com                 On-Line Applications Research
Ask me about RTEMS: a free RTOS  Huntsville AL 35805
    Support Available             (256) 722-9985

Re: GCC 4.1: Buildable on GHz machines only?

[Peter Barada]

>> Yes, but Ralf was complaining about embedded cross-compiling development 
>> for RTEMS. I have not tried to reply to Peter Barada who complains about 
>> GCC inablity to be run on embedded targets directly.
>
>Logically Peter's situation is the same as the NetBSD issue with 
>building and testing on old hosts.   He is running GNU/Linux on
>ColdFire and I suspect his ColdFire target is probably faster and better 
>equipped than the old UNIX boxes the BSD folks mentioned.

Its a 266Mhz ColdFire v4e machine, about 263 BogoMips, 1/20 the
BogoMips of my workstation, and with an NFS rootfs, it gets network
bound pretty rapidly and runs even slower compared to a NetBSD machine
with a local disk :)

-- 
Peter Barada
peter@the-baradas.com

Re: GCC 4.1: Buildable on GHz machines only?

[Joel Sherrill <joel@OARcorp.com>]

Peter Barada wrote:
>>>Yes, but Ralf was complaining about embedded cross-compiling development 
>>>for RTEMS. I have not tried to reply to Peter Barada who complains about 
>>>GCC inablity to be run on embedded targets directly.
>>
>>Logically Peter's situation is the same as the NetBSD issue with 
>>building and testing on old hosts.   He is running GNU/Linux on
>>ColdFire and I suspect his ColdFire target is probably faster and better 
>>equipped than the old UNIX boxes the BSD folks mentioned.
> 
> 
> Its a 266Mhz ColdFire v4e machine, about 263 BogoMips, 1/20 the
> BogoMips of my workstation, and with an NFS rootfs, it gets network
> bound pretty rapidly and runs even slower compared to a NetBSD machine
> with a local disk :)

I would have thought the CPU itself was comparable to or faster than
a 68040 or 68060 since they was always at much lower clock speeds.

Do you have any local disk or is everything NFS?  If so, that would
be killer for performance.  I remember an old pair of SparcStation 10's
we used to have.  Network builds vs. local disk was already like 5-10x
slower.

How much RAM?

--joel

Re: GCC 4.1: Buildable on GHz machines only?

[Peter Barada]

>> Its a 266Mhz ColdFire v4e machine, about 263 BogoMips, 1/20 the
>> BogoMips of my workstation, and with an NFS rootfs, it gets network
>> bound pretty rapidly and runs even slower compared to a NetBSD machine
>> with a local disk :)
>
>I would have thought the CPU itself was comparable to or faster than
>a 68040 or 68060 since they was always at much lower clock speeds.

Oh, its faster than the 040/060 since those topped out at 75Mhz, but
at the same time, the more restricted addressing modes/instructions
require more instructions to acheive the same amount of work, but on
the whole is faster(the imfamous RISC debate).

>Do you have any local disk or is everything NFS?  If so, that would
>be killer for performance.  I remember an old pair of SparcStation 10's
>we used to have.  Network builds vs. local disk was already like 5-10x
>slower.

Its currently NFS all the way. :)

>How much RAM?

128Mb.  I do have some experimental kernel hacks in to allow swapping
via NFS, so you can understand why it can take *days* to build stuff.

-- 
Peter Barada
peter@the-baradas.com

Re: GCC 4.1: Buildable on GHz machines only?

[Hugh Sasse]

On Tue, 17 May 2005, Joseph S. Myers wrote:
         [...]
> shortly.  All those posted (at least this month) seem to get posted with
> subject lines which do not match the normal form produced by test_summary
> and so don't get so readily found by my script which counts how many test
> results postings there are for different versions and targets.  For
> example, [Example *very* trimmed -- hgs@dmu.ac.uk]
> comparison to 4.1.0 20050416 (experimental)".  Ensuring your test results
> use the standard Subject header format makes it more likely they can
> handled properly by sites processing the gcc-testresults postings into

Is this standard documented (where?), please? I ask because the
script that generates these has few comments, so it's a little
difficult to know what will break when 'meddling' :-) with it.

I know that could sound aggressive/negative, but I don't intend that
tone.  Sometimes the most useful thing I can do is post test
results, so I'd at least like to do that right.
         [...]

Aside to those finding the criticisms hard to take: For my part, GCC
has given years of good service, but to raise a problem efficiently
means leaving out the appropriate proportion of praise.

         Thank you
         Hugh

Re: GCC 4.1: Buildable on GHz machines only?

[Toon Moene]

Peter Barada wrote:

> Its a 266Mhz ColdFire v4e machine, about 263 BogoMips, 1/20 the
> BogoMips of my workstation, and with an NFS rootfs, it gets network
> bound pretty rapidly and runs even slower compared to a NetBSD machine
> with a local disk :)

Hmmm, Ghz wise and BogoMips wise, this is about half what I have (a 550 
Mhz G4 PowerBook).

Nevertheless, you don't hear *me* complain ...

I build GCC while at work (i.e., while away from the notebook at home :-)

Try it ... it works,

-- 
Toon Moene - e-mail: toon@moene.indiv.nluug.nl - phone: +31 346 214290
Saturnushof 14, 3738 XG  Maartensdijk, The Netherlands
A maintainer of GNU Fortran 95: http://gcc.gnu.org/fortran/
News on GNU Fortran 95: http://gfortran.org/

Re: GCC 4.1: Buildable on GHz machines only?

[Joseph S. Myers]

On Tue, 17 May 2005, Hugh Sasse wrote:

> On Tue, 17 May 2005, Joseph S. Myers wrote:
>         [...]
> > shortly.  All those posted (at least this month) seem to get posted with
> > subject lines which do not match the normal form produced by test_summary
> > and so don't get so readily found by my script which counts how many test
> > results postings there are for different versions and targets.  For
> > example, [Example *very* trimmed -- hgs@dmu.ac.uk]
> > comparison to 4.1.0 20050416 (experimental)".  Ensuring your test results
> > use the standard Subject header format makes it more likely they can
> > handled properly by sites processing the gcc-testresults postings into
> 
> Is this standard documented (where?), please? I ask because the
> script that generates these has few comments, so it's a little
> difficult to know what will break when 'meddling' :-) with it.

It's a de facto standard: don't modify your Subject header from that 
test_summary generates; there are plenty of examples on gcc-testresults of 
what the headers should look like.  You can rewrite the shell script 
output by test_summary - all the test summaries sent by CodeSourcery 
Automatic Testing System rewrite the script to control the From address - 
but preserve the subject header when you do so or when you send the 
summary manually.

-- 
Joseph S. Myers               http://www.srcf.ucam.org/~jsm28/gcc/
    jsm@polyomino.org.uk (personal mail)
    joseph@codesourcery.com (CodeSourcery mail)
    jsm28@gcc.gnu.org (Bugzilla assignments and CCs)

[wwwdocs] bugs.html cleanup (was: GCC 4.1: Buildable on GHz machines )only?

[Gerald Pfeifer]

On Tue, 17 May 2005, Joel Sherrill <joel@OARcorp.com> wrote:
>> For future reference, where patches should be sent is explained here:
>> http://gcc.gnu.org/lists.html
> OK .. and Bugzilla or http://gcc.gnu.org/bugs.html references that link how?

I'm not sure we should add a link to lists.html from bugs.html, however
when reviewing the current bugs.html page I noticed that it's way too
baroque and committed the following patch.

Any further simplifications and other patches are most welcome!

Gerald

Rephrase some parts to make them shorter.  Omit some details for 
submitting bugs by e-mail.

Index: bugs.html
===================================================================
RCS file: /cvs/gcc/wwwdocs/htdocs/bugs.html,v
retrieving revision 1.93
diff -u -3 -p -r1.93 bugs.html
--- bugs.html	23 Feb 2005 22:42:09 -0000	1.93
+++ bugs.html	17 May 2005 20:47:46 -0000
@@ -53,13 +53,13 @@
 <h1><a name="report">Reporting Bugs</a></h1>
 
 <p>The main purpose of a bug report is to enable us to fix the bug. The
-most important prerequisite for this is that the report must be complete and
-self-contained, which we explain in detail below.</p>
+most important prerequisite for this is that the report must be complete
+and self-contained.</p>
 
 <p>Before you report a bug, please check the 
-<a href="#known">list of well-known bugs</a> and, <strong>if possible
-in any way, try a current development snapshot</strong>.
-If you want to report a bug with versions of GCC before 3.1 we strongly
+<a href="#known">list of well-known bugs</a> and, <strong>if possible,
+try a current development snapshot</strong>.
+If you want to report a bug with versions of GCC before 3.4 we strongly
 recommend upgrading to the current release first.</p>
 
 <p>Before reporting that GCC compiles your code incorrectly, please
@@ -116,10 +116,6 @@ three of which can be obtained from the 
   a successful compilation; this is a symptom of a hardware problem,
   not of a compiler bug (sorry)</li>
 
-  <li>E-mail messages that complement previous, incomplete bug
-  reports. Post a new, self-contained, full bug report instead, if
-  possible as a follow-up to the original bug report</li>
-
   <li>Assembly files (<code>*.s</code>) produced by the compiler, or any
   binary files, such as object files, executables, core files, or
   precompiled header files</li>
@@ -163,17 +159,6 @@ preprocessed file it generates.</p>
 <blockquote><p><code>gcc -v -save-temps <i>all-your-options
 source-file</i></code></p></blockquote>
 
-<p>Typically the preprocessed file (extension <code>.i</code> for C or
-<code>.ii</code> for C++, and <code>.f</code> if the preprocessor is used on
-Fortran files) will be large, so please compress the
-resulting file with one of the popular compression programs such as
-bzip2, gzip, zip or compress (in
-decreasing order of preference).  Use maximum compression
-(<code>-9</code>) if available.  Please include the compressed
-preprocessor output in your bug report, even if the source code is
-freely available elsewhere; it makes the job of our volunteer testers
-much easier.</p>
-
 <p>The <b>only</b> excuses to not send us the preprocessed sources are
 (i) if you've found a bug in the preprocessor, (ii) if you've reduced
 the testcase to a small file that doesn't include any other file or
@@ -186,12 +171,6 @@ then try to create a small file that tri
 it in the bug report, although you may want to post parts of it to
 point out assembly code you consider to be wrong.</p>
 
-<p>Whether to use MIME attachments or <code>uuencode</code> is up to
-you.  In any case, make sure the compiler command line, version and
-error output are in plain text, so that we don't have to decode the
-bug report in order to tell who should take care of it.  A meaningful
-subject indicating language and platform also helps.</p>
-
 <p>Please avoid posting an archive (.tar, .shar or .zip); we generally
 need just a single file to reproduce the bug (the .i/.ii/.f preprocessed
 file), and, by storing it in an archive, you're just making our
@@ -204,16 +183,6 @@ make sure the compiler version, error me
 the body of your bug report as plain text, even if needlessly
 duplicated as part of an archive.</p>
 
-<p>If you fail to supply enough information for a bug report to be
-reproduced, someone will probably ask you to post additional
-information (or just ignore your bug report, if they're in a bad day,
-so try to get it right on the first posting :-).  In this case, please
-post the additional information to the bug reporting mailing list, not
-just to the person who requested it, unless explicitly told so.  If
-possible, please include in this follow-up all the information you had
-supplied in the incomplete bug report (including the preprocessor
-output), so that the new bug report is self-contained.</p>
-
 <h2><a name="gnat">Detailed bug reporting instructions for GNAT</a></h2>
 
 <p>See the <a href="#detailed">previous section</a> for bug reporting

Re: GCC 4.1: Buildable on GHz machines only?

[Peter Barada]

>> Its a 266Mhz ColdFire v4e machine, about 263 BogoMips, 1/20 the
>> BogoMips of my workstation, and with an NFS rootfs, it gets network
>> bound pretty rapidly and runs even slower compared to a NetBSD machine
>> with a local disk :)
>
>Hmmm, Ghz wise and BogoMips wise, this is about half what I have (a 550 
>Mhz G4 PowerBook).
>
>Nevertheless, you don't hear *me* complain ...

No, but I don't have a disk, so everything has to come over the
network.  I also don't have much ram, so if I start running out of RAM,
it slows to a crawl since it can't cache any source.

>I build GCC while at work (i.e., while away from the notebook at home :-)
>
>Try it ... it works,

Huh?  I can cross-compile GCC, its all the packages that require
native configuration/building....

-- 
Peter Barada
peter@the-baradas.com

Re: GCC 4.1: Buildable on GHz machines only?

[Jonathan Wilson]

> Huh?  I can cross-compile GCC, its all the packages that require
> native configuration/building....
Is it fesable for people in this sort of situation to build GCC on a fast 
machine but with the final host and target both set to whatever the slower 
machine is (in this case coldfire)
Does GCC even support that?

Re: GCC 4.1: Buildable on GHz machines only?

[Ralf Corsepius]

On Tue, 2005-05-17 at 12:11 -0500, Joel Sherrill  wrote:
> Joe Buck wrote:
> > 
> > I used to be an embedded programmer myself, and while I cared very much
> > about the size and speed of the embedded code I wound up with, I didn't
> > care at all about being able to run the compiler itself on a machine that
> > wasn't reasonably up to date, much less trying to bootstrap the compiler
> > on an embedded target.  Is that really what we should be aiming for?
Well, an embedded programmer won't care much about this issue, but as
RTEMS maintainer, I am building and packaging the toolchains - so GCC
building times are a concern to me.

> >  As
> > opposed to just making -Os work really well?
ACK, this is an issue as well. 

ATM, I am experiencing objects being generated by GCC to increase in
size and forcing us to gradually abandon targets with tight memory
requirements. At least one cause of this seems to be GCC abandoning COFF
in favor of ELF, which seems to imply larger memory requirements.

>   If I could get better embedded
> > code by having the compiler use a lot of memory, but I could easily afford
> > a machine with that amount of memory, I would have had no complaint.
ACK.

> There are at least three classes of development I have noticed on this
> thread:
> 
>    (1) self-hosting gcc on slow but traditional hosts (e.g. 68040 UNIX
>      or old Sun's)
>    (2) self-hosted embedded development (e.g. sounds like Peter Barada)
>    (3) embedded development using regular cross-compilation
> 
> In general all are concerned about lower end CPUs than are used for
> the mainstream GCC user on GNU/Linux and other UNIces.
> 
> (1) and (2) are similar and probably have similar requirements.  They 
> would like building GCC and running it to be possible on what would
> be considered low end hardware for main-stream development purposes.
> 
> (3) is the model for RTEMS, other RTOSes, no-OS targets, and could
> be the model used by (2).  I won't include (1) because they want their
> systems to be self supporting and users will compile their own code.
> 
> We are all concerned about the time and memory required to build GCC.
> But it is a critical concern for (1) and (2) since they are on small 
> hosts.  For (3) and RTEMS, it concerns us because the RTEMS Project
> provides RPMs for  11 targets and tries to include every language
> we can possibly support (C, C++, Ada, FORTRAN, and Java).  I know for 
> the targets that it compiles on, RTEMS works well with C, C++, and Ada.
> I am unsure about the precise status of RTEMS+Java
gcj builds for most RTEMS-targets. RTEMS support for libgcj and its
infrastructure is missing and probably won't be implemented any time
soon. A usable gcj+libgcj for RTEMS is not in sight.

>  and FORTRAN is currently up for discussion.
gfortran builds fine for all CPUs, building libgfortran fails for some
CPUs, which don't meet the expectations of gfortran/f95.

Objc builds without problems for all targets.

Ada builds for all CPUs Ada supports, but suffers from general cross-
building deficits and lacks multilibs.

>   Our tool build times are thus very long
> and when we follow up by building RTEMS and add-on libraries, it
> gets longer.  We struggle to keep up which is why RTEMS reports are
> sporadic and tend to cluster nearer a release point.
Be lucky, we can't build libgcj and gnat doesn't support multilibs. I
would not expect us to handle this.

> > It therefore seems that we have two *separate* problems: one is
> that 
> > increased resource consumption makes gcc harder to use as a hosted
> > compiler on older systems, and the other is that embedded target support
> > isn't getting the attention it needs
> [..]
> > it seems sometimes they get mixed solely because too many
> > free software projects don't support cross-compilation properly, but
> > that is a bug in those projects.
> 
> You are correct.  Many free libraries have rough edges when cross-
> building.
My point is: GCC also has them, and could do better wrt. this.

Ralf

Re: GCC 4.1: Buildable on GHz machines only?

[Ranjit Mathew]

Jonathan Wilson wrote:
>>Huh?  I can cross-compile GCC, its all the packages that require
>>native configuration/building....
> 
> Is it fesable for people in this sort of situation to build GCC on a fast 
> machine but with the final host and target both set to whatever the slower 
> machine is (in this case coldfire)
> Does GCC even support that?

Yes, this is called a crossed-native build and GCC does
support it. I used to use it some time back for building
GCJ for Win32:

  http://ranjitmathew.hostingzero.com/phartz/gcj/bldgcj.html

You build a crossed-native compiler in two phases:

  1. Build a cross compiler for $TARGET on a fast box.

  2. Use the cross compiler to build a crossed-native
     compiler for $TARGET on the same box.

I used to do it for Win32 simply for the reason that
building under Linux (on the same box) was *way* faster
and less error-prone than under Win32.

I was going to suggest this to Peter myself before
I saw your message.

Ranjit.

PS: Surely this must be one of the longest threads in
recent times on the GCC list!

PPS: I do not see some of the messages, for example, a
couple of messages from Robert Dewar that seem to be
referenced in other messages.

-- 
Ranjit Mathew      Email: rmathew AT gmail DOT com

Bangalore, INDIA.    Web: http://ranjitmathew.hostingzero.com/

Re: GCC 4.1: Buildable on GHz machines only?

[Gabriel Dos Reis]

Ranjit Mathew <rmathew@gmail.com> writes:

[...]

| PS: Surely this must be one of the longest threads in
| recent times on the GCC list!

:-)

| PPS: I do not see some of the messages, for example, a
| couple of messages from Robert Dewar that seem to be
| referenced in other messages.

same here.

-- Gaby

Re: GCC 4.1: Buildable on GHz machines only?

[Richard Sandiford]

"Joseph S. Myers" <joseph@codesourcery.com> writes:
> For example, "Results for 4.1.020050506(experimental) testsuite on 
> mips64-unknown-elf" with the components of the version number all run 
> together
> [...]
> Ensuring your test results 
> use the standard Subject header format makes it more likely they can 
> handled properly by sites processing the gcc-testresults postings into 
> databases of GCC test status on different targets (such as 
> <http://www.toolchain.org/testresults/index.html> but other sites have 
> done this sort of thing before and may do so in future).

FWIW, those mips messages _were_ sent with test_summary.  I'm not
sure why they got mangled.  Are there any known issues that would
cause that?

Richard

Re: GCC 4.1: Buildable on GHz machines only?

[Richard Earnshaw]

On Tue, 2005-05-17 at 21:03, Paul Brook wrote:
[building of gfortran]
> It does require gmp/mpfr on the host, but in most cases the host is native, 
> and they are dead easy to cross compile anyway.

And it's long been my contention that we shouldn't even be doing that.  

I still feel these packages should be brought in-tree, especially since
specific minimum versions are needed.

R.

Re: GCC 4.1: Buildable on GHz machines only?

[Richard Earnshaw]

On Tue, 2005-05-17 at 23:33, Joseph S. Myers wrote:

> It's a de facto standard: don't modify your Subject header from that 
> test_summary generates; there are plenty of examples on gcc-testresults of 
> what the headers should look like.  You can rewrite the shell script 
> output by test_summary - all the test summaries sent by CodeSourcery 
> Automatic Testing System rewrite the script to control the From address - 
> but preserve the subject header when you do so or when you send the 
> summary manually.

In my experience, test_summary does the wrong thing if you build and
test a unified source tree, for example, I run it on a unified arm-elf
tree and the subject line comes out as:

Results for 6.3.50.20050330-cvs -nx testsuite on arm-unknown-elf

This number seems to come from one of the other packages in the source
tree (gdb?).  It certainly only changes when I update my binutils/gdb
sources, not when I update my gcc sources.

It would be good to get this fixed properly, but I was never much of a
shell wizard.

R.

Re: GCC 4.1: Buildable on GHz machines only?

[Karel Gardas]

On Mon, 16 May 2005, DJ Delorie wrote:

>
>> What I have problem understanding is the last sentence of this
>> paragraph in the light of your claim that it will results in
>> swapping especially when we consider developers' machines with
>> 512MB/1GB RAM, i.e. machines where memory is not "tight".
>
> Sigh, Linux works the same way.  Processes can exceed their HARD
> ulimit if there happens to be memory available, making RLIMIT_RSS
> basically useless.

Perhaps we can use --param ggc-min-expand=X --param ggc-min-heapsize=Y 
options? I've tried here:
http://gcc.gnu.org/ml/gcc/2005-05/msg00967.html

and got some interesting results which might be more similar to the 
machines with low memory.

Cheers,
Karel
--
Karel Gardas                  kgardas@objectsecurity.com
ObjectSecurity Ltd.           http://www.objectsecurity.com

Re: GCC 4.1: Buildable on GHz machines only?

[Joseph S. Myers]

On Wed, 18 May 2005, Richard Sandiford wrote:

> "Joseph S. Myers" <joseph@codesourcery.com> writes:
> > For example, "Results for 4.1.020050506(experimental) testsuite on 
> > mips64-unknown-elf" with the components of the version number all run 
> > together
> > [...]
> > Ensuring your test results 
> > use the standard Subject header format makes it more likely they can 
> > handled properly by sites processing the gcc-testresults postings into 
> > databases of GCC test status on different targets (such as 
> > <http://www.toolchain.org/testresults/index.html> but other sites have 
> > done this sort of thing before and may do so in future).
> 
> FWIW, those mips messages _were_ sent with test_summary.  I'm not
> sure why they got mangled.  Are there any known issues that would
> cause that?

What awk version are you using?  (The script tests for gawk, nawk, awk if 
$AWK is not set.)  Experimenting suggests that mawk is deficient in this 
regard: where the script extracts the version number

$2 == "version" { save = $0; $1 = ""; $2 = ""; version = $0; gsub(/^ */, 
"", version); gsub(/\r$/, "", version); $0 = save; }

mawk loses the spaces whereas gawk or nawk do not.

-- 
Joseph S. Myers               http://www.srcf.ucam.org/~jsm28/gcc/
    jsm@polyomino.org.uk (personal mail)
    joseph@codesourcery.com (CodeSourcery mail)
    jsm28@gcc.gnu.org (Bugzilla assignments and CCs)

Re: GCC 4.1: Buildable on GHz machines only?

[Joseph S. Myers]

On Wed, 18 May 2005, Richard Earnshaw wrote:

> In my experience, test_summary does the wrong thing if you build and
> test a unified source tree, for example, I run it on a unified arm-elf
> tree and the subject line comes out as:
> 
> Results for 6.3.50.20050330-cvs -nx testsuite on arm-unknown-elf
> 
> This number seems to come from one of the other packages in the source
> tree (gdb?).  It certainly only changes when I update my binutils/gdb
> sources, not when I update my gcc sources.
> 
> It would be good to get this fixed properly, but I was never much of a
> shell wizard.

As test_summary is designed to summarize GCC logs it isn't obvious what is 
correct for it to do when presented with logs from other software as well 
- when the logs are for both GCC version x and GDB version y.

I think the right approach might be for more complete version information 
in a better defined format to go in the .sum files so the script can look 
just for GCC version information and distinguish it from GDB version 
information.  This should include the configure flags from gcc -v so that 
you don't need to keep config.status around from the build if you want to 
use test_summary on logs from an installed compiler test.  
(default_gcc_version in gcc/testsuite/lib/gcc.exp does run gcc -v but 
everything but the version string only goes in gcc.log not gcc.sum.)  If 
the .sum files had well-defined lines

GCC version: ...
GCC LAST_UPDATED: ...
    (should be put in gcc -v so test_summary doesn't need the source 
     directory around)
GCC configure flags: ...
GCC BOOT_CFLAGS: ...
    (getting this from the environment when running test_summary seems 
     like a kludge; should be in gcc -v)
GCC host: ...
GCC build: ...
GCC target: ...
GDB version: ...

then test_summary could process the .sum files more generally and reliably 
and without needing other information from the source or build trees.

-- 
Joseph S. Myers               http://www.srcf.ucam.org/~jsm28/gcc/
    jsm@polyomino.org.uk (personal mail)
    joseph@codesourcery.com (CodeSourcery mail)
    jsm28@gcc.gnu.org (Bugzilla assignments and CCs)

Re: GCC 4.1: Buildable on GHz machines only?

[Robert Dewar]

Ralf Corsepius wrote:

> ATM, I am experiencing objects being generated by GCC to increase in
> size and forcing us to gradually abandon targets with tight memory
> requirements. At least one cause of this seems to be GCC abandoning COFF
> in favor of ELF, which seems to imply larger memory requirements.

I don't see at all why this should increase target memory requirements,
can you eludicate?

Re: GCC 4.1: Buildable on GHz machines only?

[Peter Barada]

>> ATM, I am experiencing objects being generated by GCC to increase in
>> size and forcing us to gradually abandon targets with tight memory
>> requirements. At least one cause of this seems to be GCC abandoning COFF
>> in favor of ELF, which seems to imply larger memory requirements.
>
>I don't see at all why this should increase target memory requirements,
>can you eludicate?

Perhaps it is that the ELF executables are larger in total size(not
code size) than the corresponding COFF executables, and if stored on
the target, then the footprint increases, causing "larger memory
requirements" of flash, not DRAM.

-- 
Peter Barada
peter@the-baradas.com

Re: GCC 4.1: Buildable on GHz machines only?

[Robert Dewar]

Peter Barada wrote:

> Perhaps it is that the ELF executables are larger in total size(not
> code size) than the corresponding COFF executables, and if stored on
> the target, then the footprint increases, causing "larger memory
> requirements" of flash, not DRAM.
> 

Possibly, though I am surprised if the difference is significant,
assuming no debug information of course!

Re: GCC 4.1: Buildable on GHz machines only?

[Richard Sandiford]

"Joseph S. Myers" <joseph@codesourcery.com> writes:
> On Wed, 18 May 2005, Richard Sandiford wrote:
>> FWIW, those mips messages _were_ sent with test_summary.  I'm not
>> sure why they got mangled.  Are there any known issues that would
>> cause that?
>
> What awk version are you using?  (The script tests for gawk, nawk, awk if 
> $AWK is not set.)  Experimenting suggests that mawk is deficient in this 
> regard: where the script extracts the version number
>
> $2 == "version" { save = $0; $1 = ""; $2 = ""; version = $0; gsub(/^ */, 
> "", version); gsub(/\r$/, "", version); $0 = save; }
>
> mawk loses the spaces whereas gawk or nawk do not.

Good catch!  I was indeed using mawk.  I've installed gawk now,
so hopefully next time it should work...

Richard

Re: GCC 4.1: Buildable on GHz machines only?

[Kai Henningsen]

peter@the-baradas.com (Peter Barada)  wrote on 17.05.05 in <20050517210315.50DFE9842C@baradas.org>:

> Its a 266Mhz ColdFire v4e machine, about 263 BogoMips, 1/20 the
> BogoMips of my workstation, and with an NFS rootfs, it gets network

BogoMips are called BogoMips because they are not comparable among  
different CPUs. All they measure is how often the CPU needs to run a  
particular near-empty loop to delay a certain time.

There usually is a small factor which can convert between BogoMips and CPU  
MHz for every CPU model. It would seem to be 1 for your ColdFire; it  
happens to be 1/2 for my Athlon (bogomips: 2287.20, cpu MHz: 1145.142).

Comparisions like yours are worse than meaningless.

MfG Kai

Re: GCC 4.1: Buildable on GHz machines only?

[Peter Barada]

>> Its a 266Mhz ColdFire v4e machine, about 263 BogoMips, 1/20 the
>> BogoMips of my workstation, and with an NFS rootfs, it gets network
>
>BogoMips are called BogoMips because they are not comparable among  
>different CPUs. All they measure is how often the CPU needs to run a  
>particular near-empty loop to delay a certain time.

I know exactly what a BogoMips is.

>There usually is a small factor which can convert between BogoMips and CPU  
>MHz for every CPU model. It would seem to be 1 for your ColdFire; it  
>happens to be 1/2 for my Athlon (bogomips: 2287.20, cpu MHz: 1145.142).
>
>Comparisions like yours are worse than meaningless.

I wouldn't call it meaningless.  I don't have other benchmark numbers
for the chip, and it was menat to show that it isn't a blazingly fast
processor (as compared to desktop machines).

-- 
Peter Barada
peter@the-baradas.com

Re: GCC 4.1: Buildable on GHz machines only?

[Kai Henningsen]

peter@the-baradas.com (Peter Barada)  wrote on 21.05.05 in <20050521133156.668989817E@baradas.org>:

> >> Its a 266Mhz ColdFire v4e machine, about 263 BogoMips, 1/20 the
> >> BogoMips of my workstation, and with an NFS rootfs, it gets network
> >
> >BogoMips are called BogoMips because they are not comparable among
> >different CPUs. All they measure is how often the CPU needs to run a
> >particular near-empty loop to delay a certain time.
>
> I know exactly what a BogoMips is.

But it seems you ignore what it means.

> >There usually is a small factor which can convert between BogoMips and CPU
> >MHz for every CPU model. It would seem to be 1 for your ColdFire; it
> >happens to be 1/2 for my Athlon (bogomips: 2287.20, cpu MHz: 1145.142).
> >
> >Comparisions like yours are worse than meaningless.
>
> I wouldn't call it meaningless.  I don't have other benchmark numbers

It's exactly as meaningful - slightly less, in fact - as just quoting the  
MHz of the chip.

It doesn't tell anything interesting about what the chip can actually do  
with those MHz.

> for the chip, and it was menat to show that it isn't a blazingly fast
> processor (as compared to desktop machines).

So quote the MHz and be done with it.


MfG Kai

Re: GCC 4.1: Buildable on GHz machines only?

[Haren Visavadia]

--- Tom Tromey wrote:
> I'm not sure what Plan B would be.  Maybe separate
> libgcj releases
> somehow.

You coulder consider just having GCJ inside GCC but
somehow get it to use GNU Classpath directly, this
would also reduce it needing to be re-sync with GNU
Classpath (which I beleive libgcj is based on it) and
allows you deliver a more up-to-date Java library to
people faster.









		
___________________________________________________________ 
How much free photo storage do you get? Store your holiday 
snaps for FREE with Yahoo! Photos http://uk.photos.yahoo.com

Re: GCC 4.1: Buildable on GHz machines only?

[Jason Mancini]

A little humor from a long time ML lurker...

Via C3-2 Nehemiah 1GHz 512MB ddr

$ ../gcc-4.0.0/configure --prefix=/home/jason/local/gcc-400 --enable-shared 
\
--enable-threads=posix --disable-checking --enable-long-long 
--enable-__cxa_atexit \ --enable-clocale=gnu --disable-libunwind-exceptions 
--enable-languages=c --with-system-zlib

$ time make bootstrap
...
3860.71user 245.24system 1:10:05elapsed 97%CPU
0inputs+0outputs (6698major+12862842minor)pagefaults 0swaps

$ strip gcc ; upx -9 gcc ; ls -l gcc
-rwxr-xr-x  1 jason jason 42672 May  5 23:07 gcc

So the bootstrap process generates a useful 10 bytes/second.  ;-)

But seriously, GCC and various language standards have kept up with modern 
hardware.
If someone takes a GCC release and hacks up a release to run on 50MHz boxes 
from years
gone by, that's great, but I think the main GCC devs shouldn't worry about 
it because
GCC is more about flexibility and extensibility than slim and trim I'd say.  
Perhaps there
should be an "embedded GCC" team that focuses on a separate light-weight 
C/C++ project.

Many thanks for GCC!
-Jason, back to lurking.

Re: GCC 4.1: Buildable on GHz machines only?

[Dan Kegel]

Peter Barada wrote:
>>>Unfortunately for some of the embedded targets(like the ColdFire V4e
>>>work I'm doing), a bootstrap is impossible due to limited memory and
>>>no usable mass-storage device on the hardware I have available, so
>>>hopefully a successful crossbuild will suffice.
>>
>>How about a successful crossbuild plus
>>passing some regression test suite,
>>e.g. gcc's, glibc's, and/or ltp's?
>>Any one of them would provide a nice reality check.
> 
> I'm open to running them if there's a *really* clear how-to to do it
> that takes into account remote hardware.

I'm not sure it qualifies as *really* clear, but my
doc on doing remote gcc and glibc test runs is at
http://kegel.com/crosstool/current/doc/crosstest-howto.html
Have you tried that yet?
It worked for me on systems with 16 MB of RAM and a
network connection.  I bet it'd work with less RAM
if you ditched the glibc tests.
- Dan


-- 
Trying to get a job as a c++ developer?  See http://kegel.com/academy/getting-hired.html

Re: GCC 4.1: Buildable on GHz machines only?

[Peter Barada]

>> Unfortunately for some of the embedded targets(like the ColdFire V4e
>> work I'm doing), a bootstrap is impossible due to limited memory and
>> no usable mass-storage device on the hardware I have available, so
>> hopefully a successful crossbuild will suffice.
>
>How about a successful crossbuild plus
>passing some regression test suite,
>e.g. gcc's, glibc's, and/or ltp's?
>Any one of them would provide a nice reality check.

I'm open to running them if there's a *really* clear how-to to do it
that takes into account remote hardware.

-- 
Peter Barada
peter@the-baradas.com

Re: GCC 4.1: Buildable on GHz machines only?

[Dan Kegel]

Peter Barada wrote:
>>>>The alternative of course is to do only crossbuilds.  Is it reasonable
>>>>to say that, for platforms where a bootstrap is no longer feasible, a
>>>>successful crossbuild is an acceptable test procedure to use instead?
>>>
>>A successful crossbuild is certainly the minimum concievable standard.
>>Perhaps one should also require bootstrapping the C compiler alone;
>>that would provide at least some sanity-checking.
> 
> 
> Unfortunately for some of the embedded targets(like the ColdFire V4e
> work I'm doing), a bootstrap is impossible due to limited memory and
> no usable mass-storage device on the hardware I have available, so
> hopefully a successful crossbuild will suffice.

How about a successful crossbuild plus
passing some regression test suite,
e.g. gcc's, glibc's, and/or ltp's?
Any one of them would provide a nice reality check.
- Dan

-- 
Trying to get a job as a c++ developer?  See http://kegel.com/academy/getting-hired.html

Re: GCC 4.1: Buildable on GHz machines only?

[Peter Barada]

>>> The alternative of course is to do only crossbuilds.  Is it reasonable
>>> to say that, for platforms where a bootstrap is no longer feasible, a
>>> successful crossbuild is an acceptable test procedure to use instead?
>>> 
>> Sure, and get flamed and trounced by Uli on glibc when you talk
>> about problems with crossbuilding.
>
>He'll flame you anyway for talking about platforms
>that are no longer relevant to the desktop.
>(Well, maybe he wouldn't.  See http://lwn.net/Articles/19297 )
>
>A successful crossbuild is certainly the minimum concievable standard.
>Perhaps one should also require bootstrapping the C compiler alone;
>that would provide at least some sanity-checking.

Unfortunately for some of the embedded targets(like the ColdFire V4e
work I'm doing), a bootstrap is impossible due to limited memory and
no usable mass-storage device on the hardware I have available, so
hopefully a successful crossbuild will suffice.

-- 
Peter Barada
peter@the-baradas.com

Re: GCC 4.1: Buildable on GHz machines only?

[Daniel Kegel]

>> The alternative of course is to do only crossbuilds.  Is it reasonable
>> to say that, for platforms where a bootstrap is no longer feasible, a
>> successful crossbuild is an acceptable test procedure to use instead?
>> 
> Sure, and get flamed and trounced by Uli on glibc when you talk
> about problems with crossbuilding.

He'll flame you anyway for talking about platforms
that are no longer relevant to the desktop.
(Well, maybe he wouldn't.  See http://lwn.net/Articles/19297 )

A successful crossbuild is certainly the minimum concievable standard.
Perhaps one should also require bootstrapping the C compiler alone;
that would provide at least some sanity-checking.