Re: Of Bounties and Mercenaries

Re: Of Bounties and Mercenaries

[Richard Kenner]

    If gcc 3.5 is ten times slower compiling programs at -O2, that's a
    good thing: it means that it does more to procude better code. 

No.  Producing better code means producing better code.  Compiling slower
means compiling slower.  One would *hope that the latter means the former,
but that has to be shown, not just implied.

    A program is compiled once but executed 100000 times. 

True, but that's often not relevant to developers, who care about how
quickly they can compile their applications.

Re: Of Bounties and Mercenaries

[Mark Hahn]

>     A program is compiled once but executed 100000 times. 
> 
> True, but that's often not relevant to developers, who care about how
> quickly they can compile their applications.

GCC *IS* quite fast.  

people still complain about it: is it not true that those who do are 
using grotesquely bloated app frameworks?  is there any value to PCH,
other than pandering to people who deliberately choose to arrange 
their code into vast mazes of twisty headers?

consider an app framework where everything that the developer is trying 
to work n resides in inter-dependent header files with boatloads of inline code.
OK, inlining is a great feaure which presents the compiler with more
opportunities to optimize;  the problem is all the reparsing, is it not?

consider an alternative: link-time, whole-program optimization.  this 
removes much of the need to have so much code inlined, and offers far more 
optimization opportunity than the all-inline-PCH approach.  and it would 
benefit *everyone*, not just header-o-philes.

was link-time (delayed) optimization considered as a more powerful
alternative to PCH?  was it rejected because of the usual paranoia about 
someone hijacking an internal compiler representation?

at my HPC center, runtime speed is somewhere between 5e3 and 2e9 more 
important than compile speed (median is probably 2e5.)

regards, mark hahn.

Re: Of Bounties and Mercenaries

[Kai Henningsen]

hahn@physics.mcmaster.ca (Mark Hahn)  wrote on 03.04.04 in <Pine.LNX.4.44.0404031440350.24108-100000@coffee.psychology.mcmaster.ca>:

> >     A program is compiled once but executed 100000 times.
> >
> > True, but that's often not relevant to developers, who care about how
> > quickly they can compile their applications.
>
> GCC *IS* quite fast.

GCC is usually fast enough for me.

OTOH, just about every compiler seems to be faster.

> people still complain about it: is it not true that those who do are
> using grotesquely bloated app frameworks?  is there any value to PCH,
> other than pandering to people who deliberately choose to arrange
> their code into vast mazes of twisty headers?

Or to people who *use* those frameworks but didn't design them?

(And when I think of stuff like the Win32 API, MacOS X/Cocoa or even  
traditional MacOS (these days known as Carbon), I'll say that *not*  
organizing that stuff into lots of header files would have been even  
*more* insane. And I suspect you wouldn't be all that happy with a single  
posix.h header, either.)

Oh, and all the above don't even use significant amounts of inline code.

> consider an alternative: link-time, whole-program optimization.  this
> removes much of the need to have so much code inlined, and offers far more
> optimization opportunity than the all-inline-PCH approach.  and it would
> benefit *everyone*, not just header-o-philes.

Sure they might be nice, but I don't see how they'd even touch the header  
problem. See above.

MfG Kai

Re: Of Bounties and Mercenaries

[Chris Lattner]

Andrew Pinski wrote:
> The other problem with this is that "Modern" "Standard" C++ has the
> following problems:
> 1. Makes a mess out of simplifying code from the compiling when the
> compiler is being miscompiled itself.

Huh?

> 2. "Modern" means templates see 1.

Double huh?

> 3. "Standard" means that we have to do more workarounds that Boost does,
> see 1.

No, you just require a reasonable baseline compiler, like GCC 3.x.  :)  I
realize that this is not a wonderful solution with GCC now, but that's
primarily due to the fact that it's so hard to cross compile GCC to
certain architectures.  In LLVM, it's a simple matter of saying -march=x86
or -march=sparcv9 to select a backend, so retargetting is much simpler
than in GCC.

> 4. GC will be still there just in a slightly different form, see 2 and
> 1.

Not true at all.  In LLVM we have absolutely no GC, and no need or desire
for one.  With C++ ctor/dtors and a simple ownership model, there is no
need for a GC at all.  Note that this makes performance bugs dramatically
easier to understand.

> 5. The number of bug reports about the C++ is high and we get more
> invalid  bugs each day, so we will then get bug reports about GCC not
> compiling on some C++ compiler, see 3.

This seems to be the same point as #4 above.  Perhaps you are trying to
say that not as many people know C++ as know C?

> Now if we figure out a good GC in that only parts of the compiler uses
> it, we will no longer have all the issues we currently have because the
> part I am talking about RTL (which is the most memory bounded part) will
> move from being in the GC into being malloc'd and maybe pool alloced
> instead so it will be relatively close in memory.

Hrm, this has absolutely nothing to do with C++.  In fact, this is all
about having a clear ownership model for memory (which none of the GCC IRs
have).  If nothing else, the GC introduced in GCC 3.0 made the compiler
easier to work on and understand.  It seems like taking that away, would,
well, be a huge regression in terms of GCC maintenance.

-Chris

-- 
http://llvm.cs.uiuc.edu/
http://www.nondot.org/~sabre/Projects/

Re: Of Bounties and Mercenaries

[Robert Dewar]

Chris Lattner wrote:
> Andrew Pinski wrote:
> 
>>The other problem with this is that "Modern" "Standard" C++ has the
>>following problems:
>>1. Makes a mess out of simplifying code from the compiling when the
>>compiler is being miscompiled itself.
> 
> 
> Huh?

If you really don't understand the above point 1, that's surprising.
Have you ever actually worked through problems in a self-compiling
bootstrapped compiler? Or are you just arguing language merits?

Re: Of Bounties and Mercenaries

[Chris Lattner]

On Tue, 6 Apr 2004, Robert Dewar wrote:

> Chris Lattner wrote:
> > Andrew Pinski wrote:
> >
> >>The other problem with this is that "Modern" "Standard" C++ has the
> >>following problems:
> >>1. Makes a mess out of simplifying code from the compiling when the
> >>compiler is being miscompiled itself.
> >
> >
> > Huh?
>
> If you really don't understand the above point 1, that's surprising.
> Have you ever actually worked through problems in a self-compiling
> bootstrapped compiler? Or are you just arguing language merits?

No, I just can't parse the english statement.  Can Andrew or someone else
restate?

-Chris

-- 
http://llvm.cs.uiuc.edu/
http://www.nondot.org/~sabre/Projects/

Re: Of Bounties and Mercenaries

[Daniel Jacobowitz]

On Tue, Apr 06, 2004 at 09:07:48AM -0500, Chris Lattner wrote:
> On Tue, 6 Apr 2004, Robert Dewar wrote:
> 
> > Chris Lattner wrote:
> > > Andrew Pinski wrote:
> > >
> > >>The other problem with this is that "Modern" "Standard" C++ has the
> > >>following problems:
> > >>1. Makes a mess out of simplifying code from the compiling when the
> > >>compiler is being miscompiled itself.
> > >
> > >
> > > Huh?
> >
> > If you really don't understand the above point 1, that's surprising.
> > Have you ever actually worked through problems in a self-compiling
> > bootstrapped compiler? Or are you just arguing language merits?
> 
> No, I just can't parse the english statement.  Can Andrew or someone else
> restate?

Sure: I believe what Andrew is saying is that when you have a failure
in the stage2 compiler, i.e. a miscompiled compiler, it becomes
substantially harder to diagnose and minimise.

Whether that's true or not I don't know.  It's plausible.

-- 
Daniel Jacobowitz
MontaVista Software                         Debian GNU/Linux Developer

Re: Of Bounties and Mercenaries

[Chris Lattner]

On Tue, 6 Apr 2004, Daniel Jacobowitz wrote:
> On Tue, Apr 06, 2004 at 09:07:48AM -0500, Chris Lattner wrote:
> > On Tue, 6 Apr 2004, Robert Dewar wrote:
> >
> > > Chris Lattner wrote:
> > > > Andrew Pinski wrote:
> > > >
> > > >>The other problem with this is that "Modern" "Standard" C++ has the
> > > >>following problems:
> > > >>1. Makes a mess out of simplifying code from the compiling when the
> > > >>compiler is being miscompiled itself.
> > > >
> > > >
> > > > Huh?
> > >
> > > If you really don't understand the above point 1, that's surprising.
> > > Have you ever actually worked through problems in a self-compiling
> > > bootstrapped compiler? Or are you just arguing language merits?
> >
> > No, I just can't parse the english statement.  Can Andrew or someone else
> > restate?
>
> Sure: I believe what Andrew is saying is that when you have a failure
> in the stage2 compiler, i.e. a miscompiled compiler, it becomes
> substantially harder to diagnose and minimise.
>
> Whether that's true or not I don't know.  It's plausible.

Ah, well I think there are two issues here.  First, bootstrapping is just
one of many tests for compiler correctness that should be done.  Many
other programs should be tested regularly in addition to the compiler
bootstrapping itself.  Currently, GCC exercises the C++ front-end very
little in a bootstrap.  I actually think that making the bootstrap be more
interesting is better: it flushes out more bugs earlier.  That said, I
really don't think there is any good substitute for doing regular
regression test runs and "program tests" (ie, compile and test a whole
bunch of random programs).

Second, I think that tracking down miscompilations is more difficult in
GCC than needbe due to lack of tools.  For example, we have a nifty tool
called bugpoint: http://llvm.cs.uiuc.edu/docs/CommandGuide/bugpoint.html
which takes a program being miscompiled and identifies the compiler pass(es)
causing the problem and reduces the input program down to a loop nest
being miscompiled, automatically.  For other types of problems (e.g.
compiler crashes/aborts) it can often narrow down the testcase to a few
instructions that trigger the problem.

I believe that using a HLL has *many* advantages compared to a low-level
language like C.  Writing higher-level code reduces the number of silly
bugs that creep in.  Using higher-level abstractions makes it easier to
design the compiler from a high-level.  Using a language that makes it
easy to use the right datastructure for a particular job means it's more
likely to be used, etc.  Even simple language features like namespaces
make it much easier for our external users to build tools that interface
with our code.  Finally, using a HLL makes the developers more productive.

In short, our experience using C++ in LLVM has been nothing but positive.

-Chris

-- 
http://llvm.cs.uiuc.edu/
http://www.nondot.org/~sabre/Projects/

Re: Of Bounties and Mercenaries

[Bernd Jendrissek]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Tue, Apr 06, 2004 at 09:07:48AM -0500, Chris Lattner wrote:
> On Tue, 6 Apr 2004, Robert Dewar wrote:
> > Chris Lattner wrote:
> > > Andrew Pinski wrote:
> > >
> > >>The other problem with this is that "Modern" "Standard" C++ has the
> > >>following problems:
> > >>1. Makes a mess out of simplifying code from the compiling when the
> > >>compiler is being miscompiled itself.
> > >
> > > Huh?
> >
> > If you really don't understand the above point 1, that's surprising.
> > Have you ever actually worked through problems in a self-compiling
> > bootstrapped compiler? Or are you just arguing language merits?

Anyone (newbies like I, for example ;) doubting the nastiness of
diagnosing a miscompiled *compiler* (not just its output) should
immediately go and read the ACM Classic: Reflections on Trusting Trust:

http://www.acm.org/classics/sep95/

The simple example of teaching a compiler to understand '\n' etc.
without having any mapping of 'n' => 012 anywhere in its source code (if
you don't grasp the profoundness of this, think about it again, until
you do), should indicate how weird a miscompiled compiler can be: it can
end up injecting bugs into your programs without the bug having any
source code!  The compiler can "learn" to bugify your code, and will
remember to do so even *after* you've eliminated the ultimate source of
the problem.

That's why it's a good idea to bootstrap GCC regularly with a non-GCC
compiler.  (I guess an ancient GCC would be a sufficiently close
approximation to a non-GCC compiler as most people will bother to get.
I happen to have gcc-2.7.2.3 around, and in my copious free time I
occasionally (in between bouts of FreeCiv) try to get gcc-2.5.8 going.)

Truly profound, this bootstrapping stuff.  Read that paper, co-newbies.
(And no, I'm not smoking anything.)

All of this also makes me want to see a *separate* C compiler that's
also free software, that is capable of bootstrapping GCC.  tcc (?)
springs to mind, but last I checked it barfed in libiberty.  On and off
I also play around with a bootstrap toolchain, which I eventually hope
to teach to compile C.  Progress is painfully slow, but it's fun!

Ah, another page springs to mind, where this guy has *done* much of the
work of bootstrapping a compiler from nothing, instead of just
daydreaming about it:

http://homepage.ntlworld.com/edmund.grimley-evans/bcompiler.html

- -- 
http://voyager.abite.co.za/~berndj/ (up again for now - yay!)
I've generally found that the fastest way to get the right answer on the net
is to confidently assert the answer you believe to be right; those who know
will immediately correct you, while if you just ask, often no answers arrive.
All it requires is a willingness to look bad on occasion.
                                               - Joe Buck on gcc@gcc.gnu.org
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.4 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iD8DBQFAc6Tn/FmLrNfLpjMRAtwdAJ9qME6cZMub7cKNFDfi2/P6j6zulwCgpjoM
w1VxOKeh7qP+KoTysniTXEE=
=wJAG
-----END PGP SIGNATURE-----

Re: Of Bounties and Mercenaries

[Tom Lord]

    > From: Bernd Jendrissek <berndj@prism.co.za>

    > That's why it's a good idea to bootstrap GCC regularly with a non-GCC
    > compiler.  (I guess an ancient GCC would be a sufficiently close
    > approximation to a non-GCC compiler as most people will bother to get.
    > I happen to have gcc-2.7.2.3 around, and in my copious free time I
    > occasionally (in between bouts of FreeCiv) try to get gcc-2.5.8 going.)

    > Truly profound, this bootstrapping stuff.  Read that paper, co-newbies.
    > (And no, I'm not smoking anything.)


Well, it's a classic paper for a good reason.

I say: give it to students.  Every generation or 2 of students should
rebuild the bootstrapping path starting from raw iron.  The students
will come out smarter and we all have a more secure foundation.

And remember, don't trust any software you get from Ken Thompson.
-t

Re: Of Bounties and Mercenaries

[Joe Buck]

On Wed, Apr 07, 2004 at 12:33:28PM -0700, Tom Lord wrote:
> 
>     > From: Bernd Jendrissek <berndj@prism.co.za>
> 
>     > That's why it's a good idea to bootstrap GCC regularly with a non-GCC
>     > compiler.  (I guess an ancient GCC would be a sufficiently close
>     > approximation to a non-GCC compiler as most people will bother to get.
>     > I happen to have gcc-2.7.2.3 around, and in my copious free time I
>     > occasionally (in between bouts of FreeCiv) try to get gcc-2.5.8 going.)
> 
>     > Truly profound, this bootstrapping stuff.  Read that paper, co-newbies.
>     > (And no, I'm not smoking anything.)
> 
> 
> Well, it's a classic paper for a good reason.
> 
> I say: give it to students.  Every generation or 2 of students should
> rebuild the bootstrapping path starting from raw iron.  The students
> will come out smarter and we all have a more secure foundation.

If you're really worried about the Ken Thompson hack, bootstrap starting
from several non-GCC compilers and determine whether you always get the
same bits.  Use cross-compilation too.  You then will prove that either
your GCC binary does not have the Thompson hack, or else that all the
free and proprietary compilers you tried have the same hack, so cleverly
designed as to be cross-platform.

Re: Of Bounties and Mercenaries

[Tom Lord]

    > From: Joe Buck <Joe.Buck@synopsys.com>

    > > Well, it's a classic paper for a good reason.

    > > I say: give it to students.  Every generation or 2 of students should
    > > rebuild the bootstrapping path starting from raw iron.  The students
    > > will come out smarter and we all have a more secure foundation.

    > If you're really worried about the Ken Thompson hack, 

Mildly worried -- not seriously.  Not _quite_ yet.  Well, ok, a
_little_ seriously.  It is a problem.

A Thompson virus injected into a major distribution would do a heck 'o
lot of damage and I haven't heard that vendors take steps to mitigate
that risk.  The cost of injection compared to the rewards of
exploitation looks a bit uncomfortable to me.  Do you disagree?  A
little Watergate-scale breaking-and-entering at SuSE/Novell, perhaps?

I hope the future deemphasizes major distributors of binary-form
platforms and that bootstrapping paths will be widely distributed,
varied, and used as the path to deployed systems.   Don't
misunderstand me, please --- major distributors: good;  major
distributors of once-compiled binaries?: not so good.

    > bootstrap starting from several non-GCC compilers and determine
    > whether you always get the same bits.  Use cross-compilation
    > too.  You then will prove that either your GCC binary does not
    > have the Thompson hack, or else that all the free and
    > proprietary compilers you tried have the same hack, so cleverly
    > designed as to be cross-platform.

Well, "same bits" is going to be hard to evaluate given differences
and chaos in code generation.  I guess you're assuming a convenient
and secure fixed-point on the path there.  The "fixed point" part is
easy but the "secure" part -- you're just moving around and slightly
obfuscating the job of a thompson virus.  As for depending on non-GCC
compilers --- well --- how many are there, really, that are capable of
being used for this?  If the number is small and they too come from a
few central sources -- you've only multiplied the cost of injecting a
Thompson virus by a small constant factor.  Ok, two breakins instead
of one.  It's quite a fragile chain of trust we're currently living
in.

And, anyway, custoemrs are currently trained to accept binaries -- not
to bootstrap.   We need to fix that upstream of those customers.

But all that aside, --- yeah -- go nuts with that kind of stuff.
Controlling the bootstrapping path is the best medicine here.  Do it
all over the place.  Lots of people in parallel.  It's the
monocultures that turn the idle curiosity into a serious risk.
Bootstrapping from raw iron should be a standard IT dept. function.

It's a great student exercise to rebuild the computing world _and_, as
a side effect, it helps mitigate the risk.   A perfect match.

Anyway, GCC-wise:  

~ Eventually, the GCC architecture is doomed.  Just too heavy and
  intractable and, sooner or later, something more clever and weildy
  will come along and very quickly displace it.  (I'm not holding my
  breath for it soon but it'll happen in our lifetimes for sure.)

~ Meanwhile, hey, a very fast but crappy-code compiler would be a huge
  boon to development.

GCC would be complemented by a parallel project to make a
simple-as-possible compiler suitable for bootstrapping GCC; also
useful as the fast compiler for edit-compile-debug cycles.  And it
would be half-a-step towards enabling a thorough Thompson-virus
killer.

Write it in forth, maybe.

Regards and, I've been posting way to much to this list lately for a
non-developer so, um, unless you want to take this particular
conversation further I should probably shut up for while.  Yrs in free
software comradery :-)
-t

Re: Of Bounties and Mercenaries

[Joe Buck]

On Wed, Apr 07, 2004 at 01:25:58PM -0700, Tom Lord wrote:
>     > bootstrap starting from several non-GCC compilers and determine
>     > whether you always get the same bits.  Use cross-compilation
>     > too.  You then will prove that either your GCC binary does not
>     > have the Thompson hack, or else that all the free and
>     > proprietary compilers you tried have the same hack, so cleverly
>     > designed as to be cross-platform.
> 
> Well, "same bits" is going to be hard to evaluate given differences
> and chaos in code generation.

You misunderstand.  "same bits" means "same bits".  gcc's three-stage
bootstrap should produce identical bits regardless of the bootstrap
compiler.  It is designed to eliminate effects caused by different
starting compilers.  The compiler compiles itself with itself.

Re: Of Bounties and Mercenaries

[Tom Lord]

    > From: Joe Buck <Joe.Buck@synopsys.com>

    > On Wed, Apr 07, 2004 at 01:25:58PM -0700, Tom Lord wrote:
    > >     > bootstrap starting from several non-GCC compilers and determine
    > >     > whether you always get the same bits.  Use cross-compilation
    > >     > too.  You then will prove that either your GCC binary does not
    > >     > have the Thompson hack, or else that all the free and
    > >     > proprietary compilers you tried have the same hack, so cleverly
    > >     > designed as to be cross-platform.

    > > Well, "same bits" is going to be hard to evaluate given differences
    > > and chaos in code generation.

    > You misunderstand.  "same bits" means "same bits".  gcc's three-stage
    > bootstrap should produce identical bits regardless of the bootstrap
    > compiler.  It is designed to eliminate effects caused by different
    > starting compilers.  The compiler compiles itself with itself.

I don't misunderstand.   That's what I meant by saying that the "fixed
point" part is easy but the "secure" part is not.

By injecting other compilers in the bootstrapping phase, which
incidentally most customers won't currently bother to do, you're just
raising the bar by a very small amount from a 1-stage thompson virus
to an n-stage thompson virus.   Since the number of candidate
compilers to inject into bootstrapping is currently very small -- as i
said: you're just multiplying the exploitation cost by a small
constant factor.

You say: "or else [...] all the free and proprietary compilers you
tried have the same hack" and I'm saying --- that's not currently
far-fetched enough to make me comfortable.  There aren't that many
other compilers I can throw in the mix there and many of them are
centrally controlled.

To really overcome the Thompson RISK, we need a combinatorics of
bootstrapping paths in theory and _practice_ that yields a _huge_
number of necessary exploits.   With lots of basic bootstrap paths
available (such as if we get a new 4 or 10 every few years, from
students), _then_ we can just start publishing and comparing binaries
and have a sense of security.

-t

Re: Of Bounties and Mercenaries

[Joe Buck]

On Wed, Apr 07, 2004 at 01:40:53PM -0700, Tom Lord wrote:
>     > You misunderstand.  "same bits" means "same bits".  gcc's three-stage
>     > bootstrap should produce identical bits regardless of the bootstrap
>     > compiler.  It is designed to eliminate effects caused by different
>     > starting compilers.  The compiler compiles itself with itself.
> 
> I don't misunderstand.   That's what I meant by saying that the "fixed
> point" part is easy but the "secure" part is not.
> 
> By injecting other compilers in the bootstrapping phase, which
> incidentally most customers won't currently bother to do, you're just
> raising the bar by a very small amount from a 1-stage thompson virus
> to an n-stage thompson virus.

Which is why I said that you could prove either that no compiler in the
set has a Thompson bug, or they all do.

> You say: "or else [...] all the free and proprietary compilers you
> tried have the same hack" and I'm saying --- that's not currently
> far-fetched enough to make me comfortable.  There aren't that many
> other compilers I can throw in the mix there and many of them are
> centrally controlled.

You're off in tinfoil hat land now, I'm afraid.  Without a theory as to
how someone could have gotten the same Thompson hack into Microsoft's
compiler, Sun's compiler, HP's compiler, and gcc, and then made sure
that the bug would keep functioning over the course of years of compiler
evolution, that's simply ridiculous.

Remember, for the Thompson hack to work, the compiler has to recognize
that it's compiling the compiler, and hack the output to reinsert two sets
of bugs into the output code.  But Thompson only had to recognize pcc.
Your hypothetical hack would have to recognize every C compiler in
existence, propagating the bugs into each one, every time, no matter
how they change.

Re: Of Bounties and Mercenaries

[Phil Edwards]

On Wed, Apr 07, 2004 at 01:55:37PM -0700, Joe Buck wrote:
> 
> Remember, for the Thompson hack to work, the compiler has to recognize
> that it's compiling the compiler, and hack the output to reinsert two sets
> of bugs into the output code.  But Thompson only had to recognize pcc.
> Your hypothetical hack would have to recognize every C compiler in
> existence, propagating the bugs into each one, every time, no matter
> how they change.

Precisely.  Thompson's hack doesn't scale in a proprietary, non-Illuminati
production environment.  Good thing, too.

Can this stupid thread stop now, or is there a /concrete/ suggestion that's
going to be made?  If this is another "but I must always have the last word"
thread, tell me now so I can just killfile it all.

-- 
Behind everything some further thing is found, forever; thus the tree behind
the bird, stone beneath soil, the sun behind Urth.  Behind our efforts, let
there be found our efforts.
              - Ascian saying, as related by Loyal to the Group of Seventeen

Re: Of Bounties and Mercenaries

[Tom Lord]

    > From: Phil Edwards <phil@jaj.com>

    > Can this stupid thread stop now, or is there a /concrete/
    > suggestion that's going to be made?  If this is another "but I
    > must always have the last word" thread, tell me now so I can
    > just killfile it all.

Killfiling is one thing.  A private act.  You can do it all on your
own without posting anything.   What you've actually done instead is
something entirely different.

-t

Re: Of Bounties and Mercenaries

[Phil Edwards]

On Wed, Apr 07, 2004 at 10:16:59PM -0700, Tom Lord wrote:
> 
>     > From: Phil Edwards <phil@jaj.com>
> 
>     > Can this stupid thread stop now, or is there a /concrete/
>     > suggestion that's going to be made?  If this is another "but I
>     > must always have the last word" thread, tell me now so I can
>     > just killfile it all.
> 
> Killfiling is one thing.  A private act.  You can do it all on your
> own without posting anything.   What you've actually done instead is
> something entirely different.

No kidding.  I'm trying to browbeat the rest of this list into not feeding
the trolls.

-- 
Behind everything some further thing is found, forever; thus the tree behind
the bird, stone beneath soil, the sun behind Urth.  Behind our efforts, let
there be found our efforts.
              - Ascian saying, as related by Loyal to the Group of Seventeen

Re: Of Bounties and Mercenaries

[Tom Lord]

    > From: Joe Buck <Joe.Buck@synopsys.COM>

    > On Wed, Apr 07, 2004 at 01:40:53PM -0700, Tom Lord wrote:
    > >     > You misunderstand.  "same bits" means "same bits".  gcc's three-stage
    > >     > bootstrap should produce identical bits regardless of the bootstrap
    > >     > compiler.  It is designed to eliminate effects caused by different
    > >     > starting compilers.  The compiler compiles itself with itself.

    > > I don't misunderstand.   That's what I meant by saying that the "fixed
    > > point" part is easy but the "secure" part is not.

    > > By injecting other compilers in the bootstrapping phase, which
    > > incidentally most customers won't currently bother to do, you're just
    > > raising the bar by a very small amount from a 1-stage thompson virus
    > > to an n-stage thompson virus.

    > Which is why I said that you could prove either that no compiler in the
    > set has a Thompson bug, or they all do.

Right.  We agree about that (which is trivial --- it's a pretty basic
factual thing).  I'm just saying that from published existing
practice, "they all" is a disturbingly small set, for practical
purposes.

Let's get more paranoid, shall we?   Better compare those binaries a
few different ways to be sure the tools you use to compare them aren't
also hosed.  Is that already done with GCC?  Where's the web page
with results?


    > > You say: "or else [...] all the free and proprietary compilers you
    > > tried have the same hack" and I'm saying --- that's not currently
    > > far-fetched enough to make me comfortable.  There aren't that many
    > > other compilers I can throw in the mix there and many of them are
    > > centrally controlled.

    > You're off in tinfoil hat land now, I'm afraid.  

Don't be afraid.  I hope and suspect you are right.  At the same time,
I think I have a not completely loony fear that you are wrong.  If I
had to bet a dollar, I'd bet you're right.  If I had to bet a million
dollars -- mmm.... I'd look for some hedges.  Is any org out there
betting a million dollars on the security of GCC as deployed across
the world?


    > Without a theory as to how someone could have gotten the same
    > Thompson hack into Microsoft's compiler, Sun's compiler, HP's
    > compiler, and gcc, and then made sure that the bug would keep
    > functioning over the course of years of compiler evolution,
    > that's simply ridiculous.

Hrm.  For one thing, I'm not aware of any ongoing effort to compare
the results of GCC bootstrapping via all those paths.  Are you?

For another thing: a 3-way attack vector?   That's not huge.  Let's
compare attack costs vs. attack rewards.   How many gazillions of
dollars are modulated by GCC-generated code?


    > Remember, for the Thompson hack to work, the compiler has to recognize
    > that it's compiling the compiler, and hack the output to reinsert two sets
    > of bugs into the output code.  But Thompson only had to recognize pcc.
    > Your hypothetical hack would have to recognize every C compiler in
    > existence, propagating the bugs into each one, every time, no matter
    > how they change.

Yeah, right.   Since the Thompson paper, noboby at all has worked on
higher-level programming techniques.  Sure.

We agree about the factual issues -- just not about our guestimates of
how they measure up against the economics.   I concede .... I'm
expressing a paranoia.  I assert: it's not so far fetched as to be
worth ignoring.

-t

Re: Of Bounties and Mercenaries

[Joe Buck]

On Wed, Apr 07, 2004 at 10:14:13PM -0700, Tom Lord wrote:
> Let's get more paranoid, shall we?  

No, let's get more real.

That's my last comment on this topic.

Re: Of Bounties and Mercenaries

[Chris Lattner]

Stephan T. Lavavej wrote:
>[Ben Elliston]
>> This should be a side goal of future GCC work: to develop it
>> in a way that makes it easier for the uninitiated to hack on
>> it.  This includes removing complicated cruft and generally
>> trying to keep the code base small and as simple as possible.

> How about writing gcc in Modern, Standard C++?

You might be interested in LLVM: http://llvm.cs.uiuc.edu/

We are doing basically that.  The other advantage is that you can also use
algorithms and data structures that make sense in the process.  In C++
it's much easier to not use linked-lists for every data structure in the
compiler. :)  Using C++ well has been a HUGE reason that LLVM has made as
much progress as it has in the time it has existed (it's only 3.5 years
old, basically the same "age" as tree-ssa).

Despite the fact that LLVM makes EXTENSIVE use of the STL and other modern
C++ features, it can already bootstrap itself.  It also has a large number
of aggressive intra- and inter- procedural optimizations, a JIT compiler,
profile-driven optimization support, aggressive alias analysis, ...

> You could do away with your crazy garbage-collected C (double ugh).

Exactly.  In LLVM, the ownership model for IR objects is trivial,
completely eliminating the need for a GC.  This and many other things
makes the optimizer *FAST*.

-Chris

-- 
http://llvm.cs.uiuc.edu/
http://www.nondot.org/~sabre/Projects/

Re: Of Bounties and Mercenaries

[Stephan T. Lavavej]

[Ben Elliston]
> This should be a side goal of future GCC work: to develop it
> in a way that makes it easier for the uninitiated to hack on
> it.  This includes removing complicated cruft and generally
> trying to keep the code base small and as simple as possible.

How about writing gcc in Modern, Standard C++?  You could do away with your
crazy garbage-collected C (double ugh).

This inflammatory suggestion was brought to you by...

Stephan T. Lavavej
http://nuwen.net

Re: Of Bounties and Mercenaries

[Daniel Berlin]

On Apr 5, 2004, at 11:08 PM, Stephan T. Lavavej wrote:

> [Ben Elliston]
>> This should be a side goal of future GCC work: to develop it
>> in a way that makes it easier for the uninitiated to hack on
>> it.  This includes removing complicated cruft and generally
>> trying to keep the code base small and as simple as possible.
>
> How about writing gcc in Modern, Standard C++?  You could do away with 
> your
> crazy garbage-collected C (double ugh).
>
> This inflammatory suggestion was brought to you by...
>
About half of us want to do this.
Another 25% want to rewrite it in some other language
The other 25% think converting it to ISO C went too far.
They also believe that english language development should have stopped 
with shakespeare, and that kids these days don't do anything right.
--Dan

Re: Of Bounties and Mercenaries

[Gabriel Dos Reis]

Daniel Berlin <dberlin@dberlin.org> writes:

| On Apr 5, 2004, at 11:08 PM, Stephan T. Lavavej wrote:
| 
| > [Ben Elliston]
| >> This should be a side goal of future GCC work: to develop it
| >> in a way that makes it easier for the uninitiated to hack on
| >> it.  This includes removing complicated cruft and generally
| >> trying to keep the code base small and as simple as possible.
| >
| > How about writing gcc in Modern, Standard C++?  You could do away
| > with your
| > crazy garbage-collected C (double ugh).
| >
| > This inflammatory suggestion was brought to you by...
| >
| About half of us want to do this.
| Another 25% want to rewrite it in some other language
| The other 25% think converting it to ISO C went too far.
| They also believe that english language development should have
| stopped with shakespeare, and that kids these days don't do anything
| right.

Very recently, I've taken a poll among the g++ maintainers on the
matter of making the front-end compilable by (but -without- requiring)
a C++ compiler.  I got one positive feedback and one over-my-dead-body.

[ The reason I brought the issue is that some people here have been
  developing an interesting C++ library that makes it  possible to
  represent C++ programs in C++ and they have been using a commercial
  compiler (early versions used GCC-3.0.x, before switching to that
  commercial compiler); as a member of that project, a C++ user and
  maintainer of   GCC, I'm of the opinion that such a library is of
  great value to the GCC/g++ user community. ]

-- Gaby
 

Re: Of Bounties and Mercenaries

[Joe Buck]

On Thu, Apr 08, 2004 at 12:27:16AM +0200, Gabriel Dos Reis wrote:
> Very recently, I've taken a poll among the g++ maintainers on the
> matter of making the front-end compilable by (but -without- requiring)
> a C++ compiler.  I got one positive feedback and one over-my-dead-body.

How close are we to that state today?  We'd need all the K&R C gone,
and would need to avoid C++ keywords, but probably the big one is
playing fast and loose with enums.

It seems that before making a judgment of either "positive feedback"
or "over my dead body" it makes sense to know the size of the changes
required.
 

Re: Of Bounties and Mercenaries

[Gabriel Dos Reis]

Joe Buck <Joe.Buck@synopsys.com> writes:

| On Thu, Apr 08, 2004 at 12:27:16AM +0200, Gabriel Dos Reis wrote:
| > Very recently, I've taken a poll among the g++ maintainers on the
| > matter of making the front-end compilable by (but -without- requiring)
| > a C++ compiler.  I got one positive feedback and one over-my-dead-body.
| 
| How close are we to that state today?  We'd need all the K&R C gone,
| and would need to avoid C++ keywords, but probably the big one is
| playing fast and loose with enums.

For the g++ part, K&R C is completely done.

I offered to produce patches to get rid of use of C++ keywords.  That
is part of what I call making the front-end compilable with a c++
compiler.

What do you mean by "loose with enums"?

-- Gaby

Re: Of Bounties and Mercenaries

[Joe Buck]

On Thu, Apr 08, 2004 at 08:31:16PM +0200, Gabriel Dos Reis wrote:
> Joe Buck <Joe.Buck@synopsys.com> writes:
> 
> | On Thu, Apr 08, 2004 at 12:27:16AM +0200, Gabriel Dos Reis wrote:
> | > Very recently, I've taken a poll among the g++ maintainers on the
> | > matter of making the front-end compilable by (but -without- requiring)
> | > a C++ compiler.  I got one positive feedback and one over-my-dead-body.
> | 
> | How close are we to that state today?  We'd need all the K&R C gone,
> | and would need to avoid C++ keywords, but probably the big one is
> | playing fast and loose with enums.
> 
> For the g++ part, K&R C is completely done.
> 
> I offered to produce patches to get rid of use of C++ keywords.  That
> is part of what I call making the front-end compilable with a c++
> compiler.
> 
> What do you mean by "loose with enums"?

C permits conversions between enums and ints that C++ does not.

Re: Of Bounties and Mercenaries

[Gabriel Dos Reis]

Joe Buck <Joe.Buck@synopsys.COM> writes:

| On Thu, Apr 08, 2004 at 08:31:16PM +0200, Gabriel Dos Reis wrote:
| > Joe Buck <Joe.Buck@synopsys.com> writes:
| > 
| > | On Thu, Apr 08, 2004 at 12:27:16AM +0200, Gabriel Dos Reis wrote:
| > | > Very recently, I've taken a poll among the g++ maintainers on the
| > | > matter of making the front-end compilable by (but -without- requiring)
| > | > a C++ compiler.  I got one positive feedback and one over-my-dead-body.
| > | 
| > | How close are we to that state today?  We'd need all the K&R C gone,
| > | and would need to avoid C++ keywords, but probably the big one is
| > | playing fast and loose with enums.
| > 
| > For the g++ part, K&R C is completely done.
| > 
| > I offered to produce patches to get rid of use of C++ keywords.  That
| > is part of what I call making the front-end compilable with a c++
| > compiler.
| > 
| > What do you mean by "loose with enums"?
| 
| C permits conversions between enums and ints that C++ does not.

Yes; I was not aware of that naming.
In fact, C++ permits implicit conversion of enum to ints and explicit
conversion of ints to enums.  I don't know how big that issue is.

-- Gaby

Re: Of Bounties and Mercenaries

[Tom Lord]

    > From: "Stephan T. Lavavej" <stl@caltech.edu>

    > [Ben Elliston]
    > > This should be a side goal of future GCC work: to develop it
    > > in a way that makes it easier for the uninitiated to hack on
    > > it.  This includes removing complicated cruft and generally
    > > trying to keep the code base small and as simple as possible.

    > How about writing gcc in Modern, Standard C++?  You could do away with your
    > crazy garbage-collected C (double ugh).

    > This inflammatory suggestion was brought to you by...

Adding to the mini-dog-pile-on-the-rabbit:  write in some vvhll (very,
very hll) that let's you write directly in graph and set notation and
then uses an ai program to generate an ml-family program that compiles
to the compiler.

Heck, if you didn't care about performance, writing it in mathematica
or some such would make many tasks a complete breeze --- so really,
it's just a question of how to optimize programs in such notation down
to a practical compiler.

I particularly like the idea of writing virtual passes and having
those partially evaluated to fold their composition into single
passes.   Give it 10 years --- GCC is dead :-)

-t

Re: Of Bounties and Mercenaries

[Andrew Pinski]

On Apr 5, 2004, at 23:08, Stephan T. Lavavej wrote:

> [Ben Elliston]
>> This should be a side goal of future GCC work: to develop it
>> in a way that makes it easier for the uninitiated to hack on
>> it.  This includes removing complicated cruft and generally
>> trying to keep the code base small and as simple as possible.
>
> How about writing gcc in Modern, Standard C++?  You could do away with 
> your
> crazy garbage-collected C (double ugh).
>
> This inflammatory suggestion was brought to you by...

The other problem with this is that "Modern" "Standard" C++ has the 
following
problems:

1.  Makes a mess out of simplifying code from the compiling when the 
compiler
is being miscompiled itself.

2.  "Modern" means templates see 1.

3.  "Standard" means that we have to do more workarounds that Boost 
does, see 1.

4.  GC will be still there just in a slightly different form, see 2 and 
1.

5.  The number of bug reports about the C++ is high and we get more 
invalid bugs
each day, so we will then get bug reports about GCC not compiling on 
some C++
compiler, see 3.

Now if we figure out a good GC in that only parts of the compiler uses 
it, we will
no longer have all the issues we currently have because the part I am 
talking about
RTL (which is the most memory bounded part) will move from being in the 
GC into being
malloc'd and maybe pool alloced instead so it will be relatively close 
in memory.


Just some thoughts about using "Modern" "Standard" C++,
Andrew Pinski

Re: Of Bounties and Mercenaries

[Scott Robert Ladd]

Stephan T. Lavavej wrote:
> How about writing gcc in Modern, Standard C++?  You could do away with your
> crazy garbage-collected C (double ugh).
> 
> This inflammatory suggestion was brought to you by...

While I'm often a C++ programmer by trade, I do not favor rewriting GCC 
in C++.

C is an excellent language -- despite being Standardized (twice!) by 
committees, it remains a clean and succint way of expressing algorithms. 
Yes, C++ has many nice object-oriented features -- and those features 
are often misused, obscuring the underlying logic of a program. It is 
very easy to be seduced by the power of higher-level abstractions, only 
to lose the efficiency and clarity of expression required for elegant 
programming.

-- 
Scott Robert Ladd
Coyote Gulch Productions (http://www.coyotegulch.com)
Software Invention for High-Performance Computing

RE: Of Bounties and Mercenaries

[Stephan T. Lavavej]

[Scott Robert Ladd]
> C is an excellent language -- despite being Standardized
> (twice!) by committees

Despite, heh.

> it remains a clean and succint way of expressing algorithms.

I don't really think so; not when the algorithms involve any sort of data
structures (which means basically most algorithms).

> Yes, C++ has many nice object-oriented features -- and those
> features are often misused, obscuring the underlying logic of
> a program.

I didn't suggest using Crummy, Common C++, where inheritance is overused and
bozos abound.

> It is very easy to be seduced by the power of higher-level
> abstractions, only to lose the efficiency and clarity of
> expression required for elegant programming.

Recently I was discussing with some friends how to implement Huffman
compression.  When a Huffman tree is being constructed, "available" nodes
are kept in a pile, and nodes with the lowest weight must be repeatedly
removed, while new nodes are inserted.

After a bit of thinking, we arrived at this:

std::priority_queue<Node *, std::vector<Node *>, boost::function<bool (Node
*, Node *)> >
available_nodes(*_1 > *_2);

Now, that's certainly... subtle, but it simplifies greatly the following
code.

And you don't ever have to use GC if you use RAII.

Stephan T. Lavavej
http://nuwen.net

Re: Of Bounties and Mercenaries

[Scott Robert Ladd]

Stephan T. Lavavej wrote:
> std::priority_queue<Node *, std::vector<Node *>, boost::function<bool (Node
> *, Node *)> >
> available_nodes(*_1 > *_2);

The above is beautiful, until you have to remember five years later what 
underlies the symbology. Minor changes cascade through C++ code, 
quietly, hidden behind template gynamistics and inheritance; subtle bugs 
live in syntactic sugar. Furthermore, such code also finds subtle and 
and annoying bugs in C++ compilers, which are complex and inconsistent 
beasts.

That's not to say that C (or any other language) is prone to problems, 
nor am I against the idea of replacing code when the sutuation warrants. 
However, I also don't see the point of replacing working, established 
code with an entirely new framework. Replacing it would take longer (and 
be more error-prone) than educating people in how GCC is already 
written. Not all old code is bad code.

> And you don't ever have to use GC if you use RAII.

On that, we're in total agreement. However, this conversation is now far 
from on-topic, and so I shall move on to other things.

-- 
Scott Robert Ladd
Coyote Gulch Productions (http://www.coyotegulch.com)
Software Invention for High-Performance Computing

Re: Of Bounties and Mercenaries

[Toon Moene]

Stephan T. Lavavej wrote:

> Recently I was discussing with some friends how to implement Huffman
> compression.  When a Huffman tree is being constructed, "available" nodes
> are kept in a pile, and nodes with the lowest weight must be repeatedly
> removed, while new nodes are inserted.
> 
> After a bit of thinking, we arrived at this:
> 
> std::priority_queue<Node *, std::vector<Node *>, boost::function<bool (Node
> *, Node *)> >
> available_nodes(*_1 > *_2);
> 
> Now, that's certainly... subtle, but it simplifies greatly the following
> code.

[ Sorry coming into this debate four days late ]

I didn't even parse this code, though I have "Accelerated C++" by Koenig 
& Moo - I was hit by the obvious: the adjective "subtle".

I have worked on too many programming projects with multiple 
contributors over the last 20 years, and the one thing you *do not want 
to have* is subtility.

Everything in the code has to be blatantly obvious, or else one of your 
coworkers is going to misinterpret it.

No amount of commenting is going to offset that.

-- 
Toon Moene - mailto:toon@moene.indiv.nluug.nl - phoneto: +31 346 214290
Saturnushof 14, 3738 XG  Maartensdijk, The Netherlands
Maintainer, GNU Fortran 77: http://gcc.gnu.org/onlinedocs/g77_news.html
GNU Fortran 95: http://gcc.gnu.org/fortran/ (under construction)

Re: Of Bounties and Mercenaries

[Alexandre Oliva]

On Apr  6, 2004, "Stephan T. Lavavej" <stl@caltech.edu> wrote:

> How about writing gcc in Modern, Standard C++?  You could do away with your
> crazy garbage-collected C (double ugh).

As much as I like C++, I don't see how switching to it would do away
with our need for garbage collection.  And then, in order to enjoy the
benefits of C++, we'd have to pretty much rewrite gcc from scratch, at
which point the recollection of something I read springs to mind:
[googles a bit] ah, yes:
http://www.joelonsoftware.com/articles/fog0000000069.html

-- 
Alexandre Oliva             http://www.ic.unicamp.br/~oliva/
Red Hat Compiler Engineer   aoliva@{redhat.com, gcc.gnu.org}
Free Software Evangelist  oliva@{lsd.ic.unicamp.br, gnu.org}

RE: Of Bounties and Mercenaries

[Stephan T. Lavavej]

[Alexandre Oliva]
> As much as I like C++, I don't see how switching to it would
> do away with our need for garbage collection.

It's pretty simple, really.  Garbage collection is a morally bankrupt idea.
Now, it's probably a step up from manual memory management, which is what C
will otherwise force you to do, but that doesn't say much.

In C++, you can use Resource Acquisition Is Initialization (RAII).  You
delegate resource management to objects; when the objects are constructed
and initialized, they acquire their resources, and when the objects are
destroyed, they release their resources.

Now, if you work with local objects all of the time, they die at the ends of
their blocks and all is well.  If you need to manage collections of things,
use STL containers, which will gather and release resources as necessary.

The interesting part is what to do with resources that do not have obvious
copy semantics (files, sockets, etc.).  Have resource manager classes that
derive from boost::noncopyable, and then maintain shared_ptr's to them.
When you need a file, or whatever, construct it (acquiring the resource)
with /new/, and immediately hand it off to a shared_ptr.  You can then copy
that shared_ptr at whim, put it in containers, pass it to functions, and it
will always refer to the same file.  When all the shared_ptr's die, then the
file is released.

This means: do not use /new/ unless you are immediately handing something
noncopyable off to a shared_ptr, and do not use /delete/.

The one exception: When you need something exotic, write your own
resource-managing class.  It can use new and delete and good old fashioned
manual memory management all it likes, as long as it stays confined to that
single class.

Using this philosophy, the resources manage themselves, rather than you
having to do it (traditional C) or having a garbage collector do it (with
the associated locality problems).

> And then, in order to enjoy the benefits of C++, we'd have to
> pretty much rewrite gcc from scratch
> http://www.joelonsoftware.com/articles/fog0000000069.html

That's funny; I always considered Fred Brooks a demigod.

Joel Spolsky misquoted Brooks, by the way.  He actually said (emphasis his):

"/plan to throw one away; you will, anyhow/"

Stephan T. Lavavej
http://nuwen.net

Re: Of Bounties and Mercenaries

[Alexandre Oliva]

On Apr  6, 2004, "Stephan T. Lavavej" <stl@caltech.edu> wrote:

> In C++, you can use Resource Acquisition Is Initialization (RAII).

Sure.  That, along with reference-counted pointers, can take care of
*some* cases in which you'd like to release memory.  It doesn't take
care of cycles, though, and they're quite pervasive in GCC.  Besides,
garbage collection can actually be more efficient than reference
counting, if it's done well.

-- 
Alexandre Oliva             http://www.ic.unicamp.br/~oliva/
Red Hat Compiler Engineer   aoliva@{redhat.com, gcc.gnu.org}
Free Software Evangelist  oliva@{lsd.ic.unicamp.br, gnu.org}

RE: Of Bounties and Mercenaries

[Stephan T. Lavavej]

[Alexandre Oliva]
> It doesn't take care of cycles, though, and they're quite
> pervasive in GCC.

Cycles are the standard objection to my argument.

It also completely misses the point.  I advocate using shared_ptr (a
reference counted pointer) to hold noncopyable resource managers - that
doesn't introduce cycles.

If you have a data structure which has cycles, then you should encapsulate
it.  For example, std::list is a doubly linked list and hence has cycles of
pointers inside of it.  But the user of std::list doesn't have to know or
care about that.

Manual memory management is a reasonable thing to do, as long as it stays
confined to the constructor, destructor, and methods of a single class that
needs to do something exotic.  (Of course, the same class can use vectors
and whatnot to do mundane things.)

Stephan T. Lavavej
http://nuwen.net

Re: Of Bounties and Mercenaries

[Robert Dewar]

> I think the fallacy in the argument is a bit different.  The idea is
> that, if I were to build a program that I knew thousands of users
> would run thousands of times, and having it run a bit faster would
> make a significant difference in how happy my users would be such that
> they'd be willing to pay more for the package, for support, whatever,
> it would make sense to squeeze as much performance as possible out of
> the compiler into the application.  From the POV of the users of my
> application, it doesn't matter if I spend hours or days building the
> application, or even tuning the compiler to find the optimal flags to
> build it: they'll be happier the faster the thing runs.

If you have days to spend, they are probably better spent profiling
your application and analyzing its speed. if there really is an inner
loop that is eating time, optimize it by hand.

Re: Of Bounties and Mercenaries

[Stephan T. Lavavej]

[Nathanael Nerode]
> My need was for a compiler which built in a sane manner
> which could be understood (and upgradided) by anyone, rather
> than a black box of untouchable black magic where mortals
> feared to tread.

Speaking of that, want to take a look at http://gcc.gnu.org/PR14666 and
http://gcc.gnu.org/PR14601 ?  It's all black magic to me...

I can provide the exact files needed to reproduce 14666 on GNU/Linux
(PR14601 needs a Windows system with MSYS).  :->

Adding the ability at configure time to build in given library and include
file search paths would solve 14601 and part of 14666.  But that would just
be a way to cover up the real problems.

Stephan T. Lavavej
http://nuwen.net

Re: Of Bounties and Mercenaries

[Nathanael Nerode]

Joe Buck wrote:
>Motivated individuals who feel that their needs aren't being addressed are
>welcome to educate themselves enough to become GCC contributors themselves.

I guess I did. :-)  My need was for a compiler which built in a sane manner
which could be understood (and upgradided) by anyone, rather than a black box
of untouchable black magic where mortals feared to tread.

I hope I've made some progress on that front (eliminating Cygnus configure
from the tree, updating stuff to autoconf 2.5x, adding comments to configure
and make scripts, reorganizing them, eliminating the exotic fixincludes
scripts, getting libada into its own directory, etc.)

I'm quite sure this qualifies as an infrastructure improvement which
benefits everyone but which most companies won't dedicate specific money
or people to, which was one of the categories people were talking about,
no?

-- 
Make sure your vote will count.
http://www.verifiedvoting.org/

Re: Of Bounties and Mercenaries

[Dan Kegel]

Ian wrote:
 > ... What Scott is talking about is how to balance the needs of the
 > free software community and the needs of the commercial interests.

I don't see any need for increased 'balance'.  The commercial
contributions increase the quality of gcc.  What's not to like
about that?  The free software community gets nothing but benefit
from them.

Joe Buck wrote:
 > Motivated individuals who feel that their needs aren't being addressed
 > are welcome to educate themselves enough to become GCC contributors
 > themselves.

Indeed.  The whole idea behind the free software community is
that if enough people want something, they can make it happen.
And more often than not, a sufficiently motivated person
can get excellent help from the community, and really make
progress.

Case in point.  I was fed up with how hard it was to build
gcc and glibc cross-toolchains, and how many bugs there were
in the ppc405 and sh4 support.  Did I whine and kvetch?
Yes, for about a year.  But since that didn't help much, and
since I really needed it, I sat down and worked on a solution
of sorts.   The evolving result is http://kegel.com/crosstool,
which actually does make it kind of easy to build many combinations
of (binutils, gcc, glibc, target CPU) targeting Linux
on any Posix-compatible workstation, and incorporates many
semi-carefully documented patches solving various problems on
particular older versions of tools or for odd platforms.
I wouldn't have been able to do it without a good search
engine, and without help from dozens of experts who patiently
answered the questions I couldn't just google or solve myself.

So Scott, relax.
No need to try to make things happen faster by adding
voting, bounties, or some mythical matchmaking service.
Anyone who wants something bad enough can already help make it
happen.
- Dan

Re: Of Bounties and Mercenaries

[Bud Davis]

> Stelios Xanthakis wrote:
> Compiler speed *should not* be an issue. If gcc 3.5 is ten times
> slower compiling programs at -O2, that's a good thing: it means
> that it does more to procude better code. 

Wow, I was preparing a long winded reply about large work groups, 
large source trees, phased integration, and how a compile time 
increase of a few hours is a big deal...when I realized !!

Great Troll !!  Got me !!


--bud davis

Re: Of Bounties and Mercenaries

[Richard Kenner]

    I made a serious (with tongue a tad in cheek) set of suggestions 
    specific to GCC, and in response to comments made herein by others. 
    There is a sincere need to address these issues, which are very 
    GCC-specific; few other packages have the breadth and depth of GCC, 
    across platforms and users. Therefore, this seems to be an appropriate 
    venue.

You are totally ignoring the fact that there are many people who *are* paying
to have GCC maintained and a number of companies very happy to receive
such payments.  But the point remains that the issues you are raising are
financial and political and this is a *technical* list.

Re: Of Bounties and Mercenaries

[Daniel Berlin]

On Apr 2, 2004, at 11:25 AM, Richard Kenner wrote:

>     I made a serious (with tongue a tad in cheek) set of suggestions
>     specific to GCC, and in response to comments made herein by others.
>     There is a sincere need to address these issues, which are very
>     GCC-specific; few other packages have the breadth and depth of GCC,
>     across platforms and users. Therefore, this seems to be an 
> appropriate
>     venue.
>
> You are totally ignoring the fact that there are many people who *are* 
> paying
> to have GCC maintained and a number of companies very happy to receive
> such payments.  But the point remains that the issues you are raising 
> are
> financial and political and this is a *technical* list.
>
It's not completely clear from the description that this is offtopic:

	• 	gcc  is a high volume list for general development discussions 
about GCC.  Anything relevant to the development or testing of GCC and 
not  covered by other mailing lists is suitable for discussion here.

This could be considered "relevant to the development of GCC, and not 
covered by other mailing lists"

Not that i care (hey, if it makes people feel better to put $5 towards 
fixing a given bug, what do i care?), even though i'd be the one to 
implement any such suggestions.
--Dan

Re: Of Bounties and Mercenaries

[Scott Robert Ladd]

Richard Kenner wrote:
> You are totally ignoring the fact that there are many people who *are* paying
> to have GCC maintained and a number of companies very happy to receive
> such payments.  But the point remains that the issues you are raising are
> financial and political and this is a *technical* list.

Odd. When someone asks about getting a specific feature added or having 
a bug fixed, Robert Dewar (for example) inevitably replies that funding 
is required. Is he banned from making such statements? I should hope not!

Any group project involves politics; to ignore that is to be ineffective.

-- 
Scott Robert Ladd
Coyote Gulch Productions (http://www.coyotegulch.com)
Software Invention for High-Performance Computing

Re: Of Bounties and Mercenaries

[Tom Lord]

Instead of bounties and mercenaries, a model which has occaisional
success but a long track record of that success being very limited and
narrowly focused, may I suggest instead the OSDL model?

From a bird's-eye view, OSDL collects money from corporate sponsors
and uses that to help re-seed the Linux-kernel commons.  It takes on
and pays for tasks that no one contributor could individually afford
and that no downstream customer is going to want to pay for directly.
It "ties together" a great deal of the commercial activity around the
kernel.  It affords the sponsors a degree of protection against the
kernel project being usurped or fragmented.  It's a funded hub of
cooperation.

GCC may be too small a scope to make real progress on your general
idea and may already be situated in the commercial world in too
complex a way.  I think you'll have trouble differentiating from
commercial GCC developers while still finding enough paying
customers/contributors to make the effort worthwhile.  It might make
conceptual sense for the companies most interested in GCC to form an
OSDL-like entity -- but I think it's unlikely because, from the
perspective of those companies, based on past content on this list,
"if the SC system ain't broken, don't fix it."

Instead, a .org that worked on several different projects, including
GCC, and that was responsive to the kinds of issues typically
neglected by the vendor-developers, might actually have a chance of
collecting funding from everyone from individuals to global corps.

Antepenultimately: be wary of legal issues. To what entity will bounty
posters send money?  What will be the legal relationship between that
entity and bounty collectors?  The right to advertise/run the thing on
"gnu.org" domains should not be presumed.  Such a system would
interact with the FSF's 503(c) status in ways that need careful
attention.

Penultimately: history does not paint a promising picture.  Several
bounty/mercenary systems have come and most have gone.  None has been
a conduit for much money.  Doesn't mean it can't be done but people
have tried and not yet succeeded in any big way.  People generally
don't shop that way and, where these systems have succeeded in the
past, it looks to me like they succeeded by being a convenient conduit
for funding in a form that yields a larger tax break than the R&D tax
credits --- those conduits being sought sometimes for commercial
reasons (to fund development of free software that won't uniquely
benefit the funder or, perhaps, just to save money) and sometimes for
political reasons (to fund development of free software whose
political impact is hoped for, for which R&D tax credits might not
apply).  You might find that you can make more money selling t-shirts
if, overall, your entity is seen to be doing a lot of public good.

Ultimately: I gather you want to pool funds from people interested in
issues they can't afford to individually address.  I think that's a
true and good niche to fill.  But two suggestions: First: Don't nickle
and dime people --- asking them to earmark every penny rather than
just giving hints about their interests makes it too hard to send a
few bucks.  Second: if pooling _isn't_ your goal then you should bag
all this bounty stuff and just be a sole proprietor GCC consultant
(which, for all I know, you already are).

-t

Re: Of Bounties and Mercenaries

[Scott Robert Ladd]

Tom Lord wrote:
> Instead of bounties and mercenaries, a model which has occaisional
> success but a long track record of that success being very limited and
> narrowly focused, may I suggest instead the OSDL model?

The OSDL is an excellent model -- but it does beg one question. Mr. 
Torvalds is now in their employ, and the Linux folk have long been 
grumpy about GCC compile times. I wonder is OSDL has or will fund 
efforts toward improveing compiler speed?

> GCC may be too small a scope to make real progress on your general
> idea and may already be situated in the commercial world in too
> complex a way.

My fear (and that of others) is that GCC has become so commercialized 
that it fails to address the needs of the community at large. I'm not 
anti-commercial -- I'm just looking for some model that meets the needs 
of certain forgotten and ignored constituencies.

> Ultimately: I gather you want to pool funds from people interested in
> issues they can't afford to individually address.  I think that's a
> true and good niche to fill.  But two suggestions: First: Don't nickle
> and dime people --- asking them to earmark every penny rather than
> just giving hints about their interests makes it too hard to send a
> few bucks.

Precisely. I should have been more clear, perhaps.

> Second: if pooling _isn't_ your goal then you should bag
> all this bounty stuff and just be a sole proprietor GCC consultant
> (which, for all I know, you already are).

As a matter of disclaimer, I have never been paid anything to work on 
GCC. I *have* been paid to work on commecial C++ compilers (not INtel, 
for those who wonder).

That *could* change, as I have some customers who very much want OpenMP. 
   And others are looking into funding as well.

-- 
Scott Robert Ladd
Coyote Gulch Productions (http://www.coyotegulch.com)
Software Invention for High-Performance Computing

Re: Of Bounties and Mercenaries

[Joe Buck]

On Fri, Apr 02, 2004 at 07:48:15PM -0500, Scott Robert Ladd wrote:
> My fear (and that of others) is that GCC has become so commercialized 
> that it fails to address the needs of the community at large.

That would assume that there is a significant divergence between commercial
interests and "community at large" interests.

Companies as well as individuals want a compiler that produces good code
without consuming excessive resources.  There certainly are some differences,
particularly when it comes to tradeoffs involving older hardware.

Motivated individuals who feel that their needs aren't being addressed are
welcome to educate themselves enough to become GCC contributors themselves.

Re: Of Bounties and Mercenaries

[Stelios Xanthakis]

On Fri, 2 Apr 2004, Scott Robert Ladd wrote:

> Tom Lord wrote:
> > Instead of bounties and mercenaries, a model which has occaisional
> > success but a long track record of that success being very limited and
> > narrowly focused, may I suggest instead the OSDL model?
> 
> The OSDL is an excellent model -- but it does beg one question. Mr. 
> Torvalds is now in their employ, and the Linux folk have long been 
> grumpy about GCC compile times. I wonder is OSDL has or will fund 
> efforts toward improveing compiler speed?
> 

Compiler speed *should not* be an issue. If gcc 3.5 is ten times
slower compiling programs at -O2, that's a good thing: it means
that it does more to procude better code. A program is compiled
once but executed 100000 times. So compile-time is *not* an issue
and it would be wrong if gcc developers dropped features because
they slow compile speed.

What would be nice is to be able to compile applications with -O0 while
developping and testing them and with -O2 for the final executable
(the one that will be used 100000 times).

Unfortunatelly this doesn't work. A progam that works at -O0 may
reveal bugs at -O2. If you could guarantee that ``if something is
ok at -O0 is also ok in -O2'' that'd be a real win.

Apart from that, i think it's up to the application progammer
to write code that does not require -O2, can be tested at -O0
and use valgrind on it.

gcc developers *should* completely ignore comments about the speed
of gcc. Do you get paid to improve the speed? No. Application
programmers should write good code. If a big corp has incompetent
people and expects from gcc to get faster at -O2, they can hire
a filthy bounty hunter ;)

Re: Of Bounties and Mercenaries

[Toon Moene]

Stelios Xanthakis wrote:

> Compiler speed *should not* be an issue. If gcc 3.5 is ten times
> slower compiling programs at -O2, that's a good thing: it means
> that it does more to procude better code. A program is compiled
> once but executed 100000 times.

Unfortunately, that depends on your niche.  In meteorology, a program is 
compiled and then used for a year before recompiling (unless an urgent 
problems rises).  That's (in our case) 365 * 8 ~ 3000 runs against 1 
compile.

However, there are developers who do almost nothing but compiling 
(versions) of their code and hardly ever run it (to the end).

Hope this helps,

-- 
Toon Moene - mailto:toon@moene.indiv.nluug.nl - phoneto: +31 346 214290
Saturnushof 14, 3738 XG  Maartensdijk, The Netherlands
Maintainer, GNU Fortran 77: http://gcc.gnu.org/onlinedocs/g77_news.html
GNU Fortran 95: http://gcc.gnu.org/fortran/ (under construction)

Re: Of Bounties and Mercenaries

[Robert Dewar]

Toon Moene wrote:

> Unfortunately, that depends on your niche.  In meteorology, a program is 
> compiled and then used for a year before recompiling (unless an urgent 
> problems rises).  That's (in our case) 365 * 8 ~ 3000 runs against 1 
> compile.

Such extreme cases are ones in which it pays to do extensive profiling
and code improvement. They also tend to be cases where the execution
time is spent in inner loops that can if necessary be optimized by
hand.

Re: Of Bounties and Mercenaries

[Robert Dewar]

> Compiler speed *should not* be an issue. If gcc 3.5 is ten times
> slower compiling programs at -O2, that's a good thing: it means
> that it does more to procude better code. A program is compiled
> once but executed 100000 times. So compile-time is *not* an issue
> and it would be wrong if gcc developers dropped features because
> they slow compile speed.

This is an extreme position which is untenable for most application
developers.

> Unfortunatelly this doesn't work. A progam that works at -O0 may
> reveal bugs at -O2. If you could guarantee that ``if something is
> ok at -O0 is also ok in -O2'' that'd be a real win.

That would be a *far* more severe restraint on generated code. Indeed
if you mean by "ok" that the code happens to "work" on at least some
test cases, it is likely that the constraint you give above is total,
i.e. that no optimization can be done at all. Most likely you are
not very familiar with compiler technology or with machine code,
or you would not make such a suggestion :-)

> Apart from that, i think it's up to the application progammer
> to write code that does not require -O2, can be tested at -O0
> and use valgrind on it.

Major testing is required under the conditions of delivery, so the
idea of doing all testing at -O0 and delivery at -O2 is flawed.
Obviously it is non-decidable whether a program is correct, so the
idea that programs like valgrind can guarantee this correctness
is naive.

> gcc developers *should* completely ignore comments about the speed
> of gcc. Do you get paid to improve the speed? No. Application
> programmers should write good code.

Well if this can be achieved merely by your exortation, why not
have these application programmers write optimal assembly code
in the first place, thus removing dependencies on the compiler
entirely :-) In fact writing code that is 100% correct according
to the language standard and thus impervious to optimization
effets is very difficult, and no tool can check that you have
achieved this goal.

> If a big corp has incompetent
> people and expects from gcc to get faster at -O2, they can hire
> a filthy bounty hunter ;)

Well there is no danger of the gcc development community paying
too much attention to this extreme advice :-)

As always the balance between code quality and compile speed is
a trade off. And not an easy one, since requirements definitely
differ. But if you are working on applications with millions
of lines of code where maximum performance is required, then
you ask for best possible code and best possible compile speed.
Of course you can't have both, so what you really ask for is
a reasonable trade off.

Actually in my experience, many/most optimizations are disappointing
and do not generate the improvements that are hoped for. Even if some
benchmarks show substantial improvement, the effect on real large
application programs is more limited.

What this says is that if you install an optimization that takes
significant compile time, then you need to be sure it is really
worth while.

It is true that machines are getting faster, but this is something
that weakens the need for both high performance and high compile
speed, so it does not necessarily change the balance that much.
On the other hand, compile speed is something that is measured
by human time, there is a big difference between a system that
requires 10 minutes to recompile and one that takes 10 hours,
or even 10 days. It changes the entire approach to working on
such a system.

A factor of 10 degradation in compile time would be entirely
unacceptable to the majority of the community. That's clear
from past discussions. Some degradation is acceptable if it
really pays off.

I find the balance of gcc fairly reasonable. A diagnostic of
this is that we find a few customers who complain about
compile time, a few who complain about performance of generated
code, but for the most part the balance meets the needs of
our (our = Ada Core Technologies here) customers. On the other
hand, we have frequently heard from Apple that the balance is
not so good for their customer base, and that they really need
better compile time.

I do suspect that gcc compilation time could be speeded up without
noticeable loss of performance. Indeed we have found a few places
where relatively simple fixes to gcc speed up compilation hugely
for certain selected programs, with little or no loss in code
performance, and we should all be looking for such opportunities.

Part of the trouble is that it's more interesting to work on
new algorithms and optimizations, than to work on profiling
and tuning old ones :-)

Re: Of Bounties and Mercenaries

[Scott Robert Ladd]

Robert Dewar wrote:
> Actually in my experience, many/most optimizations are disappointing
> and do not generate the improvements that are hoped for. Even if some
> benchmarks show substantial improvement, the effect on real large
> application programs is more limited.

I couldn't have said it better myself. One of my motivations behind 
creating Acovea is to show how nonsensical it is to throw every 
optimization at code under the assumption that it will always produce 
faster code. The key to fast code is writing good algorithms, profiling, 
and using the *right* optimizations.

> Part of the trouble is that it's more interesting to work on
> new algorithms and optimizations, than to work on profiling
> and tuning old ones :-)

Ain't it the truth. ;)

-- 
Scott Robert Ladd
Coyote Gulch Productions (http://www.coyotegulch.com)
Software Invention for High-Performance Computing

Re: Of Bounties and Mercenaries

[Scott Robert Ladd]

Stelios Xanthakis wrote:
> Compiler speed *should not* be an issue. If gcc 3.5 is ten times
> slower compiling programs at -O2, that's a good thing: it means
> that it does more to procude better code. A program is compiled
> once but executed 100000 times. So compile-time is *not* an issue
> and it would be wrong if gcc developers dropped features because
> they slow compile speed.

You must be awefully good if you only compile your programs once! I'm 
just a poor sot who has to compile over and over again as I add new 
features and test and debug... ;)

Seriously, compile speed is a very serious problem for people working on 
large applications; it gets rather onerous when every cycle takes twn 
minutes to compile, ten seconds to test, and a minute to debug; I'm much 
more productive with a fast compiler.

> gcc developers *should* completely ignore comments about the speed
> of gcc. Do you get paid to improve the speed? No. Application
> programmers should write good code. If a big corp has incompetent
> people and expects from gcc to get faster at -O2, they can hire
> a filthy bounty hunter ;)

A rather arogant attitude, if I do say so myself. I guess we're not all 
as good as you are.

I might also add that I was not advocating the position of Linux 
developers, merely stating a well-known example of people who want 
something and who might be willing to pay for it.


-- 
Scott Robert Ladd
Coyote Gulch Productions (http://www.coyotegulch.com)
Software Invention for High-Performance Computing

Re: Of Bounties and Mercenaries

[Alexandre Oliva]

On Apr  3, 2004, Scott Robert Ladd <coyote@coyotegulch.com> wrote:

> Stelios Xanthakis wrote:
>> Compiler speed *should not* be an issue. If gcc 3.5 is ten times
>> slower compiling programs at -O2, that's a good thing: it means
>> that it does more to procude better code. A program is compiled
>> once but executed 100000 times. So compile-time is *not* an issue
>> and it would be wrong if gcc developers dropped features because
>> they slow compile speed.

> You must be awefully good if you only compile your programs once!

I think the fallacy in the argument is a bit different.  The idea is
that, if I were to build a program that I knew thousands of users
would run thousands of times, and having it run a bit faster would
make a significant difference in how happy my users would be such that
they'd be willing to pay more for the package, for support, whatever,
it would make sense to squeeze as much performance as possible out of
the compiler into the application.  From the POV of the users of my
application, it doesn't matter if I spend hours or days building the
application, or even tuning the compiler to find the optimal flags to
build it: they'll be happier the faster the thing runs.

It's to me, the person who gets to build the application, that the
speed of the compiler matters.  The faster it is, the more
combinations of flags I can try with a reasonable amount of CPU time
requirements.

But then, even for the application builder, compiler speed isn't as
important as for developers.  The developer is the person that gets to
build the application a nearly-infinite number of times during the
development cycle, having to rebuild it and retest it for every
change.

But then, a compiler is a tool that is used by developers and
application builders; users are second-order customers.  Sure enough,
we want to generate code that runs as fast as possible, for the
benefit of the users, as long as this doesn't make the compiler too
slow for developers and builders to want to use it, otherwise all the
effort put into generating fast code will be wasted since the compiler
won't have any direct users.

-- 
Alexandre Oliva             http://www.ic.unicamp.br/~oliva/
Red Hat Compiler Engineer   aoliva@{redhat.com, gcc.gnu.org}
Free Software Evangelist  oliva@{lsd.ic.unicamp.br, gnu.org}

Re: Of Bounties and Mercenaries

[Ben Elliston]

Alexandre Oliva <aoliva@redhat.com> writes:

> But then, a compiler is a tool that is used by developers and
> application builders; users are second-order customers.  Sure
> enough, we want to generate code that runs as fast as possible, for
> the benefit of the users, as long as this doesn't make the compiler
> too slow for developers and builders to want to use it, otherwise
> all the effort put into generating fast code will be wasted since
> the compiler won't have any direct users.

For those who are unfamiliar with it, Proebsting's Law is an
interesting idea to ponder -- "compiler advances double computing
power every 18 years":

  http://research.microsoft.com/~toddpro/papers/law.htm

Ben

Re: Of Bounties and Mercenaries

[Mike Stump]

On Saturday, April 3, 2004, at 05:53 AM, Stelios Xanthakis wrote:
> Compiler speed *should not* be an issue.

Let me guess, you represent the voice of one developer, or did you miss 
the first by two days?

Re: Of Bounties and Mercenaries

[Tom Lord]

    > From: Scott Robert Ladd <coyote@coyotegulch.com>

    > My fear (and that of others) is that GCC has become so
    > commercialized that it fails to address the needs of the
    > community at large. I'm not anti-commercial -- I'm just looking
    > for some model that meets the needs of certain forgotten and
    > ignored constituencies.

I think you are being to narrow.

Existing commercial interests are _part_ of the community at large.

One interesting question is if you can identify needs of _theirs_ that
are being forgotten and ignored.   Perhaps some of those might even be
shared by whatever "ignored constituencies" you have in mind.

Is there, for example, a tragedy of the commons phenomenon in play in
which existing commercial interests have not yet discovered how to
organize funding for basic improvements that will benefit them all,
but will not be paid for by any third-party customer, but are
desirable to protect the future value of GCC?

A team of hackers at Intel could write a whitepaper saying "Why (and
how) ICC needs to be rewritten from scratch to keep it viable for more
than 3 years".  They could make that case to Intel and, there being a
single payer for such work, they'd only need to win one customer.

A team of hackers on the net could write a whitepaper saying "10
Sweeping Overhaul's Needed to Keep GCC viable for more than 5 years"
or another saying "What Businesses Benefit From Ensuring GCC Runs Well
on Modest/Outdated HW?" but if they want to sell those plans and don't
want to find a sugar-daddy-corp to pony up for the whole deal -- they
need to sell to N customers and get multiple payers for a single
project.

The industry is _fairly_ bad at that N-payers/Single-project pattern.
The social and organizational practice of consortium-forming is pretty
regularized now --- you can even find services who will help you do it
--- yet, nevertheless, it remains a quite heavyweight process.   Good
luck networking well enough to even enter into talks with the Right
People for such a thing.

OSDL is interesting in part because it finally manages to generalize
the process a little bit.   It's a comparatively open-ended
consortium.    Consequently, all the players are already joined at a
given table.   The start-up and infrastructure costs of a consortium
process are ammortized.    They can use OSDL as a vehicle to rapidly
and efficiently do all kinds of N-payer kernel-related projects.

What's the next step in the generalization process that OSDL has
started?

-t

RE: Of Bounties and Mercenaries

[Dave Korn]

> -----Original Message-----
> From: gcc-owner@gcc.gnu.org On Behalf Of Richard Kenner
> Sent: 02 April 2004 17:25

> such payments.  But the point remains that the issues you are 
> raising are
> financial and political and this is a *technical* list.


  It's Friday.  In fact, this side of the pond it's already
last-thing-Friday-afternoon.  So cut us some off-topic slack!


    cheers,
       DaveK
-- 
Can't think of a witty .sigline today....

Re: Of Bounties and Mercenaries

[Ian Lance Taylor]

kenner@vlsi1.ultra.nyu.edu (Richard Kenner) writes:

>     I made a serious (with tongue a tad in cheek) set of suggestions 
>     specific to GCC, and in response to comments made herein by others. 
>     There is a sincere need to address these issues, which are very 
>     GCC-specific; few other packages have the breadth and depth of GCC, 
>     across platforms and users. Therefore, this seems to be an appropriate 
>     venue.
> 
> You are totally ignoring the fact that there are many people who *are* paying
> to have GCC maintained and a number of companies very happy to receive
> such payments.  But the point remains that the issues you are raising are
> financial and political and this is a *technical* list.

I can't agree.  gcc is a consortium of commercial companies working
together on a single open source code base.  But it is also a
volunteer project supported by the Free Software Foundation for the
advancement of free software.  What Scott is talking about is how to
balance the needs of the free software community and the needs of the
commercial interests.  This is an important issue for the gcc project
as a whole.  The gcc mailing list is the correct place to discuss it.

A discussion of the general issue would indeed be off-topic for the
gcc mailing list.  But a discussion of the issue in the specific
context of gcc is on-topic.

Ian

Of Bounties and Mercenaries

[Scott Robert Ladd]

What to do, what to do: On one hand, we have a user community that needs 
improvements in GCC, but who is not paying anyone to do such work. On 
the other hand, we have the GCC developers who often lament the number 
of bugs and regressions that go unpatched.

Shall the twain ever meet?

It seems to me that both sides have the same problem: They want 
something done, but no one is paying for it. So GCC continues to have 
serious bugs, and many users are unhappy, and some unfaithful have begun 
having concourse with -- dare I say it? -- commercial compilers that 
satisfy their needs.

While I would prefer an altruistic world, I have family and animals to 
feed; my teenage daughters devour whole pizzas in a single gulp, while 
my iguana alone eats an entire produce section every week. I'm certain 
we all have our little obligations for rent, power, and other such 
"necessities."

So in whose vested interest is it to make GCC even better? Who will pay 
to make it better, stronger, and faster?

Perhaps we should vote with our pocketbooks. Bugs and feature requests 
could be tracked by value, as defined by the amount of money people 
commit on behalf of such requests. Users could "weight" their vote with 
cold, hard cash to attract the proper mercenary mindset.

Of course, such a model *does* cut out those people who have great need 
but little cash or small markets. If I develop for an embedded system, 
for example, I might offer $100 to request repair of some bug that 
annoys me -- but if a thousand people pay $10 each to fix something for 
the ia32, my poor embedded system will be out-of-luck, I think.

Ah, but no one really cares about those "fringe" systems, when the 
majority of people (the only really *important* ones) use Intel and AMD 
processors running Linux. And if no one's paying (for example) to 
improve floating-point speed and accuracy, it must not be important, eh? 
Anything important *must* have a dollar value attached, according to 
some GCC developers.

Perhaps we could invoke quotas -- for every "popular" bug fixed, the GCC 
developers would have to fix some other, less popular bug. Affirmative 
action for less-popular platforms, bugs, and requests!

How to make a buck and still take care of minority needs? An age old 
question, to be certain.

-- 
Scott Robert Ladd
Coyote Gulch Productions (http://www.coyotegulch.com)
Software Invention for High-Performance Computing

Re: Of Bounties and Mercenaries

[Michael Matz]

Hi Scott,

On Fri, 2 Apr 2004, Scott Robert Ladd wrote:

> [... non-technical things having potential to provoke a nice flame 
> war...]

You seem to have many ideas.  May I suggest to use gnu.misc.discuss to 
discuss them with people who might be interested in them and then after 
reaching consensus implement them?


Ciao,
Michael.

Re: Of Bounties and Mercenaries

[Scott Robert Ladd]

Michael Matz wrote:
>>[... non-technical things having potential to provoke a nice flame 
>>war...]

> You seem to have many ideas.  May I suggest to use gnu.misc.discuss to 
> discuss them with people who might be interested in them and then after 
> reaching consensus implement them?

I made a serious (with tongue a tad in cheek) set of suggestions 
specific to GCC, and in response to comments made herein by others. 
There is a sincere need to address these issues, which are very 
GCC-specific; few other packages have the breadth and depth of GCC, 
across platforms and users. Therefore, this seems to be an appropriate 
venue.

-- 
Scott Robert Ladd
Coyote Gulch Productions (http://www.coyotegulch.com)
Software Invention for High-Performance Computing

Re: Of Bounties and Mercenaries

[Toon Moene]

Scott Robert Ladd wrote:

 > Therefore, this seems to be an appropriate venue.

Talking about venue - are you joining us in Ottawa 
(http://www.gccsummit.org) ?

Nothing works better for those incrowd conferences than outside people 
asking hard questions (hi Chris !).

Cheers,

-- 
Toon Moene - mailto:toon@moene.indiv.nluug.nl - phoneto: +31 346 214290
Saturnushof 14, 3738 XG  Maartensdijk, The Netherlands
Maintainer, GNU Fortran 77: http://gcc.gnu.org/onlinedocs/g77_news.html
GNU Fortran 95: http://gcc.gnu.org/fortran/ (under construction)

Re: Of Bounties and Mercenaries

[Scott Robert Ladd]

Toon Moene wrote:
> Talking about venue - are you joining us in Ottawa 
> (http://www.gccsummit.org) ?
> 
> Nothing works better for those incrowd conferences than outside people 
> asking hard questions (hi Chris !).

I would love to, if I can find the time and money. At the moment, I'm 
leaning toward attending...

..Scott

-- 
Scott Robert Ladd

Re: Of Bounties and Mercenaries

[Ian Lance Taylor]

Scott Robert Ladd <coyote@coyotegulch.com> writes:

> Of course, such a model *does* cut out those people who have great
> need but little cash or small markets. If I develop for an embedded
> system, for example, I might offer $100 to request repair of some bug
> that annoys me -- but if a thousand people pay $10 each to fix
> something for the ia32, my poor embedded system will be out-of-luck, I
> think.
> 
> Ah, but no one really cares about those "fringe" systems, when the
> majority of people (the only really *important* ones) use Intel and
> AMD processors running Linux. And if no one's paying (for example) to
> improve floating-point speed and accuracy, it must not be important,
> eh? Anything important *must* have a dollar value attached, according
> to some GCC developers.

I don't have any solutions.  I'm just going to comment that I often
see the flipped version of which bugs are more important.  Bugs which
affect some particular embedded system are important to the
microprocessor vendor, or the customer who is stopped by the bug, and
they know perfectly well that the bug is unlikely to ever get fixed if
they don't pay for it.  So for them it becomes a straightforward
investment decision.  For bugs on popular platforms, on the other
hand, the hardware vendors generally comfort themselves with the
belief that since so many people use the platform, somebody else will
fix the bug.

Floating point performance is an example of a an issue so diffuse that
nobody feels responsible for fixing it.  On the other hand, Red Hat
and others (or perhaps a silent backer) are investing a great deal of
money in the tree-ssa reworking of gcc infrastructure, so this sort of
general overhaul is indeed possible.

Part of the problem is that gcc has become sufficiently complex that
it is difficult for the uninitiated to fix a bug.  Thus gcc partially
loses one of the usual advantages of free software--the fact that bugs
can be fixed by people with an itch to scratch.

Ian

Re: Of Bounties and Mercenaries

[Scott Robert Ladd]

Ian Lance Taylor wrote:
> Floating point performance is an example of a an issue so diffuse that
> nobody feels responsible for fixing it.  On the other hand, Red Hat
> and others (or perhaps a silent backer) are investing a great deal of
> money in the tree-ssa reworking of gcc infrastructure, so this sort of
> general overhaul is indeed possible.

In general, numericists may not be in a good position to influence GCC 
development. Coming from academic and government sources, they can have 
trouble allocating funds for soemthing like "improving GCC," as it is 
not part of their specific mandate.

One reason I'm educating myself about tree-ssa is so I can work on 
gfortran, a project unlikely to excite much fervor inside the halls of 
Red Hat and SuSE. The numerical market is small, and Fortran 95 poorly 
understood and appreciated outside its (surprisingly large) community.

> Part of the problem is that gcc has become sufficiently complex that
> it is difficult for the uninitiated to fix a bug.  Thus gcc partially
> loses one of the usual advantages of free software--the fact that bugs
> can be fixed by people with an itch to scratch.

I completely agree; tree-ssa is, in my mind, easier to work with and 
understand. I've looked at working on GCC PRs,a nd simply haven't the 
time to invest in understanding the non-tree-ssa architecture. The 
complexity of GCC makes it very difficult for people to simply jump in 
and help out.

-- 
Scott Robert Ladd
Coyote Gulch Productions (http://www.coyotegulch.com)
Software Invention for High-Performance Computing

Re: Of Bounties and Mercenaries

[Ben Elliston]

Scott Robert Ladd <coyote@coyotegulch.com> writes:

> > Part of the problem is that gcc has become sufficiently complex that
> > it is difficult for the uninitiated to fix a bug.  Thus gcc partially
> > loses one of the usual advantages of free software--the fact that bugs
> > can be fixed by people with an itch to scratch.
> 
> I completely agree; tree-ssa is, in my mind, easier to work with and
> understand. I've looked at working on GCC PRs,a nd simply haven't
> the time to invest in understanding the non-tree-ssa
> architecture. The complexity of GCC makes it very difficult for
> people to simply jump in and help out.

This should be a side goal of future GCC work: to develop it in a way
that makes it easier for the uninitiated to hack on it.  This includes
removing complicated cruft and generally trying to keep the code base
small and as simple as possible.

tree-ssa is a reasonably good example of this: with little prior
compiler experience, someone can pick up a book like Morgan and start
to make sense of the tree-ssa-* files.

Ben