GCC

GCC

[Steven Bosscher]

Hi,

In the "Processor Architect's Panel" at the kernel summit, GCC was
apparently discussed shortly:

"Jon 'maddog' Hall said that the various processor architectures wild
also benefit from paying more attention to gcc. The architects responded
uniformly with complaints about how difficult it is to work with the gcc
team. They all understand their interest in having gcc work will with
their processors, but actually getting patches into the gcc code base is
difficult."  (http://lwn.net/Articles/40831/)

I'm not sure why they think it is so difficult.  It would seem that if
the patch is architecture-specific and well-formed (ie. conforming to
the coding style, etc), it typically just goes in, period.  And patches
to target-independent code may go through one or two review cycles, but
again, if the patch looks good, it goes in.  At least, I got the
impression that patches are seldomly rejected.

So why would these people think it's difficult to work with the people
on this list, and to contribute code (and what can be done about it)?

Gr.
Steven

Re: GCC

[Aaron Lehmann]

On Thu, Jul 31, 2003 at 08:44:19AM +0200, Steven Bosscher wrote:
> I'm not sure why they think it is so difficult.  It would seem that if
> the patch is architecture-specific and well-formed (ie. conforming to
> the coding style, etc), it typically just goes in, period.  And patches
> to target-independent code may go through one or two review cycles, but
> again, if the patch looks good, it goes in.  At least, I got the
> impression that patches are seldomly rejected.

Copyright assignments.

Re: GCC

[Randy.Dunlap]

> On Thu, Jul 31, 2003 at 08:44:19AM +0200, Steven Bosscher wrote:
>> I'm not sure why they think it is so difficult.  It would seem that if the
>> patch is architecture-specific and well-formed (ie. conforming to the
>> coding style, etc), it typically just goes in, period.  And patches to
>> target-independent code may go through one or two review cycles, but
>> again, if the patch looks good, it goes in.  At least, I got the
>> impression that patches are seldomly rejected.
>
> Copyright assignments.

Yes, it seemed to be more about legal issues
than working with the gcc developers.

~Randy

Re: GCC

[Lars Segerlund]

  There is poor documentation, ( in the sence that it is not easily 
digested if you haven't done a couple of runs of the code ), and also 
not much in the howto getting started.

  Also the overall workings of the frontends/gcc and some major parts of 
it is not that well documented.

  Take the tree's, GENERIC and GIMPLE have just started to have decent 
documentation, which have probably excluded a lot of people from doing 
work related to that.

  It's quite a bit of an undertaking getting started in writing code for 
gcc before you get going.

  I think it is this kind of problems that people refer to as hard to 
work with, also breakage in other ports by their patches.

  / regards, Lars Segerlund.

Steven Bosscher wrote:
> Hi,
> 
> In the "Processor Architect's Panel" at the kernel summit, GCC was
> apparently discussed shortly:
> 
> "Jon 'maddog' Hall said that the various processor architectures wild
> also benefit from paying more attention to gcc. The architects responded
> uniformly with complaints about how difficult it is to work with the gcc
> team. They all understand their interest in having gcc work will with
> their processors, but actually getting patches into the gcc code base is
> difficult."  (http://lwn.net/Articles/40831/)
> 
> I'm not sure why they think it is so difficult.  It would seem that if
> the patch is architecture-specific and well-formed (ie. conforming to
> the coding style, etc), it typically just goes in, period.  And patches
> to target-independent code may go through one or two review cycles, but
> again, if the patch looks good, it goes in.  At least, I got the
> impression that patches are seldomly rejected.
> 
> So why would these people think it's difficult to work with the people
> on this list, and to contribute code (and what can be done about it)?
> 
> Gr.
> Steven
> 
> 

Re: GCC

[Gerald Pfeifer]

On Wed, 30 Jul 2003, Aaron Lehmann wrote:
> Copyright assignments.

Please note that (especially) for changes like those you mentioned also
copyright disclaimers are sufficient.

Gerald
-- 
Gerald Pfeifer (Jerry)   gerald@pfeifer.com   http://www.pfeifer.com/gerald/

Re: GCC

[Steven Bosscher]

Op do 31-07-2003, om 14:41 schreef Gerald Pfeifer:
> On Wed, 30 Jul 2003, Aaron Lehmann wrote:
> > Copyright assignments.
> 
> Please note that (especially) for changes like those you mentioned also
> copyright disclaimers are sufficient.

Where and how is this documented?  Maybe it's not clear to some people.

Gr.
Steven

Re: GCC

[cgd]

this discussion isn't really specific to gcc, it applies to, say,
binutils too.  but since gcc is where it started that's where i'm
going to leave it.  8-)


At Thu, 31 Jul 2003 06:42:29 +0000 (UTC), "Steven Bosscher" wrote:
> I'm not sure why they think it is so difficult.

I'm not a "processor architect," but I interact with some of them, and
i've tried to get some of them to be more GCC cognizant.  I am,
however, somebody who works on low-level "processor stuff," and these
are factors in my experience:

(1) the assignments *are* a pain in the butt, but the amount of pain
    can be highly variable.  Originallly, no problem to get them
    signed (though the processing lag was annoying).  Later, new
    lawyers, much wrangling to get something OK to all sides,
    ultimately consumed about 5 months...  "ugh."

(2) slow patch approval times can be a problem, especially for people
    who are new.

    Looking back at my old patch list back to when I started
    submitting stuff (mid-2000), wait-for-approval time of > a month
    wasn't uncommon.

    These days, for me at least, the time is shorter.  I don't know if
    that's the experience of new developers, though.

(3) at least when i started doing this, there wasn't that much working
    documentation about how stuff could/should be tested (i.e., run
    with this sim, which is actually likely to work, by doing these
    steps...)  That made me uncertain of what I was doing, at least a
    bit.  I think this is getting better.

So, that's for me, a low-level OS hacker who thinks himself fairly
adaptable.  8-)

Note that these are all things that people care about only if doing
the GCC work themselves.  They can be addressed easily by hiring
somebody to do a port and submit it back.  Costs some money, but then,
so does processor design.



Sounds like the panel included processor architects who had been
"reached."  I think if you actually want to reach "processor
architects" generally, there are more problems IMO:

Processor architects do processor architecture.  Typically, they don't
know GCC internals.  (Often some will have *some* compiler exposure,
but the amount varies.)  If they're making a new processor, they
probably don't have time to learn GCC internals, or even to become
involved in the GCC community.

The result of that is that they might not be cognizant of things that
they should try to avoid if they want a good GCC back-end (if they're
designing with a new architecture) or a good scheduler description (if
they're doing a new implementation of an existing architecture).

Some seem obvious, e.g., if you make your scheduling rules bizarre and
with many exceptions, it'll be difficult to create an "optimal" (or
close to) code schduler for them.  Seems obvious enough to someone who
touches GCC a lot, but

        (1) it might not be to someone who doesn't, and

        (2) at some point, you hit a wall where "difficult" is "really
            really difficult," i.e., 2 pages of instruction issue rule
            special cases can translate into much more if you want to
            model them well.  8-)

Other stuff pops into mind: e.g. if you're designing a new
architecture, and want a good GCC port, and are on the fence about
branch delay slots (or whatever): obviously GCC supports them on
multiple architectures...  Does that mean they're *easy* to cope with
in GCC (or the other tools)?  etc.  

I think these are questions that you don't know about GCC unless
you've been in it for some amount of time -- which architects aren't
likely to have put in.

A collection of wisdom related to "designing architecture to make GCC
happy" would probably help.

These types of issues are ones where, unless you get it right early on
during the architecture of your part, it may be very very difficult to
get a good GCC port years later.  I don't know that they can be
addressed well by throwing money at some GCC developers, either.  (I
mean, they probably can be addressed, but it IMO it doesn't appear to
be a good way to spend money, up front.)



cgd

Re: GCC

[cgd]

this discussion isn't really specific to gcc, it applies to, say,
binutils too.  but since gcc is where it started that's where i'm
going to leave it.  8-)


At Thu, 31 Jul 2003 06:42:29 +0000 (UTC), "Steven Bosscher" wrote:
> I'm not sure why they think it is so difficult.

I'm not a "processor architect," but I interact with some of them, and
i've tried to get some of them to be more GCC cognizant.  I am,
however, somebody who works on low-level "processor stuff," and these
are factors in my experience:

(1) the assignments *are* a pain in the butt, but the amount of pain
    can be highly variable.  Originallly, no problem to get them
    signed (though the processing lag was annoying).  Later, new
    lawyers, much wrangling to get something OK to all sides,
    ultimately consumed about 5 months...  "ugh."

(2) slow patch approval times can be a problem, especially for people
    who are new.

    Looking back at my old patch list back to when I started
    submitting stuff (mid-2000), wait-for-approval time of > a month
    wasn't uncommon.

    These days, for me at least, the time is shorter.  I don't know if
    that's the experience of new developers, though.

(3) at least when i started doing this, there wasn't that much working
    documentation about how stuff could/should be tested (i.e., run
    with this sim, which is actually likely to work, by doing these
    steps...)  That made me uncertain of what I was doing, at least a
    bit.  I think this is getting better.

So, that's for me, a low-level OS hacker who thinks himself fairly
adaptable.  8-)

Note that these are all things that people care about only if doing
the GCC work themselves.  They can be addressed easily by hiring
somebody to do a port and submit it back.  Costs some money, but then,
so does processor design.



Sounds like the panel included processor architects who had been
"reached."  I think if you actually want to reach "processor
architects" generally, there are more problems IMO:

Processor architects do processor architecture.  Typically, they don't
know GCC internals.  (Often some will have *some* compiler exposure,
but the amount varies.)  If they're making a new processor, they
probably don't have time to learn GCC internals, or even to become
involved in the GCC community.

The result of that is that they might not be cognizant of things that
they should try to avoid if they want a good GCC back-end (if they're
designing with a new architecture) or a good scheduler description (if
they're doing a new implementation of an existing architecture).

Some seem obvious, e.g., if you make your scheduling rules bizarre and
with many exceptions, it'll be difficult to create an "optimal" (or
close to) code schduler for them.  Seems obvious enough to someone who
touches GCC a lot, but

        (1) it might not be to someone who doesn't, and

        (2) at some point, you hit a wall where "difficult" is "really
            really difficult," i.e., 2 pages of instruction issue rule
            special cases can translate into much more if you want to
            model them well.  8-)

Other stuff pops into mind: e.g. if you're designing a new
architecture, and want a good GCC port, and are on the fence about
branch delay slots (or whatever): obviously GCC supports them on
multiple architectures...  Does that mean they're *easy* to cope with
in GCC (or the other tools)?  etc.  

I think these are questions that you don't know about GCC unless
you've been in it for some amount of time -- which architects aren't
likely to have put in.

A collection of wisdom related to "designing architecture to make GCC
happy" would probably help.

These types of issues are ones where, unless you get it right early on
during the architecture of your part, it may be very very difficult to
get a good GCC port years later.  I don't know that they can be
addressed well by throwing money at some GCC developers, either.  (I
mean, they probably can be addressed, but it IMO it doesn't appear to
be a good way to spend money, up front.)



cgd

Re: GCC

[David O'Brien]

On Wed, Jul 30, 2003 at 11:47:07PM -0700, Aaron Lehmann wrote:
> On Thu, Jul 31, 2003 at 08:44:19AM +0200, Steven Bosscher wrote:
> > I'm not sure why they think it is so difficult.  It would seem that if
> > the patch is architecture-specific and well-formed (ie. conforming to
> > the coding style, etc), it typically just goes in, period.  And patches
> > to target-independent code may go through one or two review cycles, but
> > again, if the patch looks good, it goes in.  At least, I got the
> > impression that patches are seldomly rejected.
> 
> Copyright assignments.

I agree with Robert Dewar about showing evidence that this is the main
problem.  AMD hired SuSE to do the GCC work for Opteron, so copyright
assignments certainly weren't a problem for AMD.  I know there are some
SuSE amd64(x86-64) patches that never got accepted -- FreeBSD hit the
same bugs which the patches would have fixed.

I think a much more accurate description would be Zack's "A Maintenance
Programmer's View of GCC" from the Ottawa GCC Summit.  My last patch
trying to add a "GCC_OPTIONS" environmental variable was for AMD and some
very large ISV's benefit.  Didn't go in, and not for copyright assignment
issues.

Re: GCC

[Zack Weinberg]

"David O'Brien" <obrien@FreeBSD.org> writes:

> I think a much more accurate description would be Zack's "A Maintenance
> Programmer's View of GCC" from the Ottawa GCC Summit.  My last patch
> trying to add a "GCC_OPTIONS" environmental variable was for AMD and some
> very large ISV's benefit.  Didn't go in, and not for copyright assignment
> issues.

That's not a great example.  There was nothing technically wrong with
your patch; rather, people have convincing arguments that we don't
want that feature.  This is not normally going to happen to
target-specific optimization patches.

I would suggest the SPE and Altivec patchsets as better examples of
the problems in this area.  Those are features that no one objects to
in the abstract -- support for two different sets of vector
instructions, and the related intrinsics -- but Aldy had to do
tremendous amounts of work to get them to the point where they were
acceptable.  Part of the problem is that the backend API is a mess;
that's the stuff I talked about in my paper.  Work is ongoing to clean
that up.  I think we'll be in much better shape in a few years.

But the other part of the problem is that the features were designed
without consulting with people who knew what the C and C++ standards
have to say in this area, and so (for example) the original Altivec
patches had vector-initializer syntax that was considered
unacceptable, which makes GCC 3.x's Altivec incompatible with the
original Motorola compilers.  The only way that situation can improve
is if GCC maintainers are brought on board in the design stages of
such projects, and can point out the problems before they become
entrenched.

zw

p.s. I wonder if the GCC_OPTIONS patch would be more acceptable if it
were processed by the driver, so that the additional options showed up
in -v output.

processor architects say it's hard to work with us?

[Joe Buck]

Steven, if you use a subject line like "GCC" on the gcc list, people
might not bother to read your message.  So here it is again.

I suspect that one issue is that the restriction that a patch be
architecture-specific might be seen as too severe, if the existing GCC
infrastructure lacks capabilities that are needed to get good results
on novel architectures.  I know that this was part of the reason for the
pgcc fork: Intel engineers got better code for the Pentium by slashing
away at the frontend/backend distinction than they could have achieved
by doing things "right".

On Thu, Jul 31, 2003 at 08:44:19AM +0200, Steven Bosscher wrote:
> In the "Processor Architect's Panel" at the kernel summit, GCC was
> apparently discussed shortly:
> 
> "Jon 'maddog' Hall said that the various processor architectures wild
> also benefit from paying more attention to gcc. The architects responded
> uniformly with complaints about how difficult it is to work with the gcc
> team. They all understand their interest in having gcc work will with
> their processors, but actually getting patches into the gcc code base is
> difficult."  (http://lwn.net/Articles/40831/)
> 
> I'm not sure why they think it is so difficult.  It would seem that if
> the patch is architecture-specific and well-formed (ie. conforming to
> the coding style, etc), it typically just goes in, period.  And patches
> to target-independent code may go through one or two review cycles, but
> again, if the patch looks good, it goes in.  At least, I got the
> impression that patches are seldomly rejected.
> 
> So why would these people think it's difficult to work with the people
> on this list, and to contribute code (and what can be done about it)?

Re: processor architects say it's hard to work with us?

[Lars Segerlund]

  Look at Cray for example, he minimized his designs, even introducing 
the notorious +0 -0 , and no division operation only 1/x in order to get 
all the speed he could get from minimal hardware, the point I am trying 
to make is that if you got simple hardware and no stalls !!! whatsoever 
!!! It's likely to move even if it's not that fast, ( look at cray again 
). ( I choose cray just to point out these two features, and because 
theyre well known, no religion).

  Today I believe that a lot of the advanced features like vector 
processing, different forms of SIMD and such are just making it's way 
into 'generic' infrastructure for compilers and other tools.

  A lot of grief seem's to have gone into preventing stalls, prefetching 
and different forms of register allocation strategies, but now there 
seem to be a bit of general support for what's needed to make use of 
these advanced features.

  So IMHO, compilers are catching up to the best of breed 'ad hoc' 
target specific compilers of the 60's and 70's, which is no mean feat 
cosidering that they are using a generic infrastructure and a lot safer 
'theoretical' ground to stand on.

  So as stated below, I believe that if anything the infrastructure to 
support advanced processor specific features must 'be there' since that 
is likely to be a large part of the problem.

  / Regards, Lars Segerlund.

Joe Buck wrote:
> Steven, if you use a subject line like "GCC" on the gcc list, people
> might not bother to read your message.  So here it is again.
> 
> I suspect that one issue is that the restriction that a patch be
> architecture-specific might be seen as too severe, if the existing GCC
> infrastructure lacks capabilities that are needed to get good results
> on novel architectures.  I know that this was part of the reason for the
> pgcc fork: Intel engineers got better code for the Pentium by slashing
> away at the frontend/backend distinction than they could have achieved
> by doing things "right".
> 
> On Thu, Jul 31, 2003 at 08:44:19AM +0200, Steven Bosscher wrote:
> 
>>In the "Processor Architect's Panel" at the kernel summit, GCC was
>>apparently discussed shortly:
>>
>>"Jon 'maddog' Hall said that the various processor architectures wild
>>also benefit from paying more attention to gcc. The architects responded
>>uniformly with complaints about how difficult it is to work with the gcc
>>team. They all understand their interest in having gcc work will with
>>their processors, but actually getting patches into the gcc code base is
>>difficult."  (http://lwn.net/Articles/40831/)
>>
>>I'm not sure why they think it is so difficult.  It would seem that if
>>the patch is architecture-specific and well-formed (ie. conforming to
>>the coding style, etc), it typically just goes in, period.  And patches
>>to target-independent code may go through one or two review cycles, but
>>again, if the patch looks good, it goes in.  At least, I got the
>>impression that patches are seldomly rejected.
>>
>>So why would these people think it's difficult to work with the people
>>on this list, and to contribute code (and what can be done about it)?
> 
> 
> 

Re: GCC

[Bernardo Innocenti]

David O'Brien wrote:

> On Wed, Jul 30, 2003 at 11:47:07PM -0700, Aaron Lehmann wrote:
>>Copyright assignments.
> 
> I agree with Robert Dewar about showing evidence that this is the main
> problem. [...]

I hereby offer myself as a living evidence against those
copyright assignments <grin>.

Being a fresh new GCC contributor, I can tell you that I have
barely survived the burden of interacting with the FSF to get
the paperwork done.

The original topic was about getting big companies to
contribute, but I'd like to stress that, in many
community-driven projects, most of the development work
comes from thousands occasional contributors.

Most programmers just want to contribute their changes
and don't care anything about legal stuff.
It's very easy to prove: just ask your co-workers and
friends whether they would be willing to go through the
copyright assignment procedure just to see their patch
applied.

I have have finally been able to get the first small
patch in after waiting for over one month. This could
have been barely acceptable some years ago. Today
there are too many projects that make it much easier
to get involved.

In my experience, contributing to the Linux kernel
has been immediately rewarded with the satisfaction of
seeing my patches applied in a matter of hours or, in
the worst case, in a few days.

I guess the problem is just to get started. When you've
finally got the assignment on file, working on GCC and
the kernel can be equally interesting. People are
generally helpful on both mailing lists and the project
infrastructure is easy to work with.

-- 
  // Bernardo Innocenti - Develer S.r.l., R&D dept.
\X/  http://www.develer.com/

Please don't send Word attachments - http://www.gnu.org/philosophy/no-word-attachments.html