A Suggestion for Release Testing

A Suggestion for Release Testing

[Scott Robert Ladd]

Given the recent problems with the 4.0.0 release and major packages like
KDE and the kernel, has anyone considered testing releases by completely
compiling a Linux system?

As one of those infamous "Gentoo users", I don't think it would be at
all difficult to build an automated test harness, running in a chroot,
that would rebuild a "standard" Linux install with a pending release of
GCC. The suite of applications would need to be relatively tight; I'd
suggest Linux, Gnome, KDE, their major appliations, and perhaps OpenOffice.

The downside is that the compile would take quite a while; even for my
dual Opteron, full builds take more than a day. On the other hand, this
might catch some problems before release.

I'm willing to implement this, if it's deemed useful.

..Scott

Re: A Suggestion for Release Testing

[Joe Buck]

On Thu, Jun 09, 2005 at 04:42:33PM -0400, Scott Robert Ladd wrote:
> Given the recent problems with the 4.0.0 release and major packages like
> KDE and the kernel, has anyone considered testing releases by completely
> compiling a Linux system?

With 4.0.0, compiling a complete GNU/Linux distribution reveals bugs in
GCC, but even more bugs in C++ software that is not valid C++.  Assuming
we can get the distros to fix the latter set of problems, it would
certainly be good for release testing if we built complete distros, or
significant chunks thereof.  It would be even better if this could be
coordinated with the people producing all that software; I think that (for
example) GCC and KDE could both do better about trying each others' early
code sooner than we do, and communicate any problems found.

Re: A Suggestion for Release Testing

[Scott Robert Ladd]

Joe Buck wrote:
> With 4.0.0, compiling a complete GNU/Linux distribution reveals bugs
> in GCC, but even more bugs in C++ software that is not valid C++.
> Assuming we can get the distros to fix the latter set of problems...

I don't have a good solution for this problem, other than education.

> ...it would certainly be good for release testing if we built
> complete distros, or significant chunks thereof.  It would be even
> better if this could be coordinated with the people producing all
> that software; I think that (for example) GCC and KDE could both do
> better about trying each others' early code sooner than we do, and
> communicate any problems found.

I'm hoping that creating a formal "core software" test might foster some
of that communication. We have to start somewhere.

..Scott

Re: A Suggestion for Release Testing

[R Hill]

Scott Robert Ladd wrote:
> Joe Buck wrote:
>> With 4.0.0, compiling a complete GNU/Linux distribution reveals bugs
>> in GCC, but even more bugs in C++ software that is not valid C++.
>> Assuming we can get the distros to fix the latter set of problems...
> 
> I don't have a good solution for this problem, other than education.

The more the better. ;)  Actually, by the time 4.0.0 was released most 
of the packages in Gentoo that had issues with the new compiler had 
already been patched and tested.  There are some great gcc porting devs 
on board and a small independent group of users that use the weekly 
snapshots attempting to build a clean system in a chroot and patching up 
what we run into.

>> ...it would certainly be good for release testing if we built
>> complete distros, or significant chunks thereof.  It would be even
>> better if this could be coordinated with the people producing all
>> that software; I think that (for example) GCC and KDE could both do
>> better about trying each others' early code sooner than we do, and
>> communicate any problems found.
> 
> I'm hoping that creating a formal "core software" test might foster some
> of that communication. We have to start somewhere.

I use Gentoo's "system" target plus a few of the essentials like the 
kernel, xorg, Firefox ;), etc. as my smoketest.  Others use whatever 
they want or need.  Between all of us we end up covering a lot of the 
distro.  LFS and DIYL could be good models to look at as well.

While some of the Gentoo userbase isn't exactly helpful as far as 
development goes ;) there's still a lot of good work being done that's 
really not being used by anyone but us.  Patches go upstream of course, 
but we're in a good position to give an overall view of just how ready a 
particular release is to be let loose in public. Opening some lines of 
communication between GCC, the producers, and the users might be an 
asset to everybody.

--de. (currently working the hell out of RC1)

Re: A Suggestion for Release Testing

[Arthur Nascimento]

2005/6/9, Scott Robert Ladd <scott.ladd@coyotegulch.com>:
> Given the recent problems with the 4.0.0 release and major packages like
> KDE and the kernel, has anyone considered testing releases by completely
> compiling a Linux system?

Yes, people do it all the time. Check Sourcemage, LFS and DIY:

http://www.sourcemage.org/
http://www.linuxfromscratch.org/
http://www.diy-linux.org/

Re: A Suggestion for Release Testing

[Scott Robert Ladd]

Arthur Nascimento wrote:
>>Given the recent problems with the 4.0.0 release and major packages like
>>KDE and the kernel, has anyone considered testing releases by completely
>>compiling a Linux system?

> Yes, people do it all the time. Check Sourcemage, LFS and DIY:
> 
> http://www.sourcemage.org/
> http://www.linuxfromscratch.org/
> http://www.diy-linux.org/

Of course there are several source-based distributions, but that's not
what I'm talking about. While they may do individual testing when a new
GCC release appears, I'm talking about a formal testing process, where
GCC would be regularly tested during development to ensure that nothing
unexpected is broken. Breaking KDE or the kernel demonstrates a serious
problem in a release, for example.

The test suite can check individual tests, but what may be needed is a
comprehensive test across several applications that comprise a whole.
Certainly GCC would be the better for being able to effectively compile
Linux, KDE, Gnome, Apache, and other "core" programs.

..Scott

Re: A Suggestion for Release Testing

[Gerald Pfeifer]

On Thu, 9 Jun 2005, Scott Robert Ladd wrote:
> Given the recent problems with the 4.0.0 release and major packages like
> KDE and the kernel, has anyone considered testing releases by completely
> compiling a Linux system?

Are you sure nobody is doing this?  Or to phrase it differently: have
you checked Bugzilla and the ChangeLogs as to what kind of reports and
patches SUSE and Red Hat have contributed to GCC 4.0 in recent months
and weeks? :-)

Note, though, that this is only one part of the equation.  A most 
significant amount of work also goes into analysing and potentially
fixing packages which do not compile any longer and submit fixes
upstream.  And once that has has been done, one has to address the
usual set of miscompilations, which, if they appear somewhere deep
in KDE or lilo, are a lot of fun to track down, so you'll also want
to perform full functional and stress testing of the resulting distro.

And all that for at least x86, x86-64, ppc, s390/s390x and ia64. ;-)

> The downside is that the compile would take quite a while; even for my
> dual Opteron, full builds take more than a day. On the other hand, this
> might catch some problems before release.
> 
> I'm willing to implement this, if it's deemed useful.

This is certainly an excellent idea.  The more testing we can get,
the better.

Gerald

Re: A Suggestion for Release Testing

[René Rebe]

[-- Attachment #1: Type: text/plain, Size: 1877 bytes --]

Hi,

On Sunday 12 June 2005 19:51, Gerald Pfeifer wrote:

> > Given the recent problems with the 4.0.0 release and major packages like
> > KDE and the kernel, has anyone considered testing releases by completely
> > compiling a Linux system?

> > I'm willing to implement this, if it's deemed useful.

Note that the T2 System Development Environmetn (http://www.t2-project.org) 
does already implement a build system with about 1600 package descriptions to 
build and bootrap a system including cross compilation and integrated distcc 
(and ccache - though this is less interesting in this context) support.

> > The downside is that the compile would take quite a while; even for my
> > dual Opteron, full builds take more than a day. On the other hand, this
> > might catch some problems before release.
> >
> > I'm willing to implement this, if it's deemed useful.
>
> This is certainly an excellent idea.  The more testing we can get,
> the better.

Bootstrapping a, reduced but including KDE, Linux desktop takes slightly more 
than a day on a, not that recent, Athlon - with OpenOffice about 7 hours 
longer. Bootrsapping all the 1600++ packages of T2 would take something in 
the range of over 4 days. Of course less on a dual opteron.

We already have contributor hosted regression testers running mostly 
permanently:

  http://t2.geeks.cl/regressions/stable/regressions.html

And the official T2 site will get a regression tester with permanent historic 
build erros and nice graph plots again soon, too.

Needless to say we support most of the Linux supported architectures to 
possibly test build on.

PS: Hardware sponsors welcome ,-)

Sincerely yours,

-- 
René Rebe - Rubensstr. 64 - 12157 Berlin (Europe / Germany)
            http://www.exactcode.de | http://www.t2-project.org
            +49 (0)30  255 897 45

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: A Suggestion for Release Testing

[Scott Robert Ladd]

Gerald Pfeifer wrote:
> Are you sure nobody is doing this?  Or to phrase it differently: have
> you checked Bugzilla and the ChangeLogs as to what kind of reports and
> patches SUSE and Red Hat have contributed to GCC 4.0 in recent months
> and weeks? :-)

My suggestion stems from the code-generation problem with KDE, and
questions I received from a couple of kernel developers who experienced
problems.

I'm not certain my suggestion is practical; it was just an idea.

..Scott

Re: A Suggestion for Release Testing

[Mike Stump]

On Jun 12, 2005, at 10:51 AM, Gerald Pfeifer wrote:
> Note, though, that this is only one part of the equation.  A most
> significant amount of work also goes into analysing and potentially
> fixing packages which do not compile any longer and submit fixes
> upstream.

:-(  Sometimes we wish that gcc had a 5 year warn cycle, followed by  
a 5 year, -fpretty-please cycle, just so that we don't loose the  
testing that building tons of software brings to the table.

Re: A Suggestion for Release Testing

[Mark Mitchell]

Scott Robert Ladd wrote:
> Given the recent problems with the 4.0.0 release and major packages like
> KDE and the kernel, has anyone considered testing releases by completely
> compiling a Linux system?

I'm all for more testing -- but I have a standard rant about it being 
easier to run tests than to fix problems.  We actually have a wealth of 
known regressions -- some pretty serious -- in Bugzilla, and plenty more 
known bugs.  Most come from real problems reported by real users on real 
code.  So, it's not like we're running out of bugs to fix.

Setting up automated regression testing is good, and hugely useful -- 
but what makes it *really* valuable is having someone who comes in every 
morning, looks at the output, and figures out who to blame, and, if 
necessary, fixes the problem herself.

-- 
Mark Mitchell
CodeSourcery, LLC
mark@codesourcery.com
(916) 791-8304

Fixing Bugs (Was: A Suggestion for Release Testing)

[Scott Robert Ladd]

Mark Mitchell wrote:
> I'm all for more testing -- but I have a standard rant about it being
>  easier to run tests than to fix problems.  We actually have a wealth
> of known regressions -- some pretty serious -- in Bugzilla, and
> plenty more known bugs.  Most come from real problems reported by
> real users on real code.  So, it's not like we're running out of bugs
> to fix.

That's true, but several factors frustrate people who want to fix bugs.

Consider, as an example, the bug/non-bug at http://gcc.gnu.org/PR323,
which was a matter of recent discussion on this list. Some rather
respectable people (e.g., Vincent Lefèvre) consider this a bug, and have
proposed solutions. Yet here's a quote from an earlier message by
Giovanni Bajo.

GB> You are mistaken, we think GCC isn't buggy about 323 because
GB> the C/C++ standards do not tell us to do better than this. If
GB> you have higher expectations about floating point and C/C++,
GB> you should file a bugreport against the C/C++ standards.

GB> Really, we can't make everybody happy. The best we can do is
GB> to adhere the international well-known ISO/ANSI standards.

The ISO Standard doesn't prevent GCC from being *better* than specified,
does it? Are we somehow breaking ISO compliance by doing math right? Is
it so wrong to try and fix a problem that frustrates many people and
makes GCC look bad?

With the attitude shown by Giovanni, there's really no point in
submitting a patch, is there? Dozens of people have reported this
problem, potential solutions exist, but any patch is going to be ignored
because the bug isn't considered a bug by the Powers That Be.

If I were to present a patch that implements the recomendations of the
Numerical C Extensions Group, would it be accepted or rejected (on the
subject alone; ignore for the moment potential technical bugs in the
submitted code)?

Documentation for the compiler itself is woefully inadequate for someone
"new". GCC may be old-hat to folks who've worked on it for years, but it
is very complex and foreign territory for most non-compiler experts.
Working on GCC requires a much broader knowledge than working on other
projects, and without some sort of tutorial or guidance, working on it
quickly becomes frustrating.

Here's an example: Building new targets and fixing some code generation
bugs involve changing the machine definitions, which are written in a
rather uncommon language. Frankly, I haven't figured out all the nuances
yet, mostly because I don't have the luxury of studying it, and I can't
find any clear and comprehensive documentation.

Gentoo provides a mentoring system for developers. This is far better
than GCC's "fix it yourself and send us the code" policy. There is no
gentle way to become involved in GCC; it's a sink-or-swim, trial by fire
environment. If I could get a half-dozen quick questions answered, and I
could submit a couple of patches that are lying about on my hard drive.

For that matter, Gentoo also has some very excellent IRC channels that
provide a lot of help, and usually *friendly* help at that. The GCC
mailing lists are useful, but there's nowhere to go and have a quick
chat about "this is confusing the heck out of me" or "how do I approach
this problem."

I have submitted patches; I have tried to help with various aspects of
GCC. I make no claim to having accomplished anything great, but I have
tried, and watched patches die of bit rot while being told that certain
bugs shouldn't be fixed.

Everything in life is a matter of give and take. People report bugs; you
want someone to fix the bugs -- perhaps GCC should be more welcoming and
helpful.

..Scott

Re: Fixing Bugs (Was: A Suggestion for Release Testing)

[Andrew Pinski]

On Jun 14, 2005, at 10:14 AM, Scott Robert Ladd wrote:

> For that matter, Gentoo also has some very excellent IRC channels that
> provide a lot of help, and usually *friendly* help at that. The GCC
> mailing lists are useful, but there's nowhere to go and have a quick
> chat about "this is confusing the heck out of me" or "how do I approach
> this problem."

This is wrong, there is an IRC channel which talks about technical 
issues
deal with developing GCC (not with though but that should be in an 
UNIX/C/C++
IRC channel instead).

Thanks,
Andrew Pinski

Re: Fixing Bugs (Was: A Suggestion for Release Testing)

[Scott Robert Ladd]

Andrew Pinski wrote:
> This is wrong, there is an IRC channel which talks about technical issues
> deal with developing GCC (not with though but that should be in an
> UNIX/C/C++ IRC channel instead).

Where?

A Google search on "GCC IRC" doesn't find much, and the few times I've
visited #gcc on FreeNode, it's been silent.

..Scott

Re: Fixing Bugs (Was: A Suggestion for Release Testing)

[Andrew Pinski]

On Jun 14, 2005, at 10:34 AM, Scott Robert Ladd wrote:

> Andrew Pinski wrote:
>> This is wrong, there is an IRC channel which talks about technical 
>> issues
>> deal with developing GCC (not with though but that should be in an
>> UNIX/C/C++ IRC channel instead).
>
> Where?
>
> A Google search on "GCC IRC" doesn't find much, and the few times I've
> visited #gcc on FreeNode, it's been silent.

FreeNode is not the only IRC server.  irc.oftc.net is where #gcc is 
hosted.

-- Pinski

Re: Fixing Bugs (Was: A Suggestion for Release Testing)

[Scott Robert Ladd]

Andrew Pinski wrote:
> FreeNode is not the only IRC server.  irc.oftc.net is where #gcc is hosted.

You are ebing positively rude. Just because I mention FreeNode doesn't
mean I'm so ignorant as to be unaware of other servers.

Where is irc.oftc.net #gcc above documented?

..Scott

Re: Fixing Bugs (Was: A Suggestion for Release Testing)

[Diego Novillo]

On Tue, Jun 14, 2005 at 10:38:47AM -0400, Andrew Pinski wrote:
> 
> On Jun 14, 2005, at 10:34 AM, Scott Robert Ladd wrote:
> 
> >Andrew Pinski wrote:
> >>This is wrong, there is an IRC channel which talks about technical 
> >>issues
> >>deal with developing GCC (not with though but that should be in an
> >>UNIX/C/C++ IRC channel instead).
> >
> >Where?
> >
> >A Google search on "GCC IRC" doesn't find much, and the few times I've
> >visited #gcc on FreeNode, it's been silent.
> 
> FreeNode is not the only IRC server.  irc.oftc.net is where #gcc is 
> hosted.
> 
Now added to the wiki.


Diego.

Re: Fixing Bugs (Was: A Suggestion for Release Testing)

[Andrew Pinski]

On Jun 14, 2005, at 10:14 AM, Scott Robert Ladd wrote:

> Here's an example: Building new targets and fixing some code generation
> bugs involve changing the machine definitions, which are written in a
> rather uncommon language. Frankly, I haven't figured out all the 
> nuances
> yet, mostly because I don't have the luxury of studying it, and I can't
> find any clear and comprehensive documentation.

No you don't need to learn a new language. You need to learn RTL which 
is a
language yes but it is based on LISP.  Also some of the recent bugs are 
on
the tree level so you just need to know GIMPLE which is really just 
simple
expressions.  So I don't see why you are complaining because code 
generation
bugs should be just reported and let the powers at be fix them.  If you 
show
that it is a regression, then it will get a higher priority, than other 
bugs.

Again I have not seen a simple bug report from you recently; just a 
simple
this goes bonkers, this is what I want GCC to output will do.

Thanks,
Andrew Pinski

Re: Fixing Bugs (Was: A Suggestion for Release Testing)

[Scott Robert Ladd]

Andrew Pinski wrote:
>
> No you don't need to learn a new language. You need to learn RTL which is a
> language yes but it is based on LISP.  Also some of the recent bugs are on
> the tree level so you just need to know GIMPLE which is really just simple
> expressions.  So I don't see why you are complaining because code
> generation bugs should be just reported and let the powers at be fix them.

Didn't Mark just complain that people should spend more time fixing bugs
than expecting others to do it for them? Mark complains that we already
have too many unresolved bugs, and I tend to agree. If the current
developers are so busy, it seems logical that we need more developers,
not bug reporters. Wasn;t that the gist of Mark's polite rant?

And to fix bugs, I'm expected to learn a variation on Lisp and GIMPLE as
well. I'm not saying that expectation is wrong, I am saying it is an
impediment to working on GCC.

In many ways, I see GCC as similar in model to the Red Cross. You have a
paid staff that handles the day-to-day business, and a horde of
volunteers who do much of the grunt work. The Red Cross provides
training and mentoring to bring people along. GCC and other free
software projects would do well to consider how non-technical volunteer
organizations succeed.

If there is a lack of people fixing bugs, don;t necessarily blame it on
people being lazy. Maybe being a volunteer GCC developer is more
difficult than it needs to be?

> Again I have not seen a simple bug report from you recently; just a simple
> this goes bonkers, this is what I want GCC to output will do.

I haven't filed many bug reports recently (though I have in the past)
because I didn't feel that the effort was justified by the result.

Not that I've given up entirely: I've recently asked about how certain
problems should be reported. For example, -floop-optimize2 is a
pessimism for many algorithms. Is that a bug, or simply a feature that
is not yet fully implemented?

..Scott

Re: Fixing Bugs (Was: A Suggestion for Release Testing)

[Robert Dewar]

Scott Robert Ladd wrote:
  the gist of Mark's polite rant?
> 
> And to fix bugs, I'm expected to learn a variation on Lisp and GIMPLE as
> well. I'm not saying that expectation is wrong, I am saying it is an
> impediment to working on GCC.

with respect, I disagree, and I think you should invest the effort
before you hazard an opinion here. Compilers are complex beasts, and
all involve intermediate languages, and of course these intermediate
languages must be learned before you can do anything.

> If there is a lack of people fixing bugs, don;t necessarily blame it on
> people being lazy. Maybe being a volunteer GCC developer is more
> difficult than it needs to be?

I think a lot of what happens is that easy bugs do get fixed. The ones
that don't are often complex, or ill-reported, and thus tend to require
a lot of knowledge to work on effectively.

> I haven't filed many bug reports recently (though I have in the past)
> because I didn't feel that the effort was justified by the result.
> 
> Not that I've given up entirely: I've recently asked about how certain
> problems should be reported. For example, -floop-optimize2 is a
> pessimism for many algorithms. Is that a bug, or simply a feature that
> is not yet fully implemented?

This is a good example of something that likely can only be effectively
worked on by someone with a considerable amount of gcc knowledge.

Re: Fixing Bugs (Was: A Suggestion for Release Testing)

[Scott Robert Ladd]

Robert Dewar wrote:
> Scott Robert Ladd wrote:
>> And to fix bugs, I'm expected to learn a variation on Lisp and GIMPLE as
>> well. I'm not saying that expectation is wrong, I am saying it is an
>> impediment to working on GCC.
> 
> with respect, I disagree, and I think you should invest the effort
> before you hazard an opinion here. Compilers are complex beasts, and
> all involve intermediate languages, and of course these intermediate
> languages must be learned before you can do anything.

Testing and bug reporting are ways people can contribute to GCC if they
haven't the time or knowledge to effect repairs themselves.

GCC isn't special; all areas of specialized knowledge have steep
learning curves, and I'm certain I could find applications that would
mystify GCC's illuminati.

Lowering the learning curve would bring in more GCC developers and get
bugs fixed faster. I know quite a few very bright people who find GCC's
current documentation inadquate and opaque, and the environment less
than inviting.

..Scott

Re: Fixing Bugs (Was: A Suggestion for Release Testing)

[Robert Dewar]

Scott Robert Ladd wrote:

> Testing and bug reporting are ways people can contribute to GCC if they
> haven't the time or knowledge to effect repairs themselves.

Sure
> 
> GCC isn't special; all areas of specialized knowledge have steep
> learning curves, and I'm certain I could find applications that would
> mystify GCC's illuminati.

Actually, as anyone working with compilers knows, they are unusually
complex pieces of software, so they are special.
> 
> Lowering the learning curve would bring in more GCC developers and get
> bugs fixed faster. I know quite a few very bright people who find GCC's
> current documentation inadquate and opaque, and the environment less
> than inviting.

Improved documentation is of course always welcome
> 
> ..Scott

Re: Fixing Bugs (Was: A Suggestion for Release Testing)

[Timothy J. Wood]

On Jun 14, 2005, at 8:13 AM, Robert Dewar wrote:
> I think a lot of what happens is that easy bugs do get fixed. The ones
> that don't are often complex, or ill-reported, and thus tend to  
> require
> a lot of knowledge to work on effectively.

   One form of mentoring would be to _not_ have the core folks fixing  
the easy
bugs.  Throw those to the newbies with some pointers at where to look.
Eventually newbies get more and more experienced, and old-guard folks  
get more
available time to work on hard issues.

   I'm not saying this doesn't happen right now -- I don't know  
enough to make
that determination :)

-tim

Re: Fixing Bugs (Was: A Suggestion for Release Testing)

[Andrew Pinski]

On Jun 14, 2005, at 10:43 PM, Timothy J. Wood wrote:

>
> On Jun 14, 2005, at 8:13 AM, Robert Dewar wrote:
>> I think a lot of what happens is that easy bugs do get fixed. The ones
>> that don't are often complex, or ill-reported, and thus tend to 
>> require
>> a lot of knowledge to work on effectively.
>
>   One form of mentoring would be to _not_ have the core folks fixing 
> the easy
> bugs.  Throw those to the newbies with some pointers at where to look.
> Eventually newbies get more and more experienced, and old-guard folks 
> get more
> available time to work on hard issues.

This is in fact how I started working on GCC, I started with some easy 
bugs
and moved my way up.  I also now a days reduce preprocessed source which
comes into GCC's bugzilla and CC the person who I think caused the 
regression.
So I look at every bug which comes in and knows when there is an easy 
one.

Maybe I should start a wiki page for bugs which might be easy to fix 
but that
will not be for this week.

-- Pinski

RE: Fixing Bugs (Was: A Suggestion for Release Testing)

[Dave Korn]

----Original Message----
>From: Scott Robert Ladd
>Sent: 14 June 2005 15:51

> In many ways, I see GCC as similar in model to the Red Cross. You have a
> paid staff that handles the day-to-day business, 

  Really?  "GCC" has "paid staff"?  Who are they?  How much does "GCC" pay
them?  Where's the web page listing the vacancies?

> and a horde of volunteers who do much of the grunt work. The Red Cross
provides
> training and mentoring to bring people along. GCC and other free
> software projects would do well to consider how non-technical volunteer
> organizations succeed.

  The Red Cross has, as you mentioned, paid staff, and indeed has other
things, such as a "budget" with which to pay them, and a "organisation", and
"premises".  As far as I'm aware, gcc has none of those things, and although
the SC could perhaps be considered the nearest thing to an embodiment of the
"organisation" of gcc, I still think the analogy is rendered invalid by this
lack of similarity, and any lesson that could be learnt from it rendered
valueless.

  Perhaps I've missed something here, because I'm mystified how you could
think that gcc development is conducted by an organisation with full-time
staff.


    cheers,
      DaveK
-- 
Can't think of a witty .sigline today....

Re: Fixing Bugs (Was: A Suggestion for Release Testing)

[Scott Robert Ladd]

Dave Korn wrote:
>   Perhaps I've missed something here, because I'm mystified how you could
> think that gcc development is conducted by an organisation with full-time
> staff.

Perhaps not a formal staff, but there is a core group of developers
whose incomes are closely tied to GCC development. Many of the people
who work on GCC are employed or funded by corporations or institutions
with Linux interests -- or these developers have companies of their own
that sell services related to GCC.

And that's a Good Thing, in my opinion. I've been paid on a few
occassions to work on GCC, although it has usually been limited ($150
for fixing a bug critical to a friend's work, for example). If you look
at past threads, requests for new features are often met with "where's
the funding?" responses.

That's not to say that all, or even most, GCC developers are
mercenaries. There exist other motivations for working on GCC --
expanding ones skills, academic curiosity, the desire to help an ignored
consituency -- yet people do have families to feed, iguanas to house,
and cars to fix. And those other motivations can often be satisfied by
volunteering to work on other projects.

My entire immediate family works for the Red Cross, both on their
payroll and as volunteers. They have a very small paid staff, funded by
donations from interested parties; much of their work is carried out by
volunteers, and the Red Cross tries very hard to respect, train, and
thank volunteers.

It's a good model.

..Scott

Re: Fixing Bugs (Was: A Suggestion for Release Testing)

[Andrew Pinski]

On Jun 14, 2005, at 10:14 AM, Scott Robert Ladd wrote:

> Gentoo provides a mentoring system for developers. This is far better
> than GCC's "fix it yourself and send us the code" policy. There is no
> gentle way to become involved in GCC; it's a sink-or-swim, trial by 
> fire
> environment. If I could get a half-dozen quick questions answered, and 
> I
> could submit a couple of patches that are lying about on my hard drive.

We don't do that, we request the source of the failing source and most 
of
the time say how to fix the original code if it was invalid code.

I have not seen one recent bug report from you so I don't understand why
you are complaining!

Thanks,
Andrew P

Re: Fixing Bugs (Was: A Suggestion for Release Testing)

[Joseph S. Myers]

On Tue, 14 Jun 2005, Scott Robert Ladd wrote:

> If I were to present a patch that implements the recomendations of the
> Numerical C Extensions Group, would it be accepted or rejected (on the
> subject alone; ignore for the moment potential technical bugs in the
> submitted code)?

I don't know to what recommendations you refer (please give a URL).  I 
would consider a patch resolving issue 323 by implementing the C90/C99 
standard requirements for how excess precision works to be desirable.  
That is, excess precision should be explicitly modeled throughout the 
compiler (rather than the x86 back end pretending it can operate on float 
and double); not just assignments but also casts and function call and 
return should discard excess precision (including cast from a type to 
itself, etc.; and see also DR#318); and arithmetic on constants should use 
excess precision the same way as at runtime; this should all be enabled by 
-ansi / -std=c89 / -std=c99 and disabled by -fno-float-store.  This would 
allow predictable operation in accordance with the standards on processors 
where excess precision is natural.

I think changing the default precision must be a matter for -m options; 
these would also affect the ABI by changing the size of long double and 
libraries written to use long double in computations of functions for 
double would cease to work with a different precision.

-- 
Joseph S. Myers               http://www.srcf.ucam.org/~jsm28/gcc/
    jsm@polyomino.org.uk (personal mail)
    joseph@codesourcery.com (CodeSourcery mail)
    jsm28@gcc.gnu.org (Bugzilla assignments and CCs)

Re: Fixing Bugs (Was: A Suggestion for Release Testing)

[Robert Dewar]

Scott Robert Ladd wrote:

> The ISO Standard doesn't prevent GCC from being *better* than specified,
> does it? Are we somehow breaking ISO compliance by doing math right? Is
> it so wrong to try and fix a problem that frustrates many people and
> makes GCC look bad?

No, but you are degrading performance unnecessarily by doing math to
conform to the Scott-Ladd standard, and we are not implementing SL C
we are implementing ISO C :-)

Of course it is perfectly reasonable to have well chosen options to
do more or less than the standard requires.
> 
> With the attitude shown by Giovanni, there's really no point in
> submitting a patch, is there? Dozens of people have reported this
> problem, potential solutions exist, but any patch is going to be ignored
> because the bug isn't considered a bug by the Powers That Be.

It would be quite wrong to patch this unconditionally. The proper
patch would be one that provided a switch and a well defined semantics
for the switch, but this needs very careful design, and careful
review by people who understand fpt well.
> 
> If I were to present a patch that implements the recomendations of the
> Numerical C Extensions Group, would it be accepted or rejected (on the
> subject alone; ignore for the moment potential technical bugs in the
> submitted code)?

Again, it would probably be better submitted under a switch. For
example on a machine like the Alpha, the impact of following these
recomendations is severe degradation of performance.
> 
> Documentation for the compiler itself is woefully inadequate for someone
> "new". GCC may be old-hat to folks who've worked on it for years, but it
> is very complex and foreign territory for most non-compiler experts.
> Working on GCC requires a much broader knowledge than working on other
> projects, and without some sort of tutorial or guidance, working on it
> quickly becomes frustrating.

Compilers are complex beasts, I don't think you would expect to be able
to wade in and fix nuclear reactors without being somewhat of a nuclear
reactor expert. You can't expect to wade in and fix compilers without
being reasonably compiler literate. For those who are compiler literate,
GCC is certainly accessible. Sure, it could be made more accessible,
principally by adding more documentation. Patches to add correct useful
additional documentation are definitely welcome.
> 
> Here's an example: Building new targets and fixing some code generation
> bugs involve changing the machine definitions, which are written in a
> rather uncommon language. Frankly, I haven't figured out all the nuances
> yet, mostly because I don't have the luxury of studying it, and I can't
> find any clear and comprehensive documentation.

These definitions are actually very clear, certainly they could not
be written in some normal programming language since they are essentially
denotational semantic definitions, for which a language like C would be
perfectly horrible. You need to invest the effort to fully understand
the machine definition language before you can do anything in this
area, and it is hard to see how it could be otherwise.
> 
> Gentoo provides a mentoring system for developers. This is far better
> than GCC's "fix it yourself and send us the code" policy. There is no
> gentle way to become involved in GCC; it's a sink-or-swim, trial by fire
> environment. If I could get a half-dozen quick questions answered, and I
> could submit a couple of patches that are lying about on my hard drive.

There is no gentle way to become involved in any compiler, they are complex
beasts, and require a considerable investment of effort.

> I have submitted patches; I have tried to help with various aspects of
> GCC. I make no claim to having accomplished anything great, but I have
> tried, and watched patches die of bit rot while being told that certain
> bugs shouldn't be fixed.

I know of no bug (i.e. behavior non-conformant with ISO C) that someone
has said should not be fixed.
> 
> Everything in life is a matter of give and take. People report bugs; you
> want someone to fix the bugs -- perhaps GCC should be more welcoming and
> helpful.

ANything to make it more welcoming and helpful is of course desirable, but
I think you are expecting too much if you expect an artifact of this
inherent complexity to be easily accessible, especially to people without
reasonably comprehensive training (i.e. at least a familiarity with compilers
at the dragon book level, and reading some of the crucial papers in the area).

Re: Fixing Bugs (Was: A Suggestion for Release Testing)

[Scott Robert Ladd]

Robert Dewar wrote:
>> If I were to present a patch that implements the recomendations of the
>> Numerical C Extensions Group, would it be accepted or rejected (on the
>> subject alone; ignore for the moment potential technical bugs in the
>> submitted code)?

> Again, it would probably be better submitted under a switch. For
> example on a machine like the Alpha, the impact of following these
> recomendations is severe degradation of performance.

Agreed. I never said it should be a default option.

> Compilers are complex beasts, I don't think you would expect to be able
> to wade in and fix nuclear reactors without being somewhat of a nuclear
> reactor expert. You can't expect to wade in and fix compilers without
> being reasonably compiler literate. For those who are compiler literate,
> GCC is certainly accessible. Sure, it could be made more accessible,
> principally by adding more documentation. Patches to add correct useful
> additional documentation are definitely welcome.

That is exactly my point. Mark chastises people for talking about
testing, implying that we are lazy for not providing patches. Perhaps
some of us wish to contribute to GCC, but lacking the knowledge to patch
the compiler, we perform testing and make bug reports.

> These definitions are actually very clear, certainly they could not
> be written in some normal programming language since they are essentially
> denotational semantic definitions, for which a language like C would be
> perfectly horrible. You need to invest the effort to fully understand
> the machine definition language before you can do anything in this
> area, and it is hard to see how it could be otherwise.

Clear to you, but not to many other bright people. And I have *never*
said there is no requirement for learning, or even criticized the
language of the machine definition files. I am saying that Mark can't
complain about a lack of bug-fixers when big-fixing is non-trivial
(which I believe agrees with your point.)

> There is no gentle way to become involved in any compiler, they are complex
> beasts, and require a considerable investment of effort.

Nope. Gentleness comes in many forms, and includes being polite to
newbies, not condescending. Providing mentors to work with people on
specific questions would do wonders for teaching GCC internals.

> I know of no bug (i.e. behavior non-conformant with ISO C) that someone
> has said should not be fixed.

No, what they do is redefine the bug such that it is no longer a bug.
Certain U.S. Presidents are quite fond of this tactic, recategorizing
things to avoid dealing with them. Bug 323 is an example of how GCC does
this.

> ANything to make it more welcoming and helpful is of course desirable, but
> I think you are expecting too much if you expect an artifact of this
> inherent complexity to be easily accessible, especially to people without
> reasonably comprehensive training (i.e. at least a familiarity with
> compilers > at the dragon book level, and reading some of the crucial papers in the
> area).

I don't deny that reality. Mark seems to feel that fixing bugs is as
easy as testing and bug reporting, and it is not. Yet people then
complain that people don't file enough bugs! It seems that no matter
what someone "outside" tries to contribute, it isn't "right".

If you want more people fixing bugs, it requires patience and respect.

..Scott

Re: Fixing Bugs (Was: A Suggestion for Release Testing)

[Mark Mitchell]

Scott Robert Ladd wrote:

> That is exactly my point. Mark chastises people for talking about
> testing, implying that we are lazy for not providing patches.

> I don't deny that reality. Mark seems to feel that fixing bugs is as
> easy as testing and bug reporting, and it is not. 

Actually, I don't agree with either of the statements ascribed to me, 
and if I conveyed that sentiment, I apologize.  To be clear, I think 
testing, automated and otherwise, and bug reporting, and bug-mastering 
are all very valuable.  My point was simply that once you have the tests 
and bug reports, someone has to fix the bugs before the users see benefit.

CodeSourcery struggles with exactly the same problem; we have worked 
hard to set up some test automation for our ARM builds, and it's working 
well, but we're not (yet!) as disciplined as we want to be about 
analyzing and fixing the failures.  I think the reason is that the 
testing is something you work hard to set up once, and then, roughly 
speaking, you just sit back and let the computer work hard forever; the 
analysis/fixing is an ongoing project that requires people to work hard 
on a regular basis.

Once I get my automated GCC bug-fixing bot finished I am going to have 
an easy life.  Unfortunately, I use GCC to build the bot, and I'm 
getting an ICE in reload...

-- 
Mark Mitchell
CodeSourcery, LLC
mark@codesourcery.com
(916) 791-8304

Re: Fixing Bugs (Was: A Suggestion for Release Testing)

[Diego Novillo]

On Tue, Jun 14, 2005 at 09:01:04AM -0700, Mark Mitchell wrote:

> Once I get my automated GCC bug-fixing bot finished I am going to have 
> an easy life.  Unfortunately, I use GCC to build the bot, and I'm 
> getting an ICE in reload...
> 
I have a patch for that.  I'm sure the napking I wrote it on was
somewhere around here...

Re: Fixing Bugs (Was: A Suggestion for Release Testing)

[Scott Robert Ladd]

Mark Mitchell wrote:
> Actually, I don't agree with either of the statements ascribed to me,
> and if I conveyed that sentiment, I apologize.

And I apologize for perhaps reading more into your statements than was
intended.

> My point was simply that once you have the tests
> and bug reports, someone has to fix the bugs before the users see benefit.

I wish I could do more, which is why I made (what I hoped were)
constructive suggestions for trying to get more people involved in
fixing bugs.

True, there are many lazy people who don't want to learn anything. I'm
more interested in attracting the people who, like myself, simply need
help getting around the more obscure corners of GCC. I still think that
Gentoo-style mentoring could be successful.

> Once I get my automated GCC bug-fixing bot finished I am going to have
> an easy life.  Unfortunately, I use GCC to build the bot, and I'm
> getting an ICE in reload...

Such software would make you rich beyond the dreams of avarice.

..Scott

Re: Fixing Bugs (Was: A Suggestion for Release Testing)

[Richard Guenther]

On 6/14/05, Scott Robert Ladd <scott.ladd@coyotegulch.com> wrote:

> I wish I could do more, which is why I made (what I hoped were)
> constructive suggestions for trying to get more people involved in
> fixing bugs.

For getting more people involved in fixing bugs they need those
bugs in plain C code (read: a testcase) presented to them, preferrably
with a (short) description what and why it is going wrong.

Useful suggestions about what one could do tend to be ignored or
end in endless back-and-forth mailings like this.  Especially if the
discussion is about such broad field as FP correctness.

It may also help to have bugs collected beyond a meta-bug
(such as 323 seems to be).

Also GCC is not the only player involved here, but glibc is as
well.

Take a break and come back with results of actual work done,
this impresses people a lot more than (repeated) ranting about
gcc development in general.

Richard.

Re: Fixing Bugs (Was: A Suggestion for Release Testing)

[Scott Robert Ladd]

Richard Guenther wrote:
> Take a break and come back with results of actual work done,
> this impresses people a lot more than (repeated) ranting about
> gcc development in general.

I have worked on GCC; not much, and probably trivial in your eyes,
but practical work nonetheless. To trivialize contributions is a great
way of driving away potential contributors.

I would like to improve floating-point in GCC; doing so scratches my
personal itch. My silly idea is to determine the best approach
*through discussion*.

..Scott

Re: Fixing Bugs (Was: A Suggestion for Release Testing)

[chris jefferson]

Scott Robert Ladd wrote:

>Richard Guenther wrote:
>  
>
>>Take a break and come back with results of actual work done,
>>this impresses people a lot more than (repeated) ranting about
>>gcc development in general.
>>    
>>
>
>I have worked on GCC; not much, and probably trivial in your eyes,
>but practical work nonetheless. To trivialize contributions is a great
>way of driving away potential contributors.
>
>I would like to improve floating-point in GCC; doing so scratches my
>personal itch. My silly idea is to determine the best approach
>*through discussion*.
>
>  
>
One thing I have come across, both in gcc and in other projects, is that 
often discussion is not the best option, but instead just writing some 
code is better.

It's very easy to have discussions go around in circles about if option 
a or option b is better, and which will lead to slowdowns, or intrusive 
changes, or whatever. It's very hard to know how well something will 
actually work, and if it will be possible, until it's actually been 
written. While it's briefly annoying to write code which then isn't used 
the first time you do it, I've quickly learned it's faster and easier 
than extensive discussions, and most good code will go through 3 or 4 
iterations before it finally settles, and need a whole bundle of tests 
writing, so writing an initial test version is not actually that big a 
time investment compared to the total amount of time something will 
take. Working code is also of course by far the most convincing argument 
:).

I have 4 completely different implementations of std::tr1::tuple lying 
around somewhere, obviously only one was actually used, but the only 
real way to know which would be best was to just write them and see how 
they looked and worked.

Chris

Re: Fixing Bugs (Was: A Suggestion for Release Testing)

[Scott Robert Ladd]

chris jefferson wrote:
> One thing I have come across, both in gcc and in other projects, is that
> often discussion is not the best option, but instead just writing some
> code is better.

There's a fine line between too much talk and not enough.

> It's very easy to have discussions go around in circles about if option
> a or option b is better, and which will lead to slowdowns, or intrusive
> changes, or whatever.

I'm more interested in "Does doing A make any sense?"

An example: In its formative stages, gfortran had a problem with certain
kinds of constants. Whether the problem needed to be solved depending on
how strictly one read the standard.

I simply went ahead and wrote a patch (actually, three of them), trying
to satisfy all sides involved. This resulted in long discussions of
whether the problem really *was* a problem or not. In the end, the
compiler was modified to behave as other Fortran 95s do (which was my
original suggestion, before people started quoting the standard), and I
wasted much of the time spent writing the original patch.

Had the problem been talked out in the beginning, I could have spent
more time working on an acceptable solution. This is one incident that
lead me to stop submitting patches; I have only so much time for GCC,
and am donating all of it out of my own pocket. Maybe others can afford
to do that, but I can't, after three months in the hospital followed by
an injury to my wife. If I'm going to contribute to GCC, it needs to be
something I know will be more than just a shot in the dark.

> While it's briefly annoying to write code which then isn't used
> the first time you do it, I've quickly learned it's faster and easier
> than extensive discussions, and most good code will go through 3 or 4
> iterations before it finally settles, and need a whole bundle of tests
> writing, so writing an initial test version is not actually that big a
> time investment compared to the total amount of time something will
> take. Working code is also of course by far the most convincing argument
> :).

Perhaps I'm too steeped in being an engineer, but in my experience,
quality upfront discussion saves a lot of time and produces better
results. I'd hate to build a bridge the way you suggest developing GCC.

..Scott

Re: Fixing Bugs (Was: A Suggestion for Release Testing)

[Aaron W. LaFramboise]

chris jefferson wrote:

> Working code is also of course by far the most convincing argument 
> :).

Boosters, FreeBSD hackers, and I'm sure tons of others are calling this
the "Bicycle shed effect."

<http://www.freebsd.org/doc/en_US.ISO8859-1/books/faq/misc.html#BIKESHED-PAINTING>

I agree.  Preliminary discussion is hugely more vulnerable to bicycle
shed color disagreements, where participants may feel that all of the
colors have not been completely examined, compared to tested working
code, where others will have more confidence that due diligence has been
applied.

In all of the open source world I have seen, posting code is always better.


Aaron W. LaFramboise

Re: Fixing Bugs (Was: A Suggestion for Release Testing)

[Scott Robert Ladd]

Aaron W. LaFramboise wrote:
> Boosters, FreeBSD hackers, and I'm sure tons of others are calling this
> the "Bicycle shed effect."
> 
> <http://www.freebsd.org/doc/en_US.ISO8859-1/books/faq/misc.html#BIKESHED-PAINTING>

If I'm building a bicycle shed, I may want to talk with others who have
done so in the past, learning from the experience and gaining their
insights. Why did they use a certain type of construction? What sort of
storage did they build in? What worked and didn't work for someone else
who has already built a shed? What did they learn from their own work?
Any shed I build will be better for such discussions.

GCC's floating-point support can be improved by considering what people
want from their math in conjunction with the characteristics of
different systems. Discussions herein have clarified and expanded my
understanding of the issues.

> In all of the open source world I have seen, posting code is always better.

Better than what? Design? Analysis? Just because some people like to
nitpick doesn't mean there shouldn't be forethought to our work.

Be that as it may, one must work within the presiding culture, and if
design and analysis are frowned upon, it is directly to code that I will go.

..Scott

Re: Fixing Bugs (Was: A Suggestion for Release Testing)

[Jonathan Wakely]

On Thu, Jun 16, 2005 at 10:30:03AM -0400, Scott Robert Ladd wrote:

> Aaron W. LaFramboise wrote:
> > Boosters, FreeBSD hackers, and I'm sure tons of others are calling this
> > the "Bicycle shed effect."
> > 
> > <http://www.freebsd.org/doc/en_US.ISO8859-1/books/faq/misc.html#BIKESHED-PAINTING>
> 
> If I'm building a bicycle shed, I may want to talk with others who have
> done so in the past, learning from the experience and gaining their
> insights. Why did they use a certain type of construction? What sort of
> storage did they build in? What worked and didn't work for someone else
> who has already built a shed? What did they learn from their own work?
> Any shed I build will be better for such discussions.

You've completely missed the point (rather appropriately).  You're
*supposed* to be building a nuclear reactor, but have got preoccupied
discussing the colour of the bicycle shed in which the staff will park
their bikes.

That's what "bicycle shed painting" refers to.

> GCC's floating-point support can be improved by considering what people
> want from their math in conjunction with the characteristics of
> different systems. Discussions herein have clarified and expanded my
> understanding of the issues.
> 
> > In all of the open source world I have seen, posting code is always better.
> 
> Better than what? Design? Analysis? Just because some people like to
> nitpick doesn't mean there shouldn't be forethought to our work.

The point Aaron is making is that discussion of such topics very rarely
results in any useful design or analysis, because every Tom, Dick and
Harry pipes up to add their 2c, the majority of which are completely
irrelevant. (Look, I'm contributing to it, and I'm completely irrelevant
to the discussion).

If this thread had resulted in any useful design or analysis noone would
mind, but it's turned into a multi-headed beast covering the etiquette
of bugmasters, the names of vapourware compiler switches, and alien
invasion (though that was very funny). Oh, and whether PR323 is a bug,
of course.

The only remaining place this thread can go (before satisfying Godwin's
rule) is for the resident markov generator to correct you all by pointing
out that the linker on MMIX platforms uses UTF-8 for the symbol names.

(what do you mean it's not friday afternoon yet?)

jon

Re: Fixing Bugs (Was: A Suggestion for Release Testing)

[Robert Dewar]

On Thu, Jun 16, 2005 at 10:30:03AM -0400, Scott Robert Ladd wrote:

 > Aaron W. LaFramboise wrote:

 >> > Boosters, FreeBSD hackers, and I'm sure tons of others are calling this
 >> > the "Bicycle shed effect."
 >> >
 >> > <http://www.freebsd.org/doc/en_US.ISO8859-1/books/faq/misc.html#BIKESHED-PAINTING>

 >
 > If I'm building a bicycle shed, I may want to talk with others who have
 > done so in the past, learning from the experience and gaining their
 > insights. Why did they use a certain type of construction? What sort of
 > storage did they build in? What worked and didn't work for someone else
 > who has already built a shed? What did they learn from their own work?
 > Any shed I build will be better for such discussions.

chuckle chuckle ...

The reference is of course to Parkinson's second law, that the discussion
time on an issue is in inverse proportion to its importance. I think
that in fact Scott has unwittingly provided a perfect reference for this
thread :-)

Re: Fixing Bugs

[Kai Henningsen]

cow@compsoc.man.ac.uk (Jonathan Wakely)  wrote on 16.06.05 in <20050616145729.GA11184@compsoc.man.ac.uk>:

> On Thu, Jun 16, 2005 at 10:30:03AM -0400, Scott Robert Ladd wrote:
>
> > Aaron W. LaFramboise wrote:
> > > Boosters, FreeBSD hackers, and I'm sure tons of others are calling this
> > > the "Bicycle shed effect."
> > >
> > > <http://www.freebsd.org/doc/en_US.ISO8859-1/books/faq/misc.html#BIKESHED
> > > -PAINTING>
> >
> > If I'm building a bicycle shed, I may want to talk with others who have
> > done so in the past, learning from the experience and gaining their
> > insights. Why did they use a certain type of construction? What sort of
> > storage did they build in? What worked and didn't work for someone else
> > who has already built a shed? What did they learn from their own work?
> > Any shed I build will be better for such discussions.
>
> You've completely missed the point (rather appropriately).  You're
> *supposed* to be building a nuclear reactor, but have got preoccupied
> discussing the colour of the bicycle shed in which the staff will park
> their bikes.
>
> That's what "bicycle shed painting" refers to.

Actually, based on that FAQ entry, no, it doesn't.

Those are two different projects. It's not a shed as part of a reactor.

It's *not* about being distracted by unimportant details; it's about  
discussing easy projects to death and just rubberstamping hard projects.

Which does not seem to describe any current problem.

MfG Kai

Re: Fixing Bugs (Was: A Suggestion for Release Testing)

[Daniel Berlin]

On Thu, 2005-06-16 at 10:30 -0400, Scott Robert Ladd wrote:
> Aaron W. LaFramboise wrote:
> > Boosters, FreeBSD hackers, and I'm sure tons of others are calling this
> > the "Bicycle shed effect."
> > 
> > <http://www.freebsd.org/doc/en_US.ISO8859-1/books/faq/misc.html#BIKESHED-PAINTING>
> 
> If I'm building a bicycle shed, I may want to talk with others who have
> done so in the past, learning from the experience and gaining their
> insights. Why did they use a certain type of construction? What sort of
> storage did they build in? What worked and didn't work for someone else
> who has already built a shed? What did they learn from their own work?
> Any shed I build will be better for such discussions.
> 
> GCC's floating-point support can be improved by considering what people
> want from their math in conjunction with the characteristics of
> different systems. Discussions herein have clarified and expanded my
> understanding of the issues.
> 
> > In all of the open source world I have seen, posting code is always better.
> 
> Better than what? Design? Analysis? Just because some people like to
> nitpick doesn't mean there shouldn't be forethought to our work.
> 
> Be that as it may, one must work within the presiding culture, and if
> design and analysis are frowned upon, it is directly to code that I will go.
Design and analysis is not frowned upon.
Endless design and analysis over minutae is.
Code and design can be revised.


> 
> ..Scott

Re: Fixing Bugs (Was: A Suggestion for Release Testing)

[Robert Dewar]

Scott Robert Ladd wrote:

> I would like to improve floating-point in GCC; doing so scratches my
> personal itch. My silly idea is to determine the best approach
> *through discussion*.

Perfectly appropriate, and you are getting lots of discussion!

Re: Fixing Bugs (Was: A Suggestion for Release Testing)

[Scott Robert Ladd]

Robert Dewar wrote:
> Scott Robert Ladd wrote:
>> I would like to improve floating-point in GCC; doing so scratches my
>> personal itch. My silly idea is to determine the best approach
>> *through discussion*.

> Perfectly appropriate, and you are getting lots of discussion!

And I'm burbling with joy!

Well, maybe not *that* enthused... :)

..Scott

Re: Fixing Bugs (Was: A Suggestion for Release Testing)

[Laurent GUERBY]

On Tue, 2005-06-14 at 09:01 -0700, Mark Mitchell wrote:
> CodeSourcery struggles with exactly the same problem; we have worked 
> hard to set up some test automation for our ARM builds, and it's working 
> well, but we're not (yet!) as disciplined as we want to be about 
> analyzing and fixing the failures.

At AdaCore (I assume it's still the rule), a patch was allowed to be
committed only *after* a special run of the regression tester with CVS
source baseline + only your patch showed a clean run (it was all
automated so you just had to provide a clean patch to the system).

That cleans-up the area of who has to do the analysis, and it's on
a patch the person just wrote so no precious brain time wasted in
refocus and analysis of random failures. If the patch is judged correct
but triggering a latent bug somewhere else, the person that will work on
it will also start from a small set of triggering conditions so an
easier task overall.

This leaves:

- the problem of interaction of two patches committed the exact same day
that have a bad interaction, but this is quite rare even for GCC.

- if you don't have enough ressources to run simulators, platform
specific problems.

The only missing piece for such a setup to be workable for GCC in the
FSF framework is some dedicated computing ressources, plus someone to do
the script work.

Laurent

Re: Fixing Bugs (Was: A Suggestion for Release Testing)

[Robert Dewar]

Laurent GUERBY wrote:

> At AdaCore (I assume it's still the rule), a patch was allowed to be
> committed only *after* a special run of the regression tester with CVS
> source baseline + only your patch showed a clean run (it was all
> automated so you just had to provide a clean patch to the system).

Yes, it's still the rule :-) and we have about 10,000 test directories
now, many with a lot of code (it's many millions of lines in all, a
good thing that machines are getting faster). In fact our test suite
seems to take somewhat over an hour to execute on fastest machines,
and that has been fairly constant!

Re: Fixing Bugs (Was: A Suggestion for Release Testing)

[James A. Morrison]

Robert Dewar <dewar@adacore.com> writes:

> Yes, it's still the rule :-) and we have about 10,000 test directories
> now, many with a lot of code (it's many millions of lines in all, a
> good thing that machines are getting faster). In fact our test suite
> seems to take somewhat over an hour to execute on fastest machines,
> and that has been fairly constant!

 It would be nice to have some of these tests integrated into GCC for the rest
of us to test with.

-- 
Thanks,
Jim

http://www.csclub.uwaterloo.ca/~ja2morri/
http://phython.blogspot.com
http://open.nit.ca/wiki/?page=jim

Re: Fixing Bugs (Was: A Suggestion for Release Testing)

[Robert Dewar]

James A. Morrison wrote:
> Robert Dewar <dewar@adacore.com> writes:
> 
> 
>>Yes, it's still the rule :-) and we have about 10,000 test directories
>>now, many with a lot of code (it's many millions of lines in all, a
>>good thing that machines are getting faster). In fact our test suite
>>seems to take somewhat over an hour to execute on fastest machines,
>>and that has been fairly constant!
> 
> 
>  It would be nice to have some of these tests integrated into GCC for the rest
> of us to test with.
> 

The great majority of these tests cannot be integrated into GCC, since
they contain proprietary customer code. As you know we are now trying
to provide test cases with fixes as we make patches to the Ada front
end, but these tests have yet to be integrated into some Ada test suite.

There are some tests among the 10,000 that could be put into such an
Ada test suite, but that's quite a lot of work, and has not yet
been carried out (it is certainly on the list to do).

Re: Fixing Bugs (Was: A Suggestion for Release Testing)

[Robert Dewar]

Scott Robert Ladd wrote:

> No, what they do is redefine the bug such that it is no longer a bug.
> Certain U.S. Presidents are quite fond of this tactic, recategorizing
> things to avoid dealing with them. Bug 323 is an example of how GCC does
> this.

I don't know of any example of this. That is a case where something is
clearly a non-conformance with the language definition, and has been
declared not a bug.

Re: Fixing Bugs (Was: A Suggestion for Release Testing)

[Giovanni Bajo]

Scott Robert Ladd <scott.ladd@coyotegulch.com> wrote:

> Consider, as an example, the bug/non-bug at http://gcc.gnu.org/PR323,
> which was a matter of recent discussion on this list. Some rather
> respectable people (e.g., Vincent Lefèvre) consider this a bug, and
> have proposed solutions. Yet here's a quote from an earlier message by
> Giovanni Bajo.
> [...]

First of all, I would consider polite to CC: me on the mail if you quote and
debate my statements.

> The ISO Standard doesn't prevent GCC from being *better* than
> specified, does it? Are we somehow breaking ISO compliance by doing
> math right? Is it so wrong to try and fix a problem that frustrates
> many people and makes GCC look bad?

Where exactly do I say that it is wrong to provide a patch that makes GCC
better in this regard? As a bugmaster, I just decided to consider this not a
bug, in the strictest meaning of "bug". If you want to file in Bugzilla an
enhancement proposal about adding options/modes about higher FPU precision, I
would not object.

Also, it seems you have the wrong belief that, if bug 323 were confirmed in
Bugzilla, a patch would automatically appear. I think the two events are
totally unrelated, especially for an issue which has been debated so much in
the past years, and where most people have already a formed opinion on the
matter.

> With the attitude shown by Giovanni, there's really no point in
> submitting a patch, is there? Dozens of people have reported this
> problem, potential solutions exist, but any patch is going to be
> ignored because the bug isn't considered a bug by the Powers That Be.

You seem to believe that a patch can be accepted in GCC only if it fixes a bug.
That is wrong: a valid patch can also add a new feature. While it is probably
hard to convince the Powers That Be (which, by the way, are surely not me) that
the default setting of GCC should be whatever Bug 323 requests, it is much much
easier that a patch adding a new command line option to request higher FP
accuracy be reviewed and accepted. Of course, people would have to *see* such a
patch. And nobody is blocking people from writing it.

I hope to have clarified my position.

Giovanni Bajo

Re: Fixing Bugs (Was: A Suggestion for Release Testing)

[Scott Robert Ladd]

Giovanni Bajo wrote:
> Scott Robert Ladd <scott.ladd@coyotegulch.com> wrote:
> First of all, I would consider polite to CC: me on the mail if you quote and
> debate my statements.

I meant no offense, and thought that I *had* CC'd you on the message.

>>The ISO Standard doesn't prevent GCC from being *better* than
>>specified, does it? Are we somehow breaking ISO compliance by doing
>>math right? Is it so wrong to try and fix a problem that frustrates
>>many people and makes GCC look bad?
> 
> Where exactly do I say that it is wrong to provide a patch that makes GCC
> better in this regard? As a bugmaster, I just decided to consider this not a
> bug, in the strictest meaning of "bug". If you want to file in Bugzilla an
> enhancement proposal about adding options/modes about higher FPU precision, I
> would not object.

I've been suggesting various ways to enhance, or at better delineate,
floating-point in GCC. Once I've got a good idea of the best way to
approach this (and I've had some excellent feedback), I will indeed
submit a patch. I even have one started...

Given the number of "duplicate filings" associated with 323, I assumed
that filing another item on the topic would be ineffective.

> Also, it seems you have the wrong belief that, if bug 323 were confirmed in
> Bugzilla, a patch would automatically appear.

No; my objection is to having people's concerns flatly rejected. It is
not clear from reading the message on 323 that a "add an option" patch
would be considered.

If anything, I'm very pleased with the way people have handled the bugs
I've reported; all but one (or more than 20) have been fixed. Good stuff.

>>With the attitude shown by Giovanni, there's really no point in
>>submitting a patch, is there? Dozens of people have reported this
>>problem, potential solutions exist, but any patch is going to be
>>ignored because the bug isn't considered a bug by the Powers That Be.
> 
> You seem to believe that a patch can be accepted in GCC only if it fixes a bug.

No, my experience is that a patch is only accepted if it has already
been approved, at least on a conceptual level. I've tried the "make a
patch without talking to anyone" approach, and wasted a lot of time.

> I hope to have clarified my position.

And I mine.

..Scott

Re: Fixing Bugs (Was: A Suggestion for Release Testing)

[Gabriel Dos Reis]

"Giovanni Bajo" <giovannibajo@libero.it> writes:

| Also, it seems you have the wrong belief that, if bug 323 were confirmed in
| Bugzilla, a patch would automatically appear. I think the two events are
| totally unrelated, especially for an issue which has been debated so much in
| the past years, and where most people have already a formed opinion on the
| matter.

That is untrue.  When something is considered not an issue, it is
usually hard to get a patch for it.

-- Gaby