Regression count, and how to keep bugs around forever

Regression count, and how to keep bugs around forever

[Steven Bosscher]

Hello,

This is a complaint about how the bug database is being managed.  It
is getting harder and harder to find bug reports to work on, because
too many old bug reports are being kept open even though there is no
sign of intent to ever resolve the report.

For example, PR18346 is a bug report in Bugzilla, and it is apparently
a regression.  It's not a high-priority regression, because it is for
a target that is neither a secondary nor a primary platform. But the
bug report of course does show up in the overview of regressions (such
as the "All regressions" link on the homepage).

Some facts about this bug report:
* the bug was reported by the only listed maintainer for this target
* the bug was confirmed by the reporter himself
* the bug has seen *no* activity at all in 3.5 years.
* the target maintainer insists that the bug report should be kept open

Literally no action has been taken for three and a half years! Not by
the target maintainer and not by anyone else.

This is an extreme case, but it is not entirely unusual that
regression PRs (even some P3 and P2 ones) have not seen any activity
for two years or more.

The bigger issue here, is that people seem to be using Bugzilla as a
kind-of TODO list for things may some day work on, but probably will
not. The result is that Bugzilla becomes increasingly hard to use over
time: The number of open reports in Bugzilla keeps increasing, the
list of so-called low-priority regression becomes less useful with
each release, and identifying regressions that people actually care
about becomes practically impossible.
(See also my earlier complaint about a similar ages old bug report
that isn't going anywhere:
http://gcc.gnu.org/ml/gcc/2007-11/msg00659.html).

The current list of "All regressions" should be a list of bugs that
people are actively trying to resolve, preferably before the release
of GCC 4.3. Instead, it is a mix of high-activity bug reports and bug
reports where even the target maintainer has been unwilling for 3.5
years to spend some time on resolving the bug report. So to pick a bug
report to work on, I need to go through the but report summaries of a
long list, trying to pick out new regressions between the old
no-one-cares P4 and P5 regressions.

Maybe it is just me, but I find it very annoying to have to wade
through long bug lists, so I just don't do this. Instead I just don't
look at P4/P5 regressions anymore at all. It's just too much trouble
to find a bug report where the reporter or the target maintainer cares
as much as you do about resolving the bug.

The victims, in the end, are GCC's users.  Once a regression report
from a user had been downgraded to P4 or P5, it disappears from the
radar into the grey mess of older reports.

Is this really how this community wants to manage its bug database?

To me, the situation is quite clear: If a bug report is open for so
long, and even the reporter and the responsible maintainer show no
sign of motivation to work on resolving the bug, I think this tells us
something about how important this bug is: Not important enough to
fix.  IMOH we should close such reports as WONTFIX or SUSPENDED to
make them less visible, so that other bug reports don't fall through
the cracks.

So I'm asking for a policy here that says when it is OK to resolve old
bug without progress as WONTFIX or SUSPENDED. Start shooting.

Gr.
Steven

Re: Regression count, and how to keep bugs around forever

[Paul Brook]

> So I'm asking for a policy here that says when it is OK to resolve old
> bug without progress as WONTFIX or SUSPENDED. Start shooting.

I think this would be a big mistake to reuse an existing state for this.

If/when someone does start caring about that particular feature it'll be 
impossible for them to distinguish between bugs that have been deliberately 
closed (typically because the cure is worse than the disease), and those that 
you've closed through apathy.

Paul

Re: Regression count, and how to keep bugs around forever

[Joe Buck]

On Wed, Dec 19, 2007 at 01:11:11AM +0000, Paul Brook wrote:
> > So I'm asking for a policy here that says when it is OK to resolve old
> > bug without progress as WONTFIX or SUSPENDED. Start shooting.
> 
> I think this would be a big mistake to reuse an existing state for this.

But this is pretty much what SUSPENDED means; it means that there's no
intent to work on the bug in the near term.

> If/when someone does start caring about that particular feature it'll be 
> impossible for them to distinguish between bugs that have been deliberately 
> closed (typically because the cure is worse than the disease), and those that 
> you've closed through apathy.

That's an argument for not using WONTFIX.

On the other hand, bugs could just be dropped to P5, and people doing
queries would exclude that.

Re: Regression count, and how to keep bugs around forever

[Paul Brook]

On Wednesday 19 December 2007, Joe Buck wrote:
> On Wed, Dec 19, 2007 at 01:11:11AM +0000, Paul Brook wrote:
> > > So I'm asking for a policy here that says when it is OK to resolve old
> > > bug without progress as WONTFIX or SUSPENDED. Start shooting.
> >
> > I think this would be a big mistake to reuse an existing state for this.
>
> But this is pretty much what SUSPENDED means; it means that there's no
> intent to work on the bug in the near term.

Ok. I did check the GCC bugzilla help pages, and they don't mention SUSPENDED 
at all :-)

Paul

Re: Regression count, and how to keep bugs around forever

[Joe Buck]

On Wed, Dec 19, 2007 at 01:25:19AM +0000, Paul Brook wrote:
> On Wednesday 19 December 2007, Joe Buck wrote:
> > On Wed, Dec 19, 2007 at 01:11:11AM +0000, Paul Brook wrote:
> > > > So I'm asking for a policy here that says when it is OK to resolve old
> > > > bug without progress as WONTFIX or SUSPENDED. Start shooting.
> > >
> > > I think this would be a big mistake to reuse an existing state for this.
> >
> > But this is pretty much what SUSPENDED means; it means that there's no
> > intent to work on the bug in the near term.
> 
> Ok. I did check the GCC bugzilla help pages, and they don't mention SUSPENDED 
> at all :-)

Patches welcome, as they say.

We have 91 bugs in the SUSPENDED state.  Many of them are odd corners of
the standard where we're waiting for the appropriate committee to decide
what the correct behavior is; others are ancient bugs that we've just
agreed to live with.  The oldest one relates to the issue of extra FPU
precision on x86.

Re: Regression count, and how to keep bugs around forever

[Joseph S. Myers]

On Wed, 19 Dec 2007, Steven Bosscher wrote:

> The bigger issue here, is that people seem to be using Bugzilla as a
> kind-of TODO list for things may some day work on, but probably will

I don't see any problem with that.  It records known issues.  We can then 
use the existing fields, or create new ones, to track how important we 
think those issues are.  I'd rather have known testsuite failures filed 
centrally in Bugzilla than scattered elsewhere.  (Once a testsuite failure 
is filed, it can be XFAILed with a comment referencing the PR, and the PR 
remain open until the bug is fixed and the XFAIL removed.  In principle 
I'd like every XFAIL relating to a GCC bug (as opposed to a bug in system 
libraries etc.) to have an open PR, and any failure present for any length 
of time to be XFAILed and filed in Bugzilla so the normal test results 
show no "expected unexpected" FAILs.)

> The current list of "All regressions" should be a list of bugs that
> people are actively trying to resolve, preferably before the release

No, "All regressions" should be a list of all regressions known.  A list 
of bugs people are actively trying to resolve is useful - indeed more 
useful - but should be called something else.

> To me, the situation is quite clear: If a bug report is open for so
> long, and even the reporter and the responsible maintainer show no
> sign of motivation to work on resolving the bug, I think this tells us
> something about how important this bug is: Not important enough to
> fix.  IMOH we should close such reports as WONTFIX or SUSPENDED to
> make them less visible, so that other bug reports don't fall through
> the cracks.
> 
> So I'm asking for a policy here that says when it is OK to resolve old
> bug without progress as WONTFIX or SUSPENDED. Start shooting.

I objected to the notion of WONTFIX six years ago when we were 
experimenting with Bugzilla, and I object to it now, with only the 
modification: if a bug relates to a feature that has been removed from 
trunk, then WONTFIX is reasonable, and likewise if it requests a fix in an 
old branch for a bug fixed on trunk where we think such a fix is too 
risky.

If a bug requests a feature we decide would be a bad idea, I think INVALID 
is right but don't really care if you use WONTFIX.

If a bug simply hasn't attracted attention, but is still a bug present in 
the current toolchain, it should be left open.  Just because one developer 
won't fix it doesn't justify a declaration that no-one will ever be 
interested in fixing it.

By design, SUSPENDED is an open status rather than a closed one.  Using it 
within the terms of the description in bugs/management.html makes sense - 
go ahead.

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: Regression count, and how to keep bugs around forever

[Joe Buck]

On Wed, Dec 19, 2007 at 01:25:19AM +0000, Paul Brook wrote:
> > Ok. I did check the GCC bugzilla help pages, and they don't mention SUSPENDED 
> > at all :-)

I wrote:
> Patches welcome, as they say.

Never mind; see
http://gcc.gnu.org/bugs/management.html

for when to use SUSPENDED.

RE: Regression count, and how to keep bugs around forever

[Weddington, Eric]

 

> -----Original Message-----
> From: Steven Bosscher [mailto:stevenb.gcc@gmail.com] 
> Sent: Tuesday, December 18, 2007 6:00 PM
> To: GCC
> Cc: hp@gcc.gnu.org
> Subject: Regression count, and how to keep bugs around forever
> 
> Maybe it is just me, but I find it very annoying to have to wade
> through long bug lists, so I just don't do this. Instead I just don't
> look at P4/P5 regressions anymore at all. It's just too much trouble
> to find a bug report where the reporter or the target maintainer cares
> as much as you do about resolving the bug.
 
I understand your frustration.

I know that some AVR target bugs might fit in your category that you
describe. However a few of us are working to resolve the situation at
least for the AVR target. If you any problems, issues, complaints, or
patches ;-) about/for the AVR target, please let me know so they can be
addressed. Thanks.

Eric Weddington
Product Manager
Atmel

Re: Regression count, and how to keep bugs around forever

[David Daney]

Joe Buck wrote:
> On Wed, Dec 19, 2007 at 01:25:19AM +0000, Paul Brook wrote:
>   
>>> Ok. I did check the GCC bugzilla help pages, and they don't mention SUSPENDED 
>>> at all :-)
>>>       
>
> I wrote:
>   
>> Patches welcome, as they say.
>>     
>
> Never mind; see
> http://gcc.gnu.org/bugs/management.html
>
> for when to use SUSPENDED.
>
>   
Perhaps we need a new state like: DORMANT -- Like new, but nobody cares
enough to do anything about it.

Really this is like NEW P5, so perhaps no new state is needed.

David Daney

Re: Regression count, and how to keep bugs around forever

[Manuel López-Ibáñez]

On 19/12/2007, Steven Bosscher <stevenb.gcc@gmail.com> wrote:
>
> The current list of "All regressions" should be a list of bugs that
> people are actively trying to resolve, preferably before the release
> of GCC 4.3. Instead, it is a mix of high-activity bug reports and bug
> reports where even the target maintainer has been unwilling for 3.5
> years to spend some time on resolving the bug report. So to pick a bug
> report to work on, I need to go through the but report summaries of a
> long list, trying to pick out new regressions between the old
> no-one-cares P4 and P5 regressions.
>

I am sorry but I don't understand how this can be possible. Old
no-one-cares have a lower ID than new ones. So if you start with the
list backwards you should always get the newer ones. Also, PRs that
are regressions for 4.3 only cannot be that old (but perhaps they are
no-one-cares).

On the other hand, there are around 1003 PRs UNCONFIRMED. Those are annoying.

> Maybe it is just me, but I find it very annoying to have to wade
> through long bug lists, so I just don't do this. Instead I just don't
> look at P4/P5 regressions anymore at all. It's just too much trouble
> to find a bug report where the reporter or the target maintainer cares
> as much as you do about resolving the bug.

Well, perhaps instead of 2 lists: Serious regressions and All
regressions. We should have 3 lists: High priority, Medium Priority,
Low priority. High priority is the same as Serious regressions, Medium
are P4 and P5 and Low priority are those that you just described
(P6?).

Anyway, I don't typically look at those lists. I create my own
customized searches and save them.

Cheers,

Manuel.

Re: Regression count, and how to keep bugs around forever

[Rask Ingemann Lambertsen]

On Wed, Dec 19, 2007 at 01:59:51AM +0100, Steven Bosscher wrote:
> The current list of "All regressions" should be a list of bugs that
> people are actively trying to resolve, preferably before the release
> of GCC 4.3.

   No, it should be exactly what it says it is. If you want an additional
list of bugs that are being actively worked on (and labelled as such),
that's fine (although I have no idea how that list would be useful).

> Instead, it is a mix of high-activity bug reports and bug
> reports where even the target maintainer has been unwilling for 3.5
> years to spend some time on resolving the bug report.

   That may be an indication that maintainership should be passed on to
someone else. I don't see how it can be an indication that the bug should
not be fixed.

> So to pick a bug
> report to work on, I need to go through the but report summaries of a
> long list, trying to pick out new regressions between the old
> no-one-cares P4 and P5 regressions.

   PR numbers are assigned in ascending order. The newest regressions have
the highest numbers. What exactly is the problem you're facing when starting
with the highest-numbered PRs?

> To me, the situation is quite clear: If a bug report is open for so
> long, and even the reporter and the responsible maintainer show no
> sign of motivation to work on resolving the bug, I think this tells us
> something about how important this bug is: Not important enough to
> fix.  IMOH we should close such reports as WONTFIX or SUSPENDED to
> make them less visible, so that other bug reports don't fall through
> the cracks.
> 
> So I'm asking for a policy here that says when it is OK to resolve old
> bug without progress as WONTFIX or SUSPENDED. Start shooting.

   Having assigned myself to and/or posted patches for some of the bugs you
want to close as WONTFIX, including four which have four-digit PR numbers,
my response is predictable: No way.

-- 
Rask Ingemann Lambertsen
Danish law requires addresses in e-mail to be logged and stored for a year

Re: Regression count, and how to keep bugs around forever

[Steven Bosscher]

On Dec 19, 2007 4:32 PM, Rask Ingemann Lambertsen <rask@sygehus.dk> wrote:
> On Wed, Dec 19, 2007 at 01:59:51AM +0100, Steven Bosscher wrote:
> > The current list of "All regressions" should be a list of bugs that
> > people are actively trying to resolve, preferably before the release
> > of GCC 4.3.
>
>    No, it should be exactly what it says it is.

I don't mind renaming it ;-)

> If you want an additional
> list of bugs that are being actively worked on (and labelled as such),
> that's fine (although I have no idea how that list would be useful).

Let's take a bug as an example case: http://gcc.gnu.org/23835

Here, there is a bug report about a huge compile time increase.  The
release manager decided  that this was not a release blocker for GCC
4.2.  So it was marked P4, and it disappeared from the radar for GCC
4.3 for everyone who only looks at the "Serious regressions".

Gr.
Steven

Re: Regression count, and how to keep bugs around forever

[Rask Ingemann Lambertsen]

On Wed, Dec 19, 2007 at 10:17:00PM +0100, Steven Bosscher wrote:
> On Dec 19, 2007 4:32 PM, Rask Ingemann Lambertsen <rask@sygehus.dk> wrote:
> 
> > If you want an additional
> > list of bugs that are being actively worked on (and labelled as such),
> > that's fine (although I have no idea how that list would be useful).
> 
> Let's take a bug as an example case: http://gcc.gnu.org/23835
> 
> Here, there is a bug report about a huge compile time increase.  The
> release manager decided  that this was not a release blocker for GCC
> 4.2.  So it was marked P4, and it disappeared from the radar for GCC
> 4.3 for everyone who only looks at the "Serious regressions".

   Right. I tend to look at the list of "All regressions" for that reason. I
have also bookmarked about 20 PRs that I'm likely to work on. It does not
come as a surprice to me that one size doesn't fit all. The list of "Serious
regressions" just gives you a peek over the release manager's shoulders. I
use it mainly as an indication of how far away regressions only mode is.

-- 
Rask Ingemann Lambertsen
Danish law requires addresses in e-mail to be logged and stored for a year

Re: Regression count, and how to keep bugs around forever

[NightStrike]

On 12/19/07, Steven Bosscher <stevenb.gcc@gmail.com> wrote:
> Let's take a bug as an example case: http://gcc.gnu.org/23835
>
> Here, there is a bug report about a huge compile time increase.  The
> release manager decided  that this was not a release blocker for GCC
> 4.2.  So it was marked P4, and it disappeared from the radar for GCC
> 4.3 for everyone who only looks at the "Serious regressions".

Under this system, do P4's and P5's ever get fixed?

Re: Regression count, and how to keep bugs around forever

[David Daney]

NightStrike wrote:
> On 12/19/07, Steven Bosscher <stevenb.gcc@gmail.com> wrote:
>   
>> Let's take a bug as an example case: http://gcc.gnu.org/23835
>>
>> Here, there is a bug report about a huge compile time increase.  The
>> release manager decided  that this was not a release blocker for GCC
>> 4.2.  So it was marked P4, and it disappeared from the radar for GCC
>> 4.3 for everyone who only looks at the "Serious regressions".
>>     
>
> Under this system, do P4's and P5's ever get fixed?
>   
Under the existing system *no* bugs get fixed unless someone wants to 
fix them.

And to answer your question:

http://gcc.gnu.org/bugzilla/buglist.cgi?query_format=advanced&short_desc_type=allwordssubstr&short_desc=&known_to_fail_type=allwordssubstr&known_to_work_type=allwordssubstr&long_desc_type=substring&long_desc=&bug_file_loc_type=allwordssubstr&bug_file_loc=&gcchost_type=allwordssubstr&gcchost=&gcctarget_type=allwordssubstr&gcctarget=&gccbuild_type=allwordssubstr&gccbuild=&keywords_type=allwords&keywords=&bug_status=RESOLVED&bug_status=VERIFIED&bug_status=CLOSED&priority=P4&priority=P5

Re: Regression count, and how to keep bugs around forever

[Mark Mitchell]

Joseph S. Myers wrote:
> On Wed, 19 Dec 2007, Steven Bosscher wrote:
> 
>> The bigger issue here, is that people seem to be using Bugzilla as a
>> kind-of TODO list for things may some day work on, but probably will
> 
> I don't see any problem with that.

Me neither.  In fact, I think there's a lot of value in a central
database of "all known bugs"; it helps users figure out whether
something they're running into is something they've run into before.

However, I am sympathetic to the idea that we need ways to see what's
important to work on now.  The problem there is that importance is in
the eye of the beholder.  The PN system expressions something about
regressions that's moderately useful, but nothing else.  I suspect that
we need more database fields, so that people could run more interesting
searches.

-- 
Mark Mitchell
CodeSourcery
mark@codesourcery.com
(650) 331-3385 x713