Re: Labelling of regressions in Bugzilla

[Jonathan Wakely <jwakely.gcc@gmail.com>]

On Wed, 15 Dec 2021 at 12:15, Richard Earnshaw wrote:
>
>
>
> On 15/12/2021 11:39, Jonathan Wakely via Gcc wrote:
> > On IRC we've been discussing some changes to Bugzilla that would give
> > a bit more structure to how we label and process regressions.
> >
> > Currently we add something like "[9/10/11/12 Regression]" to the start
> > of the summary, and then edit that when it's fixed on a branch, when
> > forking a new release branch from trunk, and when the oldest branch is
> > closed and becomes unmaintained. Finding active regressions means a
> > free-text search in the summary field.
> >
> > On IRC we discussed adding a new custom field to record which branches
> > a regression affects. This could be like the
> > known-towork/known-to-fail fields, so only allow predefined values,
> > and allow searching by "all of" and by "any of". The possible values
> > for the field would only be the major releases, not all the minor
> > releases and snapshots and vendor branches that are in the
> > known-to-work field. So just 4.9, 5, 6, 7 etc. not 4.9.4, 5.0, 5.1.0,
> > 5.1.1 etc.
> >
> > When a new branch is forked from trunk before a release all bugs that
> > have "trunk" in the regression field would automatically get "12"
> > added (or if we already used "12" instead of "trunk" they'd get "13"
> > added, either way would work).
> >
> > Unlike the current system, we wouldn't need to remove closed branches
> > from the regressions field. We do that today so the Summary field
> > doesn't get cluttered with old branch info, but if it's in a separate
> > field keeping the old data present is valuable. We would only remove a
> > branch from that field when the regression is fixed on the branch. We
> > would still be able to search for regressions in active branches
> > easily, but it would also be possible to see at a glance that a given
> > regression was present on the gcc-8 branch and never fixed there. This
> > would also help vendors who maintain older branches, as the
> > information that a regression affected an old branch would not be
> > wiped out of the summary when the branch reaches EOL.
> >
> > Jakub also suggested it would be nice to be able to enter a revision
> > into a "regressed at" field and have it auto-populate the regressions
> > list with all branches that the commit is present in. (Ideally any of
> > SVN rNNNN numbers, or git revisions, or gcc-descr rNN-NNN strings
> > could be used there). That would be useful when we bisect the
> > regression and find where it started.
> >
> > Iain pointed out a drawback of not having the regression info in the
> > Summary. Currently it does draw your attention when looking at the
> > results of a bugzilla search. Andrew noted that bug aliases are
> > automatically added to the summary, e.g. https://gcc.gnu.org/PR94404
> > shows its alias "(c++core-issues)". Maybe we could do that for
> > regressions (for the active branches only, so the result would be
> > similar to what we have today).
> >
> > Thoughts? Objections? Better ideas?
> >
>
> My immediate thought (since I tend to dislike deleting history) is why
> not have two fields?  One listing all the release branches where this
> has occurred and another for where it has now been fixed.  That way you
> can see quickly whether the regression has ever affected some versions
> of a release.

The Known To Fail field could be used for that, if we were more
diligent about using it when fixes are backported to a branch.

> Something we lack today with the single fixed in field is
> the ability to track exactly which dot releases of each branch contained
> the fix for a regression.

Yes, I use Target Milestone for the oldest dot release and then add
something like "Fixed for 9.4, 10.3, 11.2 and trunk" in the comments.
That could be more structured.

>
> Other than that, I have no other concerns at the moment.

Thanks for the feedback!

I think my concern with the idea is that when we make this info more
structured, there's a danger of giving ourselves more work to do
pushing buttons just for the sake of it. I think it's OK to make it
optional to use these fields, so that those who want to record the
rich info (e.g. me, maybe you, and Andrew Pinski when he goes on a
bugzilla tidying mission :-) can do so. The danger then is that using
them for search isn't reliable because the fields aren't used
consistently.

I think I'd still rather than the option to use them. At least for my
little corner of the project, I would aim to fill them in
consistently.