Re: release branch policy and distributions

[Carlos O'Donell <carlos@redhat.com>]

On 2/16/23 17:57, Michael Hudson-Doyle wrote:
> Would it be unreasonable to suggest a policy where performance improvements
> are not backported to release branches until say a month after they have
> been included in a glibc release? I realize this would add some overhead to
> keep track of these 'pending' backports but I personally would be happier
> consuming the release branches directly if there was this sort of policy.

Michael, Andreas, Adhemerval,

Thank you all for raising this.

I want to talk about outcomes.

The outcome I want is for there to be fewer defects in the development branch,
and by proxy fewer defects in the release branch.

(1) Do we have evidence of an increased rate of defects?

I know we have some anecdotal evidence that we recently had defects in the
rolling release branches. Have we collected that evidence to determine what
kind of action is required? Do we have a gap in our hardware or testing that
needs to be improved?

A gap here could be that we need to setup x86_64 pre-commit CI with an AVX512
system to test all the IFUNCs (which may catch nothing if the tests are missing).

Another gap here could be that we need to setup pre-commit CI to rebuild certain
packages under the modified glibc (similar to Fedora Rawhide CI).

(2) What is your distro policy for updating from the rolling release branch?

While upstream gives a policy, what is your own policy?

Example: Fedora Rawhide CI rebuilds a number of packages using the new glibc
we sync weekly, and we review the rebuild failures and their testsuite results
before putting the new glibc into Rawhide (or stable Fedora releases).
For example, rebuilding lua and running their testsuite, particularly the string
testsuite is good at detecting further string-related optimization defects.

(3) How does delaying backports impact our outcome?

One way is that we use this extra time to do additional testing that discovers
defects, and then we work with the machine maintainer, IHVs, etc, and correct
the defects.

This means that the action we want to take is not delaying, but some kind of
increase in testing. In fact delaying may solve nothing if additional
validation and verification is not carried out in that delay period.

My opinion is that delaying alone is not an outcome changing activity, and as
a steward for the project I do not want to delay code from reach our users
unless we can show that delay allowed the users to capture some value e.g.
higher stability.

How could Canonical, Gentoo or Linaro support additional upstream testing?

Can we work together to turn on more distro-specific pre-commit CI testing?

We have patchwork, it has a REST API, and we can submit test results via that
API, like we do today for i686 test results (Fedora Rawhide-based).

-- 
Cheers,
Carlos.