Glibc stable release process (Glibc 2.26.1)

Glibc stable release process (Glibc 2.26.1)

[Romain Naour]

Hello All,

As suggested by Siddhesh Poyarekar in BZ-22146 [0], I'd like to continue the
discussion about tagging a 2.26.1 release.

Glibc 2.26 introduced some regressions (notably BZ-21930 and BZ-22146) on major
architectures (x86 and x86_64) that are already fixed in the stable branch.
Without them, we can't compile C++ any code using mathematical functions (ex:
std::fpclassify() when libstdc++ is compiled with -Os).

Siddhesh Poyarekar said that is no plan for a new release since most
distributions prefer to backport patches. But for downstream users like build
tools (Buildroot, crosstool-ng, Yocto...) that use the release archives, it
means that a lot of patches need to be backported (42 at the time of writing).

However, the point is not about choosing what commit to cherry-pick.
From the downstream perspective, a commit on the maintenance branch is always
worth it. If it were not needed, it would not have been committed to the
maintenance branch in the first place.

Gabriel F. T. Gomes suggested the use of a git clone from a given hash instead
but, as explained by Yann E. Morin, using "2.26.1" is much more descriptive than
using a random hash from the stable branch.

A new dot-release is a strong indication that the most critical fixes have made
their way to the maintenance branch, and that it has been officially sanctioned
by upstream (you! ;-)), which is a very important status for downstreams.


We do understand that getting a new release out can be quite some work, and we
do acknowledge the efforts that are made to provide those releases.

Yet, being able to refer to human-readable versions is very important.

Maybe a middle ground is that a tag is pushed to the repository, just to
identify a specific sha1 as "this is 2.26.1". This is relatively lightweight (a
signed tag) and gives downstream users that need it a clear identification of a
blessed version.


Thank you!

[0] https://sourceware.org/bugzilla/show_bug.cgi?id=22146

Best regards,
Romain Naour

Re: Glibc stable release process (Glibc 2.26.1)

[Tulio Magno Quites Machado Filho]

Romain Naour <romain.naour@gmail.com> writes:

> However, the point is not about choosing what commit to cherry-pick.
> From the downstream perspective, a commit on the maintenance branch is always
> worth it. If it were not needed, it would not have been committed to the
> maintenance branch in the first place.
>
> Gabriel F. T. Gomes suggested the use of a git clone from a given hash instead
> but, as explained by Yann E. Morin, using "2.26.1" is much more descriptive than
> using a random hash from the stable branch.

It isn't random.  It helps to track all the objects a point of history has
and all its parent commits.
That information may not be important for everyone, but it is valuable for
some users.

It also helps downstream to follow the "upstream first" ideology. ;-)

> A new dot-release is a strong indication that the most critical fixes have made
> their way to the maintenance branch, and that it has been officially sanctioned
> by upstream (you! ;-)), which is a very important status for downstreams.

This behavior varies according to the community.  On glibc, a patch is only
backported to stable branches if it's already reviewed and accepted on master
and if there is consensus they're suitable to a stable branch [1].
You can track these discussions in the libc-stable mailing list. [2]

It doesn't affect only glibc 2.26.  If there are older releases downstream
not following a glibc stable branch which aren't cherry-picking backported
patches I strongly advise to start doing so in order to get bugs fixes and
security fixes because the last glibc point release was 2.14.1 in 2011.

I'm not opposing to a point release, but considering how the glibc community
has been working, I don't think it helps to make it more stable, reviewed or
officially sanctioned.

[1] https://sourceware.org/glibc/wiki/Release/#General_policy
[2] https://sourceware.org/ml/libc-stable/

-- 
Tulio Magno

Re: Glibc stable release process (Glibc 2.26.1)

[Paul Eggert]

Romain Naour wrote:
> Maybe a middle ground is that a tag is pushed to the repository, just to
> identify a specific sha1 as "this is 2.26.1". This is relatively lightweight (a
> signed tag) and gives downstream users that need it a clear identification of a
> blessed version.
That "blessing" would be problematic. How can we be expected to "bless" any 
particular commit unless we've checked it? And that checking will take some 
work, work that will slow us down on other things.

Instead of blessing, how about using the output of something like the following 
shell command to generate the version number?

git describe --match 'glibc-*' --abbrev=7 --dirty

This will generate a string that identifies the commit but is shorter and easier 
to read than an SHA hash, and so is more likely to give users that warm and 
fuzzy feeling. On my glibc repository right now, for example, it generates:

glibc-2.26-415-g4d3693e

which means there have been 415 commits since 2.26, and the latest commit can be 
abbreviated 4d3693e. A similar (but shorter) version-numbering scheme is used by 
GNU Coreutils and other GNU packages when between releases, and Gnulib has a 
script build-aux/git-version-gen that can generate it.

Re: Glibc stable release process (Glibc 2.26.1)

[Arjan van de Ven]

On 9/29/2017 1:17 PM, Romain Naour wrote:
> Hello All,
> 
> As suggested by Siddhesh Poyarekar in BZ-22146 [0], I'd like to continue the
> discussion about tagging a 2.26.1 release.
> 
> Glibc 2.26 introduced some regressions (notably BZ-21930 and BZ-22146) on major
> architectures (x86 and x86_64) that are already fixed in the stable branch.
> Without them, we can't compile C++ any code using mathematical functions (ex:
> std::fpclassify() when libstdc++ is compiled with -Os).
> 
> Siddhesh Poyarekar said that is no plan for a new release since most
> distributions prefer to backport patches. But for downstream users like build
> tools (Buildroot, crosstool-ng, Yocto...) that use the release archives, it
> means that a lot of patches need to be backported (42 at the time of writing).

as someone who does glibc for a distro... I strongly prefer tarbal releases.
There's a lot less ambiguity in terms of what is running, and distros generally
are set up to take tarbals anyway....

taking a patch or two is fine, but doing this over and over again makes it less
obvious what exact stack is running etc.

Re: Glibc stable release process (Glibc 2.26.1)

[Yann E. MORIN]

Tulio, Paul, Arjan, All,

Thanks for the fast replies. :-)

[Replying with Romain, here, as we wrote the query together]

I have also interspersed Tulio's, Paul's, and Arjan's replies...

On 2017-09-29 18:38 -0300, Tulio Magno Quites Machado Filho spake thusly:
> Romain Naour <romain.naour@gmail.com> writes:
> > However, the point is not about choosing what commit to cherry-pick.
> > From the downstream perspective, a commit on the maintenance branch is always
> > worth it. If it were not needed, it would not have been committed to the
> > maintenance branch in the first place.
> >
> > Gabriel F. T. Gomes suggested the use of a git clone from a given hash instead
> > but, as explained by Yann E. Morin, using "2.26.1" is much more descriptive than
> > using a random hash from the stable branch.
> 
> It isn't random. [snip]

Of course we know what a sha1 in a git tree means, thank you. ;-)

What Romain and I were trying to say was that we would have to
arbitrarily choose the commit on the tree. That choice would look
random.

Let's say that we decide to use today's last commit on the branch, which
is d37c951fde57e8acb320a9a7d437ba50a1fc3c8a. That's all good. But then
tomorrow, new commits get pushed to the maintenance branch, and now the
head of the same branch is 548cc83c38a91852b1e44045ead3d20ccd5db4cf
(it is now fdf58ebc60ce0eb459fd616241b52872b3571ac1; writing this mail
took much longer than I anticipated ;-] ).

So in three days time, our "choice" would look no better as if we had
randomly choosen a commit on the tree. There is nothing that makes that
one stand out more than the ones before or after. Except that at some
point in time, it was the latest. But that is useless past that very
moment.

The problem with using a sha1 is two fold. First, it is not
human-readable. While I can pretty easily wrap my head around what
2.26.1 means, d37c951fde57e8acb320a9a7d437ba50a1fc3c8a is defintiely
opaque for me.

Second, because it is opaque, any commit on a branch is semantically
equivalent to another. So, there is nothing that says "this is pretty
good, you can use it".

The alternative, that only guarantees the former, but does not help on
the latter, is to cary the now-42 patches from the branch locally.

This is ugly, because it is very easily prone to bitrot, and like
explained above, would be missing later patches, so the set of patch
would look no less randomly chosen that chosing a sha1 would be.

> That information may not be important for everyone, but it is valuable for
> some users.

And I can assure you that we do find this information very important.

> It also helps downstream to follow the "upstream first" ideology. ;-)

Here, it is just about consuming a release. Contribution is another
important topic, but that's not the point here.

Release tag or git sha1, is not really what would prevent a downstream
from carying local patches. But that's a tangent...

> > A new dot-release is a strong indication that the most critical fixes have made
> > their way to the maintenance branch, and that it has been officially sanctioned
> > by upstream (you! ;-)), which is a very important status for downstreams.
> 
> This behavior varies according to the community.  On glibc, a patch is only
> backported to stable branches if it's already reviewed and accepted on master
> and if there is consensus they're suitable to a stable branch [1].

I think that what we wrote is basically that: patches on the maintenance
branch have a reason to be there, so we as a downstream do not have to
cherry-pick those. We want to use all of them, because you already did a
good job at cherry-picking them from master! :-)



On 2017-09-29 17:26 -0700, Paul Eggert spake thusly:
> Romain Naour <romain.naour@gmail.com> writes:
> > Maybe a middle ground is that a tag is pushed to the repository, just to
> > identify a specific sha1 as "this is 2.26.1". This is relatively lightweight (a
> > signed tag) and gives downstream users that need it a clear identification of a
> > blessed version.
> That "blessing" would be problematic. How can we be expected to "bless" any
> particular commit unless we've checked it? And that checking will take some
> work, work that will slow us down on other things.

Yet, every commit that went on the branch has already been blessed, by
the very process that allowed it to be cherry-picked from master.

So, tagging the branch can happen almost at any time: it only contains
bug fixes that have already been thoroughly reviewed, as Tulio explained.

> Instead of blessing, how about using the output of something like the
> following shell command to generate the version number?
> 
> git describe --match 'glibc-*' --abbrev=7 --dirty
[--SNIP--]
> glibc-2.26-415-g4d3693e

This is indeed a nice representation. But it is not stable with time,
because 7 is not enough to uniquely identify a commit. Even it it were
today, it may no longer be the case in the future, killing
reproducibility.

But the choice of what commit to use is no less arbitrary than the
alternatives, above.

> [snip] A similar (but shorter) version-numbering scheme is
> used by GNU Coreutils and other GNU packages when between releases, and Gnulib
> has a script build-aux/git-version-gen that can generate it.

binutils and gcc, as well as the Linux kernel (for headers), which are
all other components of a toolchain, all do provide dot-releases. ;-)



On 2017-09-29 18:38 -0300, Tulio Magno Quites Machado Filho spake thusly:
> You can track these discussions in the libc-stable mailing list. [2]
> 
> It doesn't affect only glibc 2.26.  If there are older releases downstream
> not following a glibc stable branch which aren't cherry-picking backported
> patches I strongly advise to start doing so in order to get bugs fixes and
> security fixes because the last glibc point release was 2.14.1 in 2011.

That is unrelated. The point is not about cherry-picking individual patches.

Consider that not all downstreams have the manpower of a Redhat, a
Debian, a Fedora or a Ubuntu (to name a few). By providing regular
dot-releases, you also help those downstreams to update.



On 2017-09-29 17:38 -0700, Arjan van de Ven spake thusly:
> as someone who does glibc for a distro... I strongly prefer tarbal
> releases.

May we know what distribution you are refering to?

> taking a patch or two is fine, but doing this over and over again makes
> it less obvious what exact stack is running etc.

Exactly.

Cherry-picking individual patches from the stable branch also risk
missing critical fixes, or introducing regressions, as a fix may rely on
a previous one, and all sort of hard-to-see interactions between fixes.



On 2017-09-29 18:38 -0300, Tulio Magno Quites Machado Filho spake thusly:
> I'm not opposing to a point release, but considering how the glibc community
> has been working, I don't think it helps to make it more stable, reviewed or
> officially sanctioned.
> 
> [1] https://sourceware.org/glibc/wiki/Release/#General_policy

So, lemme quote this:

    Each branch has so-called interested parties, usually glibc maintainers
    in distributions where the particular branch is being used; tagging
    revisions on the release branches should result of consensus between the
    maintainer and interested parties - one workable model is that the
    maintainer suggests that he wants to release and other people check if
    they are happy with the set of patches included and the timing is
    fine-tuned; if a release is important for one of the parties (e.g.
    distribution nearing a release), they can suggest a release of new
    revision as well if it is meaningful. 

Is there a way to register the Buildroot project as an interested party?

As an interested party, we Buildroot would like to suggest that a new
revision be tagged, as it is meaningful to us:

  - at least two major regressions on two major architectures have been
    fixed, namely BZ-21930 and BZ-22146,

  - we are nearing the end of our development cycle (freeze end of
    October) and would benefit from a human-readable version tag.

(Just tell me what rules to play by, and I'll happily play by those
rules! ;-] )

Yes, we are asking you to put in a bit more of efforts to help us, that
is true. But we still believe that pushing a tag is not that much. ;-)

Anyway, we do appreciate that this is your decision to take.

Thank you for reading so far! ;-)

Regards,
Yann E. MORIN, Romain Naour.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

Re: Glibc stable release process (Glibc 2.26.1)

[Zack Weinberg]

On Sat, Sep 30, 2017 at 6:18 AM, Yann E. MORIN <yann.morin.1998@free.fr> wrote:
>
> What Romain and I were trying to say was that we would have to
> arbitrarily choose the commit on the tree. That choice would look
> random.
>
> Let's say that we decide to use today's last commit on the branch, which
> is d37c951fde57e8acb320a9a7d437ba50a1fc3c8a. That's all good. But then
> tomorrow, new commits get pushed to the maintenance branch, and now the
> head of the same branch is 548cc83c38a91852b1e44045ead3d20ccd5db4cf
> (it is now fdf58ebc60ce0eb459fd616241b52872b3571ac1; writing this mail
> took much longer than I anticipated ;-] ).
>
> So in three days time, our "choice" would look no better as if we had
> randomly choosen a commit on the tree. There is nothing that makes that
> one stand out more than the ones before or after. Except that at some
> point in time, it was the latest. But that is useless past that very
> moment.
[...]

I'm a little underslept and I'm not sure I fully understand the issue
here, but would it help if we literally just tagged point releases and
pushed tarballs to ftp.gnu.org from a cron job?  Once a month if there
have been any patches since the previous tag, perhaps?  With the
official line being that all patches on the release branches are
carefully vetted and we recommend tracking the git branch if you can,
but this is easier for some downstream organizations so we offer this
as well.

zw

Re: Glibc stable release process (Glibc 2.26.1)

[Arjan van de Ven]

On 9/30/2017 4:57 AM, Zack Weinberg wrote:
> 
> I'm a little underslept and I'm not sure I fully understand the issue
> here, but would it help if we literally just tagged point releases and
> pushed tarballs to ftp.gnu.org from a cron job?  Once a month if there
> have been any patches since the previous tag, perhaps?  With the
> official line being that all patches on the release branches are
> carefully vetted and we recommend tracking the git branch if you can,
> but this is easier for some downstream organizations so we offer this
> as well.

with my distro hat on, yes I would appreciate this already a lot.
I'll consume these (and likely other distros will as well) as very
good anchor points.

It also leads to, say, a CVE be able to list "fixed in 2.26.5"
and everyone (and all more importantly, all tools that we all use
to cross reference our distros to CVE databases) will know if things
are already fixed, or if someone needs to take a look for a fix
to backport outside of the releases

Re: Glibc stable release process (Glibc 2.26.1)

[Arjan van de Ven]

On 9/30/2017 3:18 AM, Yann E. MORIN wrote:
> 
> On 2017-09-29 17:38 -0700, Arjan van de Ven spake thusly:
>> as someone who does glibc for a distro... I strongly prefer tarbal
>> releases.
> 
> May we know what distribution you are refering to?

Clear Linux (http://www.clearlinux.org)

Re: Glibc stable release process (Glibc 2.26.1)

[Florian Weimer]

On 09/30/2017 04:27 PM, Arjan van de Ven wrote:
> It also leads to, say, a CVE be able to list "fixed in 2.26.5"
> and everyone (and all more importantly, all tools that we all use
> to cross reference our distros to CVE databases) will know if things
> are already fixed, or if someone needs to take a look for a fix
> to backport outside of the releases

There aren't supposed to be any merges on release branches, and we 
should enforce that with an update hook (we currently don't).

If we do that and tag the master branch right after it is opened for 
development, then you could use “git describe” to generate unique 
version numbers for every commit on a stable branch.

Publishing tarballs just to turn 2.26-40 into 2.26.5 is a waste of 
resources, in my opinion.

Thanks,
Florian

Re: Glibc stable release process (Glibc 2.26.1)

[Yann E. MORIN]

Zack, All,

On 2017-09-30 07:57 -0400, Zack Weinberg spake thusly:
> On Sat, Sep 30, 2017 at 6:18 AM, Yann E. MORIN <yann.morin.1998@free.fr> wrote:
> > What Romain and I were trying to say was that we would have to
> > arbitrarily choose the commit on the tree. That choice would look
> > random.
> >
> > Let's say that we decide to use today's last commit on the branch, which
> > is d37c951fde57e8acb320a9a7d437ba50a1fc3c8a. That's all good. But then
> > tomorrow, new commits get pushed to the maintenance branch, and now the
> > head of the same branch is 548cc83c38a91852b1e44045ead3d20ccd5db4cf
> > (it is now fdf58ebc60ce0eb459fd616241b52872b3571ac1; writing this mail
> > took much longer than I anticipated ;-] ).
> >
> > So in three days time, our "choice" would look no better as if we had
> > randomly choosen a commit on the tree. There is nothing that makes that
> > one stand out more than the ones before or after. Except that at some
> > point in time, it was the latest. But that is useless past that very
> > moment.
> [...]
> 
> I'm a little underslept and I'm not sure I fully understand the issue
> here, but would it help if we literally just tagged point releases and
> pushed tarballs to ftp.gnu.org from a cron job?  Once a month if there
> have been any patches since the previous tag, perhaps?

With my and Romain's Buildroot hats on, yes, that would be awesome! :-)

> With the
> official line being that all patches on the release branches are
> carefully vetted and we recommend tracking the git branch if you can,
> but this is easier for some downstream organizations so we offer this
> as well.

Totally in line with you on this.

Thank you! :-)

Regards,
Yann E. MORIN, Romain Naour.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

Re: Glibc stable release process (Glibc 2.26.1)

[Andreas K. Huettel]

[-- Attachment #1: Type: text/plain, Size: 1526 bytes --]

Am Samstag, 30. September 2017, 02:38:53 CEST schrieb Arjan van de Ven:
> On 9/29/2017 1:17 PM, Romain Naour wrote:
> > Hello All,
> > 
> > As suggested by Siddhesh Poyarekar in BZ-22146 [0], I'd like to continue
> > the discussion about tagging a 2.26.1 release.
> > 
> > Glibc 2.26 introduced some regressions (notably BZ-21930 and BZ-22146) on
> > major architectures (x86 and x86_64) that are already fixed in the stable
> > branch. Without them, we can't compile C++ any code using mathematical
> > functions (ex: std::fpclassify() when libstdc++ is compiled with -Os).
> > 
> > Siddhesh Poyarekar said that is no plan for a new release since most
> > distributions prefer to backport patches. But for downstream users like
> > build tools (Buildroot, crosstool-ng, Yocto...) that use the release
> > archives, it means that a lot of patches need to be backported (42 at the
> > time of writing).
> as someone who does glibc for a distro... I strongly prefer tarbal releases.
> There's a lot less ambiguity in terms of what is running, and distros
> generally are set up to take tarbals anyway....
> 
> taking a patch or two is fine, but doing this over and over again makes it
> less obvious what exact stack is running etc.

+1 (another distro voice, Gentoo)

Even if we apply some patches downstream, having a common reference point, as 
e.g. a 2.26.1 release, would be nice. 

-- 
Dr. Andreas K. Hüttel
tel. +49 151 241 67748 (mobile)
e-mail mail@akhuettel.de
http://www.akhuettel.de/

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Glibc stable release process (Glibc 2.26.1)

[Andreas K. Huettel]

[-- Attachment #1: Type: text/plain, Size: 714 bytes --]

Am Samstag, 30. September 2017, 13:57:55 CEST schrieb Zack Weinberg:
> [...]
> 
> I'm a little underslept and I'm not sure I fully understand the issue
> here, but would it help if we literally just tagged point releases and
> pushed tarballs to ftp.gnu.org from a cron job?  Once a month if there
> have been any patches since the previous tag, perhaps?  With the
> official line being that all patches on the release branches are
> carefully vetted and we recommend tracking the git branch if you can,
> but this is easier for some downstream organizations so we offer this
> as well.

Yes please!!!

-- 
Andreas K. Hüttel
dilfridge@gentoo.org
Gentoo Linux developer (council, perl, libreoffice)

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Glibc stable release process (Glibc 2.26.1)

[Siddhesh Poyarekar]

On Saturday 30 September 2017 05:27 PM, Zack Weinberg wrote:
> I'm a little underslept and I'm not sure I fully understand the issue
> here, but would it help if we literally just tagged point releases and
> pushed tarballs to ftp.gnu.org from a cron job?  Once a month if there
> have been any patches since the previous tag, perhaps?  With the
> official line being that all patches on the release branches are
> carefully vetted and we recommend tracking the git branch if you can,
> but this is easier for some downstream organizations so we offer this
> as well.

That is probably a waste of resources and also not entirely secure since
it would preclude signing packages.

As a past Fedora maintainer, the feedback I got from a number of package
maintainers and testers in the Fedora community was that it was easier
to bisect bad patches when they were backported piece by piece as
opposed to looking at two tarballs, getting their tags, downloading
upstream sources, making scratch packages for them and then running
tests on them.  Given that Debian/Ubuntu follows a similar structure, I
suppose they would have similar problems.

That said, there seem to be at least 3 projects that seem to want this
(Gentoo, Clear Linux, Buildroot project) so I am inclined towards doing
a 2.26.1 point release for them unless anybody has a strong objection to
it.  I'll set the release date to somewhere in the middle of October
(I'm flying back home to India from the US West Coast, so I'm likely
going to be in zombie state for a few days and fighting backlog) and
look to add the aarch64 falkor routines to it as well once they're reviewed.

Siddhesh

Re: Glibc stable release process (Glibc 2.26.1)

[Andreas K. Huettel]

Am Sonntag, 1. Oktober 2017, 01:36:54 CEST schrieb Siddhesh Poyarekar:
> On Saturday 30 September 2017 05:27 PM, Zack Weinberg wrote:
> > I'm a little underslept and I'm not sure I fully understand the issue
> > here, but would it help if we literally just tagged point releases and
> > pushed tarballs to ftp.gnu.org from a cron job?  Once a month if there
> > have been any patches since the previous tag, perhaps?  With the
> > official line being that all patches on the release branches are
> > carefully vetted and we recommend tracking the git branch if you can,
> > but this is easier for some downstream organizations so we offer this
> > as well.
> 
> That is probably a waste of resources and also not entirely secure since
> it would preclude signing packages.
> 
> As a past Fedora maintainer, the feedback I got from a number of package
> maintainers and testers in the Fedora community was that it was easier
> to bisect bad patches when they were backported piece by piece as
> opposed to looking at two tarballs, getting their tags, downloading
> upstream sources, making scratch packages for them and then running
> tests on them.  Given that Debian/Ubuntu follows a similar structure, I
> suppose they would have similar problems.

To be honest, if I were a long-time glibc distro maintainer I'd probably agree 
with you and prefer hand-picking. Starting from a tag / tarball is something I 
prefer because I'm not that versed with things yet.

-- 
Andreas K. Hüttel
dilfridge@gentoo.org
Gentoo Linux developer (council, perl, libreoffice)

Re: Glibc stable release process (Glibc 2.26.1)

[Arjan van de Ven]

On 10/1/2017 12:59 PM, Andreas K. Huettel wrote:
> Am Sonntag, 1. Oktober 2017, 01:36:54 CEST schrieb Siddhesh Poyarekar:
>> On Saturday 30 September 2017 05:27 PM, Zack Weinberg wrote:
>>> I'm a little underslept and I'm not sure I fully understand the issue
>>> here, but would it help if we literally just tagged point releases and
>>> pushed tarballs to ftp.gnu.org from a cron job?  Once a month if there
>>> have been any patches since the previous tag, perhaps?  With the
>>> official line being that all patches on the release branches are
>>> carefully vetted and we recommend tracking the git branch if you can,
>>> but this is easier for some downstream organizations so we offer this
>>> as well.
>>
>> That is probably a waste of resources and also not entirely secure since
>> it would preclude signing packages.
>>
>> As a past Fedora maintainer, the feedback I got from a number of package
>> maintainers and testers in the Fedora community was that it was easier
>> to bisect bad patches when they were backported piece by piece as
>> opposed to looking at two tarballs, getting their tags, downloading
>> upstream sources, making scratch packages for them and then running
>> tests on them.  Given that Debian/Ubuntu follows a similar structure, I
>> suppose they would have similar problems.
> 
> To be honest, if I were a long-time glibc distro maintainer I'd probably agree
> with you and prefer hand-picking. Starting from a tag / tarball is something I
> prefer because I'm not that versed with things yet.
> 

I've been doing distro work for some 17 years now (including at RH way back);
while the packaging side of it is sort of a noop (you tool that anyway), the clarity
and unambiguity of version numbers (in light of CVEs) is something users like, it
sets a common baseline they don't have to look further at.

bisecting glibc patches outside of distro packages is mostly beyond end users'
capability.

Re: Glibc stable release process (Glibc 2.26.1)

[Tulio Magno Quites Machado Filho]

"Yann E. MORIN" <yann.morin.1998@free.fr> writes:

> Let's say that we decide to use today's last commit on the branch, which
> is d37c951fde57e8acb320a9a7d437ba50a1fc3c8a. That's all good. But then
> tomorrow, new commits get pushed to the maintenance branch, and now the
> head of the same branch is 548cc83c38a91852b1e44045ead3d20ccd5db4cf
> (it is now fdf58ebc60ce0eb459fd616241b52872b3571ac1; writing this mail
> took much longer than I anticipated ;-] ).
>
> So in three days time, our "choice" would look no better as if we had
> randomly choosen a commit on the tree. There is nothing that makes that
> one stand out more than the ones before or after. Except that at some
> point in time, it was the latest. But that is useless past that very
> moment.

Tarballs and tags will have the same problem.

IMHO, the only solution to this is to streamline the process of distributing
the source from upstream to downstream.

> The alternative, that only guarantees the former, but does not help on
> the latter, is to cary the now-42 patches from the branch locally.
>
> This is ugly, because it is very easily prone to bitrot, and like
> explained above, would be missing later patches, so the set of patch
> would look no less randomly chosen that chosing a sha1 would be.

Agreed, although I still believe that carrying a tarball without extra patches
will cause the same issues.

> On 2017-09-29 18:38 -0300, Tulio Magno Quites Machado Filho spake thusly:
>> I'm not opposing to a point release, but considering how the glibc community
>> has been working, I don't think it helps to make it more stable, reviewed or
>> officially sanctioned.
>> 
>> [1] https://sourceware.org/glibc/wiki/Release/#General_policy
>
> So, lemme quote this:
>
>     Each branch has so-called interested parties, usually glibc maintainers
>     in distributions where the particular branch is being used; tagging
>     revisions on the release branches should result of consensus between the
>     maintainer and interested parties - one workable model is that the
>     maintainer suggests that he wants to release and other people check if
>     they are happy with the set of patches included and the timing is
>     fine-tuned; if a release is important for one of the parties (e.g.
>     distribution nearing a release), they can suggest a release of new
>     revision as well if it is meaningful. 
>
> Is there a way to register the Buildroot project as an interested party?

https://sourceware.org/glibc/wiki/Release/#Distribution_Branch_Mapping

But I also suggest to subscribe to https://sourceware.org/ml/libc-stable/ 

-- 
Tulio Magno

Re: Glibc stable release process (Glibc 2.26.1)

[Florian Weimer]

On 10/01/2017 09:59 PM, Andreas K. Huettel wrote:
> To be honest, if I were a long-time glibc distro maintainer I'd probably agree
> with you and prefer hand-picking. Starting from a tag / tarball is something I
> prefer because I'm not that versed with things yet.

We continuously rebase Fedora on top of the upstream stable release 
branch for that Fedora release (but we do not switch branches within a 
release).

I doubt there is a clear preference, and each approach has its 
advantages and disadvantages.

I still don't understand why you need tarballs for releases, though.  or 
put differently, the difference between glibc 2.26.5 and glibc 2.26-40 
seems rather minor to me, and producing the tarballs is quite a bit of 
work for us.

Regarding security backports, you really need to read and understand our 
announcement of significant issues anyway.  People keep rediscovering 
semantically dependent patches in glibc 2.19 for the CVE-2015-7547 fix 
because the posted patch applies without conflicts without them, and 
this despite we clearly named those patches in the release announcement. 
  This is why I'm wary of pretending further that things are simple. 
They are not.

Thanks,
Florian

Re: Glibc stable release process (Glibc 2.26.1)

[Siddhesh Poyarekar]

On Monday 02 October 2017 01:50 AM, Arjan van de Ven wrote:
> I've been doing distro work for some 17 years now (including at RH way
> back);
> while the packaging side of it is sort of a noop (you tool that anyway),
> the clarity
> and unambiguity of version numbers (in light of CVEs) is something users
> like, it
> sets a common baseline they don't have to look further at.

In my experience, users look at distro release numbers and notes first
and then upstream if they don't find the necessary information there.
So the idea that they'd look for upstream news to answer questions about
their own distros seems inside out.

> bisecting glibc patches outside of distro packages is mostly beyond end
> users' capability.

Agreed and most users don't even look at the packages, they look at
release notes.  I was referring to the QE teams and their convenience of
backing out individual patches in the spec file as opposed to figuring
out bisecting patches in git.  They're much more comfortable doing the
former since that is what they do all the time for all packages.

Siddhesh

Re: Glibc stable release process (Glibc 2.26.1)

[Siddhesh Poyarekar]

On Tuesday 03 October 2017 12:52 AM, Florian Weimer wrote:
> We continuously rebase Fedora on top of the upstream stable release
> branch for that Fedora release (but we do not switch branches within a
> release).

Interesting that the Fedora community actually agreed to do that because
there was a significant amount of flaming in the past due to this very
reason.

> I still don't understand why you need tarballs for releases, though.  or
> put differently, the difference between glibc 2.26.5 and glibc 2.26-40
> seems rather minor to me, and producing the tarballs is quite a bit of
> work for us.

Right.  To be clear, even if I agree to do one now, it should not be
binding on any future release managers to do the same for their release
branch.  Maybe it makes sense to have a targeted discussion like this
with the final decision being that of the release manager for that release.

Siddhesh

Re: Glibc stable release process (Glibc 2.26.1)

[Arjan van de Ven]

On 10/2/2017 9:49 PM, Siddhesh Poyarekar wrote:
> 
> Agreed and most users don't even look at the packages, they look at
> release notes.  


actually users increasingly use automated scanners to check for CVE compliance, and in first order those
look at version numbers.
Sure for some of the bigger distros those scanners also know which minor package build
a CVE got fixed, so perhaps Fedora does not care ;-)

Re: Glibc stable release process (Glibc 2.26.1)

[Gabriel F. T. Gomes]

On 03 Oct 2017, Siddhesh Poyarekar wrote:

>On Tuesday 03 October 2017 12:52 AM, Florian Weimer wrote:
>>
>> I still don't understand why you need tarballs for releases, though.  or
>> put differently, the difference between glibc 2.26.5 and glibc 2.26-40
>> seems rather minor to me, and producing the tarballs is quite a bit of
>> work for us.  
>
>Right.  To be clear, even if I agree to do one now, it should not be
>binding on any future release managers to do the same for their release
>branch.  Maybe it makes sense to have a targeted discussion like this
>with the final decision being that of the release manager for that release.

If you actually agree to do one now, are we going to have something
similar to a code freeze?  I'm asking because I believe some patches
should probably be considered release blockers, such as those in these
threads:

  - https://sourceware.org/ml/libc-alpha/2017-09/msg00486.html
    (already committed on the master branch)
  - https://sourceware.org/ml/libc-alpha/2017-10/msg00102.html
    (not yet on master, but probably relevant to the point-release)

Maybe there are more, and eventually others will emerge, so I have no idea
what criteria should be used to decide what goes in and what stays out of
this point release.

I agree that a targeted discussion about the need for a point release (as
you suggested) makes sense, but then it probably would also make sense to
discuss the relevance of particular patches as release blockers, which adds
to the work required for the release.

Re: Glibc stable release process (Glibc 2.26.1)

[Andreas K. Huettel]

[-- Attachment #1: Type: text/plain, Size: 1582 bytes --]

Am Dienstag, 3. Oktober 2017, 16:14:52 CEST schrieb Gabriel F. T. Gomes:
> On 03 Oct 2017, Siddhesh Poyarekar wrote:
> >On Tuesday 03 October 2017 12:52 AM, Florian Weimer wrote:
> >> I still don't understand why you need tarballs for releases, though.  or
> >> put differently, the difference between glibc 2.26.5 and glibc 2.26-40
> >> seems rather minor to me, and producing the tarballs is quite a bit of
> >> work for us.
> >
> >Right.  To be clear, even if I agree to do one now, it should not be
> >binding on any future release managers to do the same for their release
> >branch.  Maybe it makes sense to have a targeted discussion like this
> >with the final decision being that of the release manager for that release.
> 
> If you actually agree to do one now, are we going to have something
> similar to a code freeze?

Given that your previous recommendation was "stay with the tip of the release 
branch", that sounds unnecessary. My recommendation from my experience as 
packager of other software would be, 
* keep adding backports to the release branches as careful as now
* tag (and tar?) point releases from the release branches at regular 
intervals, say every two months.
This might make releases more comparable between distributions. 

Of course, alternatively, every distro can do the tag and tar just themselves. 
And and in any case add patches in between... but the next point release gives 
a common reference point then again.

-- 
Andreas K. Hüttel
dilfridge@gentoo.org
Gentoo Linux developer (council, perl, libreoffice)

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Glibc stable release process (Glibc 2.26.1)

[Adhemerval Zanella]

On 03/10/2017 11:57, Andreas K. Huettel wrote:
> Am Dienstag, 3. Oktober 2017, 16:14:52 CEST schrieb Gabriel F. T. Gomes:
>> On 03 Oct 2017, Siddhesh Poyarekar wrote:
>>> On Tuesday 03 October 2017 12:52 AM, Florian Weimer wrote:
>>>> I still don't understand why you need tarballs for releases, though.  or
>>>> put differently, the difference between glibc 2.26.5 and glibc 2.26-40
>>>> seems rather minor to me, and producing the tarballs is quite a bit of
>>>> work for us.
>>>
>>> Right.  To be clear, even if I agree to do one now, it should not be
>>> binding on any future release managers to do the same for their release
>>> branch.  Maybe it makes sense to have a targeted discussion like this
>>> with the final decision being that of the release manager for that release.
>>
>> If you actually agree to do one now, are we going to have something
>> similar to a code freeze?
> 
> Given that your previous recommendation was "stay with the tip of the release 
> branch", that sounds unnecessary. My recommendation from my experience as 
> packager of other software would be, 
> * keep adding backports to the release branches as careful as now
> * tag (and tar?) point releases from the release branches at regular 
> intervals, say every two months.
> This might make releases more comparable between distributions. 
> 
> Of course, alternatively, every distro can do the tag and tar just themselves. 
> And and in any case add patches in between... but the next point release gives 
> a common reference point then again.
> 

The point release of already released versions should be based on defined
branch for ABI stabilization.  Trying to use master branch for point
releases can potentially add symbols that might resulting in releases
with different ABI and this is a potentially source of problems.

Also, keep in mind that release branches are maintained by volunteers that
might not have the required bandwidth to keep active schedule releases.
I do not know how viable would be to have this automated, but even though
it would required infrastructure work we might not have for current work.

I see that for 2.26 for the particular C++ issues might be a good thing
to aim for a 2.26.1, but I see that point releases should always be a
community discussion in concordance with release manager time disposal.

Re: Glibc stable release process (Glibc 2.26.1)

[Jonathan Nieder]

Siddhesh Poyarekar wrote:

> Agreed and most users don't even look at the packages, they look at
> release notes.  I was referring to the QE teams and their convenience of
> backing out individual patches in the spec file as opposed to figuring
> out bisecting patches in git.  They're much more comfortable doing the
> former since that is what they do all the time for all packages.

FWIW this is not true in Debian.  It sounds like it's mostly a Red Hat
specific thing.

Debian follows stable branches for some packages (e.g., Linux) and
does not for some others (e.g., Git).  It comes down to how close
upstream is to Debian's point of view on which patches are important
enough to apply to the stable branch.

I don't know where glibc falls on that spectrum.  It may be that
Debian would be able to consume glibc point releases as updates to an
already released Debian release.  It's likely that Debian wouldn't.
(That said, I am not saying tagged point releases would *hurt* Debian
--- this reply is only about the question of whether they would help.)

Thanks,
Jonathan

Re: Glibc stable release process (Glibc 2.26.1)

[Siddhesh Poyarekar]

On Tuesday 03 October 2017 07:44 PM, Gabriel F. T. Gomes wrote:
> If you actually agree to do one now, are we going to have something
> similar to a code freeze?  I'm asking because I believe some patches
> should probably be considered release blockers, such as those in these
> threads:
> 
>   - https://sourceware.org/ml/libc-alpha/2017-09/msg00486.html
>     (already committed on the master branch)
>   - https://sourceware.org/ml/libc-alpha/2017-10/msg00102.html
>     (not yet on master, but probably relevant to the point-release)
>
> Maybe there are more, and eventually others will emerge, so I have no idea
> what criteria should be used to decide what goes in and what stays out of
> this point release.

There's no strict limitation on what goes into a stable branch, but I
think the following guidelines should be followed if in doubt:

0. Must not break ABI on that branch
1. Reasonably limited in scope (i.e. isolated to an architecture, bug
   fixes, CVE fixes, small feature additions)
2. Should not affect translation strings (flexible, could be overridden
   for serious bugs/CVEs)

> I agree that a targeted discussion about the need for a point release (as
> you suggested) makes sense, but then it probably would also make sense to
> discuss the relevance of particular patches as release blockers, which adds
> to the work required for the release.

I propose just a 1 week freeze to stabilize the branch before a point
release, the 1 month freeze again does not make sense since we're not
waiting for translations or detailed architecture tests given the
limited scope of the change.  The current guidelines put point releases
and full releases in the same category, so I'd like to hear more
opinions from maintainers on this.

Siddhesh

Re: Glibc stable release process (Glibc 2.26.1)

[Siddhesh Poyarekar]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On Tuesday 03 October 2017 08:27 PM, Andreas K. Huettel wrote:
> Given that your previous recommendation was "stay with the tip of
> the release branch", that sounds unnecessary. My recommendation
> from my experience as packager of other software would be, * keep
> adding backports to the release branches as careful as now * tag
> (and tar?) point releases from the release branches at regular 
> intervals, say every two months. This might make releases more
> comparable between distributions.

I think the difference there is on who has the responsibility of
testing the release tarball.  When we bless a release as upstream, we
should at least be doing some minimal testing before throwing it over
the wall.  If a distribution builds a tarball off the branch and uses
it, it becomes their responsibility to test :)

Siddhesh
-----BEGIN PGP SIGNATURE-----

iQFMBAEBCAA2FiEEvHxzcmN+wQxX16plecQ9+/HPIYcFAlnUgQUYHHNpZGRoZXNo
QHNvdXJjZXdhcmUub3JnAAoJEHnEPfvxzyGHp5EH/1I+8s2iPlltknhLsxYrxC9c
THAijpZs8yr+vBat+T8MsfTxN92YDbxGTPH3yQuU5riThJs0q6DGNj+MIKBA76ZI
zzZeXav3yCHRMm/WIRp0kysUHhnXj3GB976nJart2HjcrZEgYdPQok50/Xdhd4po
8ORMcST0g2lSnHltAX5fEcj51W1N57Av3ykdNJdaSLS2PELC0e3mWJb0zEcNUm+l
PmWN37IlY0lxCKPMhHrndafSIEQ2Thk36bMIq0dKDEF5hHrDl0MXWtyVNW0wsmLL
oi0cgfThzmD9k2UYleut/+IMFgOeB6YxgXppfi3RYuJd4vJLCkZ6yXVM2KdKZ+Q=
=MdDO
-----END PGP SIGNATURE-----

Re: Glibc stable release process (Glibc 2.26.1)

[Andreas Schwab]

On Okt 04 2017, Siddhesh Poyarekar <siddhesh@sourceware.org> wrote:

> 0. Must not break ABI on that branch

We must not break the ABI on any branch.  What you probably mean is that
we must not extend the ABI on the release branch.

Andreas.

-- 
Andreas Schwab, SUSE Labs, schwab@suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

Re: Glibc stable release process (Glibc 2.26.1)

[Siddhesh Poyarekar]

On Wednesday 04 October 2017 02:06 PM, Andreas Schwab wrote:
> On Okt 04 2017, Siddhesh Poyarekar <siddhesh@sourceware.org> wrote:
> 
>> 0. Must not break ABI on that branch
> 
> We must not break the ABI on any branch.  What you probably mean is that
> we must not extend the ABI on the release branch.

Sorry yes, that's what I meant.

Siddhesh

Re: Glibc stable release process (Glibc 2.26.1)

[Arjan van de Ven]

On 10/4/2017 1:36 AM, Andreas Schwab wrote:
> On Okt 04 2017, Siddhesh Poyarekar <siddhesh@sourceware.org> wrote:
> 
>> 0. Must not break ABI on that branch
> 
> We must not break the ABI on any branch.  What you probably mean is that
> we must not extend the ABI on the release branch.

and no new symvers

I'd expect this to be both backward (like usual) but also forward compatible
as a goal.

now if a CVE requires a new symver, I'm sure that will trump that rule

Re: Glibc stable release process (Glibc 2.26.1)

[Joseph Myers]

On Wed, 4 Oct 2017, Siddhesh Poyarekar wrote:

> >   - https://sourceware.org/ml/libc-alpha/2017-10/msg00102.html
> >     (not yet on master, but probably relevant to the point-release)
> >
> > Maybe there are more, and eventually others will emerge, so I have no idea
> > what criteria should be used to decide what goes in and what stays out of
> > this point release.
> 
> There's no strict limitation on what goes into a stable branch, but I
> think the following guidelines should be followed if in doubt:
> 
> 0. Must not break ABI on that branch
> 1. Reasonably limited in scope (i.e. isolated to an architecture, bug
>    fixes, CVE fixes, small feature additions)
> 2. Should not affect translation strings (flexible, could be overridden
>    for serious bugs/CVEs)

I would suggest:

3. Wait a few days after committing to master before cherry-picking to the 
branch, in case any problems with the patch show up on master (the above 
patch being a case in point).  When applying a patch to a branch, make 
sure to watch out for any followup patches that were applied to master to 
fix such problems, and cherry-pick those to the branch as well.

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: Glibc stable release process (Glibc 2.26.1)

[Joseph Myers]

On Wed, 4 Oct 2017, Andreas Schwab wrote:

> On Okt 04 2017, Siddhesh Poyarekar <siddhesh@sourceware.org> wrote:
> 
> > 0. Must not break ABI on that branch
> 
> We must not break the ABI on any branch.  What you probably mean is that
> we must not extend the ABI on the release branch.

But we should not rule out applying fixes for past ABI breakage to a 
release branch.  (E.g. 
<https://sourceware.org/ml/libc-alpha/2017-08/msg00900.html> - I'm not 
asserting that particular patch is worth backporting, or would be 
appropriate to apply to a release branch rather than a smaller fix for the 
ABI issue, simply that fixing such a past ABI breakage on a release 
branch, and thereby restoring a symbol that was previously wrongly lost, 
is appropriate.)

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: Glibc stable release process (Glibc 2.26.1)

[Joseph Myers]

On Wed, 4 Oct 2017, Arjan van de Ven wrote:

> On 10/4/2017 1:36 AM, Andreas Schwab wrote:
> > On Okt 04 2017, Siddhesh Poyarekar <siddhesh@sourceware.org> wrote:
> > 
> > > 0. Must not break ABI on that branch
> > 
> > We must not break the ABI on any branch.  What you probably mean is that
> > we must not extend the ABI on the release branch.
> 
> and no new symvers
> 
> I'd expect this to be both backward (like usual) but also forward compatible
> as a goal.
> 
> now if a CVE requires a new symver, I'm sure that will trump that rule

Even new GLIBC_PRIVATE symbols are risky for package updates on live 
systems (as you get the situation where a process had loaded libc.so not 
providing the symbol, then glibc is upgraded, then that process loads 
a newer libm.so which expects a GLIBC_PRIVATE symbol from libc.so that 
only newer libc.so provides - a real issue that appeared with backports of 
nan unbounded stack usage fixes).

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: Glibc stable release process (Glibc 2.26.1)

[Gabriel F. T. Gomes]

On 04 Oct 2017, Joseph Myers wrote:

>On Wed, 4 Oct 2017, Siddhesh Poyarekar wrote:
>
>> >   - https://sourceware.org/ml/libc-alpha/2017-10/msg00102.html
>> >     (not yet on master, but probably relevant to the point-release)
>> >
>> > Maybe there are more, and eventually others will emerge, so I have no idea
>> > what criteria should be used to decide what goes in and what stays out of
>> > this point release.  
>> 
>> There's no strict limitation on what goes into a stable branch, but I
>> think the following guidelines should be followed if in doubt:
>> 
>> 0. Must not break ABI on that branch
>> 1. Reasonably limited in scope (i.e. isolated to an architecture, bug
>>    fixes, CVE fixes, small feature additions)
>> 2. Should not affect translation strings (flexible, could be overridden
>>    for serious bugs/CVEs)  
>
>I would suggest:
>
>3. Wait a few days after committing to master before cherry-picking to the 
>branch, in case any problems with the patch show up on master (the above 
>patch being a case in point).  When applying a patch to a branch, make 
>sure to watch out for any followup patches that were applied to master to 
>fix such problems, and cherry-pick those to the branch as well.

Totally agree.  I felt unnecessarily rushed.

Re: Glibc stable release process (Glibc 2.26.1)

[Florian Weimer]

On 10/04/2017 02:52 PM, Arjan van de Ven wrote:
> On 10/4/2017 1:36 AM, Andreas Schwab wrote:
>> On Okt 04 2017, Siddhesh Poyarekar <siddhesh@sourceware.org> wrote:
>>
>>> 0. Must not break ABI on that branch
>>
>> We must not break the ABI on any branch.  What you probably mean is that
>> we must not extend the ABI on the release branch.
> 
> and no new symvers
> 
> I'd expect this to be both backward (like usual) but also forward 
> compatible
> as a goal.
> 
> now if a CVE requires a new symver, I'm sure that will trump that rule

No, if a new symbol version is required, we cannot backport the fix.  We 
need to leave it unfixed or come up with a different fix.

Thanks,
Florian

Re: Glibc stable release process (Glibc 2.26.1)

[Andreas Schwab]

On Okt 04 2017, Joseph Myers <joseph@codesourcery.com> wrote:

> Even new GLIBC_PRIVATE symbols are risky for package updates on live 
> systems

Same holds for ABI changes for such symbols.

Andreas.

-- 
Andreas Schwab, SUSE Labs, schwab@suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

Re: Glibc stable release process (Glibc 2.26.1)

[Carlos O'Donell]

On 10/04/2017 06:32 AM, Florian Weimer wrote:
> On 10/04/2017 02:52 PM, Arjan van de Ven wrote:
>> On 10/4/2017 1:36 AM, Andreas Schwab wrote:
>>> On Okt 04 2017, Siddhesh Poyarekar <siddhesh@sourceware.org>
>>> wrote:
>>> 
>>>> 0. Must not break ABI on that branch
>>> 
>>> We must not break the ABI on any branch.  What you probably mean
>>> is that we must not extend the ABI on the release branch.
>> 
>> and no new symvers
>> 
>> I'd expect this to be both backward (like usual) but also forward
>> compatible as a goal.
>> 
>> now if a CVE requires a new symver, I'm sure that will trump that
>> rule
> 
> No, if a new symbol version is required, we cannot backport the fix.
> We need to leave it unfixed or come up with a different fix.

Agreed. We should not be adding new symbol versions to stable branches.

Likewise as Joseph and Andreas mentioned, we should not be changing
the semantics of GLIBC_PRIVATE symbols, nor adding new ones, they can
and will crash processes as glibc is updated.

-- 
Cheers,
Carlos.

Re: Glibc stable release process (Glibc 2.26.1)

[Yann E. MORIN]

Siddhesh, All,

On 2017-10-04 12:04 +0530, Siddhesh Poyarekar spake thusly:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
> 
> On Tuesday 03 October 2017 08:27 PM, Andreas K. Huettel wrote:
> > Given that your previous recommendation was "stay with the tip of
> > the release branch", that sounds unnecessary. My recommendation
> > from my experience as packager of other software would be, * keep
> > adding backports to the release branches as careful as now * tag
> > (and tar?) point releases from the release branches at regular 
> > intervals, say every two months.

Having regular dot-releases would be awesome, yes! Thanks! :-)

I think every two months is a bit long, esepecially for the .1 release.
Most of the critical bugs are largely identified and fixed in the first
month after the release, so maybe a .1 one month after the release,
then every two months for the next ones sounds more sensible?

> > This might make releases more
> > comparable between distributions.
> 
> I think the difference there is on who has the responsibility of
> testing the release tarball.  When we bless a release as upstream, we
> should at least be doing some minimal testing before throwing it over
> the wall.

As was previously discussed, it was my understanding that backports
from master to the stable branch undergo a rather strict process, so
the stable branch should only see pretty solid changes coming in in
the first place.

As such, it makes sense that the testing that has to be done on the
branch be limited when compared to a full release.

>  If a distribution builds a tarball off the branch and uses
> it, it becomes their responsibility to test :)

Agreed. :-)

Regards,
Yann E. MORIN.

> Siddhesh
> -----BEGIN PGP SIGNATURE-----
> 
> iQFMBAEBCAA2FiEEvHxzcmN+wQxX16plecQ9+/HPIYcFAlnUgQUYHHNpZGRoZXNo
> QHNvdXJjZXdhcmUub3JnAAoJEHnEPfvxzyGHp5EH/1I+8s2iPlltknhLsxYrxC9c
> THAijpZs8yr+vBat+T8MsfTxN92YDbxGTPH3yQuU5riThJs0q6DGNj+MIKBA76ZI
> zzZeXav3yCHRMm/WIRp0kysUHhnXj3GB976nJart2HjcrZEgYdPQok50/Xdhd4po
> 8ORMcST0g2lSnHltAX5fEcj51W1N57Av3ykdNJdaSLS2PELC0e3mWJb0zEcNUm+l
> PmWN37IlY0lxCKPMhHrndafSIEQ2Thk36bMIq0dKDEF5hHrDl0MXWtyVNW0wsmLL
> oi0cgfThzmD9k2UYleut/+IMFgOeB6YxgXppfi3RYuJd4vJLCkZ6yXVM2KdKZ+Q=
> =MdDO
> -----END PGP SIGNATURE-----

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

Re: Glibc stable release process (Glibc 2.26.1)

[Siddhesh Poyarekar]

On Wednesday 04 October 2017 09:57 PM, Yann E. MORIN wrote:
>> Having regular dot-releases would be awesome, yes! Thanks! :-)

Wait, I didn't volunteer for time boxed point releases, I have
volunteered only for 2.26.1.  Further releases would be based on how
long this one takes and if I have enough time for it.  For 2.27 and
beyond, it depends on the release manager for those releases.

Siddhesh

Re: Glibc stable release process (Glibc 2.26.1)

[Carlos O'Donell]

On 10/04/2017 09:02 PM, Siddhesh Poyarekar wrote:
> On Wednesday 04 October 2017 09:57 PM, Yann E. MORIN wrote:
>>> Having regular dot-releases would be awesome, yes! Thanks! :-)
> 
> Wait, I didn't volunteer for time boxed point releases, I have
> volunteered only for 2.26.1.  Further releases would be based on how
> long this one takes and if I have enough time for it.  For 2.27 and
> beyond, it depends on the release manager for those releases.

Agreed, there is a danger here, we don't want to set a precedent
that the glibc release manager is always going to do a point release.

In fact I don't really understand the community requirements for this
at all, and I'm writing up some details on a proposal that might be
more flexible.

My key point is that the glibc community does not have the resources
to test and make point releases, but that the downstream, if they want
it, can coordinate this with a little help from us.

While I like the *idea* of point releases, a point release has a meaning
and it shall not be thrown over the wall.


-- 
Cheers,
Carlos.

Re: Glibc stable release process (Glibc 2.26.1)

[Florian Weimer]

On 10/03/2017 06:53 AM, Siddhesh Poyarekar wrote:
> On Tuesday 03 October 2017 12:52 AM, Florian Weimer wrote:
>> We continuously rebase Fedora on top of the upstream stable release
>> branch for that Fedora release (but we do not switch branches within a
>> release).
> 
> Interesting that the Fedora community actually agreed to do that because
> there was a significant amount of flaming in the past due to this very
> reason.

The Fedora community isn't really in a position to complain.  We didn't 
have much choice because maintaining patches in RPM is so painful.  I've 
been working on new patch management tools with spec file auto-editing 
and other frills, so the pendulum might well swing into the other 
direction in the future.

Thanks,
Florian

Re: Glibc stable release process (Glibc 2.26.1)

[Carlos O'Donell]

On 10/09/2017 06:57 AM, Florian Weimer wrote:
> On 10/03/2017 06:53 AM, Siddhesh Poyarekar wrote:
>> On Tuesday 03 October 2017 12:52 AM, Florian Weimer wrote:
>>> We continuously rebase Fedora on top of the upstream stable
>>> release branch for that Fedora release (but we do not switch
>>> branches within a release).
>> 
>> Interesting that the Fedora community actually agreed to do that
>> because there was a significant amount of flaming in the past due
>> to this very reason.
> 
> The Fedora community isn't really in a position to complain.  We
> didn't have much choice because maintaining patches in RPM is so
> painful.  I've been working on new patch management tools with spec
> file auto-editing and other frills, so the pendulum might well swing
> into the other direction in the future.

There isn't even any complaints. We took this to FESCo for an official
ruling, and I wrote up the rationale.

https://fedoraproject.org/wiki/Glibc#Usage_of_Fedora_Rawhide

This is slightly on topic, that Fedora doesn't need specially named
point releases.

-- 
Cheers,
Carlos.

Re: Glibc stable release process (Glibc 2.26.1)

[Dmitry V. Levin]

[-- Attachment #1: Type: text/plain, Size: 2147 bytes --]

On Fri, Sep 29, 2017 at 05:26:23PM -0700, Paul Eggert wrote:
> Romain Naour wrote:
> > Maybe a middle ground is that a tag is pushed to the repository, just to
> > identify a specific sha1 as "this is 2.26.1". This is relatively lightweight (a
> > signed tag) and gives downstream users that need it a clear identification of a
> > blessed version.
> That "blessing" would be problematic. How can we be expected to "bless" any 
> particular commit unless we've checked it? And that checking will take some 
> work, work that will slow us down on other things.
> 
> Instead of blessing, how about using the output of something like the following 
> shell command to generate the version number?
> 
> git describe --match 'glibc-*' --abbrev=7 --dirty
> 
> This will generate a string that identifies the commit but is shorter and easier 
> to read than an SHA hash, and so is more likely to give users that warm and 
> fuzzy feeling. On my glibc repository right now, for example, it generates:
> 
> glibc-2.26-415-g4d3693e
> 
> which means there have been 415 commits since 2.26, and the latest commit can be 
> abbreviated 4d3693e. A similar (but shorter) version-numbering scheme is used by 
> GNU Coreutils and other GNU packages when between releases, and Gnulib has a 
> script build-aux/git-version-gen that can generate it.

We (ALT) adopted this scheme for most of the packages that are built from
git snapshots, and I think this is the way to go.  git is available for
more than 12 years, why some distro builders need an officially released
tarball made from release/2.26/master branch that would be no better than
current glibc-2.26-58-gf725563 (aka 2.26.0.58.f725563 in git-version-gen's
notation) is beyond my understanding.

It would be a different story if somebody volunteered to do a proper
release testing of point releases similar to what's currently done
or releases, see e.g. https://sourceware.org/glibc/wiki/Release/2.26

Are those projects that want point release tarballs are ready to do
a proper pre-release testing?  If yes, then it makes sense to do
point releases.


-- 
ldv

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 801 bytes --]

Re: Glibc stable release process (Glibc 2.26.1)

[Florian Weimer]

On 10/12/2017 09:29 PM, Dmitry V. Levin wrote:
> On Fri, Sep 29, 2017 at 05:26:23PM -0700, Paul Eggert wrote:
>> Romain Naour wrote:
>>> Maybe a middle ground is that a tag is pushed to the repository, just to
>>> identify a specific sha1 as "this is 2.26.1". This is relatively lightweight (a
>>> signed tag) and gives downstream users that need it a clear identification of a
>>> blessed version.
>> That "blessing" would be problematic. How can we be expected to "bless" any
>> particular commit unless we've checked it? And that checking will take some
>> work, work that will slow us down on other things.
>>
>> Instead of blessing, how about using the output of something like the following
>> shell command to generate the version number?
>>
>> git describe --match 'glibc-*' --abbrev=7 --dirty
>>
>> This will generate a string that identifies the commit but is shorter and easier
>> to read than an SHA hash, and so is more likely to give users that warm and
>> fuzzy feeling. On my glibc repository right now, for example, it generates:
>>
>> glibc-2.26-415-g4d3693e
>>
>> which means there have been 415 commits since 2.26, and the latest commit can be
>> abbreviated 4d3693e. A similar (but shorter) version-numbering scheme is used by
>> GNU Coreutils and other GNU packages when between releases, and Gnulib has a
>> script build-aux/git-version-gen that can generate it.
> 
> We (ALT) adopted this scheme for most of the packages that are built from
> git snapshots, and I think this is the way to go.  git is available for
> more than 12 years, why some distro builders need an officially released
> tarball made from release/2.26/master branch that would be no better than
> current glibc-2.26-58-gf725563 (aka 2.26.0.58.f725563 in git-version-gen's
> notation) is beyond my understanding.

Right, I share your puzzlement.

The only thing I think we could improve is that we should tag the first 
tag after a release branches with something like glibc-X.Y.9000, so that 
the git describe output is actually unique between the release branch 
and the next development branch (also, we should have update hooks which 
reject commits with multiple parents on the master branches).

I'd propose to add the following tags:

00cdcf5a4110f7ac68651f5662693c82f7bffaca   glibc-2.26.9000
58557c229319a3b8d2eefdb62e7df95089eabe37   glibc-2.25.9000
e720d3d9fea2058adb3de2905f1a399ad3e812ff   glibc-2.24.9000
c6f391dbf9b3b33e5bc0599dd96c40a0eff2f02b   glibc-2.23.9000
1b15ff4810748abee11b949e6faa115f3f2d20f4   glibc-2.22.9000
d5a8b70560cf758218c13bef6f1dd93ce216424f   glibc-2.21.9000
21c83793a223666b8cfe438d81615941896b355c   glibc-2.20.9000
d5b396c1c89ed3026fc89bfcdd72b14d59972e45   glibc-2.19.9000
6c1fd795711bb510cffaab5ad2ab2739bb8db210   glibc-2.18.9000
c758a6861537815c759cba2018a3b1abb1943842   glibc-2.17.9000
75f0d3040a2c2de8842bfa7a09e11da1a73e17d0   glibc-2.16.9000

The latter has also a glibc-2.16.0 tag, but I think that's misleading. 
The .9000 part (or any large number, really) confers that it's an 
artificial number, and it will sort correctly in most version number 
orderings.

Would these new tags break something for you?  They won't change 
anything on the release branches, only on the master branch.

If you don't see a problem with that, I'm going to suggest this on a 
separate thread (it's too buried here to receive proper attention).

Thanks,
Florian

Re: Glibc stable release process (Glibc 2.26.1)

[Dmitry V. Levin]

[-- Attachment #1: Type: text/plain, Size: 3794 bytes --]

On Thu, Oct 12, 2017 at 09:59:09PM +0200, Florian Weimer wrote:
> On 10/12/2017 09:29 PM, Dmitry V. Levin wrote:
> > On Fri, Sep 29, 2017 at 05:26:23PM -0700, Paul Eggert wrote:
> >> Romain Naour wrote:
> >>> Maybe a middle ground is that a tag is pushed to the repository, just to
> >>> identify a specific sha1 as "this is 2.26.1". This is relatively lightweight (a
> >>> signed tag) and gives downstream users that need it a clear identification of a
> >>> blessed version.
> >> That "blessing" would be problematic. How can we be expected to "bless" any
> >> particular commit unless we've checked it? And that checking will take some
> >> work, work that will slow us down on other things.
> >>
> >> Instead of blessing, how about using the output of something like the following
> >> shell command to generate the version number?
> >>
> >> git describe --match 'glibc-*' --abbrev=7 --dirty
> >>
> >> This will generate a string that identifies the commit but is shorter and easier
> >> to read than an SHA hash, and so is more likely to give users that warm and
> >> fuzzy feeling. On my glibc repository right now, for example, it generates:
> >>
> >> glibc-2.26-415-g4d3693e
> >>
> >> which means there have been 415 commits since 2.26, and the latest commit can be
> >> abbreviated 4d3693e. A similar (but shorter) version-numbering scheme is used by
> >> GNU Coreutils and other GNU packages when between releases, and Gnulib has a
> >> script build-aux/git-version-gen that can generate it.
> > 
> > We (ALT) adopted this scheme for most of the packages that are built from
> > git snapshots, and I think this is the way to go.  git is available for
> > more than 12 years, why some distro builders need an officially released
> > tarball made from release/2.26/master branch that would be no better than
> > current glibc-2.26-58-gf725563 (aka 2.26.0.58.f725563 in git-version-gen's
> > notation) is beyond my understanding.
> 
> Right, I share your puzzlement.
> 
> The only thing I think we could improve is that we should tag the first 
> tag after a release branches with something like glibc-X.Y.9000, so that 
> the git describe output is actually unique between the release branch 
> and the next development branch (also, we should have update hooks which 
> reject commits with multiple parents on the master branches).
> 
> I'd propose to add the following tags:
> 
> 00cdcf5a4110f7ac68651f5662693c82f7bffaca   glibc-2.26.9000
> 58557c229319a3b8d2eefdb62e7df95089eabe37   glibc-2.25.9000
> e720d3d9fea2058adb3de2905f1a399ad3e812ff   glibc-2.24.9000
> c6f391dbf9b3b33e5bc0599dd96c40a0eff2f02b   glibc-2.23.9000
> 1b15ff4810748abee11b949e6faa115f3f2d20f4   glibc-2.22.9000
> d5a8b70560cf758218c13bef6f1dd93ce216424f   glibc-2.21.9000
> 21c83793a223666b8cfe438d81615941896b355c   glibc-2.20.9000
> d5b396c1c89ed3026fc89bfcdd72b14d59972e45   glibc-2.19.9000
> 6c1fd795711bb510cffaab5ad2ab2739bb8db210   glibc-2.18.9000
> c758a6861537815c759cba2018a3b1abb1943842   glibc-2.17.9000

The first commit after glibc-2.17 is 2c8bfe7d6f22c4a599894846bf1715d93b295f53.

> 75f0d3040a2c2de8842bfa7a09e11da1a73e17d0   glibc-2.16.9000
> 
> The latter has also a glibc-2.16.0 tag, but I think that's misleading. 

That's because glibc-2.16 was incomplete and glibc-2.16.0 is the real tag
of 2.16 release.  The first commit after 2.16 is the one tagged as
glibc-2.16-ports-merge.

> The .9000 part (or any large number, really) confers that it's an 
> artificial number, and it will sort correctly in most version number 
> orderings.
> 
> Would these new tags break something for you?  They won't change 
> anything on the release branches, only on the master branch.

These new tags are fine, thanks!


-- 
ldv

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 801 bytes --]

Re: Glibc stable release process (Glibc 2.26.1)

[Joseph Myers]

On Thu, 12 Oct 2017, Florian Weimer wrote:

> I'd propose to add the following tags:

You mean annotated tags (as otherwise git describe will ignore them by 
default)?

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: Glibc stable release process (Glibc 2.26.1)

[Florian Weimer]

* Joseph Myers:

> On Thu, 12 Oct 2017, Florian Weimer wrote:
>
>> I'd propose to add the following tags:
>
> You mean annotated tags (as otherwise git describe will ignore them by 
> default)?

Right, annotated tags.

Re: Glibc stable release process (Glibc 2.26.1)

[Siddhesh Poyarekar]

On Friday 13 October 2017 01:29 AM, Florian Weimer wrote:
> If you don't see a problem with that, I'm going to suggest this on a
> separate thread (it's too buried here to receive proper attention).

This is a good idea in general and a trivial point to add to the release
process.  Do you intend to propose this as a full replacement for point
releases (i.e. never do it) or just an added convenience for those who
don't care for point releases or a fallback for when the release manager
(or one of the package signers, i.e. Carlos, Joseph, me, etc.) does not
want to spend time on a point release?

Siddhesh

Re: Glibc stable release process (Glibc 2.26.1)

[Florian Weimer]

On 10/16/2017 02:48 PM, Siddhesh Poyarekar wrote:
> On Friday 13 October 2017 01:29 AM, Florian Weimer wrote:
>> If you don't see a problem with that, I'm going to suggest this on a
>> separate thread (it's too buried here to receive proper attention).
> 
> This is a good idea in general and a trivial point to add to the release
> process.  Do you intend to propose this as a full replacement for point
> releases (i.e. never do it) or just an added convenience for those who
> don't care for point releases or a fallback for when the release manager
> (or one of the package signers, i.e. Carlos, Joseph, me, etc.) does not
> want to spend time on a point release?

I think it's a sufficient replacement for point releases, but I suppose 
others will disagree.

Thanks,
Florian

Re: Glibc stable release process (Glibc 2.26.1)

[Yann E. MORIN]

Hello All! :-)

On 2017-10-16 15:20 +0200, Florian Weimer spake thusly:
> On 10/16/2017 02:48 PM, Siddhesh Poyarekar wrote:
> >On Friday 13 October 2017 01:29 AM, Florian Weimer wrote:
> >>If you don't see a problem with that, I'm going to suggest this on a
> >>separate thread (it's too buried here to receive proper attention).
> >
> >This is a good idea in general and a trivial point to add to the release
> >process.  Do you intend to propose this as a full replacement for point
> >releases (i.e. never do it) or just an added convenience for those who
> >don't care for point releases or a fallback for when the release manager
> >(or one of the package signers, i.e. Carlos, Joseph, me, etc.) does not
> >want to spend time on a point release?
> 
> I think it's a sufficient replacement for point releases, but I suppose
> others will disagree.

Well, I won't re-hash the explanations I expressed, but I'll just state
that a point-release would be better, IMHO. It is not necessary to put
out tarballs, though; a tag would be enough.

I'll be at ELCE in Prague next week, so if anyone goes there, I'd be
happy to talk about this around a ${beverage}. ;-)

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

Re: Glibc stable release process (Glibc 2.26.1)

[Florian Weimer]

On 10/16/2017 05:55 PM, Yann E. MORIN wrote:
> Hello All! :-)
> 
> On 2017-10-16 15:20 +0200, Florian Weimer spake thusly:
>> On 10/16/2017 02:48 PM, Siddhesh Poyarekar wrote:
>>> On Friday 13 October 2017 01:29 AM, Florian Weimer wrote:
>>>> If you don't see a problem with that, I'm going to suggest this on a
>>>> separate thread (it's too buried here to receive proper attention).
>>>
>>> This is a good idea in general and a trivial point to add to the release
>>> process.  Do you intend to propose this as a full replacement for point
>>> releases (i.e. never do it) or just an added convenience for those who
>>> don't care for point releases or a fallback for when the release manager
>>> (or one of the package signers, i.e. Carlos, Joseph, me, etc.) does not
>>> want to spend time on a point release?
>>
>> I think it's a sufficient replacement for point releases, but I suppose
>> others will disagree.
> 
> Well, I won't re-hash the explanations I expressed, but I'll just state
> that a point-release would be better, IMHO. It is not necessary to put
> out tarballs, though; a tag would be enough.

Why do you think a tag would be superior to the output from “git 
describe”?  Because we can change it subsequently to point something else?

Thanks,
Florian

Re: Glibc stable release process (Glibc 2.26.1)

[Yann E. MORIN]

Florian, All,

On 2017-10-16 18:52 +0200, Florian Weimer spake thusly:
> On 10/16/2017 05:55 PM, Yann E. MORIN wrote:
> >Hello All! :-)
> >
> >On 2017-10-16 15:20 +0200, Florian Weimer spake thusly:
> >>On 10/16/2017 02:48 PM, Siddhesh Poyarekar wrote:
> >>>On Friday 13 October 2017 01:29 AM, Florian Weimer wrote:
> >>>>If you don't see a problem with that, I'm going to suggest this on a
> >>>>separate thread (it's too buried here to receive proper attention).
> >>>
> >>>This is a good idea in general and a trivial point to add to the release
> >>>process.  Do you intend to propose this as a full replacement for point
> >>>releases (i.e. never do it) or just an added convenience for those who
> >>>don't care for point releases or a fallback for when the release manager
> >>>(or one of the package signers, i.e. Carlos, Joseph, me, etc.) does not
> >>>want to spend time on a point release?
> >>
> >>I think it's a sufficient replacement for point releases, but I suppose
> >>others will disagree.
> >
> >Well, I won't re-hash the explanations I expressed, but I'll just state
> >that a point-release would be better, IMHO. It is not necessary to put
> >out tarballs, though; a tag would be enough.
> 
> Why do you think a tag would be superior to the output from “git describe”?
> Because we can change it subsequently to point something else?

TL;DR:
The difference is where the identification comes from:
  - a tag is generated is one location: upstream;
  - git-describe is local and arbitrary to each downstream.


Full text:


A tag is semantically immutable [0]; as soon as it exists, a tag will
always exist, and always point to the same commit. The name of the tag
bears sematic as well; it is a human-readable sequencing. But most
important of all, a tag is generated by _upstream_.


git-describe on the other hand is run by a _downstream_ to generate a
seemingly human-readable version number. The problem is not that the
output of git-describe is not immutable; the problem is to choose the
actual commit to run it on.


The problem is not about representation of the version string. It is
actually choosing what commit to use. As a downstream, we don't have
many options when we need to choose a commit on the maintenance branch
[1], i.e. either one of:
  - use the HEAD of the branch
  - use any one commit between the tag and the HEAD

Let's say we choose HEAD now, and then tomorrow you push 5 new commits.
Then the HEAD we choose today will tomorrow be seen just as randomly
chosen as any other commit. It will tomorrow not mean much more than
if we had clicked on the "I feel lucky" button then.


As a consequence, it is very difficult to express what glibc version is
running on a system, because it would only ever report '2.26' (or am I
mislead?), so it will be difficult to state "2.26 works in that setup"
when someone else states "2.26 does not work in that setup", just
because it is in fact not the same '2.26', being cut off of different
commits. A tag would actually have glibc version be '2.26.1' (or
whatever) so it would be easier to compare.


Thanks! ;-)

Regards,
Yann E. MORIN.

[0] yes, technically, you can move/remove a tag. That's just bad; don't
    do it! ;-)

[1] I will suppose that all commits that go to the maintenance branch
    are all fixing stuff (bug fix, security fix) and never a new feature
    or an API/ABI breakage or whatever. Just actual fixes. As such, they
    are all equal in importance: a fix is a fix; by the mere existence
    of it, it is required.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

Re: Glibc stable release process (Glibc 2.26.1)

[Jonathan Nieder]

Hi,

Yann E. MORIN wrote:

> The problem is not about representation of the version string. It is
> actually choosing what commit to use. As a downstream, we don't have
> many options when we need to choose a commit on the maintenance branch
> [1], i.e. either one of:
>   - use the HEAD of the branch
>   - use any one commit between the tag and the HEAD
>
> Let's say we choose HEAD now, and then tomorrow you push 5 new commits.
> Then the HEAD we choose today will tomorrow be seen just as randomly
> chosen as any other commit. It will tomorrow not mean much more than
> if we had clicked on the "I feel lucky" button then.

I don't really follow.  Would the same problem exist when choosing
between 2.26.1, 2.26.2, 2.26.3, etc?  Why isn't "use the latest
version from the branch" always the right answer?

> As a consequence, it is very difficult to express what glibc version is
> running on a system, because it would only ever report '2.26' (or am I
> mislead?),

Are you talking about version numbers from the package manager or from
somewhere else?

Thanks,
Jonathan

Re: Glibc stable release process (Glibc 2.26.1)

[Paul Eggert]

On 10/16/2017 10:34 AM, Yann E. MORIN wrote:
> The difference is where the identification comes from:
>    - a tag is generated is one location: upstream;
>    - git-describe is local and arbitrary to each downstream.
>
If you're just cloning from glibc upstream, then git-describe is not 
arbitrary to each downstream; it's the same for each downstream copy, 
and it's a portable identifier for whatever commit you choose.

If you're not just cloning, then you need a version number of your own 
anyway, as your version won't exactly match any upstream tags.

So I don't see the advantage of point-release tags for the scenario you 
describe.

Re: Glibc stable release process (Glibc 2.26.1)

[Romain Naour]

Hi All,

Le 16/10/2017 à 19:52, Jonathan Nieder a écrit :
> Hi,
> 
> Yann E. MORIN wrote:
> 
>> The problem is not about representation of the version string. It is
>> actually choosing what commit to use. As a downstream, we don't have
>> many options when we need to choose a commit on the maintenance branch
>> [1], i.e. either one of:
>>   - use the HEAD of the branch
>>   - use any one commit between the tag and the HEAD
>>
>> Let's say we choose HEAD now, and then tomorrow you push 5 new commits.
>> Then the HEAD we choose today will tomorrow be seen just as randomly
>> chosen as any other commit. It will tomorrow not mean much more than
>> if we had clicked on the "I feel lucky" button then.
> 
> I don't really follow.  Would the same problem exist when choosing
> between 2.26.1, 2.26.2, 2.26.3, etc?  Why isn't "use the latest
> version from the branch" always the right answer?
> 
>> As a consequence, it is very difficult to express what glibc version is
>> running on a system, because it would only ever report '2.26' (or am I
>> mislead?),
> 
> Are you talking about version numbers from the package manager or from
> somewhere else?

Here is the version reported by the libc on an embedded Linux system (i.e not a
Linux distro) without any package manager:

# /lib/libc.so.6
GNU C Library (Buildroot) stable release version 2.26, by Roland McGrath et al.
Copyright (C) 2017 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.
Compiled by GNU CC version 7.2.0.
Available extensions:
	crypt add-on version 2.1 by Michael Glad and others
	GNU Libidn by Simon Josefsson
	Native POSIX Threads Library by Ulrich Drepper et al
	BIND-8.2.3-T5B
libc ABIs: UNIQUE IFUNC
For bug reporting instructions, please see:
<http://www.gnu.org/software/libc/bugs.html>.

So without any help from the build system, we can't say what's the libc version.
This can be ether the official Glibc 2.26 release or the glibc-2.26-58-gf725563.

In the end, git describe is not enough. Making a stable release require updating
verison.h and create a git tag.

Note: See I'm using gcc 7.2.0, not gcc-7_1_0-release-372-g1bd23ca.

Best regards,
Romain

> 
> Thanks,
> Jonathan
> 

Re: Glibc stable release process (Glibc 2.26.1)

[Carlos O'Donell]

On 10/16/2017 01:58 PM, Romain Naour wrote:
> So without any help from the build system, we can't say what's the libc version.
> This can be ether the official Glibc 2.26 release or the glibc-2.26-58-gf725563.

You must consult the distribution's package database to know what you have
installed.

-- 
Cheers,
Carlos.

Re: Glibc stable release process (Glibc 2.26.1)

[Florian Weimer]

On 10/16/2017 10:58 PM, Romain Naour wrote:
> Here is the version reported by the libc on an embedded Linux system (i.e not a
> Linux distro) without any package manager:
> 
> # /lib/libc.so.6
> GNU C Library (Buildroot) stable release version 2.26, by Roland McGrath et al.
> Copyright (C) 2017 Free Software Foundation, Inc.
> This is free software; see the source for copying conditions.
> There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
> PARTICULAR PURPOSE.
> Compiled by GNU CC version 7.2.0.
> Available extensions:
> 	crypt add-on version 2.1 by Michael Glad and others
> 	GNU Libidn by Simon Josefsson
> 	Native POSIX Threads Library by Ulrich Drepper et al
> 	BIND-8.2.3-T5B
> libc ABIs: UNIQUE IFUNC
> For bug reporting instructions, please see:
> <http://www.gnu.org/software/libc/bugs.html>.
> 
> So without any help from the build system, we can't say what's the libc version.
> This can be ether the official Glibc 2.26 release or the glibc-2.26-58-gf725563.

Is there any system which builds from unmodified glibc sources, without 
any local changes and backports, and promises to remain this way for the 
future?

I don't think so.

Even if we had point releases, you'd still have to check the 
corresponding sources to see what the local modifications are.  Point 
releases really do not help you here.

Thanks,
Florian

Re: Glibc stable release process (Glibc 2.26.1)

[Carlos O'Donell]

On 09/29/2017 01:17 PM, Romain Naour wrote:
> We do understand that getting a new release out can be quite some work, and we
> do acknowledge the efforts that are made to provide those releases.
> 
> Yet, being able to refer to human-readable versions is very important.
> 
> Maybe a middle ground is that a tag is pushed to the repository, just to
> identify a specific sha1 as "this is 2.26.1". This is relatively lightweight (a
> signed tag) and gives downstream users that need it a clear identification of a
> blessed version.

I waited a while for this thread to die down, before summarizing.

Making releases requires resources, people, hardware, attention, and
more it has an opportunity cost.

The release process as we have it today is the realization of a minimized
process that attains the following important goals:

* Make a time boxed release every 6 months that is tested on all
  supported architectures by the machine maintainers themselves, and
  provides a strong foundation upon which distributions can build their
  operating system around. This release is the *basis* for ABI and API
  stability guarantees given to our users.

* Continue the stabilization efforts on a rolling branch that receives
  ABI/API stable changes. Each commit should be more stable than the
  last.

I worked with several downstream maintainers to get them involved in the
upstream stable branches. If you see something broken, please go upstream.
Cherry pick from master to the stable branch and sync again with downstream.
Debian does it. Fedora does it. Gentoo does it. Patching locally stops
becoming expedient once you get to pull in other well tested patches that
other distros have also tested.

In summary:

- Giving a name, or releasing a named point release from the release branches
  implies something it is not, namely a fully tested release. The release branch
  is incrementally tested and ready for distribution use. If distributions
  want to work together to coordinate and name specific release branch commits
  as being important, we can help facilitate that.

- Between Florian Weimer, Dmitry V. Levin, and others, tags have been added to
  git, along with Release process adjustments to keep adding these tags, which
  should add more descriptive names to all the branches. Having meaningful names
  for things is important.

-- 
Cheers,
Carlos.

Re: Glibc stable release process (Glibc 2.26.1)

[Yann E. MORIN]

Carlos, All,

On 2017-10-17 14:19 -0700, Carlos O'Donell spake thusly:
> On 09/29/2017 01:17 PM, Romain Naour wrote:
> > We do understand that getting a new release out can be quite some work, and we
> > do acknowledge the efforts that are made to provide those releases.
> > 
> > Yet, being able to refer to human-readable versions is very important.
> > 
> > Maybe a middle ground is that a tag is pushed to the repository, just to
> > identify a specific sha1 as "this is 2.26.1". This is relatively lightweight (a
> > signed tag) and gives downstream users that need it a clear identification of a
> > blessed version.
> 
> I waited a while for this thread to die down, before summarizing.

Thank you for this summary; it is much appreciated. Thanks also to all
involved in the discussion. :-)

Regards,
Yann E. MORIN.

> Making releases requires resources, people, hardware, attention, and
> more it has an opportunity cost.
> 
> The release process as we have it today is the realization of a minimized
> process that attains the following important goals:
> 
> * Make a time boxed release every 6 months that is tested on all
>   supported architectures by the machine maintainers themselves, and
>   provides a strong foundation upon which distributions can build their
>   operating system around. This release is the *basis* for ABI and API
>   stability guarantees given to our users.
> 
> * Continue the stabilization efforts on a rolling branch that receives
>   ABI/API stable changes. Each commit should be more stable than the
>   last.
> 
> I worked with several downstream maintainers to get them involved in the
> upstream stable branches. If you see something broken, please go upstream.
> Cherry pick from master to the stable branch and sync again with downstream.
> Debian does it. Fedora does it. Gentoo does it. Patching locally stops
> becoming expedient once you get to pull in other well tested patches that
> other distros have also tested.
> 
> In summary:
> 
> - Giving a name, or releasing a named point release from the release branches
>   implies something it is not, namely a fully tested release. The release branch
>   is incrementally tested and ready for distribution use. If distributions
>   want to work together to coordinate and name specific release branch commits
>   as being important, we can help facilitate that.
> 
> - Between Florian Weimer, Dmitry V. Levin, and others, tags have been added to
>   git, along with Release process adjustments to keep adding these tags, which
>   should add more descriptive names to all the branches. Having meaningful names
>   for things is important.
> 
> -- 
> Cheers,
> Carlos.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

Re: Glibc stable release process (Glibc 2.26.1)

[Romain Naour]

Hi All,

Le 17/10/2017 à 23:31, Yann E. MORIN a écrit :
> Carlos, All,
> 
> On 2017-10-17 14:19 -0700, Carlos O'Donell spake thusly:
>> On 09/29/2017 01:17 PM, Romain Naour wrote:
>>> We do understand that getting a new release out can be quite some work, and we
>>> do acknowledge the efforts that are made to provide those releases.
>>>
>>> Yet, being able to refer to human-readable versions is very important.
>>>
>>> Maybe a middle ground is that a tag is pushed to the repository, just to
>>> identify a specific sha1 as "this is 2.26.1". This is relatively lightweight (a
>>> signed tag) and gives downstream users that need it a clear identification of a
>>> blessed version.
>>
>> I waited a while for this thread to die down, before summarizing.
> 
> Thank you for this summary; it is much appreciated. Thanks also to all
> involved in the discussion. :-)

Yes, thank you all for the discussion :)

> 
> Regards,
> Yann E. MORIN.
> 
>> Making releases requires resources, people, hardware, attention, and
>> more it has an opportunity cost.

Indeed, I understand that.

>>
>> The release process as we have it today is the realization of a minimized
>> process that attains the following important goals:
>>
>> * Make a time boxed release every 6 months that is tested on all
>>   supported architectures by the machine maintainers themselves, and
>>   provides a strong foundation upon which distributions can build their
>>   operating system around. This release is the *basis* for ABI and API
>>   stability guarantees given to our users.
>>
>> * Continue the stabilization efforts on a rolling branch that receives
>>   ABI/API stable changes. Each commit should be more stable than the
>>   last.
>>
>> I worked with several downstream maintainers to get them involved in the
>> upstream stable branches. If you see something broken, please go upstream.
>> Cherry pick from master to the stable branch and sync again with downstream.
>> Debian does it. Fedora does it. Gentoo does it. Patching locally stops
>> becoming expedient once you get to pull in other well tested patches that
>> other distros have also tested.
>>
>> In summary:
>>
>> - Giving a name, or releasing a named point release from the release branches
>>   implies something it is not, namely a fully tested release. The release branch
>>   is incrementally tested and ready for distribution use. If distributions
>>   want to work together to coordinate and name specific release branch commits
>>   as being important, we can help facilitate that.
>>
>> - Between Florian Weimer, Dmitry V. Levin, and others, tags have been added to
>>   git, along with Release process adjustments to keep adding these tags, which
>>   should add more descriptive names to all the branches. Having meaningful names
>>   for things is important.

Ok, thanks for the summary.

I'll try to give some feedback from Buildroot before the next glibc release.

Best regards,
Romain

>>
>> -- 
>> Cheers,
>> Carlos.
>