versioning

versioning

[Ulrich Drepper]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

The reason I've stayed with version 2.3.2 for so long is that, as the
last release cycle clearly showed, it does not make much sense to put
out test releases, wait, and then make a release.  Nobody[*] tests it
anyway.  So if the code builds for the people on this list, it is good.
 And it should always build, maybe one or two days breakage, but that
has been a rare occasion.

Therefore I want to go to a time-based version numbering.  This will
help to pinpoint the base version a bug reporter is using easier.  The
sub-release number should perhaps be bumped every week, maybe every two
week.  I.e., we'd have 2.3.3 next, a week later 2.3.4.  I stepping to
2.4 is a bit too irritating.

This will lead to high numbers, sure.  But we can also say that after
2.3.N, for N == 51 or so, we automatically go to 2.4.  This will give a
predictable schedule, especially if some interface changes are
associated with a minor version jump (like dropping LT in 2.4).

I think this will work fine since I don't see any big changes on the
horizon.  We are mainly in maintenance mode, with optimizations and
adjustment for recent additions/changes to specifications.  None of this
requires a separate development tree as we used to have in the past.


If somebody sees a reason why this shouldn't be done, speak up now.
Otherwise I'll put the mechanisms in place to get this process going.


[*] Well, one or two people do, but this is insignificant given the
number of users.

- -- 
➧ Ulrich Drepper ➧ Red Hat, Inc. ➧ 444 Castro St ➧ Mountain View, CA ❖
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.3 (GNU/Linux)

iD8DBQE/yEXf2ijCOnn/RHQRAtW8AJ9a3stIcaG5c6nwHq04La0Ux1XObgCfcS3Q
wGIuKoHdDeWvmDx6WvK9TRM=
=QGbk
-----END PGP SIGNATURE-----

Re: versioning

[Andreas Jaeger]

[-- Attachment #1: Type: text/plain, Size: 2362 bytes --]

Ulrich Drepper <drepper@redhat.com> writes:

> The reason I've stayed with version 2.3.2 for so long is that, as the
> last release cycle clearly showed, it does not make much sense to put
> out test releases, wait, and then make a release.  Nobody[*] tests it
> anyway.  So if the code builds for the people on this list, it is good.
>  And it should always build, maybe one or two days breakage, but that
> has been a rare occasion.
>
> Therefore I want to go to a time-based version numbering.  This will
> help to pinpoint the base version a bug reporter is using easier.  The
> sub-release number should perhaps be bumped every week, maybe every two
> week.  I.e., we'd have 2.3.3 next, a week later 2.3.4.  I stepping to
> 2.4 is a bit too irritating.

Yes, we need more releases.  Distributions, e.g. both Red Hat and SuSE
use some CVS version of the day and which shows that going back to the
latest release is not the right thing to do.

> This will lead to high numbers, sure.  But we can also say that after
> 2.3.N, for N == 51 or so, we automatically go to 2.4.  This will give a
> predictable schedule, especially if some interface changes are
> associated with a minor version jump (like dropping LT in 2.4).


> I think this will work fine since I don't see any big changes on the
> horizon.  We are mainly in maintenance mode, with optimizations and
> adjustment for recent additions/changes to specifications.  None of this
> requires a separate development tree as we used to have in the past.

I would still do some prominent official releases.  So, e.g. 2.4.0
could be an official release with announcement, tarball etc.
Everything else would be development releases.  We just make an
official release either time-driven, e.g. every half year, or feature
driven.

> If somebody sees a reason why this shouldn't be done, speak up now.
> Otherwise I'll put the mechanisms in place to get this process going.

In general I would say go ahead - but let's discuss the "official
release" issue first.


> [*] Well, one or two people do, but this is insignificant given the
> number of users.


Andreas
-- 
 Andreas Jaeger, aj@suse.de, http://www.suse.de/~aj
  SuSE Linux AG, Deutschherrnstr. 15-19, 90429 Nürnberg, Germany
   GPG fingerprint = 93A3 365E CE47 B889 DF7F  FED1 389A 563C C272 A126

[-- Attachment #2: Type: application/pgp-signature, Size: 188 bytes --]

Re: versioning

[Ulrich Drepper]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Andreas Jaeger wrote:

> Yes, we need more releases.

I disagree.  We don't need releases.  Just anchor points which help
pin-pointing problems easier.


> I would still do some prominent official releases.

Why?  You just say what you want but not why.  I simply don't see any
reason to do something extraordinary.  The minor version will be bump
when it is the time.  This won't be different from any other bump.
Everybody who is distribution glibc should _always_ use the top of the tree.

- -- 
➧ Ulrich Drepper ➧ Red Hat, Inc. ➧ 444 Castro St ➧ Mountain View, CA ❖
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.3 (GNU/Linux)

iD8DBQE/yN4u2ijCOnn/RHQRAmj/AJ4+yFULQ2lJi+R9DZdZW+lirzRzcACfT6Yt
m0rUxsjorLNKVeL+0GOkjsA=
=pGj/
-----END PGP SIGNATURE-----

Re: versioning

[Andreas Jaeger]

[-- Attachment #1: Type: text/plain, Size: 1023 bytes --]

Ulrich Drepper <drepper@redhat.com> writes:

> Andreas Jaeger wrote:
>
>> Yes, we need more releases.
>
> I disagree.  We don't need releases.  Just anchor points which help
> pin-pointing problems easier.
>
>
>> I would still do some prominent official releases.
>
> Why?  You just say what you want but not why.  I simply don't see any
> reason to do something extraordinary.  The minor version will be bump
> when it is the time.  This won't be different from any other bump.

I want to have some major release that we put on ftp.gnu.org as
tarball and announce properly.  I'm under the impression that the
bi-weekly "anchor-points" are not put up anywhere.

> Everybody who is distribution glibc should _always_ use the top of the tree.

There're outside some people that do not use cvs,

Andreas
-- 
 Andreas Jaeger, aj@suse.de, http://www.suse.de/~aj
  SuSE Linux AG, Deutschherrnstr. 15-19, 90429 Nürnberg, Germany
   GPG fingerprint = 93A3 365E CE47 B889 DF7F  FED1 389A 563C C272 A126

[-- Attachment #2: Type: application/pgp-signature, Size: 188 bytes --]

Re: versioning

[Ulrich Drepper]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Andreas Jaeger wrote:

> I want to have some major release that we put on ftp.gnu.org as
> tarball and announce properly.  I'm under the impression that the
> bi-weekly "anchor-points" are not put up anywhere.

Somebody interested in this can surely create a cronjob to do just that.
 There is no reason why this shouldn't be possible.

I personally think tarballs are a completely unsuitable method for
distributing glibc.  It's OK for some other projects, but every time
such a glibc tarball has been created, it was already outdated.

The main and only useful method to distribute glibc is via cvs and it
has been clearly shown in the last months that this is perfectly acceptable.

And re announcements, again the question is "why?".  Everybody who has
to provide glibc for a larger group or is interested for herself will
notice or be pointed to the new policy and know that, say, every Monday
morning there is a new "release".  But, if this was the only motivation
for updating glibc in the past, that person has done an extremely poor
job[*] by not including all the important changes which went in since
the release.  Therefore, announcements serve no purpose (except ego
building on the authors' side) since everybody who reads the text and
grabbed the tarball also had to go to the current sources.


To summarize: IMO tarballs are the wrong form of distribution and
announcements are misleading since they "announce" a code base which by
itself is not the recommended one.


> There're outside some people that do not use cvs,

Then those have to go with the times unless somebody volunteers to
create tarballs (which is a mistake but I won't stop anybody making it).
 Using CVS is not the right way to distribute for all projects.  But the
glibc sources are stable and do not break.  At least not in ways which
would be noticed in the way releases were prepared in the past.


[*] E.g., those people who still report problems against the 2.3.2 tarball.

- -- 
➧ Ulrich Drepper ➧ Red Hat, Inc. ➧ 444 Castro St ➧ Mountain View, CA ❖
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.3 (GNU/Linux)

iD8DBQE/yOt+2ijCOnn/RHQRAkCHAKCAueKHYWz/jFHy+whp1ENUPsa6fQCfeX98
vMng+Rj/qoFDjIHQUO2xaIw=
=f7qn
-----END PGP SIGNATURE-----

Re: versioning

[Jeff Bailey]

[-- Attachment #1: Type: text/plain, Size: 897 bytes --]

On Sat, 2003-11-29 at 02:08, Ulrich Drepper wrote:

> Therefore I want to go to a time-based version numbering.  This will
> help to pinpoint the base version a bug reporter is using easier.  The
> sub-release number should perhaps be bumped every week, maybe every two
> week.  I.e., we'd have 2.3.3 next, a week later 2.3.4.  I stepping to
> 2.4 is a bit too irritating.

If we're looking at a time-based scheme, perhaps the date can be
incorporated easily.  The middle digit could easily represent the number
of years since 2000, the final digit the week number, so we'd be at
2.3.49 now.

That takes care of the wrapping issue, and provides useful date
information at a quick glance.

Tks,
Jeff Bailey

-- 
In the United States, there isn't a government database that hasn't been
misused by the very people entrusted with keeping its information safe.
 - Bruce Schneier

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: versioning

[Roland McGrath]

I don't like this plan.  I would like to fully address all the concerns
that motivated it, but I think we can find different and better ways to
accomplish that.  I hope you will take the time to help me understand fully.

It is indeed useful to have manually designated releases, "formal" releases
if you like.  The reason that makes these desireable for any piece of
software is that making such a release is the way that we, the maintainers,
announce that the state of the code is "good and stable"--something people
can expect to upgrade to without unknown new pain, and that they might want
to upgrade to if they are going to put off the next upgrade for some period
of time beyond when the very next usable thing can be had.  It is not
really the case that the development code is "always good with rare, brief
exceptions"--it is often a moving target for significant periods in the
natural course of development and maintenance (a few weeks of regex
instability here, a few weeks of dynamic linker instability there) and
there is no reason it shouldn't be.  

Moreover, in the case of glibc there is a more compelling reason to want
clearly distinguished releases.  That is, these are the points of API/ABI
stability.  There is no reason we should not add new symbols or new
versions of existing symbols with the relative freedom that we do now.
During the course of development, we may add a new interface and then
decide to change its details a week or three later.  We have always made
clear that new and changed interfaces are subject to continual change
between releases.  It should remain so.  Bottom line, to me what a "formal"
glibc release means is a closing of the given GLIBC_x.y.z version set and a
promise to provide binary compatibility for correct programs built from
that point forward.  I presume that the reasons why it's bad for us, the
glibc maintainers, to potentially support a proliferation of symbol
versions are obvious, both the technical ones (version name explosion
slowing searches and inflating binaries, symbol table and text inflation
adding overhead) and the practical ones (cruft in the sources, future
maintenance burden of compatibility versions).

For these reasons, I object to abandoning formal glibc releases.  
Now I would like to explore the root issues that motivated Ulrich's plan.

I quite agree that the old plan of making official test releases, waiting,
and declaring them tested and good enough before each official release, is
not working any more and has not been working well for quite some time.
The situation for some time now has been that some given state of
development from cvs is taken as the true basis for the whole-system
distribution versions of glibc such as Debian's and Fedora/Red Hat's called
2.3.2-NN and so forth.  The given snapshot chosen by each distribution's
glibc package maintainers is what actually receives a lot of testing and
gets deployed.  Overall, this is a perfectly fine state of affairs and
exactly the way we want to get leverage from distribution-specific efforts.
The problem is that this has not been feeding back into the mainline
development/release activities in any orderly or well-understood way.

I think the situation we want is that a formal glibc release is a statement
after the fact.  That is, when a given snapshot state has gotten good
testing via its use in whole system distributions and been declared stable
in practice, then we christen that state with a release number.  That
should be a wholly painless and easy process for any authorized maintainer
to do, i.e. running a script that goes quickly and maybe doing a GPG
signature.  There is no reason this shouldn't happen frequently, at least a
hell of a lot more frequently than glibc releases have been heretofore.

What I would like to see is a regular system of snapshots.  This means that
GLIBC_i.j.k.yyyymmdd means exactly the same set of sources to everyone.
(That's easy to do with tags or just with a clear specification of 0:00 UTC
on each YYYY/MM/DD as the moment of repository state we're referring to.)
tarballs (whether with or without CVS/ subdirs) are useful for some people,
and it's not hard to make them available automatically, so we should do so.
It is my hope that the glibc packages in development versions of each
distribution can be based on canonical dated snapshots with vastly smaller
numbers of additional distribution-specific patches than what we see now.

I would like to do whatever needs to be done to make the physical details
of cutting a release as trivial and painless as possible.  None of this
automation should be at all hard.  (We can start with dumping the "make dist"
cruft and using simple scripts around cvs operations to make tarballs.)
Whenever there is consensus that GLIBC_i.j.k.yyyymmdd is a stable point,
it can be retroactively declared GLIBC_i.j.(k+1), diddled, tagged, and
released with the push of a button.

If there are root motivations other than the ones that I have raised,
please describe them explicitly.  Once I'm sure that all these are on the
table, I would be happy to discuss how the suggestions I have made here do
or don't address all the important issues.


Thanks,
Roland

Re: versioning

[Andreas Jaeger]

[-- Attachment #1: Type: text/plain, Size: 7168 bytes --]

Roland McGrath <roland@redhat.com> writes:

> I don't like this plan.  I would like to fully address all the concerns
> that motivated it, but I think we can find different and better ways to
> accomplish that.  I hope you will take the time to help me understand fully.
>
> It is indeed useful to have manually designated releases, "formal" releases
> if you like.  The reason that makes these desireable for any piece of
> software is that making such a release is the way that we, the maintainers,
> announce that the state of the code is "good and stable"--something people
> can expect to upgrade to without unknown new pain, and that they might want
> to upgrade to if they are going to put off the next upgrade for some period
> of time beyond when the very next usable thing can be had.  It is not
> really the case that the development code is "always good with rare, brief
> exceptions"--it is often a moving target for significant periods in the
> natural course of development and maintenance (a few weeks of regex
> instability here, a few weeks of dynamic linker instability there) and
> there is no reason it shouldn't be.  
>
> Moreover, in the case of glibc there is a more compelling reason to want
> clearly distinguished releases.  That is, these are the points of API/ABI
> stability.  There is no reason we should not add new symbols or new
> versions of existing symbols with the relative freedom that we do now.
> During the course of development, we may add a new interface and then
> decide to change its details a week or three later.  We have always made
> clear that new and changed interfaces are subject to continual change
> between releases.  It should remain so.  Bottom line, to me what a "formal"
> glibc release means is a closing of the given GLIBC_x.y.z version set and a
> promise to provide binary compatibility for correct programs built from
> that point forward.  I presume that the reasons why it's bad for us, the
> glibc maintainers, to potentially support a proliferation of symbol
> versions are obvious, both the technical ones (version name explosion
> slowing searches and inflating binaries, symbol table and text inflation
> adding overhead) and the practical ones (cruft in the sources, future
> maintenance burden of compatibility versions).
>
> For these reasons, I object to abandoning formal glibc releases.  

I agree with these stability issues.  To have a stable version (ABI
wise) every two weeks restrains us.

> Now I would like to explore the root issues that motivated Ulrich's plan.
>
> I quite agree that the old plan of making official test releases, waiting,
> and declaring them tested and good enough before each official release, is
> not working any more and has not been working well for quite some time.
> The situation for some time now has been that some given state of
> development from cvs is taken as the true basis for the whole-system
> distribution versions of glibc such as Debian's and Fedora/Red Hat's called
> 2.3.2-NN and so forth.  The given snapshot chosen by each distribution's
> glibc package maintainers is what actually receives a lot of testing and
> gets deployed.  Overall, this is a perfectly fine state of affairs and
> exactly the way we want to get leverage from distribution-specific efforts.
> The problem is that this has not been feeding back into the mainline
> development/release activities in any orderly or well-understood way.
>
>
> I think the situation we want is that a formal glibc release is a statement
> after the fact.  That is, when a given snapshot state has gotten good
> testing via its use in whole system distributions and been declared stable
> in practice, then we christen that state with a release number.  That
> should be a wholly painless and easy process for any authorized maintainer
> to do, i.e. running a script that goes quickly and maybe doing a GPG
> signature.  There is no reason this shouldn't happen frequently, at least a
> hell of a lot more frequently than glibc releases have been heretofore.

I see some practical problem with that.  What we at SuSE do is to take
the latest release or a CVS version, and use that as basis.  When we
get to a release of our distribution, we only add those patches from
CVS that we consider important, e.g. we might not add the latest new
feature since it's unstable but add all bug fixes.  So, our SuSE glibc
is basically the CVS snapshot of day X plus some of the later patches.
I have no problem calling the CVS snapshot a release but it might
contain known bugs.

What might work is a release branch.  I would make a branch point and
port all bugfixes from mainline to this branch and then release a
glibc version - let's say a few weeks (2-4) after the branch.  The
next distributor would do the same at some point of time - and we
might even see several release branches at one time :-(.

> What I would like to see is a regular system of snapshots.  This means that
> GLIBC_i.j.k.yyyymmdd means exactly the same set of sources to everyone.

That's a good idea.  We enhanced our glibc already so that it reports
a day:
$ /lib/libc.so.6 
GNU C Library stable release version 2.3.2 (20030827), by Roland McGrath et al.
[...]

> (That's easy to do with tags or just with a clear specification of 0:00 UTC
> on each YYYY/MM/DD as the moment of repository state we're referring to.)
> tarballs (whether with or without CVS/ subdirs) are useful for some people,
> and it's not hard to make them available automatically, so we should do so.
> It is my hope that the glibc packages in development versions of each
> distribution can be based on canonical dated snapshots with vastly smaller
> numbers of additional distribution-specific patches than what we see now.

That could be done, I agree.  We should put the date in the installed
glibc, e.g. as above.  But what's the difference between a version
number and a date in this regard?

> I would like to do whatever needs to be done to make the physical details
> of cutting a release as trivial and painless as possible.  None of this
> automation should be at all hard.  (We can start with dumping the "make dist"
> cruft and using simple scripts around cvs operations to make tarballs.)
> Whenever there is consensus that GLIBC_i.j.k.yyyymmdd is a stable point,
> it can be retroactively declared GLIBC_i.j.(k+1), diddled, tagged, and
> released with the push of a button.

Again we have a ABI stability issue here.  If a version is declared
stable after some weeks, we might have changed the ABI...

> If there are root motivations other than the ones that I have raised,
> please describe them explicitly.  Once I'm sure that all these are on the
> table, I would be happy to discuss how the suggestions I have made here do
> or don't address all the important issues.

Thanks for your suggestions!

Andreas
-- 
 Andreas Jaeger, aj@suse.de, http://www.suse.de/~aj
  SuSE Linux AG, Deutschherrnstr. 15-19, 90429 Nürnberg, Germany
   GPG fingerprint = 93A3 365E CE47 B889 DF7F  FED1 389A 563C C272 A126

[-- Attachment #2: Type: application/pgp-signature, Size: 188 bytes --]

Re: versioning

[Roland McGrath]

> I agree with these stability issues.  To have a stable version (ABI
> wise) every two weeks restrains us.
[...]
> That could be done, I agree.  We should put the date in the installed
> glibc, e.g. as above.  But what's the difference between a version
> number and a date in this regard?

The difference between a date and a version number is what convention
we associate with them.  Basically, the de facto convention is that
version numbers are associated with firm APIs and ABIs.  The reason I
suggested snapshot dates as a nomenclature is to have something that is
clearly not a deliberate stable point of API/ABI development.  That
makes quite distinct the meaning of the next assigned version number,
which can correspond to symbol version set names denoting newly-firmed
ABIs.  My motivation is to have agreement that is clear amongst
ourselves, and clear to users, when ABIs are firm and when they are
not.  I am open to whatever details get us there.

> What might work is a release branch.  I would make a branch point and
> port all bugfixes from mainline to this branch and then release a
> glibc version - let's say a few weeks (2-4) after the branch.  The
> next distributor would do the same at some point of time - and we
> might even see several release branches at one time :-(.

My suggestion about retroactively declaring releases from prior
snapshots was simply motivated by the idea of formalizing existing
communal practice.  I didn't suggest branches because we have not (as a
community project) been using them in a meaningful way before now.  I
think doing so might well be a mighty fine thing.  I am only looking
for whatever solutions will in fact be a useful formalization of how
people do now work, or would prefer to be able to work, together on glibc.

The reality is that each distribution maintainer does have a release
branch.  There is no shame in having several such release branches--if
they are actively maintained branches representing the releases major
distributions are actually using.  These forks exist already in the
world and in the daily work of the people who participate here in
communal glibc development.  Each distribution's glibc maintenance
operates similarly to what you described for SuSE.  Jakub does that for
Fedora/Red Hat.  Others on this list do the same for Debian.  This has
always been done separate from the community glibc source repository.
If those maintainers would like to use branches in the shared public
repository for this, I think that would be great.  Jakub maintains Red
Hat's branch of GCC this way, and that seems to be working out well.  I
think keeping the different branches in a shared repository is a great
way to facilitate everyone's tracking of development and maintenance.
I figure anything that streamlines the process of sharing code is a
boon to overall efficiency of maintenance.  But this can only be
worthwhile if the distribution maintainers want to do it.  I do not
speak for any of them.  If people don't want to do it, it's moot.

An important component of Ulrich's original notion was the need to
recognize the way glibc release, distribution, and maintenance is in
fact taking place.  We don't need bureaucracy to help the communal
effort catch up.  We need to start with the separate efforts that have
been doing the job in the real world, leverage those, formalize what
really does take place already, and collectively improve it from there.

There will probably always be differences between the glibc release
branches used by different distributions.  Each has its own
compatibility obligations, usage circumstances, and policy decisions
that may well never be agreed as appropriate for the official community
standard glibc code base.  That is fine.  What we are concerned with
here is that which can be shared.

An official release branch is only useful if the distribution
maintainers find it of practical aid to their productivity and
effectiveness.  It's not to anyone's benefit if maintaining the release
branch per se is an extra chore with no inherent utility.  If there are
multiple distribution release branches, then I would hope that it would
be natural and easy to maintain a communal release branch that contains
exactly the fixes that everyone is going to use in their own releases.
If the forks keep in synch this way so that when code is taken into a
branch, its text matches exactly with the shared release branch, the
diffs between a distribution's glibc and the canonical release branch
will be just the minimum that they have to be.  Then the tip of the
shared branch can be named an official release yielding canonical
tarball contents, and each different flavor of glibc source package
will need only that and a small distribution-specific amount of
patching.  I think it would be great to do this in the future.  But
there is no point in me specifying how it would be.  The distribution
maintainers have to say what they want to do.  I am eager to hear.

We have not really had "release freezes" (at least lately).  But there
are "quiet" periods where ABIs are not being perturbed and everyone is
focussed on stabilizing changes.  For example, since version.h was
bumped to 2.3.3, there were no ABI additions and there were lots of
necessary bug fixes in recent weeks.  Only in the last few days have
2.3.4 symbols been added.  Up until then, the trunk could have been
said to have been in release freeze for 2.3.3 (if we had such a formal
process, which we don't).  If we did make release branches, we might
not want to have made one before 2004-1-13 or so.  That is, the way
branches are done should follow what development is doing at any given
time, not try to be rigid.  OTOH, if we had a formal process of
freezing ABI changes to version sets, we'd better have done that
earlier, since OS distributions shipped with GLIBC_2.3.3 version sets
before then.  To me that emphasizes the need to better formalize what
is going on now, without trying to slow it down.

I've written a lot.  But I think we really just need to do a few
straightforward things to cooperate in a more efficient way that helps
all of our work.  If everyone is committed, I think making it happen
will come easily, and I look forward to facilitating that.


Thanks,
Roland