Re: Releasing bzip2 1.0.7 and 1.1.0 ? Was: [PATCH] bzip2: Fix return value when combining --test,-t and -q.

[Mark Wielaard <mark@klomp.org>]

Hi,

On Tue, 2019-06-25 at 14:08 -0500, Federico Mena Quintero wrote:
> On Tue, 2019-06-25 at 20:09 +0200, Mark Wielaard wrote:
> 
> > With that I think we are ready to do a bzip2 1.0.7 release simply
> > by
> > running
> > ./prepare-release.sh 1.0.7 and ./update-release.sh 1.0.7.
> 
> I like this script!  Thanks for writing it!

It is a bit of a hack to be honest. I mostly did it this way because
there is no modern build system yet :) I don't know how Julian did
releases. But there are way too many embedded version strings and dates
in the sources (and they don't even all use the same format!)

> > So if the above sounds like a good plan, then we can execute it
> > now.
> > Push out bzip2 1.0.7 and put the 1.0.x branch in hibernation, just
> > for
> > security or major bug fixes. Then we can concentrate on making sure
> > that a bzip2 1.1.X (or maybe even call it 2.0?) has a modern build
> > system and adds things like O_CLOEXEC support, etc.
> 
> I'm sorry for not replying to your mails earlier.  At work they just
> moved us to another email provider, and I've had some trouble fixing
> my
> email setup (SMTP servers, etc.) to make everything work as before.
> 
> I am... also ready to release 1.0.7 :-P  The work you've done in
> updating the release scripts is valuable, and I'd like to propose
> this
> instead:
> 
> * Put prepare-release.sh in the gitlab repository, and adjust it for
> the changes there.

If we are moving things around and have a modern build system it might
be better to actually remove most of the embedded versions/dates from
the sources and where they are really needed use configure to set them.
There actually already seems to be a merge request for that:
https://gitlab.com/federicomenaquintero/bzip2/merge_requests
(Although for autotools, which already has been removed again?)

> * Release 1.0.7 from the gitlab repo (it already has the NEWS file
> with
> updates, while I think your repo needs a CHANGES to be
> written?).  I've
> already tested that this works; "ninja dist" takes care of it.

Yes, the CHANGES file is the only thing the prepare-release.sh script
currently makes you do by hand (because there are no older tags yet, it
could in theory be replaced with a simple git shortlog), but there are
only 5 meaningful changes:

* Fix undefined behavior in the macros SET_BH, CLEAR_BH, & ISSET_BH

* bzip2: Fix return value when combining --test,-t and -q.

* bzip2recover: Fix buffer overflow for large argv[0] 

* bzip2recover: Fix use after free issue with outFile (CVE-2016-3189)

* Make sure nSelectors is not out of range (CVE-2019-12900)

> * Keep the soname changes from the gitlab repo.  I think the original
> Makefile-libbz2_so is incorrect in how it sets the soname (or at
> least
> libtool thinks so), and both openSUSE and Fedora managed to change it
> to a single digit instead of "1.0".  I *think* Debian, and distros
> which picked up their patches from there, simply never changed this
> and
> that's why they kept the "1.0" in place.

libtool doesn't support it, but libtool has to work on lots of unix-
like systems. The SONAME isn't technically wrong. It is just not what
we consider good practice these days. Unfortunately lots of distros
rely on this particular SONAME (at least everything based on Debian,
including Ubuntu and PureOS) use libbz2.so.1.0 and they simply cannot
change it, or they break backward compatibility. So I think any new
build system should at least provide the option of keeping the old
SONAME.

> Whatever we change the soname to, it's going to be incompatible with
> some distros - and they are already used to patching things, anyway. 
> Maybe purely for my own convenience I'd like to remove the patches
> from
> openSUSE - the current soname in the gitlab repo allows for that.

Yes, and I think lots of other distros (Fedora and CentOS based ones at
least) would also like that.

> I would be fine with releasing 1.0.7 from the sourceware repo, and
> then
> a subsequent 1.1 with all the changes from gitlab - but I think this
> will cause extra work for the distros.  First they'll have to see
> which
> of their patches need to be removed, and they'll still need to patch
> the build system.  And with 1.1 released, they'll need to do it all
> over again.
>
> I think we can minimize the work for distros if we update the patches
> and the build system in a single release.  They already know how to
> handle Meson more or less automatically, while leaving the original
> bzip2 Makefiles in place means that each distro has to patch to their
> own hand-rolled build system all over again.

I am not too concerned about the (larger) distros. Although I think it
would be good to at least provide a simple option to change the SONAME.
The "classic" build does that by simply editing/patching it, what the
distros probably already do. But the distros will probably just yell
and scream at us, and figure it out. I am more concerned about users we
don't really know about, maybe on non-GNU/Linux systems that have just
been using 1.0.6 for years as is. Putting out something with a new
build system, that by default generates an incompatible SONAME and some
changes that maybe need newer features not easily detected on those
systems might cause them to not upgrade at all.

I realize I am probably overly conservative, but I really like to keep
a 1.0.x branch going just for ancient users (I expect the distro to
switch to a more modern release, once they figure out the SONAME
thingy). And if we have that ancient branch, where we can just put
security fixes on, and nothing else, then we can be even more creative
on the new modern 1.1.x (2.0.x?) branch.

> [I've sent a few subscribe requests to this mailing list as
> federico@gnome.org, but it's not sending me the confirmation mail...
> do you know who may be able to kick the list software?]

That is odd. You could try contacting overseers@sourceware.org or go on
irc.freenode.net #overseers to see if someone knows what is going on.

Cheers,

Mark