* [PATCH] bzip2: Fix return value when combining --test,-t and -q. @ 2019-01-01 0:00 Mark Wielaard 2019-01-01 0:00 ` Santiago Ruano Rincón 0 siblings, 1 reply; 14+ messages in thread From: Mark Wielaard @ 2019-01-01 0:00 UTC (permalink / raw) To: bzip2-devel Cc: Anibal Monsalve Salazar, Santiago Ruano Rincón, Anthony Fok, Mark Wielaard Hi, bzip2 lost its domain and got a new home at https://sourceware.org/bzip2/ It also didn't see a release for a very long time. Causing various patches used by distros to not have been integrated upstream. We are trying to collect them all and do a new release. The following patch comes from Debian. Please let us know if we missed some others. Thanks, Mark When passing -q to get quiet output --test would not display an error message, but would also suppress the exit 2 code to indicate the file was corrupt. Only suppress the error message with -q, not the exit value. This patch comes from Debian. "bunzip2 -qt returns 0 for corrupt archives" https://bugs.debian.org/279025 --- bzip2.c | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/bzip2.c b/bzip2.c index 854a2bb..63649f6 100644 --- a/bzip2.c +++ b/bzip2.c @@ -2003,12 +2003,14 @@ IntNative main ( IntNative argc, Char *argv[] ) testf ( aa->name ); } } - if (testFailsExist && noisy) { - fprintf ( stderr, - "\n" - "You can use the `bzip2recover' program to attempt to recover\n" - "data from undamaged sections of corrupted files.\n\n" - ); + if (testFailsExist) { + if (noisy) { + fprintf ( stderr, + "\n" + "You can use the `bzip2recover' program to attempt to recover\n" + "data from undamaged sections of corrupted files.\n\n" + ); + } setExit(2); exit(exitValue); } -- 1.8.3.1 ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH] bzip2: Fix return value when combining --test,-t and -q. 2019-01-01 0:00 [PATCH] bzip2: Fix return value when combining --test,-t and -q Mark Wielaard @ 2019-01-01 0:00 ` Santiago Ruano Rincón 2019-01-01 0:00 ` Mark Wielaard 0 siblings, 1 reply; 14+ messages in thread From: Santiago Ruano Rincón @ 2019-01-01 0:00 UTC (permalink / raw) To: Mark Wielaard Cc: bzip2-devel, Anibal Monsalve Salazar, Anthony Fok, Federico Mena Quintero [-- Attachment #1: Type: text/plain, Size: 678 bytes --] Hi, El 24/06/19 a las 09:40, Mark Wielaard escribió: > Hi, > > bzip2 lost its domain and got a new home at https://sourceware.org/bzip2/ > It also didn't see a release for a very long time. Causing various patches > used by distros to not have been integrated upstream. We are trying to > collect them all and do a new release. Thanks for this. > > The following patch comes from Debian. > Please let us know if we missed some others. For information, Federico Mena (in CC) is taking over the development of bzip2: https://people.gnome.org/~federico/blog/maintaining-bzip2.html You should consider coordinating your efforts! Cheers, -- Santiago [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 228 bytes --] ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH] bzip2: Fix return value when combining --test,-t and -q. 2019-01-01 0:00 ` Santiago Ruano Rincón @ 2019-01-01 0:00 ` Mark Wielaard 2019-01-01 0:00 ` Releasing bzip2 1.0.7 and 1.1.0 ? Was: " Mark Wielaard 2019-01-01 0:00 ` Some cherry-picks (Was: [PATCH] bzip2: Fix return value when combining --test,-t and -q.) Mark Wielaard 0 siblings, 2 replies; 14+ messages in thread From: Mark Wielaard @ 2019-01-01 0:00 UTC (permalink / raw) To: Santiago Ruano Rincón Cc: bzip2-devel, Anibal Monsalve Salazar, Anthony Fok, Federico Mena Quintero On Mon, 2019-06-24 at 10:31 +0200, Santiago Ruano Rincón wrote: > For information, Federico Mena (in CC) is taking over the development > of > bzip2: https://people.gnome.org/~federico/blog/maintaining-bzip2.html > > You should consider coordinating your efforts! Thanks. I assumed Federico was already on the bzip2-devel mailinglist. We did email earlier to discuss bzip2 maintenance and that all the infrastructure was already setup on the new sourceware.org bzip2 project. But it seems we did some duplicate work. Sorry for the miscommunication. It looks like we picked at least similar patches for the C sources, so those look mostly identical. I'll go over the remaining differences and try to cherry-pick or merge them into the bzip2 git repo. Some other files have been moved around, so it is harder to see whether we did the same for the manual and other docs. And I explicitly didn't touch the build system. I was looking at the autotools setup opensuse uses, but that also uses libtool which would mean changing the bzip.so verion/name. Which would be OK for Fedora and OpenSuse, since they already changed it long ago, but it would not be good for Debian, which uses the upstream libbz2.so.1.0 name. Also the buildbot, release scripts and the automatic updating of the homepage/manual is somewhat tied to the current setup. So I was thinking, first do a new 1.0.7 release with just the distro fixes and the updated project home, and then look at updating the rest of the build infrastructure. Cheers, Mark ^ permalink raw reply [flat|nested] 14+ messages in thread
* Releasing bzip2 1.0.7 and 1.1.0 ? Was: [PATCH] bzip2: Fix return value when combining --test,-t and -q. 2019-01-01 0:00 ` Mark Wielaard @ 2019-01-01 0:00 ` Mark Wielaard 2019-01-01 0:00 ` Mark Wielaard 2019-01-01 0:00 ` Some cherry-picks (Was: [PATCH] bzip2: Fix return value when combining --test,-t and -q.) Mark Wielaard 1 sibling, 1 reply; 14+ messages in thread From: Mark Wielaard @ 2019-01-01 0:00 UTC (permalink / raw) To: Santiago Ruano Rincón Cc: bzip2-devel, Anibal Monsalve Salazar, Anthony Fok, Federico Mena Quintero Hi, On Mon, 2019-06-24 at 11:34 +0200, Mark Wielaard wrote: > So I was thinking, first do a new 1.0.7 release with just the distro > fixes and the updated project home, and then look at updating the > rest of the build infrastructure. I have been pondering what to do about the duplicate efforts to get a bzip2 1.0.7 release out. I am really conservative and really think it would be good to do a release as is from the sourceware.org bzip2.git with just the security/bug fixes. And I think it would not be good to force the SO_NAME issue by updating the build system simultaneously. I am also reluctant to include the "e" and/or O_CLOEXEC fix as is, because a) not all distros have it currently (or different variants of it) and b) it might not exist on all platforms, so would need some kind of check for that first. On the other hand, I do actually think it would be good to have a more modern build system, and to force the SO_NAME issue (although I think we should provide upstream support for changing it back to libbz2.so.1.0 for those people who have been using it and don't want to break backward compatibility). And using newer, more modern, possibly GNU/Linux only features in 2019 would be a good thing. So what about we do a conservative release of 1.0.7 from the current sourceware.org bzip2.git and simultaneously do a 1.1.0 release from the gitlab hosted git repo? Referencing each other as the "classic" and "modern" bzip2 versions. Then we can keep the sourceware.org bzip2.git around for ultra conservative releases based on the current build "system" (that basically only takes new security issues), using 1.0.8, 1.0.9, etc. But encourage people towards the 1.1.1, 1.1.2, etc. releases for a more modern experience, if at all possible. Cheers, Mark ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Releasing bzip2 1.0.7 and 1.1.0 ? Was: [PATCH] bzip2: Fix return value when combining --test,-t and -q. 2019-01-01 0:00 ` Releasing bzip2 1.0.7 and 1.1.0 ? Was: " Mark Wielaard @ 2019-01-01 0:00 ` Mark Wielaard 2019-01-01 0:00 ` Federico Mena Quintero 0 siblings, 1 reply; 14+ messages in thread From: Mark Wielaard @ 2019-01-01 0:00 UTC (permalink / raw) To: Santiago Ruano Rincón Cc: bzip2-devel, Anibal Monsalve Salazar, Anthony Fok, Federico Mena Quintero Hi, On Tue, 2019-06-25 at 02:05 +0200, Mark Wielaard wrote: > So what about we do a conservative release of 1.0.7 from the current > sourceware.org bzip2.git and simultaneously do a 1.1.0 release from the > gitlab hosted git repo? Referencing each other as the "classic" and > "modern" bzip2 versions. > > Then we can keep the sourceware.org bzip2.git around for ultra > conservative releases based on the current build "system" (that > basically only takes new security issues), using 1.0.8, 1.0.9, etc. But > encourage people towards the 1.1.1, 1.1.2, etc. releases for a more > modern experience, if at all possible. I just pushed the following commit: commit f1e937776c5f331e24cc63a9d8e7ae9445a76761 Author: Mark Wielaard <mark@klomp.org> Date: Tue Jun 25 19:22:37 2019 +0200 Add prepare-release.sh script. Script to run to prepare a new release. It will update the release number and tell you to update the CHANGES file and to double check everything looks before doing the release commit and tagging. Afterwards you probably want to run release-update.sh to upload the release and update the website at https://sourceware.org/bzip2/ There are embedded version strings and dates in a couple of places. To keep the script simple remove some that aren't absolutely necessary. README now just points to CHANGES. README.COMPILATION.PROBLEMS only mentions the version once at the top. bzip2.c only mentions the version once when doing --version. manual.xml now doesn't have any embedded versions, just uses &bz-version; everywhere. With that I think we are ready to do a bzip2 1.0.7 release simply by running ./prepare-release.sh 1.0.7 and ./update-release.sh 1.0.7. So if the above sounds like a good plan, then we can execute it now. Push out bzip2 1.0.7 and put the 1.0.x branch in hibernation, just for security or major bug fixes. Then we can concentrate on making sure that a bzip2 1.1.X (or maybe even call it 2.0?) has a modern build system and adds things like O_CLOEXEC support, etc. Cheers, Mark ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Releasing bzip2 1.0.7 and 1.1.0 ? Was: [PATCH] bzip2: Fix return value when combining --test,-t and -q. 2019-01-01 0:00 ` Mark Wielaard @ 2019-01-01 0:00 ` Federico Mena Quintero 2019-01-01 0:00 ` Mark Wielaard 0 siblings, 1 reply; 14+ messages in thread From: Federico Mena Quintero @ 2019-01-01 0:00 UTC (permalink / raw) To: Mark Wielaard, Santiago Ruano Rincón Cc: bzip2-devel, Anibal Monsalve Salazar, Anthony Fok On Tue, 2019-06-25 at 20:09 +0200, Mark Wielaard wrote: > With that I think we are ready to do a bzip2 1.0.7 release simply by > running > ./prepare-release.sh 1.0.7 and ./update-release.sh 1.0.7. I like this script! Thanks for writing it! > So if the above sounds like a good plan, then we can execute it now. > Push out bzip2 1.0.7 and put the 1.0.x branch in hibernation, just > for > security or major bug fixes. Then we can concentrate on making sure > that a bzip2 1.1.X (or maybe even call it 2.0?) has a modern build > system and adds things like O_CLOEXEC support, etc. I'm sorry for not replying to your mails earlier. At work they just moved us to another email provider, and I've had some trouble fixing my email setup (SMTP servers, etc.) to make everything work as before. I am... also ready to release 1.0.7 :-P The work you've done in updating the release scripts is valuable, and I'd like to propose this instead: * Put prepare-release.sh in the gitlab repository, and adjust it for the changes there. * Release 1.0.7 from the gitlab repo (it already has the NEWS file with updates, while I think your repo needs a CHANGES to be written?). I've already tested that this works; "ninja dist" takes care of it. * Keep the soname changes from the gitlab repo. I think the original Makefile-libbz2_so is incorrect in how it sets the soname (or at least libtool thinks so), and both openSUSE and Fedora managed to change it to a single digit instead of "1.0". I *think* Debian, and distros which picked up their patches from there, simply never changed this and that's why they kept the "1.0" in place. Whatever we change the soname to, it's going to be incompatible with some distros - and they are already used to patching things, anyway. Maybe purely for my own convenience I'd like to remove the patches from openSUSE - the current soname in the gitlab repo allows for that. I would be fine with releasing 1.0.7 from the sourceware repo, and then a subsequent 1.1 with all the changes from gitlab - but I think this will cause extra work for the distros. First they'll have to see which of their patches need to be removed, and they'll still need to patch the build system. And with 1.1 released, they'll need to do it all over again. I think we can minimize the work for distros if we update the patches and the build system in a single release. They already know how to handle Meson more or less automatically, while leaving the original bzip2 Makefiles in place means that each distro has to patch to their own hand-rolled build system all over again. [I've sent a few subscribe requests to this mailing list as federico@gnome.org, but it's not sending me the confirmation mail... do you know who may be able to kick the list software?] Federico ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Releasing bzip2 1.0.7 and 1.1.0 ? Was: [PATCH] bzip2: Fix return value when combining --test,-t and -q. 2019-01-01 0:00 ` Federico Mena Quintero @ 2019-01-01 0:00 ` Mark Wielaard 2019-01-01 0:00 ` Federico Mena Quintero 0 siblings, 1 reply; 14+ messages in thread From: Mark Wielaard @ 2019-01-01 0:00 UTC (permalink / raw) To: Federico Mena Quintero, Santiago Ruano Rincón Cc: bzip2-devel, Anibal Monsalve Salazar, Anthony Fok Hi, On Tue, 2019-06-25 at 14:08 -0500, Federico Mena Quintero wrote: > On Tue, 2019-06-25 at 20:09 +0200, Mark Wielaard wrote: > > > With that I think we are ready to do a bzip2 1.0.7 release simply > > by > > running > > ./prepare-release.sh 1.0.7 and ./update-release.sh 1.0.7. > > I like this script! Thanks for writing it! It is a bit of a hack to be honest. I mostly did it this way because there is no modern build system yet :) I don't know how Julian did releases. But there are way too many embedded version strings and dates in the sources (and they don't even all use the same format!) > > So if the above sounds like a good plan, then we can execute it > > now. > > Push out bzip2 1.0.7 and put the 1.0.x branch in hibernation, just > > for > > security or major bug fixes. Then we can concentrate on making sure > > that a bzip2 1.1.X (or maybe even call it 2.0?) has a modern build > > system and adds things like O_CLOEXEC support, etc. > > I'm sorry for not replying to your mails earlier. At work they just > moved us to another email provider, and I've had some trouble fixing > my > email setup (SMTP servers, etc.) to make everything work as before. > > I am... also ready to release 1.0.7 :-P The work you've done in > updating the release scripts is valuable, and I'd like to propose > this > instead: > > * Put prepare-release.sh in the gitlab repository, and adjust it for > the changes there. If we are moving things around and have a modern build system it might be better to actually remove most of the embedded versions/dates from the sources and where they are really needed use configure to set them. There actually already seems to be a merge request for that: https://gitlab.com/federicomenaquintero/bzip2/merge_requests (Although for autotools, which already has been removed again?) > * Release 1.0.7 from the gitlab repo (it already has the NEWS file > with > updates, while I think your repo needs a CHANGES to be > written?). I've > already tested that this works; "ninja dist" takes care of it. Yes, the CHANGES file is the only thing the prepare-release.sh script currently makes you do by hand (because there are no older tags yet, it could in theory be replaced with a simple git shortlog), but there are only 5 meaningful changes: * Fix undefined behavior in the macros SET_BH, CLEAR_BH, & ISSET_BH * bzip2: Fix return value when combining --test,-t and -q. * bzip2recover: Fix buffer overflow for large argv[0] * bzip2recover: Fix use after free issue with outFile (CVE-2016-3189) * Make sure nSelectors is not out of range (CVE-2019-12900) > * Keep the soname changes from the gitlab repo. I think the original > Makefile-libbz2_so is incorrect in how it sets the soname (or at > least > libtool thinks so), and both openSUSE and Fedora managed to change it > to a single digit instead of "1.0". I *think* Debian, and distros > which picked up their patches from there, simply never changed this > and > that's why they kept the "1.0" in place. libtool doesn't support it, but libtool has to work on lots of unix- like systems. The SONAME isn't technically wrong. It is just not what we consider good practice these days. Unfortunately lots of distros rely on this particular SONAME (at least everything based on Debian, including Ubuntu and PureOS) use libbz2.so.1.0 and they simply cannot change it, or they break backward compatibility. So I think any new build system should at least provide the option of keeping the old SONAME. > Whatever we change the soname to, it's going to be incompatible with > some distros - and they are already used to patching things, anyway. > Maybe purely for my own convenience I'd like to remove the patches > from > openSUSE - the current soname in the gitlab repo allows for that. Yes, and I think lots of other distros (Fedora and CentOS based ones at least) would also like that. > I would be fine with releasing 1.0.7 from the sourceware repo, and > then > a subsequent 1.1 with all the changes from gitlab - but I think this > will cause extra work for the distros. First they'll have to see > which > of their patches need to be removed, and they'll still need to patch > the build system. And with 1.1 released, they'll need to do it all > over again. > > I think we can minimize the work for distros if we update the patches > and the build system in a single release. They already know how to > handle Meson more or less automatically, while leaving the original > bzip2 Makefiles in place means that each distro has to patch to their > own hand-rolled build system all over again. I am not too concerned about the (larger) distros. Although I think it would be good to at least provide a simple option to change the SONAME. The "classic" build does that by simply editing/patching it, what the distros probably already do. But the distros will probably just yell and scream at us, and figure it out. I am more concerned about users we don't really know about, maybe on non-GNU/Linux systems that have just been using 1.0.6 for years as is. Putting out something with a new build system, that by default generates an incompatible SONAME and some changes that maybe need newer features not easily detected on those systems might cause them to not upgrade at all. I realize I am probably overly conservative, but I really like to keep a 1.0.x branch going just for ancient users (I expect the distro to switch to a more modern release, once they figure out the SONAME thingy). And if we have that ancient branch, where we can just put security fixes on, and nothing else, then we can be even more creative on the new modern 1.1.x (2.0.x?) branch. > [I've sent a few subscribe requests to this mailing list as > federico@gnome.org, but it's not sending me the confirmation mail... > do you know who may be able to kick the list software?] That is odd. You could try contacting overseers@sourceware.org or go on irc.freenode.net #overseers to see if someone knows what is going on. Cheers, Mark ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Releasing bzip2 1.0.7 and 1.1.0 ? Was: [PATCH] bzip2: Fix return value when combining --test,-t and -q. 2019-01-01 0:00 ` Mark Wielaard @ 2019-01-01 0:00 ` Federico Mena Quintero 2019-01-01 0:00 ` Mark Wielaard 0 siblings, 1 reply; 14+ messages in thread From: Federico Mena Quintero @ 2019-01-01 0:00 UTC (permalink / raw) To: Mark Wielaard, Santiago Ruano Rincón Cc: bzip2-devel, Anibal Monsalve Salazar, Anthony Fok On Wed, 2019-06-26 at 01:04 +0200, Mark Wielaard wrote: > If we are moving things around and have a modern build system it > might > be better to actually remove most of the embedded versions/dates from > the sources and where they are really needed use configure to set > them. > There actually already seems to be a merge request for that: Yes, I want to update that MR tomorrow for the new build stuff. Just today I did it for BZ_VERSION in the header file; I think just the docs are missing the changes from that MR now. > I realize I am probably overly conservative, but I really like to > keep a 1.0.x branch going just for ancient users (I expect the distro > to > switch to a more modern release, once they figure out the SONAME > thingy). And if we have that ancient branch, where we can just put > security fixes on, and nothing else, then we can be even more > creative on the new modern 1.1.x (2.0.x?) branch. I like this idea more and more. And now I feel sheepish, because it's basically the same approach I took for librsvg 2.40.20, the last C version without Rust code in it. This is from that version's release notes: Version 2.40.20 - Except for emergencies, this will be the LAST RELEASE of the librsvg-2.40.x series. We are moving to 2.41, which is vastly improved over the 2.40 series. The API/ABI there remain unchaged, so we strongly encourage you to upgrade your sources and binaries to librsvg-2.41.x. I fetched from the sourceware repo onto my gitlab repo to see the changes, and indeed the sources are practically identical (the build stuff and docs are of course changed). What do you think of me doing this: - Create a "bzip2-1.0" branch with the commits and contents of the sourceware repo. - Sync the sources? Are there any changes in the gitlab master branch that you *don't* want? - Release 1.0.7 out of the bzip2-1.0 branch; deprecate that branch immediately. Advertise it as targeting people who embed bzip2-1.0.6 in their sources and probably already have customizations to the build system themselves. - Continue with the current development in the master branch; I'll probably release 1.1.0 as an experimental release with the new build stuff and all the fixes. Advertise this as targeted to distros and everything else. - Rust goes in a separate branch as usual; it's going to need some big changes to the new build system anyway. Federico ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Releasing bzip2 1.0.7 and 1.1.0 ? Was: [PATCH] bzip2: Fix return value when combining --test,-t and -q. 2019-01-01 0:00 ` Federico Mena Quintero @ 2019-01-01 0:00 ` Mark Wielaard 2019-01-01 0:00 ` Mark Wielaard 0 siblings, 1 reply; 14+ messages in thread From: Mark Wielaard @ 2019-01-01 0:00 UTC (permalink / raw) To: Federico Mena Quintero, Santiago Ruano Rincón Cc: bzip2-devel, Anibal Monsalve Salazar, Anthony Fok On Tue, 2019-06-25 at 21:22 -0500, Federico Mena Quintero wrote: > What do you think of me doing this: > > - Create a "bzip2-1.0" branch with the commits and contents of the > sourceware repo. > > - Sync the sources? Are there any changes in the gitlab master > branch > that you *don't* want? > > - Release 1.0.7 out of the bzip2-1.0 branch; deprecate that branch > immediately. Advertise it as targeting people who embed bzip2-1.0.6 > in > their sources and probably already have customizations to the build > system themselves. > > - Continue with the current development in the master branch; I'll > probably release 1.1.0 as an experimental release with the new build > stuff and all the fixes. Advertise this as targeted to distros and > everything else. > > - Rust goes in a separate branch as usual; it's going to need some > big > changes to the new build system anyway. I would like to do the release from the sourceware.org git repo first as is. And only then create a 1.0.x branch from it. That way we have something that simply replaces the bzip.org releases and establishes https://sourceware.org/bzip2/ as new (or technically old) home for bzip2, have an official download location, updated regenerated manuals, etc. Then we can tweak the homepage through bzip2-htdocs.git to point to a future development tree if you don't like to use the sourceware bzip2.git repo as main development platform. But I like to keep a (mirror) of the 1.0.x branch on sourceware so that it is easy to keep making (emergency) releases from it if necessary. I don't expect there to be many of those though. Given that the last 10 years provided 5 meaningful bug/security issues, there might be a bzip2 1.0.x release maybe once every 2 years. I don't think we want any of the other changes from the gitlab tree because they are all potential build issues, not purely bug fixes. e.g. even the simple #if BZ_UNIX scares me a little. I am sorry we didn't properly discuss and communicated what we expected from resurrecting/maintaining bzip2. Especially since we seem to be somewhat opposite developer/personalities. I am probably a conservative minimalist, while you seem more of a maximalist modernist. But I hope we can make cooperation work, because we probably need both types of people/developers to maintain bzip2 for the next 20 years :) For me the most important thing is that we provide a more stable home for bzip2 than bzip.org was. And sourceware is that to me because it existed even before bzip2, bzip2 had its home originally on sourceware.org/bzip2, it is community managed and the site, git sources and downloads/releases are widely mirrored. And that we don't break anything, not even accidentally, which is why my 1.0.7 release really only contains the minimal amount of changes just to update the homepage, release number and fix any known bug/security issues, but absolutely nothing more. For you it is probably more important to have a modern build system, a place to hack on it that makes it easy to exchange patches, automagically mixes them with "pull requests" and issue tracking, making the barrier to entry for people as low as possible, make it easy to clean up and refactor the code, etc. all as long as it doesn't break ABI of course. So that the code/community lives again to provide some innovation. Those things bite a little. Ideally they wouldn't of course, so we get both stability and progress. But I would have never guessed someone would pick a personal repo on a commercial (partially proprietary) run platform. While you probably cannot imagine why someone would think a place that offers bugzilla as issue tracker and primarily uses mailinglists to exchange patches would be a good home for the project. The conservative in me already freaks out about a simple change like you just pushed to change the format of the version string returned by BZ2_bzlibVersion (). What if someone parses it and expects that comma! screams a little voice inside my head! Even though of course any code that does that is obviously broken, and if we cannot even change that, then what innovation can we do? Anyway, I hope that explains why I care so deeply about what is there now in the sourceware bzip2.git repo, and why I think we should keep releasing that branch as is through sourceware.org. Even if we then turn it into a branch that will never really change again and we point the homepage repo sources links, to somewhere else for all future development (in which case I would happily setup a mirror to make sure the sources, releases and updated documentation are also available on sourceware). I do think you take the SONAME change too lightly though. But I don't have a good answer for it. So my conservative reaction is DO NOT CHANGE IT! Even though I think OpenSuse, Fedora and some other distros were right to "fix" it to just contain one version digit. But sad fact is that they were also wrong, because upstream is always right (even if it is wrong) when it comes to compatibility. Changing the SONAME is breaking ABI. And a lot of other distros didn't change it, and so cannot without breaking ABI for the foreseeable future (I noticed even the freedesktop flatpak runtime contains bzip2.so.1.0). Cheers, Mark ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Releasing bzip2 1.0.7 and 1.1.0 ? Was: [PATCH] bzip2: Fix return value when combining --test,-t and -q. 2019-01-01 0:00 ` Mark Wielaard @ 2019-01-01 0:00 ` Mark Wielaard 2019-01-01 0:00 ` Federico Mena Quintero 0 siblings, 1 reply; 14+ messages in thread From: Mark Wielaard @ 2019-01-01 0:00 UTC (permalink / raw) To: Federico Mena Quintero, Santiago Ruano Rincón Cc: bzip2-devel, Anibal Monsalve Salazar, Anthony Fok On Wed, 2019-06-26 at 12:55 +0200, Mark Wielaard wrote: > On Tue, 2019-06-25 at 21:22 -0500, Federico Mena Quintero wrote: > > What do you think of me doing this: > > > > - Create a "bzip2-1.0" branch with the commits and contents of the > > sourceware repo. > > > > - Sync the sources? Are there any changes in the gitlab master > > branch > > that you *don't* want? > > > > - Release 1.0.7 out of the bzip2-1.0 branch; deprecate that branch > > immediately. Advertise it as targeting people who embed bzip2-1.0.6 > > in > > their sources and probably already have customizations to the build > > system themselves. > > > > - Continue with the current development in the master branch; I'll > > probably release 1.1.0 as an experimental release with the new build > > stuff and all the fixes. Advertise this as targeted to distros and > > everything else. > > > > - Rust goes in a separate branch as usual; it's going to need some > > big > > changes to the new build system anyway. > > I would like to do the release from the sourceware.org git repo first > as is. And only then create a 1.0.x branch from it. That way we have > something that simply replaces the bzip.org releases and establishes > https://sourceware.org/bzip2/ as new (or technically old) home for > bzip2, have an official download location, updated regenerated manuals, > etc. Then we can tweak the homepage through bzip2-htdocs.git to point > to a future development tree if you don't like to use the sourceware > bzip2.git repo as main development platform. OK, lets go with this plan. There is this outstanding CVE and people have been asking for a release to resolve it. I'll run the prepare-release.sh and release-update.sh scripts creating the 1.0.7 release and will then update bzip-htdocs.git to add a pointer to the homepage for the current development future 1.1.x development branch https://gitlab.com/federicomenaquintero/bzip2 for people who want a new build system and more fixes. I'll sent mail to the list when done. Cheers, Mark ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Releasing bzip2 1.0.7 and 1.1.0 ? Was: [PATCH] bzip2: Fix return value when combining --test,-t and -q. 2019-01-01 0:00 ` Mark Wielaard @ 2019-01-01 0:00 ` Federico Mena Quintero 2019-01-01 0:00 ` Mark Wielaard 0 siblings, 1 reply; 14+ messages in thread From: Federico Mena Quintero @ 2019-01-01 0:00 UTC (permalink / raw) To: Mark Wielaard, Santiago Ruano Rincón Cc: bzip2-devel, Anibal Monsalve Salazar, Anthony Fok On Thu, 2019-06-27 at 20:00 +0200, Mark Wielaard wrote: > > I'll run the prepare-release.sh and release-update.sh scripts > creating > the 1.0.7 release and will then update bzip-htdocs.git to add a > pointer > to the homepage for the current development future 1.1.x development > branch https://gitlab.com/federicomenaquintero/bzip2 for people who > want a new build system and more fixes. > Thank you! Once you are done, I'll push a branch to the gitlab repo with those changes. In the meantime I'll blog about your plans to keep the 1.0.7 code there, and about 1.1.x being the version with the new build systems. Federico ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Releasing bzip2 1.0.7 and 1.1.0 ? Was: [PATCH] bzip2: Fix return value when combining --test,-t and -q. 2019-01-01 0:00 ` Federico Mena Quintero @ 2019-01-01 0:00 ` Mark Wielaard 0 siblings, 0 replies; 14+ messages in thread From: Mark Wielaard @ 2019-01-01 0:00 UTC (permalink / raw) To: Federico Mena Quintero, Santiago Ruano Rincón Cc: bzip2-devel, Anibal Monsalve Salazar, Anthony Fok On Thu, 2019-06-27 at 13:29 -0500, Federico Mena Quintero wrote: > On Thu, 2019-06-27 at 20:00 +0200, Mark Wielaard wrote: > > > > I'll run the prepare-release.sh and release-update.sh scripts > > creating > > the 1.0.7 release and will then update bzip-htdocs.git to add a > > pointer > > to the homepage for the current development future 1.1.x development > > branch https://gitlab.com/federicomenaquintero/bzip2 for people who > > want a new build system and more fixes. > > > > Thank you! Once you are done, I'll push a branch to the gitlab repo > with those changes. In the meantime I'll blog about your plans to keep > the 1.0.7 code there, and about 1.1.x being the version with the new > build systems. Thanks. I also posted a blog with just the release announcement and a link to your blog: https://gnu.wildebeest.org/blog/mjw/2019/06/27/bzip2-1-0-7/ ^ permalink raw reply [flat|nested] 14+ messages in thread
* Some cherry-picks (Was: [PATCH] bzip2: Fix return value when combining --test,-t and -q.) 2019-01-01 0:00 ` Mark Wielaard 2019-01-01 0:00 ` Releasing bzip2 1.0.7 and 1.1.0 ? Was: " Mark Wielaard @ 2019-01-01 0:00 ` Mark Wielaard 2019-01-01 0:00 ` Federico Mena Quintero 1 sibling, 1 reply; 14+ messages in thread From: Mark Wielaard @ 2019-01-01 0:00 UTC (permalink / raw) To: Santiago Ruano Rincón Cc: bzip2-devel, Anibal Monsalve Salazar, Anthony Fok, Federico Mena Quintero On Mon, 2019-06-24 at 11:34 +0200, Mark Wielaard wrote: > On Mon, 2019-06-24 at 10:31 +0200, Santiago Ruano Rincón wrote: > > For information, Federico Mena (in CC) is taking over the development > > of > > bzip2: https://people.gnome.org/~federico/blog/maintaining-bzip2.html > > > > You should consider coordinating your efforts! > > Thanks. I assumed Federico was already on the bzip2-devel mailinglist. > We did email earlier to discuss bzip2 maintenance and that all the > infrastructure was already setup on the new sourceware.org bzip2 > project. But it seems we did some duplicate work. Sorry for the > miscommunication. > > It looks like we picked at least similar patches for the C sources, so > those look mostly identical. I'll go over the remaining differences and > try to cherry-pick or merge them into the bzip2 git repo. I cherry-picked the following: commit ff986850159a1ea0c75617ffa792d1bb2069856e Author: Federico Mena Quintero <federico@gnome.org> Date: Wed May 29 17:14:27 2019 -0500 Change a magic number (6) for a constant (BZ_N_GROUPS). decompress.c (BZ2_decompress): Check nGroups against BZ_N_GROUPS. commit 7ed62bfb46e87a9e878712603469440e6882b184 Author: Albert Astals Cid <aacid@kde.org> Date: Tue May 28 19:35:18 2019 +0200 Make sure nSelectors is not out of range nSelectors is used in a loop from 0 to nSelectors to access selectorMtf which is UChar selectorMtf[BZ_MAX_SELECTORS]; so if nSelectors is bigger than BZ_MAX_SELECTORS it'll do an invalid memory access Fixes out of bounds access discovered while fuzzying karchive This was reported as CVE-2019-12900 BZ2_decompress in decompress.c in bzip2 through 1.0.6 has an out-of-bounds write when there are many selectors. commit 16f2c753f9959e8d7c7e1fa771b8ccc5821427aa Author: Paul Kehrer <paul.l.kehrer@gmail.com> Date: Sat Jun 8 10:06:40 2019 -0400 Fix undefined behavior in the macros SET_BH, CLEAR_BH, & ISSET_BH These macros contain this pattern: 1 << ((Int32_value) & 31 This causes the undefined behavior sanitizers in clang and gcc to complain because the shift, while ultimately stored to an unsigned variable, is done as a signed value. Adding a cast to unsigned for the int32 value resolves this issue. That makes the sources almost identical, modulo some whitespace issues (inconsistent use of tab/space as indent). And some Windows specific tweaks that I am not able to test (but they are probably correct though ). The only remaining difference between the trees (for the C sources) is the fix for O_CLOEXEC. I would like to better understand the (different) Debian solution for that: https://sources.debian.org/patches/bzip2/1.0.6-9/bzip2recover-race-open-output.diff/ Cheers, Mark ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Some cherry-picks (Was: [PATCH] bzip2: Fix return value when combining --test,-t and -q.) 2019-01-01 0:00 ` Some cherry-picks (Was: [PATCH] bzip2: Fix return value when combining --test,-t and -q.) Mark Wielaard @ 2019-01-01 0:00 ` Federico Mena Quintero 0 siblings, 0 replies; 14+ messages in thread From: Federico Mena Quintero @ 2019-01-01 0:00 UTC (permalink / raw) To: Mark Wielaard, Santiago Ruano Rincón Cc: bzip2-devel, Anibal Monsalve Salazar, Anthony Fok On Mon, 2019-06-24 at 15:55 +0200, Mark Wielaard wrote: > The only remaining difference between the trees (for the C sources) > is the fix for O_CLOEXEC. I would like to better understand the > (different) Debian solution for that: > > https://sources.debian.org/patches/bzip2/1.0.6-9/bzip2recover-race-open-output.diff/ This fix is partially correct, and I've pushed it to the repository in gitlab with one change: - Use practically the same fopen_output_safely() that is used in bzip2.c. This has a change to *not* use IntNative, to avoid cutting&pasting even more #ifdefs from bzip2.c. To make the fix fully correct, it would actually print different errors when the output file exists, versus when it cannot be opened due to an I/O error. But that can wait. I think the Debian patch may be confusing because it maintains the description "fix unsafe race condition in opening output files" from the bug report for bzip2-0.9.5... back then it *was* for a minor race condition in bzip2, but these days it's only to avoid overwriting files in bzip2recover. Federico ^ permalink raw reply [flat|nested] 14+ messages in thread
end of thread, other threads:[~2019-06-27 19:08 UTC | newest] Thread overview: 14+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2019-01-01 0:00 [PATCH] bzip2: Fix return value when combining --test,-t and -q Mark Wielaard 2019-01-01 0:00 ` Santiago Ruano Rincón 2019-01-01 0:00 ` Mark Wielaard 2019-01-01 0:00 ` Releasing bzip2 1.0.7 and 1.1.0 ? Was: " Mark Wielaard 2019-01-01 0:00 ` Mark Wielaard 2019-01-01 0:00 ` Federico Mena Quintero 2019-01-01 0:00 ` Mark Wielaard 2019-01-01 0:00 ` Federico Mena Quintero 2019-01-01 0:00 ` Mark Wielaard 2019-01-01 0:00 ` Mark Wielaard 2019-01-01 0:00 ` Federico Mena Quintero 2019-01-01 0:00 ` Mark Wielaard 2019-01-01 0:00 ` Some cherry-picks (Was: [PATCH] bzip2: Fix return value when combining --test,-t and -q.) Mark Wielaard 2019-01-01 0:00 ` Federico Mena Quintero
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).