From: Federico Mena Quintero <federico@gnome.org>
To: Mark Wielaard <mark@klomp.org>, bzip2-devel@sourceware.org
Cc: Julian Seward <jseward@acm.org>
Subject: Re: bzip2 1.0.7 released
Date: Tue, 01 Jan 2019 00:00:00 -0000 [thread overview]
Message-ID: <1b88cafb93f6b37ced1c8bbfb04a115cf6a15200.camel@gnome.org> (raw)
In-Reply-To: <20190627205837.GD9273@wildebeest.org>
On Thu, 2019-06-27 at 22:58 +0200, Mark Wielaard wrote:
> The problem with the file in the report is that it does contain some
> nSelectors that are slightly larger than BZ_MAX_SELECTORS.
There's a discussion here:
https://gitlab.com/federicomenaquintero/bzip2/commit/74de1e2e6ffc9d51ef9824db71a8ffee5962cdbc
It has links to the lbzip2 bug, and the commit that fixed it.
Summary: a bug in lbzip2 was causing it to write more than bzip2's
allowed number of selectors; it got fixed in December 2018.
If lbzip2's goals is to produce files which are compatible with bzip2,
it can be argued that indeed it was producing broken files for a while.
> -#define BZ_MAX_SELECTORS (2 + (900000 / BZ_G_SIZE))
> +#define BZ_MAX_SELECTORS (7 + (900000 / BZ_G_SIZE))
>
>
>
> But of course I cannot tell why increasing the max with 5 is correct.
> It might well be that the file is invalid. Before the fix bunzip2
> would overwrite some memory after the selectorMtf array. So it might
> be the file decompressed by accident in the past.
The comments in the lbzip2 commit that fixes this refer to the padding
required to make the block size a multiple of 8 bits. I don't know
enough about bzip2's format yet, or how the move-to-front algorithm
works, to be able to tell if *that* simple fix for bzip2 of increasing
BZ_MAX_SELECTORS is enough to make bzip2 able to read all of lbzip2's
files.
I think some distros ship lbzip2 instead of bzip2. If it was
generating non-fully-compatible files... due to a bug in bzip2... it's
probably fair to allow these files in bzip2 and be done with it. I'd
still like to hear some confirmation (from Julian or the lbzip2
authors?) on whether the analysis above is correct.
Federico
next prev parent reply other threads:[~2019-06-27 22:44 UTC|newest]
Thread overview: 16+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-01-01 0:00 Mark Wielaard
2019-01-01 0:00 ` Mark Wielaard
2019-01-01 0:00 ` Federico Mena Quintero [this message]
2019-01-01 0:00 ` Mark Wielaard
2019-01-01 0:00 ` Federico Mena Quintero
2019-01-01 0:00 ` Julian Seward
2019-01-01 0:00 ` Mark Wielaard
2019-01-01 0:00 ` Alternative nSelectors patch (Was: bzip2 1.0.7 released) Mark Wielaard
2019-01-01 0:00 ` Julian Seward
2019-01-01 0:00 ` Mark Wielaard
2019-01-01 0:00 ` Mark Wielaard
2019-01-01 0:00 ` bzip2 test suite " Mark Wielaard
2019-01-01 0:00 ` bzip2 1.0.7 released Mark Wielaard
2019-01-01 0:00 ` Federico Mena Quintero
2019-01-01 0:00 ` Mark Wielaard
2019-01-01 0:00 ` Jeffrey Walton
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1b88cafb93f6b37ced1c8bbfb04a115cf6a15200.camel@gnome.org \
--to=federico@gnome.org \
--cc=bzip2-devel@sourceware.org \
--cc=jseward@acm.org \
--cc=mark@klomp.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).