On Fri, 2019-06-28 at 13:10 +0200, Mark Wielaard wrote:
> >    In particular, the three test files in the tarball merely serve to verify
> >    that the build didn't fail in some obvious way.  They are in no way a
> >    comprehensive test set.
> 
> Yes, having a more comprehensive test set would be great.
> Especially having file encoded with other bzip2 encoders.

I don't claim it is comprehensive at all, but I have gone through all
the other bzip2 encoder/decoder projects I could find and collected the
samples/tests they used. To not clutter up the main repo I have setup a
new repository with a test driver.

https://sourceware.org/git/?p=bzip2-tests.git;a=summary
git clone git://sourceware.org/git/bzip2-tests.git

It only has one commit:

commit 3e71fa8acda490a6b288833d759eb4129ad573e2
Author: Mark Wielaard <mark@klomp.org>
Date:   Sun Jun 30 20:56:33 2019 +0200

    Initial bzip2 test suite.
    
    Contains test files from the commons-compress, dotnetzip, go, lbzip2
    and pyflate projects. Each test file has either an associated md5 sum
    to check it decompressed correctly. Or it is named bz2.bad to indicate
    it cannot be decompressed (or might need --force to decompress).
    
    The run-tests.sh test wrapper runs a bzip2 binary in a couple of
    configurations, optionally under valgrind, on all the test files in
    the tree.

See the attached README to see how to run it.

Even though it might not be comprehensive, it is a start. And you can
use the ./run-tests.sh test runner to run over a full directory tree.
See the end of the README file.

It already contains a testcase (from lbzip2) that shows the original
issue with bzip2 1.0.6
(compiled with gcc -fsanitize=undefined -fno-sanitize-recover):

Processing lbzip2/32767.bz2
  Decompress...
decompress.c:299:24: runtime error: index 18002 out of bounds for type 'UChar [18002]'
!!! bad decompress result 1

And with bzip 1.0.7 it does fail as follows:

Processing lbzip2/32767.bz2
  Decompress...

bzip2: Data integrity error when decompressing.
!!! bad decompress result 2

Sadly it also shows that the fix we have isn't complete.
Even with that fix it fails as above.

I think I have a fix for that though. Will post in a minute.

I'll try to integrate test suite with the buildbots (although some will
have to at least use --without-valgrind because testing is really,
really, really slow under valgrind).

Cheers,

Mark