From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 57304 invoked by alias); 30 Jun 2019 23:05:19 -0000 Mailing-List: contact bzip2-devel-help@sourceware.org; run by ezmlm Precedence: bulk List-Post: List-Help: List-Subscribe: List-Id: Sender: bzip2-devel-owner@sourceware.org Received: (qmail 57295 invoked by uid 89); 30 Jun 2019 23:05:19 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Checked: by ClamAV 0.100.3 on sourceware.org X-Virus-Found: No X-Spam-SWARE-Status: No, score=-8.9 required=5.0 tests=AWL,BAYES_00,GIT_PATCH_1,SPF_PASS autolearn=ham version=3.3.1 spammy=runner, clutter, integrate, Contains X-Spam-Status: No, score=-8.9 required=5.0 tests=AWL,BAYES_00,GIT_PATCH_1,SPF_PASS autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on sourceware.org X-Spam-Level: X-HELO: gnu.wildebeest.org Message-ID: Subject: bzip2 test suite (Was: bzip2 1.0.7 released) From: Mark Wielaard To: jseward@acm.org, Federico Mena Quintero , bzip2-devel@sourceware.org Date: Tue, 01 Jan 2019 00:00:00 -0000 In-Reply-To: <9998ca428c4c7f895a543aa91941e58efb0d5291.camel@klomp.org> References: <20190627205837.GD9273@wildebeest.org> <0a2331bc6d0c8500c2c45df1e3ebe01b49ad5831.camel@klomp.org> <8c4d5cf2479253406dacdee122692cc77771afb9.camel@gnome.org> <9998ca428c4c7f895a543aa91941e58efb0d5291.camel@klomp.org> Content-Type: multipart/mixed; boundary="=-77PbeaT8L9CdHa0V4SzJ" X-Mailer: Evolution 3.28.5 (3.28.5-2.el7) Mime-Version: 1.0 X-Spam-Flag: NO X-SW-Source: 2019-q2/txt/msg00034.txt.bz2 --=-77PbeaT8L9CdHa0V4SzJ Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Content-length: 2448 On Fri, 2019-06-28 at 13:10 +0200, Mark Wielaard wrote: > > In particular, the three test files in the tarball merely serve to v= erify > > that the build didn't fail in some obvious way. They are in no way a > > comprehensive test set. >=20 > Yes, having a more comprehensive test set would be great. > Especially having file encoded with other bzip2 encoders. I don't claim it is comprehensive at all, but I have gone through all the other bzip2 encoder/decoder projects I could find and collected the samples/tests they used. To not clutter up the main repo I have setup a new repository with a test driver. https://sourceware.org/git/?p=3Dbzip2-tests.git;a=3Dsummary git clone git://sourceware.org/git/bzip2-tests.git It only has one commit: commit 3e71fa8acda490a6b288833d759eb4129ad573e2 Author: Mark Wielaard Date: Sun Jun 30 20:56:33 2019 +0200 Initial bzip2 test suite. =20=20=20=20 Contains test files from the commons-compress, dotnetzip, go, lbzip2 and pyflate projects. Each test file has either an associated md5 sum to check it decompressed correctly. Or it is named bz2.bad to indicate it cannot be decompressed (or might need --force to decompress). =20=20=20=20 The run-tests.sh test wrapper runs a bzip2 binary in a couple of configurations, optionally under valgrind, on all the test files in the tree. See the attached README to see how to run it. Even though it might not be comprehensive, it is a start. And you can use the ./run-tests.sh test runner to run over a full directory tree. See the end of the README file. It already contains a testcase (from lbzip2) that shows the original issue with bzip2 1.0.6 (compiled with gcc -fsanitize=3Dundefined -fno-sanitize-recover): Processing lbzip2/32767.bz2 Decompress... decompress.c:299:24: runtime error: index 18002 out of bounds for type 'UCh= ar [18002]' !!! bad decompress result 1 And with bzip 1.0.7 it does fail as follows: Processing lbzip2/32767.bz2 Decompress... bzip2: Data integrity error when decompressing. !!! bad decompress result 2 Sadly it also shows that the fix we have isn't complete. Even with that fix it fails as above. I think I have a fix for that though. Will post in a minute. I'll try to integrate test suite with the buildbots (although some will have to at least use --without-valgrind because testing is really, really, really slow under valgrind). Cheers, Mark --=-77PbeaT8L9CdHa0V4SzJ Content-Disposition: attachment; filename="README" Content-Type: text/plain; name="README"; charset="UTF-8" Content-Transfer-Encoding: base64 Content-length: 2924 PSBCWjIgdGVzdCBmaWxlIGNvbGxlY3Rpb24gPQoKVGhpcyBpcyBhIGNvbGxl Y3Rpb24gb2YgImludGVyZXN0aW5nIiAuYnoyIGZpbGVzIHRoYXQgY2FuIGJl IHVzZWQgdG8KdGVzdCBiemlwMiB3b3JrcyBjb3JyZWN0bHkuIFRoZXkgY29t ZSBmcm9tIGRpZmZlcmVudCBwcm9qZWN0cy4KCkVhY2ggZGlyZWN0b3J5IHNo b3VsZCBjb250YWluIGEgUkVBRE1FIGZpbGUgZXhwbGFpbmluZyB3aGVyZSB0 aGUgLmJ6MgpmaWxlcyBvcmlnaW5hbGx5IGNhbWUgZnJvbS4gUGx1cyBhIHJl ZmVyZW5jZSB0byB0aGUgKEZyZWUgU29mdHdhcmUpCmxpY2Vuc2UgdGhhdCB0 aGUgcHJvamVjdCBmaWxlcyB3ZXJlIGRpc3RyaWJ1dGVkIHVuZGVyLgoKU29t ZSBmaWxlcyBhcmUgZGVsaWJlcmF0ZWx5IGJhZCwgYW5kIGFyZSB1c2UgdG8g c2VlIGhvdyBiemlwMiBoYW5kbGVzCmNvcnJ1cHQgZmlsZXMuIFRoZXkgYXJl IGV4cGxpY2l0bHkgbm90IGludGVuZGVkIHRvIGRlY29tcHJlc3MgY29ycmVj dGx5LApidXQgdG8gY2F0Y2ggZXJyb3JzIGluIGJ6aXAyIHRyeWluZyB0byBk ZWFsIHdpdGggZGVsaWJlcmF0ZWx5IGJhZCBkYXRhLgpBbGwgc3VjaCBmaWxl cyBoYXZlIGEgbmFtZSBlbmRpbmcgaW4gLmJ6Mi5iYWQuCgpBbGwgbm9uZSBi YWQgZmlsZXMgZW5kIGluIGJ6Mi4gQW5kIHNob3VsZCBjb21lIHdpdGggYSAu bWQ1IGZpbGUgZm9yCnRoZSBvcmlnaW5hbCBpbnB1dCBmaWxlLiBUaGUgLm1k NSBmaWxlIGlzIHVzZWQgdG8gY2hlY2sgdGhhdCBiemlwMgpjb3VsZCBjb3Jy ZWN0bHkgZGVjb21wcmVzcyB0aGUgZmlsZS4gVGhlIG9yaWdpbmFsIChub24t Y29tcHJlc3NlZCkKZmlsZXMgYXJlIGRlbGliZXJhdGVseSBub3QgY2hlY2tl ZCBpbi4KCkEgLm1kNSBzdW0gaXMgZ2VuZXJhdGVkIGJ5OgogIG1kNXN1bSA8 IGZpbGUgPiBmaWxlLm1kNQoKVGhpcyBnZW5lcmF0ZXMgYSAubWQ1IGZpbGUg dGhhdCBkb2Vzbid0IGNhcnJ5IGEgZmlsZSBuYW1lIChidXQganVzdCAiLSIp LgpUaGV5IGNhbiB0aGVuIGJlIGNoZWNrZWQgYWdhaW4gd2l0aDoKICBtZDVz dW0gLS1jaGVjayBmaWxlLm1kNSA8IGZpbGUKCk5vdGUgZG8gTk9UIG5hbWUg YSBmaWxlIGVuZGluZyBpbiAudGVzdGZpbGVjb3B5IG9yIC50ZXN0ZmlsZWNv cHkuYnoyLgpUaG9zZSB3aWxsIGF1dG9tYXRpY2FsbHkgYmUgY2xlYW5lZCBi eSB1cCB0aGUgdGVzdGZyYW1ld29yay4KClRoZXJlIGlzIGEgc2ltcGxlIGJh c2ggc2NyaXB0IHRvIHJ1biB0aGUgdGVzdHM6CgpydW4tdGVzdHMgWy0tYnpp cDI9YnppcDItY29tbWFuZF0gWy0td2l0aG91dC12YWxncmluZF0KICAgICAg ICAgIFstLWlnbm9yZS1tZDVdIFstLXRlc3RzLWRpcj0vcGF0aC90by9iemlw Mi10ZXN0cy1kaXJdCgpJdCB3aWxsIGJ5IGRlZmF1bHQgdGVzdCB3aXRoIHRo ZSBjb21tYW5kICdiemlwMicsIHJ1bm5pbmcgdW5kZXIKdmFsZ3JpbmQgKGlm IGluc3RhbGxlZCBvbiB0aGUgc3lzdGVtKSwgY2hlY2tpbmcgbWQ1IHN1bSBm aWxlcwphZnRlciBkZWNvbXByZXNzaW9uIHVzaW5nIHRoZSBjdXJyZW50IGRp cmVjdG9yeSAoIi4iKSB0byBmaW5kCmFueSAuYnoyIG9yIC5iejIuYmFkIGZp bGVzIChhbmQgLm1kNSBmaWxlcyBpZiBjaGVja2VkKS4KCkZvciBlYWNoIC5i ejIgZmlsZSBmb3VuZCBpdCBpcyBkZWNvbXByZXNzZWQsIHJlY29tcHJlc3Nl ZCBhbmQKZGVjb21wcmVzc2VkIGFnYWluLiBPbmNlIHdpdGggdGhlIGRlZmF1 bHQgYnppcDIgc2V0dGluZ3MgYW5kCm9uY2UgaW4gLS1zbWFsbCAoLXMpIG1v ZGUuCgpGb3IgZWFjaCAuYnoyLmJhZCBmaWxlIGRlY29tcHJlc3Npb24gaXMg dHJpZWQgdHdpY2UgYWxzby4gSW4KZGVmYXVsdCBtb2RlIGFuZCBzbWFsbCBt b2RlLiBUaGUgYnppcDIgYmluYXJ5IGlzIGV4cGVjdGVkIHRvCnJldHVybiBl aXRoZXIgMSBvciAyIGFzIGV4aXQgc3RhdHVzLiBBbnkgb3RoZXIgZXhpdCBj b2RlIGlzCmludGVycHJldGVkIGFzIGZhaWx1cmUuCgpJZiB5b3UganVzdCB3 YW50IHRvIGNoZWNrIGEgZGlyZWN0b3J5IChhbmQgYW55IHN1YmRpcmVjdG9y aWVzKQpmdWxsIG9mIChrbm93biBnb29kKSAuYnoyIGZpbGVzIHlvdSBjYW4g aW52b2tlIHRoZSBzY3JpcHQgYXM6CgogIC4vcnVuLXRlc3QgLS1pZ25vcmUt bWQ1IC0tdGVzdHMtZGlyPS9kaXIvZnVsbC9vZmYvYnoyL2ZpbGVzCgo= --=-77PbeaT8L9CdHa0V4SzJ--