Here is a smaller reproducible test case (I did the preparation part in 64 bit cygwin): - download http://releases.ubuntu.com/16.10/ubuntu-16.10-desktop-i386.iso - download http://releases.ubuntu.com/16.10/ubuntu-16.10-desktop-amd64.iso $ md5sum ubuntu-16.10-desktop-i386.iso ubuntu-16.10-desktop-amd64.iso e9e9a6c6b3c8c265788f4e726af25994 *ubuntu-16.10-desktop-i386.iso 3f50877c05121f7fd8544bef2d722824 *ubuntu-16.10-desktop-amd64.iso $ cat ubuntu-16.10-desktop-i386.iso ubuntu-16.10-desktop-amd64.iso > test.iso $ md5sum test.iso d6e8cb13ce36e2c04961004782eec139 *test.iso $ lrzip -v test.iso The following options are in effect for this COMPRESSION. Threading is ENABLED. Number of CPUs detected: 4 Detected 17160601600 bytes ram Compression level 7 Nice Value: 19 Show Progress Verbose Temporary Directory set as: /tmp/ Compression mode is: LZMA. LZO Compressibility testing enabled Heuristically Computed Compression Window: 109 = 10900MB Output filename is: test.iso.lrz File size: 3204448256 Will take 1 pass Beginning rzip pre-processing phase test.iso - Compression Ratio: 1.352. Average Compression Speed: 15.357MB/s. Total time: 00:03:18.85 $ lrzip -i test.iso.lrz test.iso.lrz: lrzip version: 0.6 file Compression: rzip + lzma Decompressed file size: 3204448256 Compressed file size: 2370391407 Compression ratio: 1.352 MD5 used for integrity testing MD5: d6e8cb13ce36e2c04961004782eec139 in 32 bit cygwin: $ TMP=/cygdrive/d/tmp/ lrzip -v -d -o - /cygdrive/c/cygwin64/home/stein/test.iso.lrz | tee >(md5sum --tag) >(sha1sum --tag) > /dev/null The following options are in effect for this DECOMPRESSION. Threading is ENABLED. Number of CPUs detected: 4 Detected 17160601600 bytes ram Compression level 7 Nice Value: 19 Show Progress Verbose Output Filename Specified: - Temporary Directory set as: /cygdrive/d/tmp/ Outputting to stdout. Detected lrzip version 0.6 file. MD5 being used for integrity testing. Decompressing... Unable to decompress entirely in ram, will use physical files Dumping temporary file to control->outFILE. Average DeCompression Speed: 29.105MB/s Dumping temporary file to control->outFILE. [OK] - 3204448256 bytes Total time: 00:01:45.07 SHA1 (-) = da39a3ee5e6b4b0d3255bfef95601890afd80709 MD5 (-) = d41d8cd98f00b204e9800998ecf8427e The MD5 sum is wrong. (both md5 and sha1 values are the hashes of an empty file, so the data is lost somewhere) I attach the cygcheck -s -v -r output again (with cygwin 2.7.0-0.1 , I later checked with 2.7.0-0.2 too, got the same results) Regards, David On 1 February 2017 at 22:25, David Balažic wrote: > I tried the 2.7.0-0.1 test release and now the behavior changed. > > Before I got consistently a wrong MD5 sum of 8bd6ad48f2cea6a710af70b434d57673 > With this release I get c43a02c309fa5e0abe778201e9ceec46. > > So something changed. > > Either the problem is in cygwin or lrzip, I guess. > > Regards, > David > > > On 30 January 2017 at 16:23, David Balažic wrote: >> I tried in Ubuntu 32 bit (both the packaged lrzip and a self compiled >> one) and there the problem does not happen, so it looks like either: >> - bad lrzip in cygwin >> - cygwin pipe issues? >> >> Regards, >> David >> >> >> On 25 January 2017 at 23:15, David Balažic wrote: >>> Hi! >>> >>> The 32 bit version of lrzip 0.631-1 contains a bug that corrupts the >>> decompressed dat in some circumstances. >>> >>> I reproduced the problem on 2 PCs (the md5sum of the broken output was >>> the same on both systems). >>> >>> I seems to happen when the (de)compressed file size is bigger than the >>> available RAM (note that the 32 bit version uses max 4GB in any case) >>> and lrzip resorts to using a temporary file. >>> >>> See below for reproducing: >>> >>> $ lrzip -i sda.img.lrz2 >>> sda.img.lrz2: >>> lrzip version: 0.6 file >>> Compression: rzip + lzma >>> Decompressed file size: 64017212928 >>> Compressed file size: 7210541304 >>> Compression ratio: 8.878 >>> MD5 used for integrity testing >>> MD5: 6594f5b0d22efd345003260054165842 >>> >>> $ date; df -h ; TMP=/cygdrive/i/t/tmp/ lrzip -v -d -o - >>> sda.img.lrz2 | tee >(md5sum --tag) >(sha1sum --tag) > /dev/null ; >>> date >>> Tue Jan 24 21:29:01 CET 2017 >>> Filesystem Size Used Avail Use% Mounted on >>> C:/cygwin 114G 94G 21G 83% / >>> D: 541G 534G 7.1G 99% /cygdrive/d >>> I: 391G 279G 113G 72% /cygdrive/i >>> Q: 60G 57G 2.8G 96% /cygdrive/q >>> The following options are in effect for this DECOMPRESSION. >>> Threading is ENABLED. Number of CPUs detected: 4 >>> Detected 17160601600 bytes ram >>> Compression level 7 >>> Nice Value: 19 >>> Show Progress >>> Verbose >>> Output Filename Specified: - >>> Temporary Directory set as: /cygdrive/i/t/tmp/ >>> Outputting to stdout. >>> Detected lrzip version 0.6 file. >>> MD5 being used for integrity testing. >>> Decompressing... >>> Unable to decompress entirely in ram, will use physical files >>> Dumping temporary file to control->outFILE. >>> >>> [1]+ Stopped TMP=/cygdrive/i/t/tmp/ lrzip -v -d -o - >>> sda.img.lrz2 | tee >(md5sum --tag) >(sha1sum --tag) > /dev/null >>> Tue Jan 24 21:31:39 CET 2017 >>> >>> stein@hofer8 /cygdrive/i/Zotac_bak >>> $ fg >>> TMP=/cygdrive/i/t/tmp/ lrzip -v -d -o - sda.img.lrz2 | tee >(md5sum >>> --tag) >(sha1sum --tag) > /dev/null >>> Dumping temporary file to control->outFILE. >>> Dumping temporary file to control->outFILE. >>> Dumping temporary file to control->outFILE. >>> Dumping temporary file to control->outFILE. >>> Dumping temporary file to control->outFILE. >>> Dumping temporary file to control->outFILE. >>> Dumping temporary file to control->outFILE. >>> Dumping temporary file to control->outFILE. >>> Dumping temporary file to control->outFILE. >>> Dumping temporary file to control->outFILE. >>> Dumping temporary file to control->outFILE. >>> Dumping temporary file to control->outFILE. >>> Dumping temporary file to control->outFILE. >>> Dumping temporary file to control->outFILE. >>> Dumping temporary file to control->outFILE. >>> Dumping temporary file to control->outFILE. >>> Dumping temporary file to control->outFILE. >>> >>> Average DeCompression Speed: 0.668MB/s >>> Dumping temporary file to control->outFILE. >>> [OK] - 64017212928 bytes >>> Total time: 25:22:26.25 >>> SHA1 (-) = 6c519210541eb128c03b7c0f803adb2b46ee2a72 >>> MD5 (-) = 8bd6ad48f2cea6a710af70b434d57673 >>> >>> >>> The correct md5sum is 6594f5b0d22efd345003260054165842. >>> >>> >>> Simply decompressing the file (lrzip -d -o sda.img sda.img.lrz2) to >>> filesystem works fine, only when piped to stdout the problem happens. >>> >>> The 64 bit version does not have this problem. >>> >>> >>> I will check if the same problem happens with the native linux build >>> of lrzip (it takes a day...). >>> >>> >>> I tried to reproduce the problem with a smaller file, but there it did >>> not happen. Maybe my first test file has some corruption that causes >>> this (unlikely). >>> >>> Some version information (complete cygcheck -s -v -r output attached): >>> >>> base-cygwin 3.8-1 >>> cygwin 2.6.1-1 >>> lrzip 0.631-1 >>> >>> Regards, >>> David