From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 90808 invoked by alias); 30 Jan 2017 15:23:53 -0000 Mailing-List: contact cygwin-help@cygwin.com; run by ezmlm Precedence: bulk List-Id: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: cygwin-owner@cygwin.com Mail-Followup-To: cygwin@cygwin.com Received: (qmail 90788 invoked by uid 89); 30 Jan 2017 15:23:52 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-1.1 required=5.0 tests=BAYES_00,FREEMAIL_ENVFROM_END_DIGIT,FREEMAIL_FROM,RCVD_IN_DNSWL_NONE,RCVD_IN_SORBS_SPAM,SPF_PASS autolearn=no version=3.3.2 spammy=Stopped, PCs, base-cygwin, basecygwin X-HELO: mail-ot0-f172.google.com Received: from mail-ot0-f172.google.com (HELO mail-ot0-f172.google.com) (74.125.82.172) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Mon, 30 Jan 2017 15:23:51 +0000 Received: by mail-ot0-f172.google.com with SMTP id 73so244732611otj.0 for ; Mon, 30 Jan 2017 07:23:50 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:content-transfer-encoding; bh=Ghdo+kTu/GSzDwmGnFDRBNccHAdAM5uNQjV5Xa5cW7Q=; b=anG9X/MGboH3sufRJygzkJHOGrP0u73kSou3b08rexlyRYqV6WjWrBu6tWE+HHxyFX AgqEkdkSa1XBOKr+KfkM/9LaXeTLT18rYQffhsfcpEdbjR5a1QD7+IRQLJF8OAolQPII /+bynGFfWtaoYvdKKXve4eGMsPVP9FzkVtr8ed1OAhZjmtcBTMhIExVbW3EA+Ln5TSDX VMTzWXShL432Kvh67HgmlvfCvsOzTjrV0Y/NiAMrz8lyqyLL7nCozbm9gkzA/CopC0r+ rifqRGz3k+veMI8Y3skcbCNZkvbvTuUhAv1CSIMaUG+rA1OG4NDREiCe9hjaMXcdiHWf uYsA== X-Gm-Message-State: AIkVDXKQOnIMhJ58dCYOWsr9xn0zjbcdwMpASYF26/shxmWrrPTMxQHa6hk73xpgEOiHPYhtod2E7zlhJ4Jv7A== X-Received: by 10.157.49.38 with SMTP id e35mr10021211otc.196.1485789829315; Mon, 30 Jan 2017 07:23:49 -0800 (PST) MIME-Version: 1.0 Received: by 10.157.50.229 with HTTP; Mon, 30 Jan 2017 07:23:48 -0800 (PST) In-Reply-To: References: From: =?UTF-8?Q?David_Bala=C5=BEic?= Date: Mon, 30 Jan 2017 15:23:00 -0000 Message-ID: Subject: Re: Bug in lrzip 0.631-1 (32 bit version) with -d -o - options To: cygwin@cygwin.com Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-IsSubscribed: yes X-SW-Source: 2017-01/txt/msg00388.txt.bz2 I tried in Ubuntu 32 bit (both the packaged lrzip and a self compiled one) and there the problem does not happen, so it looks like either: - bad lrzip in cygwin - cygwin pipe issues? Regards, David On 25 January 2017 at 23:15, David Bala=C5=BEic wrote: > Hi! > > The 32 bit version of lrzip 0.631-1 contains a bug that corrupts the > decompressed dat in some circumstances. > > I reproduced the problem on 2 PCs (the md5sum of the broken output was > the same on both systems). > > I seems to happen when the (de)compressed file size is bigger than the > available RAM (note that the 32 bit version uses max 4GB in any case) > and lrzip resorts to using a temporary file. > > See below for reproducing: > > $ lrzip -i sda.img.lrz2 > sda.img.lrz2: > lrzip version: 0.6 file > Compression: rzip + lzma > Decompressed file size: 64017212928 > Compressed file size: 7210541304 > Compression ratio: 8.878 > MD5 used for integrity testing > MD5: 6594f5b0d22efd345003260054165842 > > $ date; df -h ; TMP=3D/cygdrive/i/t/tmp/ lrzip -v -d -o - > sda.img.lrz2 | tee >(md5sum --tag) >(sha1sum --tag) > /dev/null ; > date > Tue Jan 24 21:29:01 CET 2017 > Filesystem Size Used Avail Use% Mounted on > C:/cygwin 114G 94G 21G 83% / > D: 541G 534G 7.1G 99% /cygdrive/d > I: 391G 279G 113G 72% /cygdrive/i > Q: 60G 57G 2.8G 96% /cygdrive/q > The following options are in effect for this DECOMPRESSION. > Threading is ENABLED. Number of CPUs detected: 4 > Detected 17160601600 bytes ram > Compression level 7 > Nice Value: 19 > Show Progress > Verbose > Output Filename Specified: - > Temporary Directory set as: /cygdrive/i/t/tmp/ > Outputting to stdout. > Detected lrzip version 0.6 file. > MD5 being used for integrity testing. > Decompressing... > Unable to decompress entirely in ram, will use physical files > Dumping temporary file to control->outFILE. > > [1]+ Stopped TMP=3D/cygdrive/i/t/tmp/ lrzip -v -d -o - > sda.img.lrz2 | tee >(md5sum --tag) >(sha1sum --tag) > /dev/null > Tue Jan 24 21:31:39 CET 2017 > > stein@hofer8 /cygdrive/i/Zotac_bak > $ fg > TMP=3D/cygdrive/i/t/tmp/ lrzip -v -d -o - sda.img.lrz2 | tee >(md5sum > --tag) >(sha1sum --tag) > /dev/null > Dumping temporary file to control->outFILE. > Dumping temporary file to control->outFILE. > Dumping temporary file to control->outFILE. > Dumping temporary file to control->outFILE. > Dumping temporary file to control->outFILE. > Dumping temporary file to control->outFILE. > Dumping temporary file to control->outFILE. > Dumping temporary file to control->outFILE. > Dumping temporary file to control->outFILE. > Dumping temporary file to control->outFILE. > Dumping temporary file to control->outFILE. > Dumping temporary file to control->outFILE. > Dumping temporary file to control->outFILE. > Dumping temporary file to control->outFILE. > Dumping temporary file to control->outFILE. > Dumping temporary file to control->outFILE. > Dumping temporary file to control->outFILE. > > Average DeCompression Speed: 0.668MB/s > Dumping temporary file to control->outFILE. > [OK] - 64017212928 bytes > Total time: 25:22:26.25 > SHA1 (-) =3D 6c519210541eb128c03b7c0f803adb2b46ee2a72 > MD5 (-) =3D 8bd6ad48f2cea6a710af70b434d57673 > > > The correct md5sum is 6594f5b0d22efd345003260054165842. > > > Simply decompressing the file (lrzip -d -o sda.img sda.img.lrz2) to > filesystem works fine, only when piped to stdout the problem happens. > > The 64 bit version does not have this problem. > > > I will check if the same problem happens with the native linux build > of lrzip (it takes a day...). > > > I tried to reproduce the problem with a smaller file, but there it did > not happen. Maybe my first test file has some corruption that causes > this (unlikely). > > Some version information (complete cygcheck -s -v -r output attached): > > base-cygwin 3.8-1 > cygwin 2.6.1-1 > lrzip 0.631-1 > > Regards, > David -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple