Re: bug/deficiency in unzip: large files not supported?

Re: bug/deficiency in unzip: large files not supported?

[Brent]

On 2014-11-03 15:19:05-0600, Yaakov Selkowitz wrote:


>On 2014-10-30 19:01, Brent wrote:


>>I have encountered an inconsistency between cygwin's zip and unzip programs that I think reflects a bug (or incomplete implementation) in unzip.
>>The issue: zip can successfully archive large (> 4 GiB) files that unzip cannot extract.

>Could you please try again with the new upset-6.0-11?

Yaakov: I just downloaded unzip-6.0-11.  (Freudian slip with the upset?!).  Thanks much for the update!


I can verify that cygwin unzip passes my large file tests perfectly now.  I used a 5 GiB sized file filled with pseudo random data, but that should exceed the old 4 GiB file size limitation well enough.  I used all 4 combinations of Java and cygwin to archive and extract that file, verifying each time that the extraction perfectly reproduced the original large file, byte for byte.

Any thoughts on the bug that I found with cygwin unzip regarding its unicode handling?  In particular, cygwin unzip seems to work with cygwin zip, but cannot extract archives produced by multiple other mainstream zip programs.  My last email detailing this is
    https://cygwin.com/ml/cygwin/2014-11/msg00023.html

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: bug/deficiency in unzip: large files not supported?

[Yaakov Selkowitz]

On 2014-11-03 21:14, Brent wrote:
> Any thoughts on the bug that I found with cygwin unzip regarding its unicode handling?  In particular, cygwin unzip seems to work with cygwin zip, but cannot extract archives produced by multiple other mainstream zip programs.  My last email detailing this is
>      https://cygwin.com/ml/cygwin/2014-11/msg00023.html

Have you tried this again with 6.0-11 too?  Unless I'm doing something 
wrong, I can't reproduce your error with it.

--
Yaakov


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: bug/deficiency in unzip: large files not supported?

[Andrey Repin]

Greetings, Yaakov Selkowitz!

>> Any thoughts on the bug that I found with cygwin unzip regarding its unicode handling?  In particular, cygwin unzip seems to work with cygwin zip, but cannot extract archives produced by multiple other mainstream zip programs.  My last email detailing this is
>>      https://cygwin.com/ml/cygwin/2014-11/msg00023.html

> Have you tried this again with 6.0-11 too?  Unless I'm doing something 
> wrong, I can't reproduce your error with it.

Can this be related to locale settings?
I didn't see Brent mentioning his locale settings, though.


--
WBR,
Andrey Repin (anrdaemon@yandex.ru) 05.11.2014, <3:50>

Sorry for my terrible english...


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: bug/deficiency in unzip: large files not supported?

[Yaakov Selkowitz]

On 2014-10-30 19:01, Brent wrote:
> I have encountered an inconsistency between cygwin's zip and unzip programs that I think reflects a bug (or incomplete implementation) in unzip.
>
> The issue: zip can successfully archive large (> 4 GiB) files that unzip cannot extract.

Could you please try again with the new upset-6.0-11?

--
Yaakov

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

bug/deficiency in unzip: large files not supported?

[Brent]

I have encountered an inconsistency between cygwin's zip and unzip programs that I think reflects a bug (or incomplete implementation) in unzip.

First, I am using the very latest release of cygwin 64 bit:
    $ uname -a
    CYGWIN_NT-6.1 yhbrent 1.7.32(0.274/5/3) 2014-08-13 23:06 x86_64 Cygwin
And when I installed from setup-x86_64.exe, I added the zip and unzip programs from the Archive section.  Running setup-x86_64.exe just now, I see that the version numbers of what I have installed are:
    zip: 3.0-12
    unzip: 6.0-1

The issue: zip can successfully archive large (> 4 GiB) files that unzip cannot extract.

Consider this cygwin shell session I just did:

    $ ls -al bigFile.txt
    -rwx------+ 1 yhbrent None 5368709120 Oct 30 18:35 bigFile.txt

    $ zip test.zip bigFile.txt
      adding: bigFile.txt (deflated 0%)

    $ mkdir ./tmp

    $ mv test.zip ./tmp/

    $ cd ./tmp/

    $ unzip test.zip
    Archive:  test.zip
    error: End-of-centdir-64 signature not where expected (prepended bytes?)
      (attempting to process anyway)
    warning [test.zip]:  1068461579 extra bytes at beginning or within zipfile
      (attempting to process anyway)
       skipping: bigFile.txt             need PK compat. v4.5 (can do v2.1)

    $ echo $?
    81

In other words, I had a big (5 GiB) file, zip successfuly archved it, but when I tried to unzip it, I got error output and an error exit code of 81.

I know that zip really did work in archiving that large file, because I was able to successfuly extract its payload using 7zip, and confirmed that the result was the same (diff ../bigFile.txt ./bigFile.txt printed out nothing).

Does the error output "need PK compat. v4.5" mean that unzip fails to support the zip64 extensions needed to handle large files?

Given that zip clearly has zip64 support, it is a big issue that you can archive stuff that subsequently cannot be extracted.                             

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple