* xz archiver has a problem compressing a large volume in multithread mode
@ 2017-11-13 7:50 Tetsuji Rai via cygwin
[not found] ` <5A0A6513.7030808@tlinx.org>
0 siblings, 1 reply; 3+ messages in thread
From: Tetsuji Rai via cygwin @ 2017-11-13 7:50 UTC (permalink / raw)
To: cygwin
Hi all,
I have many music files, so I often archive music directory with "tar
-cJvf music.tar.xz music" with environment variable XZ_OPTS="-T 3" (I
have 4 threaded first generation i7).   The archive is about 30GB.Â
Until some weeks ago it worked fine.  However since I updated to the
latest version of cygwin64 some weeks ago, I have a trouble extracting
from that archive at some point. "tar -xJvf music.tar.xz" says "archive
file is corrupt" or something and quits.  This is reproducible, and
gzip has no problem at all (ie, "tar -czvf music.tar.gz music" works fine.
So I want to ask anyone has a similar problem?  I want developers to
fix it.
Thanks in advance.
--
Problem reports: http://cygwin.com/problems.html
FAQ: http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: xz archiver has a problem compressing a large volume in multithread mode
[not found] ` <5A0A6513.7030808@tlinx.org>
@ 2017-11-14 4:16 ` Tetsuji Rai via cygwin
2017-11-15 5:41 ` Tetsuji Rai via cygwin
1 sibling, 0 replies; 3+ messages in thread
From: Tetsuji Rai via cygwin @ 2017-11-14 4:16 UTC (permalink / raw)
To: L A Walsh, cygwin
Not yet, because it takes a long time on my machine (1st generation i7).
Today I will try it.
--
Problem reports: http://cygwin.com/problems.html
FAQ: http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: xz archiver has a problem compressing a large volume in multithread mode
[not found] ` <5A0A6513.7030808@tlinx.org>
2017-11-14 4:16 ` Tetsuji Rai via cygwin
@ 2017-11-15 5:41 ` Tetsuji Rai via cygwin
1 sibling, 0 replies; 3+ messages in thread
From: Tetsuji Rai via cygwin @ 2017-11-15 5:41 UTC (permalink / raw)
To: L A Walsh, cygwin
On 11/14/2017 12:37, L A Walsh wrote:
> Tetsuji Rai via cygwin wrote:
>> Hi all,
>>
>> I have many music files, so I often archive music directory with "tar
>> -cJvf music.tar.xz music" with environment variable XZ_OPTS="-T 3" (I
>> have 4 threaded first generation i7).   The archive is about 30GB.
>> Until some weeks ago it worked fine.  However since I updated to the
>> latest version of cygwin64 some weeks ago, I have a trouble extracting
>> from that archive at some point. "tar -xJvf music.tar.xz" says "archive
>> file is corrupt" or something and quits.  T
> ---
> Â Â Have you tried it with one xz thread?...
>
> BTW to compress/decompress w/xz, I usually use
>  tar caf /tmp/archive.txz /music_archive  to compress and
>  tar xaf /tmp/archive.txz                 to uncompress
>
> (but those should be unrelated to your problem).
>
> I just tried "-T 3" on my version of tar and with "xz"
> and it doesn't make a difference in time, but does result
> in slightly larger file sizes -- which is characteristic
> of it using multiple threads. However, tar only produces
> 1 thread, so it never gets a chance to run things in multiple
> threads.
>
> Also --
>
> Besides, music isn't "real compressible"Â -- especially
> if you are compressing mp3 or flac, but the general purpose
> compression algorithms don't do a great job of compressing
> music.
>
> Compressing mp3->mp3+xz saved about 2.8%
> Compressing flac->flac+xz saved about 0.82% (<less than 1%)
>
> Compressing raw waves:
> xz files were 32% LARGER than flac.
>
> So what type of music are you compressing?
>
> BTW -- sending this to you in addition to list, since
> the list has been flakey for me recently -- seems to
> have a bug rejecting my domain.
>
>
>
Now I found single compression doesn't harm at all. Multithreaded
compression is the problem.  Actually, compression of mp3 files doesn't
mean anything.....so I will just use tar for archive.   It's much faster :)
--
Problem reports: http://cygwin.com/problems.html
FAQ: http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2017-11-15 5:41 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-11-13 7:50 xz archiver has a problem compressing a large volume in multithread mode Tetsuji Rai via cygwin
[not found] ` <5A0A6513.7030808@tlinx.org>
2017-11-14 4:16 ` Tetsuji Rai via cygwin
2017-11-15 5:41 ` Tetsuji Rai via cygwin
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).