* gzip 1.4-1 creates corrupt archive
@ 2012-08-25 11:18 Peter
2012-08-26 3:49 ` K Stahl
2012-08-28 2:43 ` Warren Young
0 siblings, 2 replies; 8+ messages in thread
From: Peter @ 2012-08-25 11:18 UTC (permalink / raw)
To: cygwin
To trim a log file on my windows box I tried the lines below in a bash
script. (my bash is 4.1.10(4))
cat "$LOG" | gzip > "$LOG.old.gz"
> "$LOG"
The resulting archive was corrupt.
Directly compressing like gzip "$LOG" works OK but is not suitable in my
case as the logging service constantly writes to the file
I also tried the -c and -a options of gzip but did not succeed.
From my investigation I suspect there is something with textmode and binmode
but I do not fully understand how to deal with it.
What I finally found is that bzip2 works in the above line.
So my solution is to use bzip2 instead of gzip.
So my question is: why can't gzip do it automatically if bzip2 can?
--
Problem reports: http://cygwin.com/problems.html
FAQ: http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: gzip 1.4-1 creates corrupt archive
2012-08-25 11:18 gzip 1.4-1 creates corrupt archive Peter
@ 2012-08-26 3:49 ` K Stahl
2012-08-26 4:39 ` Ryan Johnson
2012-08-28 2:43 ` Warren Young
1 sibling, 1 reply; 8+ messages in thread
From: K Stahl @ 2012-08-26 3:49 UTC (permalink / raw)
To: cygwin
> cat "$LOG" | gzip > "$LOG.old.gz" > "$LOG"
Don't have access to my machine at the moment, but try adding the '-c'
options to gzip (e.g. gzip -c ...)
--
Problem reports: http://cygwin.com/problems.html
FAQ: http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: gzip 1.4-1 creates corrupt archive
2012-08-26 3:49 ` K Stahl
@ 2012-08-26 4:39 ` Ryan Johnson
2012-08-26 6:25 ` K Stahl
0 siblings, 1 reply; 8+ messages in thread
From: Ryan Johnson @ 2012-08-26 4:39 UTC (permalink / raw)
To: cygwin
On 25/08/2012 11:45 AM, K Stahl wrote:
>> cat "$LOG" | gzip > "$LOG.old.gz" > "$LOG"
> Don't have access to my machine at the moment, but try adding the '-c'
> options to gzip (e.g. gzip -c ...)
He said he tried that and it didn't work...
Ryan
--
Problem reports: http://cygwin.com/problems.html
FAQ: http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: gzip 1.4-1 creates corrupt archive
2012-08-26 4:39 ` Ryan Johnson
@ 2012-08-26 6:25 ` K Stahl
2012-08-27 13:06 ` Earnie Boyd
0 siblings, 1 reply; 8+ messages in thread
From: K Stahl @ 2012-08-26 6:25 UTC (permalink / raw)
To: cygwin
Missed that section, was reading from my phone. Anyway, how about
instead of chastising someone for trying to help, you come up with an
alternative solution?
--
Problem reports: http://cygwin.com/problems.html
FAQ: http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: gzip 1.4-1 creates corrupt archive
2012-08-26 6:25 ` K Stahl
@ 2012-08-27 13:06 ` Earnie Boyd
2012-08-27 14:08 ` Ryan Johnson
0 siblings, 1 reply; 8+ messages in thread
From: Earnie Boyd @ 2012-08-27 13:06 UTC (permalink / raw)
To: cygwin
On Sun, Aug 26, 2012 at 12:38 AM, K Stahl wrote:
> Missed that section, was reading from my phone. Anyway, how about
> instead of chastising someone for trying to help, you come up with an
> alternative solution?
Stating that you missed something in a post isn't really chastising as
much as it is criticism that you need to pay attention more. Reading
from a phone isn't an excuse for miscommunication.
--
Earnie
-- https://sites.google.com/site/earnieboyd
--
Problem reports: http://cygwin.com/problems.html
FAQ: http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: gzip 1.4-1 creates corrupt archive
2012-08-27 13:06 ` Earnie Boyd
@ 2012-08-27 14:08 ` Ryan Johnson
2012-08-28 0:09 ` KHMan
0 siblings, 1 reply; 8+ messages in thread
From: Ryan Johnson @ 2012-08-27 14:08 UTC (permalink / raw)
To: cygwin
On 27/08/2012 7:29 AM, Earnie Boyd wrote:
> On Sun, Aug 26, 2012 at 12:38 AM, K Stahl wrote:
>> Missed that section, was reading from my phone. Anyway, how about
>> instead of chastising someone for trying to help, you come up with an
>> alternative solution?
> Stating that you missed something in a post isn't really chastising as
> much as it is criticism that you need to pay attention more. Reading
> from a phone isn't an excuse for miscommunication.
>
*sigh*
It was actually neither. Somebody who seemed to have a clue how to
diagnose this weirdness happened to miss a relevant detail. Happens to
all of us, but I figured if I pointed it out early on it would save the
OP a round trip latency in getting to the next phase of diagnosis, since
the advice given had already been tried.
Unfortunately, it backfired and introduced 2+ rounds of latency instead.
Back on topic, I can't repro using text or binary (executable) files,
which makes me wonder whether the issue is the constant appending the OP
mentioned. Perhaps gzip reads twice from the growing log file in a way
that confuses it, and bzip2 doesn't? Perhaps there's some weirdness with
atimes that makes gzip think the file is unchanged when it actually has
grown?
Ryan
--
Problem reports: http://cygwin.com/problems.html
FAQ: http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: gzip 1.4-1 creates corrupt archive
2012-08-27 14:08 ` Ryan Johnson
@ 2012-08-28 0:09 ` KHMan
0 siblings, 0 replies; 8+ messages in thread
From: KHMan @ 2012-08-28 0:09 UTC (permalink / raw)
To: cygwin
On 8/27/2012 8:01 PM, Ryan Johnson wrote:
> On 27/08/2012 7:29 AM, Earnie Boyd wrote:
>> On Sun, Aug 26, 2012 at 12:38 AM, K Stahl wrote:
>>> [snip snip snip]
>> [snip snip snip]
> [snip snip snip]
>
> Back on topic, I can't repro using text or binary (executable)
> files, which makes me wonder whether the issue is the constant
> appending the OP mentioned. Perhaps gzip reads twice from the
> growing log file in a way that confuses it, and bzip2 doesn't?
> Perhaps there's some weirdness with atimes that makes gzip think
> the file is unchanged when it actually has grown?
Same for me on text and binary files. I think OP should test
non-updated text and binary files, that would certainly help to
prove or disprove some theories.
Also failed to create a corrupted gzip file by manually appending
from another terminal during compression. After all, gzip is
reading from stdin only what it gets from cat, and gzip has been
handling stdin/stdout since forever. Sounds simple enough...
OP gave (reproduced as follows):
> To trim a log file on my windows box I tried the lines
> below in a bash script. (my bash is 4.1.10(4))
cat "$LOG" | gzip > "$LOG.old.gz"
> "$LOG"
What was the error message reported by gzip for the corrupted
file? gzip has a very simple file structure, if OP can share a
corrupt file, it should be pretty easy to learn exactly what
aspect of the gzip file is broken.
--
Cheers,
Kein-Hong Man (esq.)
Kuala Lumpur, Malaysia
--
Problem reports: http://cygwin.com/problems.html
FAQ: http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: gzip 1.4-1 creates corrupt archive
2012-08-25 11:18 gzip 1.4-1 creates corrupt archive Peter
2012-08-26 3:49 ` K Stahl
@ 2012-08-28 2:43 ` Warren Young
1 sibling, 0 replies; 8+ messages in thread
From: Warren Young @ 2012-08-28 2:43 UTC (permalink / raw)
To: Cygwin-L
On 8/25/2012 12:11 AM, Peter wrote:
>
> cat "$LOG" | gzip > "$LOG.old.gz"
Generally speaking, cat with a single file is a sign you're not using
redirection to its fullest. In this case, you get the same effect with
much less overhead with:
gzip < "$LOG" > "$LOG.old.gz"
> The resulting archive was corrupt.
Evidenced by...?
I don't disbelieve you, but can't you at least produce a "hello world"
example file that shows the problem and send it to the list, or put it
on a public web server somewhere?
>From my investigation I suspect there is something with textmode and binmode
> but I do not fully understand how to deal with it.
Why do you need the textmode hack?
That is to say, what non-Cygwin tools are you using that truly cannot
cope with Unix LF line endings, thereby requiring Cygwin to do strange
things in order to appease the native tools?
(Oh, and in case anyone reading this things I'm being a hypocrite after
my arguments in the SQLite/svn thread, I'll reiterate that I don't like
interop hacks and would like to avoid them wherever possible. I'm
hoping to find out that it is actually possible here.)
--
Problem reports: http://cygwin.com/problems.html
FAQ: http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2012-08-27 18:29 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2012-08-25 11:18 gzip 1.4-1 creates corrupt archive Peter
2012-08-26 3:49 ` K Stahl
2012-08-26 4:39 ` Ryan Johnson
2012-08-26 6:25 ` K Stahl
2012-08-27 13:06 ` Earnie Boyd
2012-08-27 14:08 ` Ryan Johnson
2012-08-28 0:09 ` KHMan
2012-08-28 2:43 ` Warren Young
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).