public inbox for cygwin@cygwin.com
 help / color / mirror / Atom feed
* RE: Cat and Head Problems with Binary Files
       [not found] <4.1.19990722175718.00de9820@mailhost.teknowledge.com>
@ 1999-07-23  9:35 ` Jeffry T Ross
  1999-07-24 22:14   ` Fergus Henderson
  1999-07-31 18:34   ` Jeffry T Ross
  0 siblings, 2 replies; 16+ messages in thread
From: Jeffry T Ross @ 1999-07-23  9:35 UTC (permalink / raw)
  To: cygwin

Randy

I thought the suggestion to use dd was a good, although a bit excessive
when one only wants to move bytes from a file to a pipe without any
conversion. So I tried it:

dd if=test1 > test2
test1 was 1296 bytes and test2 became 1301 bytes.

So then I tried dd directly to a file.
dd if=test1 of=test2
this worked fine.

So now I don't suspect cat but the piping and redirection.
The question is how does cygwin implement it's piping and redirection?
Is this problem imposed by the underlying WIN/DOS or is it a bug in BASH?

The pipe shouldn't be trying to interpret the data going through it.
That's the job of the process at the end of the pipe, be it the terminal
or another program.



							jtr@mitre.org
 

-----Original Message-----
From: Randall Schulz [ mailto:rschulz@teknowledge.com ]
Sent: Thursday, July 22, 1999 9:24 PM
To: jtr@mitre.org; cygwin@sourceware.cygnus.com
Subject: RE: Cat and Head Problems with Binary Files


JTR,

Who ever said head and cat were the way to deal with a problem like this?
Head may have a -c option, but it's indended use is to capture a certain
number of *lines* of the beginning of a file.

For what you want to do, you should try dd. It's tailor made for this sort
of thing. I knows not of lines, but every other manner of file copying
based on regular file subdivisions is its forte. It can also perform some
transformations such as case conversion, character code translation, byte
swapping, etc.

Check it out!

Randy Schulz
Teknowledge Corp.
Palo Alto, CA USA


At 05:49 PM 7/22/99 , Jeffry T Ross wrote:
>Who ever said that cat and head are textutils?
>On the Unix side of the world they're commonly used
>on binary files.  If you have a 10gig file of binary
>data, what the easiest way to get a 10k chunk?
>
>How about: head -c 10000 bigfile > littlefile
>
>This works in Unix because Unix thinks all files are
>binary, and that's because all files are binary.  The
>notion of text files is a bogus limitation imposed by
>Microsoft.
>
>Is there a reason why having cat treat all files as binary
>would cause erroneous performance when cat was used on a
>file you'd consider to be text?
>
>
>Thanks,
>jtr@mitre.org


--
Want to unsubscribe from this list?
Send a message to cygwin-unsubscribe@sourceware.cygnus.com

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: Cat and Head Problems with Binary Files
  1999-07-23  9:35 ` Cat and Head Problems with Binary Files Jeffry T Ross
@ 1999-07-24 22:14   ` Fergus Henderson
  1999-07-31 18:34     ` Fergus Henderson
  1999-07-31 18:34   ` Jeffry T Ross
  1 sibling, 1 reply; 16+ messages in thread
From: Fergus Henderson @ 1999-07-24 22:14 UTC (permalink / raw)
  To: Jeffry T Ross; +Cc: cygwin

On 23-Jul-1999, Jeffry T Ross <jtr@mitre.org> wrote:
> Randy
> 
> I thought the suggestion to use dd was a good, although a bit excessive
> when one only wants to move bytes from a file to a pipe without any
> conversion. So I tried it:
> 
> dd if=test1 > test2
> test1 was 1296 bytes and test2 became 1301 bytes.
> 
> So then I tried dd directly to a file.
> dd if=test1 of=test2
> this worked fine.
>
> So now I don't suspect cat but the piping and redirection.

I think the symptom that you report above is a bug in `dd'
(i.e. `dd' has not been properly ported to Cygwin).

> The question is how does cygwin implement it's piping and redirection?

By default, stdin, stdout, and stderr are assumed to use text mode.

If a program such as `dd' wishes to use stdin/stdout/stderr in binary
mode, then it should be modified to do so.

The issue here is basically the same as passing extra flags to open()
and fopen() to say that files should be opened in binary mode.
It's a little bit more complicated, though, since stdin/stdout/stderr
are already open.  However, there should be a way of changing the mode
of an already open file.  I don't recall off-hand what it is, though.
(I vaguely recall some mention of a setmode() function, but I could be
way off base with that...)

> Is this problem imposed by the underlying WIN/DOS or is it a bug in BASH?

Bash doesn't know whether the underlying program (e.g. `dd') will use
the stream in text or binary mode, so there's nothing bash can do.
I suppose you could add special syntax for redirection in binary mode,
so that the user could type e.g. "dd if=test1 >@ test2", but
it makes much more sense for it to be the program's responsibility to
use its files in the right mode rather than for this to be the user's
responsibility.

-- 
Fergus Henderson <fjh@cs.mu.oz.au>  |  "I have always known that the pursuit
WWW: < http://www.cs.mu.oz.au/~fjh >  |  of excellence is a lethal habit"
PGP: finger fjh@128.250.37.3        |     -- the last words of T. S. Garp.

--
Want to unsubscribe from this list?
Send a message to cygwin-unsubscribe@sourceware.cygnus.com

^ permalink raw reply	[flat|nested] 16+ messages in thread

* RE: Cat and Head Problems with Binary Files
  1999-07-23  9:35 ` Cat and Head Problems with Binary Files Jeffry T Ross
  1999-07-24 22:14   ` Fergus Henderson
@ 1999-07-31 18:34   ` Jeffry T Ross
  1 sibling, 0 replies; 16+ messages in thread
From: Jeffry T Ross @ 1999-07-31 18:34 UTC (permalink / raw)
  To: cygwin

Randy

I thought the suggestion to use dd was a good, although a bit excessive
when one only wants to move bytes from a file to a pipe without any
conversion. So I tried it:

dd if=test1 > test2
test1 was 1296 bytes and test2 became 1301 bytes.

So then I tried dd directly to a file.
dd if=test1 of=test2
this worked fine.

So now I don't suspect cat but the piping and redirection.
The question is how does cygwin implement it's piping and redirection?
Is this problem imposed by the underlying WIN/DOS or is it a bug in BASH?

The pipe shouldn't be trying to interpret the data going through it.
That's the job of the process at the end of the pipe, be it the terminal
or another program.



							jtr@mitre.org
 

-----Original Message-----
From: Randall Schulz [ mailto:rschulz@teknowledge.com ]
Sent: Thursday, July 22, 1999 9:24 PM
To: jtr@mitre.org; cygwin@sourceware.cygnus.com
Subject: RE: Cat and Head Problems with Binary Files


JTR,

Who ever said head and cat were the way to deal with a problem like this?
Head may have a -c option, but it's indended use is to capture a certain
number of *lines* of the beginning of a file.

For what you want to do, you should try dd. It's tailor made for this sort
of thing. I knows not of lines, but every other manner of file copying
based on regular file subdivisions is its forte. It can also perform some
transformations such as case conversion, character code translation, byte
swapping, etc.

Check it out!

Randy Schulz
Teknowledge Corp.
Palo Alto, CA USA


At 05:49 PM 7/22/99 , Jeffry T Ross wrote:
>Who ever said that cat and head are textutils?
>On the Unix side of the world they're commonly used
>on binary files.  If you have a 10gig file of binary
>data, what the easiest way to get a 10k chunk?
>
>How about: head -c 10000 bigfile > littlefile
>
>This works in Unix because Unix thinks all files are
>binary, and that's because all files are binary.  The
>notion of text files is a bogus limitation imposed by
>Microsoft.
>
>Is there a reason why having cat treat all files as binary
>would cause erroneous performance when cat was used on a
>file you'd consider to be text?
>
>
>Thanks,
>jtr@mitre.org


--
Want to unsubscribe from this list?
Send a message to cygwin-unsubscribe@sourceware.cygnus.com

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: Cat and Head Problems with Binary Files
  1999-07-24 22:14   ` Fergus Henderson
@ 1999-07-31 18:34     ` Fergus Henderson
  0 siblings, 0 replies; 16+ messages in thread
From: Fergus Henderson @ 1999-07-31 18:34 UTC (permalink / raw)
  To: Jeffry T Ross; +Cc: cygwin

On 23-Jul-1999, Jeffry T Ross <jtr@mitre.org> wrote:
> Randy
> 
> I thought the suggestion to use dd was a good, although a bit excessive
> when one only wants to move bytes from a file to a pipe without any
> conversion. So I tried it:
> 
> dd if=test1 > test2
> test1 was 1296 bytes and test2 became 1301 bytes.
> 
> So then I tried dd directly to a file.
> dd if=test1 of=test2
> this worked fine.
>
> So now I don't suspect cat but the piping and redirection.

I think the symptom that you report above is a bug in `dd'
(i.e. `dd' has not been properly ported to Cygwin).

> The question is how does cygwin implement it's piping and redirection?

By default, stdin, stdout, and stderr are assumed to use text mode.

If a program such as `dd' wishes to use stdin/stdout/stderr in binary
mode, then it should be modified to do so.

The issue here is basically the same as passing extra flags to open()
and fopen() to say that files should be opened in binary mode.
It's a little bit more complicated, though, since stdin/stdout/stderr
are already open.  However, there should be a way of changing the mode
of an already open file.  I don't recall off-hand what it is, though.
(I vaguely recall some mention of a setmode() function, but I could be
way off base with that...)

> Is this problem imposed by the underlying WIN/DOS or is it a bug in BASH?

Bash doesn't know whether the underlying program (e.g. `dd') will use
the stream in text or binary mode, so there's nothing bash can do.
I suppose you could add special syntax for redirection in binary mode,
so that the user could type e.g. "dd if=test1 >@ test2", but
it makes much more sense for it to be the program's responsibility to
use its files in the right mode rather than for this to be the user's
responsibility.

-- 
Fergus Henderson <fjh@cs.mu.oz.au>  |  "I have always known that the pursuit
WWW: < http://www.cs.mu.oz.au/~fjh >  |  of excellence is a lethal habit"
PGP: finger fjh@128.250.37.3        |     -- the last words of T. S. Garp.

--
Want to unsubscribe from this list?
Send a message to cygwin-unsubscribe@sourceware.cygnus.com

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Cat and Head Problems with Binary Files
  1999-07-22 17:10 Jeffry T Ross
@ 1999-07-31 18:34 ` Jeffry T Ross
  0 siblings, 0 replies; 16+ messages in thread
From: Jeffry T Ross @ 1999-07-31 18:34 UTC (permalink / raw)
  To: cygwin

I've seen a lot of discussion about cat's handling of binary files.
What I didn't see and would like to know is if the next version of
cygwin will allow me to correctly pipe or redirect the output from
cat and head if the source is a binary file?  Also, why were these
programs implemented this way to begin with?


I'm trying to pipe a binary file to a program I wrote and either
the pipe/redirect is breaking or cat/head is dying at arbitrary
points in the binary files.  I suspect that my binary files contains
what looks like an EOF.  This is similar to other threads I've seen
on this list.


Thanks,
jtr@mitre.org
 

--
Want to unsubscribe from this list?
Send a message to cygwin-unsubscribe@sourceware.cygnus.com

^ permalink raw reply	[flat|nested] 16+ messages in thread

* RE: Cat and Head Problems with Binary Files
  1999-07-29  5:09 Earnie Boyd
@ 1999-07-31 18:34 ` Earnie Boyd
  0 siblings, 0 replies; 16+ messages in thread
From: Earnie Boyd @ 1999-07-31 18:34 UTC (permalink / raw)
  To: jtr, cygwin

--- Jeffry T Ross <jtr@mitre.org> wrote:
> Randy
> 
> I thought the suggestion to use dd was a good, although a bit excessive
> when one only wants to move bytes from a file to a pipe without any
> conversion. So I tried it:
> 
> dd if=test1 > test2
> test1 was 1296 bytes and test2 became 1301 bytes.
> 
> So then I tried dd directly to a file.
> dd if=test1 of=test2
> this worked fine.
> 
> So now I don't suspect cat but the piping and redirection.
> The question is how does cygwin implement it's piping and redirection?
> Is this problem imposed by the underlying WIN/DOS or is it a bug in BASH?
> 
> The pipe shouldn't be trying to interpret the data going through it.
> That's the job of the process at the end of the pipe, be it the terminal
> or another program.
> 
> 
> 
> 							jtr@mitre.org
>  
Hi Jeffry,

What version of cygwin1.dll are you using?  Supply the output of `uname -a' and
I'll know the answer to that.

The stock b20.1 version has a problem with the default behaviors of the 
[no]binary switch for the CYGWIN variable.  If you don't have at least the Jan
15th snapshot then go to the site or mirror and install that version.

The [no]binmode switch affect the processing mode (text vs binary) for pipes
and redirects.  The default is binmode which will pass the \r character through
the pipe and not add the \r character on the redirect.  However, the default
binmode and the specified binmode are different in the stock b20.1 version. 
This created some interesting results.

Good Luck,
===
Earnie Boyd < mailto:earnie_boyd@yahoo.com >

Newbies, please visit
< http://www.freeyellow.com/members5/gw32/index.html >

(If you respond to the list, then please don't cc me)
_____________________________________________________________
Do You Yahoo!?
Free instant messaging and more at http://messenger.yahoo.com


--
Want to unsubscribe from this list?
Send a message to cygwin-unsubscribe@sourceware.cygnus.com

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: Cat and Head Problems with Binary Files
  1999-07-23 18:19   ` Geoffrey Noer
@ 1999-07-31 18:34     ` Geoffrey Noer
  0 siblings, 0 replies; 16+ messages in thread
From: Geoffrey Noer @ 1999-07-31 18:34 UTC (permalink / raw)
  To: Jeffry T Ross; +Cc: cygwin

On Thu, Jul 22, 1999, Jeffry T Ross wrote:
>
> Who ever said that cat and head are textutils?
> On the Unix side of the world they're commonly used
> on binary files.

Yes, I consider the current state to be broken for this
reason but nobody has volunteered to fix it so...

In DJGPP I believe there is a set of patches that addresses this
problem by adding a flag to all of these programs so that
people can specify whether they are (for example) catting a
binary or text file.  These patches have not yet been applied/enabled
for the Cygwin ports.

If someone would volunteer to make them work for Cygwin and submit the
changes to the FSF maintainers (and perhaps to the cygwin-developers
list), I'm sure many people would be appreciative.

Best regards,

-- 
Geoffrey Noer		Email: noer@cygnus.com
Cygnus Solutions

--
Want to unsubscribe from this list?
Send a message to cygwin-unsubscribe@sourceware.cygnus.com

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: Cat and Head Problems with Binary Files
  1999-07-22 18:05   ` $Bill Luebkert
@ 1999-07-31 18:34     ` $Bill Luebkert
  0 siblings, 0 replies; 16+ messages in thread
From: $Bill Luebkert @ 1999-07-31 18:34 UTC (permalink / raw)
  To: jtr; +Cc: cygwin

Jeffry T Ross wrote:
> 
> Who ever said that cat and head are textutils?
> On the Unix side of the world they're commonly used
> on binary files.  If you have a 10gig file of binary
> data, what the easiest way to get a 10k chunk?
> 
> How about: head -c 10000 bigfile > littlefile
> 
> This works in Unix because Unix thinks all files are
> binary, and that's because all files are binary.  The
> notion of text files is a bogus limitation imposed by
> Microsoft.
> 
> Is there a reason why having cat treat all files as binary
> would cause erroneous performance when cat was used on a
> file you'd consider to be text?

I have a slightly different opinion:

I could just as easily say that cat and text are text utils since 
they operate on text files (even extended 8-bit character text files
which one could call binary).  There is such a thing as 8-bit text.

The point is, on UNIX files are strings of bytes, not binary and not text, 
just a bunch of 8-bit characters strung together in a file.  UNIX doesn't 
care if those bytes are 'Text" or 'Binary' after all, they are/could be both.

A byte is a byte, of course of course, unless of course the byte is 
really two little-endian nybbles. :)

-- 
  ,-/-  __      _  _         $Bill Luebkert   ICQ=14439852
 (_/   /  )    // //       DBE Collectibles   http://www.wgn.net/~dbe/
  / ) /--<  o // //      Mailto:dbe@wgn.net   http://dbecoll.webjump.com/
-/-' /___/_<_</_</_    http://www.freeyellow.com/members/dbecoll/

--
Want to unsubscribe from this list?
Send a message to cygwin-unsubscribe@sourceware.cygnus.com

^ permalink raw reply	[flat|nested] 16+ messages in thread

* RE: Cat and Head Problems with Binary Files
  1999-07-22 17:49 ` Jeffry T Ross
  1999-07-22 18:05   ` $Bill Luebkert
  1999-07-23 18:19   ` Geoffrey Noer
@ 1999-07-31 18:34   ` Jeffry T Ross
  2 siblings, 0 replies; 16+ messages in thread
From: Jeffry T Ross @ 1999-07-31 18:34 UTC (permalink / raw)
  To: cygwin

Who ever said that cat and head are textutils?
On the Unix side of the world they're commonly used
on binary files.  If you have a 10gig file of binary
data, what the easiest way to get a 10k chunk?

How about: head -c 10000 bigfile > littlefile

This works in Unix because Unix thinks all files are
binary, and that's because all files are binary.  The
notion of text files is a bogus limitation imposed by
Microsoft.

Is there a reason why having cat treat all files as binary
would cause erroneous performance when cat was used on a
file you'd consider to be text?


Thanks,
jtr@mitre.org
 

-----Original Message-----
From: Earnie Boyd [ mailto:earnie_boyd@yahoo.com ]
Sent: Thursday, July 22, 1999 8:31 PM
To: jtr@mitre.org; cygwin@sourceware.cygnus.com
Subject: Re: Cat and Head Problems with Binary Files


--- Jeffry T Ross <jtr@mitre.org> wrote:
> I've seen a lot of discussion about cat's handling of binary files.
> What I didn't see and would like to know is if the next version of
> cygwin will allow me to correctly pipe or redirect the output from
> cat and head if the source is a binary file?  Also, why were these
> programs implemented this way to begin with?
> 

They are textutils for text files.  If you want something different, that is
the reason it's OpenSource.


> 
> I'm trying to pipe a binary file to a program I wrote and either
> the pipe/redirect is breaking or cat/head is dying at arbitrary
> points in the binary files.  I suspect that my binary files contains
> what looks like an EOF.  This is similar to other threads I've seen
> on this list.
> 

Definitly.  In this case, eiter modify the sources and recompile or use binary
mounts which will default to binary file processing.  WARNING: You'll have to
change all \r\n line endings in scripts to \n.


===
Earnie Boyd < mailto:earnie_boyd@yahoo.com >

Newbies, please visit
< http://www.freeyellow.com/members5/gw32/index.html >

(If you respond to the list, then please don't cc me)
_________________________________________________________
Do You Yahoo!?
Get your free @yahoo.com address at http://mail.yahoo.com


--
Want to unsubscribe from this list?
Send a message to cygwin-unsubscribe@sourceware.cygnus.com

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: Cat and Head Problems with Binary Files
  1999-07-22 17:30 Earnie Boyd
  1999-07-22 17:49 ` Jeffry T Ross
@ 1999-07-31 18:34 ` Earnie Boyd
  1 sibling, 0 replies; 16+ messages in thread
From: Earnie Boyd @ 1999-07-31 18:34 UTC (permalink / raw)
  To: jtr, cygwin

--- Jeffry T Ross <jtr@mitre.org> wrote:
> I've seen a lot of discussion about cat's handling of binary files.
> What I didn't see and would like to know is if the next version of
> cygwin will allow me to correctly pipe or redirect the output from
> cat and head if the source is a binary file?  Also, why were these
> programs implemented this way to begin with?
> 

They are textutils for text files.  If you want something different, that is
the reason it's OpenSource.


> 
> I'm trying to pipe a binary file to a program I wrote and either
> the pipe/redirect is breaking or cat/head is dying at arbitrary
> points in the binary files.  I suspect that my binary files contains
> what looks like an EOF.  This is similar to other threads I've seen
> on this list.
> 

Definitly.  In this case, eiter modify the sources and recompile or use binary
mounts which will default to binary file processing.  WARNING: You'll have to
change all \r\n line endings in scripts to \n.


===
Earnie Boyd < mailto:earnie_boyd@yahoo.com >

Newbies, please visit
< http://www.freeyellow.com/members5/gw32/index.html >

(If you respond to the list, then please don't cc me)
_________________________________________________________
Do You Yahoo!?
Get your free @yahoo.com address at http://mail.yahoo.com


--
Want to unsubscribe from this list?
Send a message to cygwin-unsubscribe@sourceware.cygnus.com

^ permalink raw reply	[flat|nested] 16+ messages in thread

* RE: Cat and Head Problems with Binary Files
@ 1999-07-29  5:09 Earnie Boyd
  1999-07-31 18:34 ` Earnie Boyd
  0 siblings, 1 reply; 16+ messages in thread
From: Earnie Boyd @ 1999-07-29  5:09 UTC (permalink / raw)
  To: jtr, cygwin

--- Jeffry T Ross <jtr@mitre.org> wrote:
> Randy
> 
> I thought the suggestion to use dd was a good, although a bit excessive
> when one only wants to move bytes from a file to a pipe without any
> conversion. So I tried it:
> 
> dd if=test1 > test2
> test1 was 1296 bytes and test2 became 1301 bytes.
> 
> So then I tried dd directly to a file.
> dd if=test1 of=test2
> this worked fine.
> 
> So now I don't suspect cat but the piping and redirection.
> The question is how does cygwin implement it's piping and redirection?
> Is this problem imposed by the underlying WIN/DOS or is it a bug in BASH?
> 
> The pipe shouldn't be trying to interpret the data going through it.
> That's the job of the process at the end of the pipe, be it the terminal
> or another program.
> 
> 
> 
> 							jtr@mitre.org
>  
Hi Jeffry,

What version of cygwin1.dll are you using?  Supply the output of `uname -a' and
I'll know the answer to that.

The stock b20.1 version has a problem with the default behaviors of the 
[no]binary switch for the CYGWIN variable.  If you don't have at least the Jan
15th snapshot then go to the site or mirror and install that version.

The [no]binmode switch affect the processing mode (text vs binary) for pipes
and redirects.  The default is binmode which will pass the \r character through
the pipe and not add the \r character on the redirect.  However, the default
binmode and the specified binmode are different in the stock b20.1 version. 
This created some interesting results.

Good Luck,
===
Earnie Boyd < mailto:earnie_boyd@yahoo.com >

Newbies, please visit
< http://www.freeyellow.com/members5/gw32/index.html >

(If you respond to the list, then please don't cc me)
_____________________________________________________________
Do You Yahoo!?
Free instant messaging and more at http://messenger.yahoo.com


--
Want to unsubscribe from this list?
Send a message to cygwin-unsubscribe@sourceware.cygnus.com

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: Cat and Head Problems with Binary Files
  1999-07-22 17:49 ` Jeffry T Ross
  1999-07-22 18:05   ` $Bill Luebkert
@ 1999-07-23 18:19   ` Geoffrey Noer
  1999-07-31 18:34     ` Geoffrey Noer
  1999-07-31 18:34   ` Jeffry T Ross
  2 siblings, 1 reply; 16+ messages in thread
From: Geoffrey Noer @ 1999-07-23 18:19 UTC (permalink / raw)
  To: Jeffry T Ross; +Cc: cygwin

On Thu, Jul 22, 1999, Jeffry T Ross wrote:
>
> Who ever said that cat and head are textutils?
> On the Unix side of the world they're commonly used
> on binary files.

Yes, I consider the current state to be broken for this
reason but nobody has volunteered to fix it so...

In DJGPP I believe there is a set of patches that addresses this
problem by adding a flag to all of these programs so that
people can specify whether they are (for example) catting a
binary or text file.  These patches have not yet been applied/enabled
for the Cygwin ports.

If someone would volunteer to make them work for Cygwin and submit the
changes to the FSF maintainers (and perhaps to the cygwin-developers
list), I'm sure many people would be appreciative.

Best regards,

-- 
Geoffrey Noer		Email: noer@cygnus.com
Cygnus Solutions

--
Want to unsubscribe from this list?
Send a message to cygwin-unsubscribe@sourceware.cygnus.com

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: Cat and Head Problems with Binary Files
  1999-07-22 17:49 ` Jeffry T Ross
@ 1999-07-22 18:05   ` $Bill Luebkert
  1999-07-31 18:34     ` $Bill Luebkert
  1999-07-23 18:19   ` Geoffrey Noer
  1999-07-31 18:34   ` Jeffry T Ross
  2 siblings, 1 reply; 16+ messages in thread
From: $Bill Luebkert @ 1999-07-22 18:05 UTC (permalink / raw)
  To: jtr; +Cc: cygwin

Jeffry T Ross wrote:
> 
> Who ever said that cat and head are textutils?
> On the Unix side of the world they're commonly used
> on binary files.  If you have a 10gig file of binary
> data, what the easiest way to get a 10k chunk?
> 
> How about: head -c 10000 bigfile > littlefile
> 
> This works in Unix because Unix thinks all files are
> binary, and that's because all files are binary.  The
> notion of text files is a bogus limitation imposed by
> Microsoft.
> 
> Is there a reason why having cat treat all files as binary
> would cause erroneous performance when cat was used on a
> file you'd consider to be text?

I have a slightly different opinion:

I could just as easily say that cat and text are text utils since 
they operate on text files (even extended 8-bit character text files
which one could call binary).  There is such a thing as 8-bit text.

The point is, on UNIX files are strings of bytes, not binary and not text, 
just a bunch of 8-bit characters strung together in a file.  UNIX doesn't 
care if those bytes are 'Text" or 'Binary' after all, they are/could be both.

A byte is a byte, of course of course, unless of course the byte is 
really two little-endian nybbles. :)

-- 
  ,-/-  __      _  _         $Bill Luebkert   ICQ=14439852
 (_/   /  )    // //       DBE Collectibles   http://www.wgn.net/~dbe/
  / ) /--<  o // //      Mailto:dbe@wgn.net   http://dbecoll.webjump.com/
-/-' /___/_<_</_</_    http://www.freeyellow.com/members/dbecoll/

--
Want to unsubscribe from this list?
Send a message to cygwin-unsubscribe@sourceware.cygnus.com

^ permalink raw reply	[flat|nested] 16+ messages in thread

* RE: Cat and Head Problems with Binary Files
  1999-07-22 17:30 Earnie Boyd
@ 1999-07-22 17:49 ` Jeffry T Ross
  1999-07-22 18:05   ` $Bill Luebkert
                     ` (2 more replies)
  1999-07-31 18:34 ` Earnie Boyd
  1 sibling, 3 replies; 16+ messages in thread
From: Jeffry T Ross @ 1999-07-22 17:49 UTC (permalink / raw)
  To: cygwin

Who ever said that cat and head are textutils?
On the Unix side of the world they're commonly used
on binary files.  If you have a 10gig file of binary
data, what the easiest way to get a 10k chunk?

How about: head -c 10000 bigfile > littlefile

This works in Unix because Unix thinks all files are
binary, and that's because all files are binary.  The
notion of text files is a bogus limitation imposed by
Microsoft.

Is there a reason why having cat treat all files as binary
would cause erroneous performance when cat was used on a
file you'd consider to be text?


Thanks,
jtr@mitre.org
 

-----Original Message-----
From: Earnie Boyd [ mailto:earnie_boyd@yahoo.com ]
Sent: Thursday, July 22, 1999 8:31 PM
To: jtr@mitre.org; cygwin@sourceware.cygnus.com
Subject: Re: Cat and Head Problems with Binary Files


--- Jeffry T Ross <jtr@mitre.org> wrote:
> I've seen a lot of discussion about cat's handling of binary files.
> What I didn't see and would like to know is if the next version of
> cygwin will allow me to correctly pipe or redirect the output from
> cat and head if the source is a binary file?  Also, why were these
> programs implemented this way to begin with?
> 

They are textutils for text files.  If you want something different, that is
the reason it's OpenSource.


> 
> I'm trying to pipe a binary file to a program I wrote and either
> the pipe/redirect is breaking or cat/head is dying at arbitrary
> points in the binary files.  I suspect that my binary files contains
> what looks like an EOF.  This is similar to other threads I've seen
> on this list.
> 

Definitly.  In this case, eiter modify the sources and recompile or use binary
mounts which will default to binary file processing.  WARNING: You'll have to
change all \r\n line endings in scripts to \n.


===
Earnie Boyd < mailto:earnie_boyd@yahoo.com >

Newbies, please visit
< http://www.freeyellow.com/members5/gw32/index.html >

(If you respond to the list, then please don't cc me)
_________________________________________________________
Do You Yahoo!?
Get your free @yahoo.com address at http://mail.yahoo.com


--
Want to unsubscribe from this list?
Send a message to cygwin-unsubscribe@sourceware.cygnus.com

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: Cat and Head Problems with Binary Files
@ 1999-07-22 17:30 Earnie Boyd
  1999-07-22 17:49 ` Jeffry T Ross
  1999-07-31 18:34 ` Earnie Boyd
  0 siblings, 2 replies; 16+ messages in thread
From: Earnie Boyd @ 1999-07-22 17:30 UTC (permalink / raw)
  To: jtr, cygwin

--- Jeffry T Ross <jtr@mitre.org> wrote:
> I've seen a lot of discussion about cat's handling of binary files.
> What I didn't see and would like to know is if the next version of
> cygwin will allow me to correctly pipe or redirect the output from
> cat and head if the source is a binary file?  Also, why were these
> programs implemented this way to begin with?
> 

They are textutils for text files.  If you want something different, that is
the reason it's OpenSource.


> 
> I'm trying to pipe a binary file to a program I wrote and either
> the pipe/redirect is breaking or cat/head is dying at arbitrary
> points in the binary files.  I suspect that my binary files contains
> what looks like an EOF.  This is similar to other threads I've seen
> on this list.
> 

Definitly.  In this case, eiter modify the sources and recompile or use binary
mounts which will default to binary file processing.  WARNING: You'll have to
change all \r\n line endings in scripts to \n.


===
Earnie Boyd < mailto:earnie_boyd@yahoo.com >

Newbies, please visit
< http://www.freeyellow.com/members5/gw32/index.html >

(If you respond to the list, then please don't cc me)
_________________________________________________________
Do You Yahoo!?
Get your free @yahoo.com address at http://mail.yahoo.com


--
Want to unsubscribe from this list?
Send a message to cygwin-unsubscribe@sourceware.cygnus.com

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Cat and Head Problems with Binary Files
@ 1999-07-22 17:10 Jeffry T Ross
  1999-07-31 18:34 ` Jeffry T Ross
  0 siblings, 1 reply; 16+ messages in thread
From: Jeffry T Ross @ 1999-07-22 17:10 UTC (permalink / raw)
  To: cygwin

I've seen a lot of discussion about cat's handling of binary files.
What I didn't see and would like to know is if the next version of
cygwin will allow me to correctly pipe or redirect the output from
cat and head if the source is a binary file?  Also, why were these
programs implemented this way to begin with?


I'm trying to pipe a binary file to a program I wrote and either
the pipe/redirect is breaking or cat/head is dying at arbitrary
points in the binary files.  I suspect that my binary files contains
what looks like an EOF.  This is similar to other threads I've seen
on this list.


Thanks,
jtr@mitre.org
 

--
Want to unsubscribe from this list?
Send a message to cygwin-unsubscribe@sourceware.cygnus.com

^ permalink raw reply	[flat|nested] 16+ messages in thread

end of thread, other threads:[~1999-07-31 18:34 UTC | newest]

Thread overview: 16+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
     [not found] <4.1.19990722175718.00de9820@mailhost.teknowledge.com>
1999-07-23  9:35 ` Cat and Head Problems with Binary Files Jeffry T Ross
1999-07-24 22:14   ` Fergus Henderson
1999-07-31 18:34     ` Fergus Henderson
1999-07-31 18:34   ` Jeffry T Ross
1999-07-29  5:09 Earnie Boyd
1999-07-31 18:34 ` Earnie Boyd
  -- strict thread matches above, loose matches on Subject: below --
1999-07-22 17:30 Earnie Boyd
1999-07-22 17:49 ` Jeffry T Ross
1999-07-22 18:05   ` $Bill Luebkert
1999-07-31 18:34     ` $Bill Luebkert
1999-07-23 18:19   ` Geoffrey Noer
1999-07-31 18:34     ` Geoffrey Noer
1999-07-31 18:34   ` Jeffry T Ross
1999-07-31 18:34 ` Earnie Boyd
1999-07-22 17:10 Jeffry T Ross
1999-07-31 18:34 ` Jeffry T Ross

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).