File reading problem with -mno-cygwin

File reading problem with -mno-cygwin

[Andrew DeFaria]

[-- Attachment #1: Type: text/plain, Size: 3276 bytes --]

OK, this one's really got me stumped and I need to resolve this quickly. 
I'd really appreciate any help that can be offered. I'm getting 
difference in the behavior of fread (I believe) depending on whether 
I've compiled with -mno-cygwin or not. I think this example is simple 
enough. Here's the code:

#include <stdlib.h>
#include <stdio.h>

#define BUFLEN (1 << 8)

int main (void) {
  FILE *fp;
  unsigned char buf [BUFLEN];
  long bytes_processed = 0;
  int bytes_read;
  int i = 0;
  int j = 0;

  if ((fp = fopen ("sf", "r")) == NULL) {
    printf ("Unable to open sf file\n");
    exit (1);
  } /* if */

  while ((bytes_read = fread (buf, 1, BUFLEN, fp)) > 0) {
    j = j + bytes_read;
    printf ("Pass #%d; %d bytes read this pass; %d total bytes processed 
so far\n", ++i, bytes_read, j);
  } /* while */
}

Pretty simple eh? Here's how I compile it:

$ gcc foo.c -o foo
$ gcc -mno-cygwin foo.c -o foo.no_cygwin

This little example requires the attached file, named sf. It's merely 
/etc/group packed into a file format with some headers and a checksum. 
Those additions add binary data to the file which I think is causing the 
problems in some strange way because if I use just a plain ASCII copy of 
/etc/group as the "sf" file it works OK. However with the attached "sf" 
file (with binary data in it) I get the following differing results:

$ foo
Pass #1; 256 bytes read this pass; 256 total bytes processed so far
Pass #2; 256 bytes read this pass; 512 total bytes processed so far
Pass #3; 256 bytes read this pass; 768 total bytes processed so far
Pass #4; 256 bytes read this pass; 1024 total bytes processed so far
Pass #5; 256 bytes read this pass; 1280 total bytes processed so far
Pass #6; 256 bytes read this pass; 1536 total bytes processed so far
Pass #7; 256 bytes read this pass; 1792 total bytes processed so far
Pass #8; 256 bytes read this pass; 2048 total bytes processed so far
Pass #9; 256 bytes read this pass; 2304 total bytes processed so far
Pass #10; 256 bytes read this pass; 2560 total bytes processed so far
Pass #11; 256 bytes read this pass; 2816 total bytes processed so far
Pass #12; 4 bytes read this pass; 2820 total bytes processed so far
$ foo.no_cygwin
Pass #1; 256 bytes read this pass; 256 total bytes processed so far
Pass #2; 256 bytes read this pass; 512 total bytes processed so far
Pass #3; 256 bytes read this pass; 768 total bytes processed so far
Pass #4; 256 bytes read this pass; 1024 total bytes processed so far
Pass #5; 256 bytes read this pass; 1280 total bytes processed so far
Pass #6; 256 bytes read this pass; 1536 total bytes processed so far
Pass #7; 256 bytes read this pass; 1792 total bytes processed so far
Pass #8; 256 bytes read this pass; 2048 total bytes processed so far
Pass #9; 256 bytes read this pass; 2304 total bytes processed so far
Pass #10; 256 bytes read this pass; 2560 total bytes processed so far
Pass #11; 252 bytes read this pass; 2812 total bytes processed so far

As you can see, with -mno-cygwin I only make 11 passes of the loop and 
only read a total of 2812 bytes. However without -mno-cygwin I make 12 
passes and read a total of 2820 bytes. The question is why?!? The other 
question is: Is there a way to get the -mno-cygwin to behave properly.

Thanks in advance.

[-- Attachment #2: sf --]
[-- Type: application/octet-stream, Size: 2820 bytes --]

[-- Attachment #3: Type: text/plain, Size: 214 bytes --]

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: File reading problem with -mno-cygwin

[Corinna Vinschen]

On Tue, Sep 03, 2002 at 07:33:20PM -0700, Andrew DeFaria wrote:
>  if ((fp = fopen ("sf", "r")) == NULL) {

Try          fopen ("sf", "rb")

Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Developer                                mailto:cygwin@cygwin.com
Red Hat, Inc.

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Proposal for new CYGWIN flag: notextmode masks away O_TEXT

[Dan Vasaru]

My apologies if this has already been discussed:

I would just love a CYGWIN variable setting that would make cygwin ignore
the O_TEXT flag from all programs, thus bypassing any translation code in
cygwin1.dll.
May I suggest "notextmode", if not taken already :)

At work, we're aiming for maximum UNIX compatibility between all files
across win32/linux/solaris. All this O_TEXT stuff really, really gets in the
way. All our mounts are binmode, yet the fact that some programs set the
O_TEXT flag, and cygwin doesn't ignore it, will at some point add a stray
carriage return (see various threads on CRLF) in various pipes and
redirects.


We'd rather patch/wrap all non-cygwin utilities in our tool chain (i.e.
cl.exe) to output binmode only. Thinking of it, one could add an lf-ize
utility that invokes a program and adds CR to input while removing them from
output. This would keep the pipes clear of carriage returns.

lfize cl.exe | grep -v "^$"
lfize cat textfile | sed | awk

I freely admit not offering much thought to CTRL+Z.

Dan


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: Proposal for new cygwin flag: notextmode masks away O_TEXT

[Christopher Faylor]

On Wed, Sep 04, 2002 at 11:57:15PM +0200, Dan Vasaru wrote:
>My apologies if this has already been discussed:
>
>I would just love a CYGWIN variable setting that would make cygwin ignore
>the O_TEXT flag from all programs, thus bypassing any translation code in
>cygwin1.dll.
>May I suggest "notextmode", if not taken already :)
>
>At work, we're aiming for maximum UNIX compatibility between all files
>across win32/linux/solaris. All this O_TEXT stuff really, really gets in the
>way. All our mounts are binmode, yet the fact that some programs set the
>O_TEXT flag, and cygwin doesn't ignore it, will at some point add a stray
>carriage return (see various threads on CRLF) in various pipes and
>redirects.

No, it won't add a stray CR unless the file is opened as O_TEXT for
output, which no programs in the cygwin release should be doing.  O_TEXT
for input specifically means that CRLFs are translated to LFs
internally.

It will *interpret* a CRLF as a LF, however, which has its own
ramifcations.

>We'd rather patch/wrap all non-cygwin utilities in our tool chain (i.e.
>cl.exe) to output binmode only. Thinking of it, one could add an lf-ize
>utility that invokes a program and adds CR to input while removing them from
>output. This would keep the pipes clear of carriage returns.

Why would you want to *add* a CR to input?  A program which uses O_TEXT
for input will still consider lines ending only with LF to be properly
terminated.

>lfize cl.exe | grep -v "^$"
>lfize cat textfile | sed | awk
>
>I freely admit not offering much thought to CTRL+Z.

I have a feeling that one of us still isn't getting something here.

If you are willing to write a program which wraps every DOS program
which outputs CRLF, then do that, mount everything with -b and you
should be all set.

cgf

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: Proposal for new cygwin flag: notextmode masks away O_TEXT

[dvasaru]

Chris,

You're right of course, and I will not push for this very hard, but such a flag would ensure compatible results across platforms:

file.txt:
i have crlf\r\n
and another one\r\n

solaris> sed 's/crlf$/nocr/' file.txt
i have crlf\r\n
and another one\r\n

cygwin> sed 's/crlf$/nocr/' file.txt
i have nocrlf\n
and another one\n

proposed:
cygwin-notextmode> sed 's/crlf$/nocrlf/' file.txt
i have crlf\r\n
and another one\r\n

Read also my comments below.

Quoting Christopher Faylor <cgf@redhat.com>:

> On Wed, Sep 04, 2002 at 11:57:15PM +0200, Dan Vasaru wrote:

> >way. All our mounts are binmode, yet the fact that some programs set the
> >O_TEXT flag, and cygwin doesn't ignore it, will at some point add a stray
> >carriage return (see various threads on CRLF) in various pipes and
> >redirects.
> 
> No, it won't add a stray CR unless the file is opened as O_TEXT for
> output, which no programs in the cygwin release should be doing.  O_TEXT
> for input specifically means that CRLFs are translated to LFs
> internally.

Mea culpa, it turns out the stray CR did happen on a text mount. 

> It will *interpret* a CRLF as a LF, however, which has its own
> ramifcations.

notextmode comes in handy here :).

[...]
> Why would you want to *add* a CR to input?  A program which uses O_TEXT
> for input will still consider lines ending only with LF to be properly
> terminated.

That would be for input to non-cygwin software that demands CR (notably notepad).
 
[...]
> I have a feeling that one of us still isn't getting something here.
> If you are willing to write a program which wraps every DOS program
> which outputs CRLF, then do that, mount everything with -b and you
> should be all set.
 
Point taken, and we shall do this. 

Thanks,
    dan.



--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: Proposal for new cygwin flag: notextmode masks away O_TEXT

[Christopher Faylor]

On Thu, Sep 05, 2002 at 12:39:51PM +0200, dvasaru@broadpark.no wrote:
>You're right of course, and I will not push for this very hard, but
>such a flag would ensure compatible results across platforms:
>
>file.txt:
>i have crlf\r\n
>and another one\r\n
>
>solaris> sed 's/crlf$/nocr/' file.txt
>i have crlf\r\n
>and another one\r\n

Works fine for me on a binmode or textmode mount, which is by
design.  sed should open its input in textmode and it's output
in binary mode.

>cygwin> sed 's/crlf$/nocr/' file.txt
>i have nocrlf\n
>and another one\n
>
>proposed:
>cygwin-notextmode> sed 's/crlf$/nocrlf/' file.txt
>i have crlf\r\n
>and another one\r\n
>
>Read also my comments below.

I think you have the tools you need, so I do not want to complicate
the binmode/textmode logic in cygwin with YA option.

That's it for me.

cgf

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

RE: Proposal for new cygwin flag: notextmode masks away O_TEXT

[Dan Vasaru]

Quoting Christopher Faylor <cgf@redhat.com>:

>Works fine for me on a binmode or textmode mount, which is by
>design.  sed should open its input in textmode and it's output
>in binary mode.

I understand that. Is there any flavour of unix that does provide CRLF
translation ?
I think not, and that's what I'm hinting at when talking about (bug-to-bug)
compatibility.

>I think you have the tools you need, so I do not want to complicate
>the binmode/textmode logic in cygwin with YA option.

I'll buy that. Just wish the option was there.

Dan



--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/