RE: mount type, compiler barf, and sym links

RE: mount type, compiler barf, and sym links

[Earnie Boyd]

---=?iso-8859-1?Q?Bj=F8rn_Hell_Larsen?= <blarsen@intervett.no> wrote:
>
> 
> Earnie writes:
> > It is rather easy for the porter to fix with supplying the
appropriate
> > file processing mode switches to the open/fopen functions.  Treat
all
> > text files as text and treat all binary files as binary.  
> 
> The problem with this, again, it that creating strict divisions
> between "text" and "binary" files, and "text" and "binary" tools
> at times doesn't make much sense.
> 
> For instance, in the GNU-Win32 distribution, the tools "cat" and
"grep"
> are classified as "text tools", allowing them to handle MS files
> with CRLF line separation and CTRL-Z end-of-file markers correctly.
> 
> Unfortunately, this also means that I won't be able to use the
> tools the way I'm used to from my Unix days, i.e.
> 
> 	cat part* | tar tzvf -
> 
> fails miserably, and 
> 
> 	MATCHES=`grep -c pattern binaryfile`
> 
> won't do what I expected it to do.
> 
> I would much prefer a single global setting that I could toggle,
> that would make all tools work in complete binary mode, under
> the assumption that I *know* about the MS/Unix file format
> differences, and would prefer to be left to deal with them
> myself.
> 
> 
> 
> Bjorn
> 
> 

Bjorn, you've just given proof that some of the utilities that come
with the CDK aren't completely compliant/ported yet either.  I just
added at line 646 of textutils-1.22/src/cat.c:

#ifdef __CYGWIN32__
    setmode( input_desc, O_BINARY );
    setmode( output_desc, O_BINARY );
#endif


and the result is exactly as _I_ would expect it to be.  The results
were also the same regardless of the cygwin mounted filesystem binary
mode switch.

If CYGWIN32=tty and the file contains \n line endings then a \r is
added before being displayed.  I.E.:

// This line ends here ->
// This is the next line.

If CYGWIN32=notty or is not set and the file contains \n line endings
then the file is displayed with no \r ever happening.  I.E.:

// This line ends here ->
                         // This is the next line.

If the file contains \r\n line endings then the file is displayed
exactly the same way regardless of the tty setting. I.E.:

// This line ends here ->
// This is the next line.


The UNIX filesystem treats all files as binary files and the /dev/tty
adds the \r to the output display for the \n it receives or has it as
an option.  The MSDOS/WIN32 filesystem has two methods/modes of file
processing known as text or binary and the output display doesn't add
a \r for the \n.  

Now we have to give thought to how to handle the differences.  In my
modification to cat.c I chose to treat everything as binary since that
is the way it would have been working in UNIX.  The result is correct
for text!=binary as well as text=binary eliminating the dependency on
this switch/parameter.

Let me apologize to you and the list for fanning flames.  But, I
intend to sing this song until there is no need to sing it again.  As
can be seen from the support for this position this is a much needed
concept.  The only problem with it is understanding when it should be
text and when it should be binary.  I tend to set file descriptors to
binary processing mode unless I'm absolutely positive it is supposed
to be treated as text (e.g.: a shell script file).


==
-                  \\||//
-------------o0O0--Earnie--0O0o--------------
--          earnie_boyd@yahoo.com          --
-- http://www.freeyellow.com/members5/gw32 --
----------------ooo0O--O0ooo-----------------

PS: Newbie's, you should visit my page.
_________________________________________________________
DO YOU YAHOO!?
Get your free @yahoo.com address at http://mail.yahoo.com

-
For help on using this list (especially unsubscribing), send a message to
"gnu-win32-request@cygnus.com" with one line of text: "help".

Re: mount type, compiler barf, and sym links

[Benjamin Riefenstahl]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 1686 bytes --]

Hi Bjørn,


Bjørn Hell Larsen wrote:
> For instance, in the GNU-Win32 distribution, the tools "cat" and "grep"
> are classified as "text tools",

In practice cat is commonly used to handle binary files. Also cat does
not process files in any way. It's pointless to convert input from
external to internal representation just to output the very same data
with the backwards conversion. Cat should use binary mode.

[grep using text mode]
>         MATCHES=`grep -c pattern binaryfile`
> 
> won't do what I expected it to do.

In which way? As far as I see this example should work the same as under
Unix. Other uses of grep might have different behaviour than on Unix.
But in most cases that I can imagine off the top of my head, I don't
have any trouble arguing that grep should not really be used in those
cases in the first place.

> I would much prefer a single global setting that I could toggle,
> that would make all tools work in complete binary mode, under
> the assumption that I *know* about the MS/Unix file format
> differences, and would prefer to be left to deal with them
> myself.

That is what binary mounts do. At the expense that tools that deal with
and process text files (like bash) suddenly break, when they get fed
native text, even though the tool is written correctly. IOW binary
mounts favour broken tools instead of correct tools.


so long, benny
======================================
Benjamin Riefenstahl (benny@crocodial.de)
Crocodial Communications EntwicklungsGmbH
Ruhrstraße 61, D-22761 Hamburg, Germany
-
For help on using this list (especially unsubscribing), send a message to
"gnu-win32-request@cygnus.com" with one line of text: "help".

Re: mount type, compiler barf, and sym links

[Geoffrey Noer]

I included a lot of your explanation in the FAQ.  Hopefully it will
help answer this more thoroughly in the future...
-gjn

On Mon, Oct 19, 1998 at 05:10:04AM -0700, Earnie Boyd wrote:
>
> The real problem lies in the way that UNIX vs WIN32 create files.  To
> UNIX a file is a file and what the file contains is what ever the
> program/programmer/user told it to put into it.  To WIN32 a file is
> also a file and what the file contains depends not only on the
> program/programmer/user but also the file processing mode.  
[...]

-- 
Geoffrey Noer
noer@cygnus.com
-
For help on using this list (especially unsubscribing), send a message to
"gnu-win32-request@cygnus.com" with one line of text: "help".

RE: mount type, compiler barf, and sym links

[Bjørn Hell Larsen]

Earnie writes:
> It is rather easy for the porter to fix with supplying the appropriate
> file processing mode switches to the open/fopen functions.  Treat all
> text files as text and treat all binary files as binary.  

The problem with this, again, it that creating strict divisions
between "text" and "binary" files, and "text" and "binary" tools
at times doesn't make much sense.

For instance, in the GNU-Win32 distribution, the tools "cat" and "grep"
are classified as "text tools", allowing them to handle MS files
with CRLF line separation and CTRL-Z end-of-file markers correctly.

Unfortunately, this also means that I won't be able to use the
tools the way I'm used to from my Unix days, i.e.

	cat part* | tar tzvf -

fails miserably, and 

	MATCHES=`grep -c pattern binaryfile`

won't do what I expected it to do.

I would much prefer a single global setting that I could toggle,
that would make all tools work in complete binary mode, under
the assumption that I *know* about the MS/Unix file format
differences, and would prefer to be left to deal with them
myself.



Bjorn

-
For help on using this list (especially unsubscribing), send a message to
"gnu-win32-request@cygnus.com" with one line of text: "help".

Re: mount type, compiler barf, and sym links

[Charles Wilson]

Stephen L Arnold wrote:

> Howdy:
>
> I can't keep up with the volume (and much of what I do read is
> pretty obtuse to me anyway); sorry if this seems like a stupid
> question.
>
> I've had the B19/coolview/egcs setup working for a few weeks (at
> least for compiling my stupid homework problems - all console mode
> stuff).  I guess I always had my root partition mounted text mode.
> After reading some of the posts lately on the subject, I got the
> impression I should use binary mounts, so when I created some new
> mount points for /tmp, /var, my ntemacs directory (for symbolic
> links), etc, I used the -b switch.  I also changed my root mount
> point in the registry to the binary type.  Suddenly my homework
> wouldn't compile; I got some weird error message about undefined
> symbols in stdio.h & stddef.h (all my source files are LF only).
>
> I changed the mount type back and everything compiles fine again.
> What was the problem?  Something to do with CR/LF translation?
> Would somebody care to explain it in terms a self-taught guy (with
> a geophysics background) can understand?

Sounds to me like all of your text files (.h's, &tc.) were CR/LF translated
when you originally untarred them. This can happen if you (1) use WinZip or
some other windows-centric program to untar.gz the archives, such as the
egcs tar.gz file (2) you untar.gz'ed using cygwin-based tar, but the
destination location was a text mount, so the low-level file routines did a
CR/LF conversion for you. I think the easiest way to fix this, without
disturbing a setup that already is properly configured, is to get rmhatm
from ftp://insomnia.be/johanvg/rmhatm-0.2.tar.gz

Be warned: I haven't tried this program myself, but it is a Perl program
that will
_recursively_ (!! careful !!) descend into your directory structure, and
convert all text files from DOS/MAC to Unix format. (Of course, you'll have
to mount everything binary _before_ running the conversion.

The other way to fix it is to mount everything binary, and then reinstall
egcs from the original tar.gz file.

> Which types of mounts do
> I want for what types of scenarios and why?
>
> I also created some sym links in /bin that point to win32 versions
> of vim and emacs (as well as linking /bin/sh to bash).  Do I need
> to explicitly link to prog1.exe or just prog1 (the links in /bin
> seem to work fine as sh, gvim, and emacs)?  Are there any hidden
> implications to using/not using the DOS .exe file extension under
> bash?
>
> Thanks in advance for any answers, Steve

I dunno about all this stuff.

-Chuck


-
For help on using this list (especially unsubscribing), send a message to
"gnu-win32-request@cygnus.com" with one line of text: "help".

Re: mount type, compiler barf, and sym links

[Earnie Boyd]

Hi Stephen,

You haven't read my "Personal Reflections" on this yet, have you?  The
binary mount switch has been over emphasised as _THE_ way to do
things.  It makes the porter lazy in dealing with the issue.

The real problem lies in the way that UNIX vs WIN32 create files.  To
UNIX a file is a file and what the file contains is what ever the
program/programmer/user told it to put into it.  To WIN32 a file is
also a file and what the file contains depends not only on the
program/programmer/user but also the file processing mode.  

If processing in text mode then certain values of data are treated
specially.  A \n (new line) written to the file will prepend a \r
(carriage return) so that if you `printf("Hello\n") you infact get
"Hello\r\n".  Upon reading this combination, the \r is removed and the
number of bytes returned by the read is 1 less than was actually read
which confuses programs dependant on ftell() and fseek().  A Ctrl-Z
encountered while reading a file sets the End Of File flags even
though it truly isn't the end of file.

These problems have been avoided globally by setting the default
processing mode to binary.  But that creates a different problem.  In
binary mode the program will receive all of the data in the file
including a \r.  The programs will then not deal with these properly
so you have to remove the \r from the text files, especially scripts
and startup resource files.  This is a porter "cop out" forcing the
user to deal with the \r for the porter.

It is rather easy for the porter to fix with supplying the appropriate
file processing mode switches to the open/fopen functions.  Treat all
text files as text and treat all binary files as binary.  The switches
are existant on UNIX open/fopen functions and will just ignore the
switch as it has no meaning to UNIX but has been defined by ANSI.

I hope this answers that question.  In a previous post I pleaded with
the porters not to depend on the binary mode mount setting in hopes of
getting the ported programs usable with out futzing with the registry.

Now for the symbollic links:  IMHO, it is always best to be specific. 
The files vim.exe and vim can coexist in the directory.  So if vim
without the extension was ever created your link would then point to
that file.  Since your using the Win32 version of vim then I find it
easiest to just edit the link file and add the .exe to the reference. 
However, it is not necessary, as you already know to do this.  Just be
aware that if it stops working look at the directory containing the
physical file.

---Stephen L Arnold <sarnold@coyote.rain.org> wrote:
>
> Howdy:
> 
> I can't keep up with the volume (and much of what I do read is 
> pretty obtuse to me anyway); sorry if this seems like a stupid 
> question.
> 
> I've had the B19/coolview/egcs setup working for a few weeks (at 
> least for compiling my stupid homework problems - all console mode 
> stuff).  I guess I always had my root partition mounted text mode.  
> After reading some of the posts lately on the subject, I got the 
> impression I should use binary mounts, so when I created some new 
> mount points for /tmp, /var, my ntemacs directory (for symbolic 
> links), etc, I used the -b switch.  I also changed my root mount 
> point in the registry to the binary type.  Suddenly my homework 
> wouldn't compile; I got some weird error message about undefined 
> symbols in stdio.h & stddef.h (all my source files are LF only).
> 
> I changed the mount type back and everything compiles fine again.  
> What was the problem?  Something to do with CR/LF translation?  
> Would somebody care to explain it in terms a self-taught guy (with 
> a geophysics background) can understand?  Which types of mounts do 
> I want for what types of scenarios and why?
> 
> I also created some sym links in /bin that point to win32 versions 
> of vim and emacs (as well as linking /bin/sh to bash).  Do I need 
> to explicitly link to prog1.exe or just prog1 (the links in /bin 
> seem to work fine as sh, gvim, and emacs)?  Are there any hidden 
> implications to using/not using the DOS .exe file extension under 
> bash?
> 
> Thanks in advance for any answers, Steve
> 
> 
> *************************************************************
> Steve Arnold                            sarnold@earthling.net
> http://www.rain.org/~sarnold
> 
> Fatal error:  MS Windows detected, deleting DOS partition...
> -
> For help on using this list (especially unsubscribing), send a
message to
> "gnu-win32-request@cygnus.com" with one line of text: "help".
> 

==
-                \\||//
-----------o0O0--Earnie--0O0o------------
--        earnie_boyd@yahoo.com        --
-- http://freeyellow.com/members5/gw32 --
--------------ooo0O--O0ooo---------------

PS: Newbie's, you should visit my page.
_________________________________________________________
DO YOU YAHOO!?
Get your free @yahoo.com address at http://mail.yahoo.com

-
For help on using this list (especially unsubscribing), send a message to
"gnu-win32-request@cygnus.com" with one line of text: "help".

mount type, compiler barf, and sym links

[Stephen L Arnold]

Howdy:

I can't keep up with the volume (and much of what I do read is 
pretty obtuse to me anyway); sorry if this seems like a stupid 
question.

I've had the B19/coolview/egcs setup working for a few weeks (at 
least for compiling my stupid homework problems - all console mode 
stuff).  I guess I always had my root partition mounted text mode.  
After reading some of the posts lately on the subject, I got the 
impression I should use binary mounts, so when I created some new 
mount points for /tmp, /var, my ntemacs directory (for symbolic 
links), etc, I used the -b switch.  I also changed my root mount 
point in the registry to the binary type.  Suddenly my homework 
wouldn't compile; I got some weird error message about undefined 
symbols in stdio.h & stddef.h (all my source files are LF only).

I changed the mount type back and everything compiles fine again.  
What was the problem?  Something to do with CR/LF translation?  
Would somebody care to explain it in terms a self-taught guy (with 
a geophysics background) can understand?  Which types of mounts do 
I want for what types of scenarios and why?

I also created some sym links in /bin that point to win32 versions 
of vim and emacs (as well as linking /bin/sh to bash).  Do I need 
to explicitly link to prog1.exe or just prog1 (the links in /bin 
seem to work fine as sh, gvim, and emacs)?  Are there any hidden 
implications to using/not using the DOS .exe file extension under 
bash?

Thanks in advance for any answers, Steve


*************************************************************
Steve Arnold                            sarnold@earthling.net
http://www.rain.org/~sarnold

Fatal error:  MS Windows detected, deleting DOS partition...
-
For help on using this list (especially unsubscribing), send a message to
"gnu-win32-request@cygnus.com" with one line of text: "help".