Sorry "people" (NOT MY taxonomy!!), but igncr IS flawed

Sorry "people" (NOT MY taxonomy!!), but igncr IS flawed

[manu0507]

Hi all,

Notwithstanding the completely preposterous "reply" by Eric Blake (more of
an idiotic acrimony, actually) to my previous post (see
http://old.nabble.com/Igncr-ineffective--tt32983438.html ), there does seem
to be a problem in dealing with Win's CR/LF line endings in "unusual" lines,
at least on Win7-64 (or, to be really precise, on my Win7-64).
The lines where CR/LFs appear not to be properly converted to LFs seem to be
empty lines (except for the CR/LF, of course), as well as some other
"unusual" constructs (lines ending with ";;CR/LF" in particular).

To work around the problem, I'm writing an application that would convert
all CR/LF-ending text files into LF-ending ones... but it's not really
trivial, because telling binary files that should be left untouched from
text files that should be converted is difficult: even the very first file
in GDB's sources ("configure") contains a '\a', i.e. a "not-text" byte.

I don't think that believing that anybody who knows anything beyond
Unix/Linux must surely be an idiot (as Blake seems to suggest) is
constructive - after all, we all strive to produce applications that would
make a difference, whatever the platform.
Cygwin is a beautiful piece of work, no doubt about it... but believing it's
bug-free just because it's inspired by Unix/Linux is counter-productive at
best, and preposterous at worst.

So, to get back to the real subject: is there a correction to be had to the
CR/LF problem?

Many thanks,

Emanuel

-- 
View this message in context: http://old.nabble.com/Sorry-%22people%22-%28NOT-MY-taxonomy%21%21%29%2C-but-igncr-IS-flawed-tp32989786p32989786.html
Sent from the Cygwin list mailing list archive at Nabble.com.


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Sorry "people" (NOT MY taxonomy!!), but igncr IS flawed

[Jeremy Bopp]

On 12/16/2011 11:13 PM, manu0507 wrote:
> 
> Hi all,
> 
> Notwithstanding the completely preposterous "reply" by Eric Blake (more of
> an idiotic acrimony, actually) to my previous post (see
> http://old.nabble.com/Igncr-ineffective--tt32983438.html ), there does seem
> to be a problem in dealing with Win's CR/LF line endings in "unusual" lines,
> at least on Win7-64 (or, to be really precise, on my Win7-64).
> The lines where CR/LFs appear not to be properly converted to LFs seem to be
> empty lines (except for the CR/LF, of course), as well as some other
> "unusual" constructs (lines ending with ";;CR/LF" in particular).

I don't have Cygwin available at the moment, so I can't try running
scripts as you describe right now.  However, the claim that a line
consisting of only a CR/LF causing problems with the igncr option makes
me pretty suspicious.  "Empty" lines are pretty darn common in bash
scripts, and I would expect to have seen many reports of problems with
igncr reported here by now if that option didn't correctly handle those
lines.

Can you send a representative example script that elicits this problem
for you?  A simple test case would go a long way to addressing the
issue.  From the sound of things, no one else has reproduced your issue yet.

Perhaps the lines that are giving you trouble are actually ended with
CR/CR/LF.  Have you examined the problematic scripts with a hex editor
or simply "od -c" to verify the line endings?

> To work around the problem, I'm writing an application that would convert
> all CR/LF-ending text files into LF-ending ones... but it's not really
> trivial, because telling binary files that should be left untouched from
> text files that should be converted is difficult: even the very first file
> in GDB's sources ("configure") contains a '\a', i.e. a "not-text" byte.

While it's not a complete solution by itself, I hope you're using the
dos2unix or d2u programs to handle the conversion.  You may also be able
to make use of the file program from the file package to help identify
files that are appropriate for conversion.  Given that the igncr option
is only useful for bash and maybe sh, scripts for those are probably the
only ones you want to convert, and the file program should be able to
identify them for you.

-Jeremy

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Sorry "people" (NOT MY taxonomy!!), but igncr IS flawed

[Christopher Faylor]

On Sat, Dec 17, 2011 at 12:21:40AM -0600, Jeremy Bopp wrote:
>On 12/16/2011 11:13 PM, manu0507 wrote:
>>Notwithstanding the completely preposterous "reply" by Eric Blake (more
>>of an idiotic acrimony, actually) to my previous post (see
>>http://old.nabble.com/Igncr-ineffective--tt32983438.html ), there does
>>seem to be a problem in dealing with Win's CR/LF line endings in
>>"unusual" lines, at least on Win7-64 (or, to be really precise, on my
>>Win7-64).  The lines where CR/LFs appear not to be properly converted
>>to LFs seem to be empty lines (except for the CR/LF, of course), as
>>well as some other "unusual" constructs (lines ending with ";;CR/LF" in
>>particular).
>
>I don't have Cygwin available at the moment, so I can't try running
>scripts as you describe right now.  However, the claim that a line
>consisting of only a CR/LF causing problems with the igncr option makes
>me pretty suspicious.  "Empty" lines are pretty darn common in bash
>scripts, and I would expect to have seen many reports of problems with
>igncr reported here by now if that option didn't correctly handle those
>lines.
>
>Can you send a representative example script that elicits this problem
>for you?  A simple test case would go a long way to addressing the
>issue.  From the sound of things, no one else has reproduced your issue
>yet.
>
>Perhaps the lines that are giving you trouble are actually ended with
>CR/CR/LF.  Have you examined the problematic scripts with a hex editor
>or simply "od -c" to verify the line endings?
>
>>To work around the problem, I'm writing an application that would
>>convert all CR/LF-ending text files into LF-ending ones...  but it's
>>not really trivial, because telling binary files that should be left
>>untouched from text files that should be converted is difficult: even
>>the very first file in GDB's sources ("configure") contains a '\a',
>>i.e.  a "not-text" byte.
>
>While it's not a complete solution by itself, I hope you're using the
>dos2unix or d2u programs to handle the conversion.  You may also be
>able to make use of the file program from the file package to help
>identify files that are appropriate for conversion.  Given that the
>igncr option is only useful for bash and maybe sh, scripts for those
>are probably the only ones you want to convert, and the file program
>should be able to identify them for you.

Hats off to you, Jeremy, for your civility in responding to this rant.

cgf

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Sorry "people" (NOT MY taxonomy!!), but igncr IS flawed

[manu0507]

Hello Jeremy,

Many thanks for your reply. To be sure, I was quite incredulous myself bash
would have problems with empty lines, but that's what happens for me.

To be precise:
- I'm running Win7-64, Home Premium SP1
- I built Cygwin by running setup.exe with all the default options. The bash
is

Manu@Holland ~
$ bash --version
GNU bash, version 4.1.10(4)-release (i686-pc-cygwin)

- the script causing trouble comes in the GNU gdb distribution
gdb-7.3.1.tar.gz
- Notepad++ (sorry, a Win app) shows me that the problematic empty line
really contains just "[CR][LF]"
- needless to say, as I reported, ALL lines end that way (and ARE handled
correctly for most of them!), but that one is the first where "[CR][LF]" is
not preceded by anything in the line
- I'm uploading the script in case you'd find an opportunity to run it in
your Cygwin - in mine it stops with

Manu@Holland /cygdrive/d/Installs/gdb_7_3_1/gdb-7.3.1
$ ./configure
./configure: line 14: $'\r': command not found
./configure: line 29: syntax error near unexpected token `newline'
'/configure: line 29: `     ;;

Many thanks and Merry Christmas,

Emanuel


Jeremy Bopp-3 wrote:
> 
> On 12/16/2011 11:13 PM, manu0507 wrote:
>> 
>> Hi all,
>> 
>> Notwithstanding the completely preposterous "reply" by Eric Blake (more
>> of
>> an idiotic acrimony, actually) to my previous post (see
>> http://old.nabble.com/Igncr-ineffective--tt32983438.html ), there does
>> seem
>> to be a problem in dealing with Win's CR/LF line endings in "unusual"
>> lines,
>> at least on Win7-64 (or, to be really precise, on my Win7-64).
>> The lines where CR/LFs appear not to be properly converted to LFs seem to
>> be
>> empty lines (except for the CR/LF, of course), as well as some other
>> "unusual" constructs (lines ending with ";;CR/LF" in particular).
> 
> I don't have Cygwin available at the moment, so I can't try running
> scripts as you describe right now.  However, the claim that a line
> consisting of only a CR/LF causing problems with the igncr option makes
> me pretty suspicious.  "Empty" lines are pretty darn common in bash
> scripts, and I would expect to have seen many reports of problems with
> igncr reported here by now if that option didn't correctly handle those
> lines.
> 
> Can you send a representative example script that elicits this problem
> for you?  A simple test case would go a long way to addressing the
> issue.  From the sound of things, no one else has reproduced your issue
> yet.
> 
> Perhaps the lines that are giving you trouble are actually ended with
> CR/CR/LF.  Have you examined the problematic scripts with a hex editor
> or simply "od -c" to verify the line endings?
> 
>> To work around the problem, I'm writing an application that would convert
>> all CR/LF-ending text files into LF-ending ones... but it's not really
>> trivial, because telling binary files that should be left untouched from
>> text files that should be converted is difficult: even the very first
>> file
>> in GDB's sources ("configure") contains a '\a', i.e. a "not-text" byte.
> 
> While it's not a complete solution by itself, I hope you're using the
> dos2unix or d2u programs to handle the conversion.  You may also be able
> to make use of the file program from the file package to help identify
> files that are appropriate for conversion.  Given that the igncr option
> is only useful for bash and maybe sh, scripts for those are probably the
> only ones you want to convert, and the file program should be able to
> identify them for you.
> 
> -Jeremy
> 
> --
> Problem reports:       http://cygwin.com/problems.html
> FAQ:                   http://cygwin.com/faq/
> Documentation:         http://cygwin.com/docs.html
> Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
> 
> 
> 
http://old.nabble.com/file/p32994383/configure configure 
-- 
View this message in context: http://old.nabble.com/Sorry-%22people%22-%28NOT-MY-taxonomy%21%21%29%2C-but-igncr-IS-flawed-tp32989786p32994383.html
Sent from the Cygwin list mailing list archive at Nabble.com.


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Sorry "people" (NOT MY taxonomy!!), but igncr IS flawed

[manu0507]

Dear Christopher,

PLEASE read the three posts (the second one in particular) in the thread
referenced below, and THEN decide who started the "rant". Thank you.

Merry Christmas,

Emanuel



Christopher Faylor-8 wrote:
> 
> On Sat, Dec 17, 2011 at 12:21:40AM -0600, Jeremy Bopp wrote:
>>On 12/16/2011 11:13 PM, manu0507 wrote:
>>>Notwithstanding the completely preposterous "reply" by Eric Blake (more
>>>of an idiotic acrimony, actually) to my previous post (see
>>>http://old.nabble.com/Igncr-ineffective--tt32983438.html ), there does
>>>seem to be a problem in dealing with Win's CR/LF line endings in
>>>"unusual" lines, at least on Win7-64 (or, to be really precise, on my
>>>Win7-64).  The lines where CR/LFs appear not to be properly converted
>>>to LFs seem to be empty lines (except for the CR/LF, of course), as
>>>well as some other "unusual" constructs (lines ending with ";;CR/LF" in
>>>particular).
>>
>>I don't have Cygwin available at the moment, so I can't try running
>>scripts as you describe right now.  However, the claim that a line
>>consisting of only a CR/LF causing problems with the igncr option makes
>>me pretty suspicious.  "Empty" lines are pretty darn common in bash
>>scripts, and I would expect to have seen many reports of problems with
>>igncr reported here by now if that option didn't correctly handle those
>>lines.
>>[...]
> 
> Hats off to you, Jeremy, for your civility in responding to this rant.
> 
> cgf
> 
> --
> Problem reports:       http://cygwin.com/problems.html
> FAQ:                   http://cygwin.com/faq/
> Documentation:         http://cygwin.com/docs.html
> Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
> 
> 
> 

-- 
View this message in context: http://old.nabble.com/Sorry-%22people%22-%28NOT-MY-taxonomy%21%21%29%2C-but-igncr-IS-flawed-tp32989786p32994403.html
Sent from the Cygwin list mailing list archive at Nabble.com.


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Sorry "people" (NOT MY taxonomy!!), but igncr IS flawed

[Dave Korn]

On 17/12/2011 15:50, manu0507 wrote:

> - the script causing trouble comes in the GNU gdb distribution
> gdb-7.3.1.tar.gz

  Hah!  I know what's happened: you used a windows program such as winzip or
similar to unpack the archive, and it's gone and "helpfully" munged all the
line endings for you.  Use 'tar xvfz gdb-7.3.1.tar.gz' at the Cygwin shell
command-line and you'll get a correctly unpacked version.  (There definitely
aren't CRLFs in the upstream release of the configure scripts.)  Windows
GUI-based archivers are well known for causing this problem.

  (Also, if you think that Eric's reply was "acrimonious", you're just having
some sort of using-language-in-different-ways communication problem.  I saw it
earlier and it seemed perfectly polite and proper to me, I honestly can't see
what about it would offend you.)

    cheers,
      DaveK

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Sorry "people" (NOT MY taxonomy!!), but igncr IS flawed

[Lee]

On 12/17/11, Dave Korn wrote:
> On 17/12/2011 15:50, manu0507 wrote:
>
>> - the script causing trouble comes in the GNU gdb distribution
>> gdb-7.3.1.tar.gz
>
>   Hah!  I know what's happened: you used a windows program such as winzip or
> similar to unpack the archive, and it's gone and "helpfully" munged all the
> line endings for you.  Use 'tar xvfz gdb-7.3.1.tar.gz' at the Cygwin shell
> command-line and you'll get a correctly unpacked version.  (There definitely
> aren't CRLFs in the upstream release of the configure scripts.)  Windows
> GUI-based archivers are well known for causing this problem.

7-Zip (from sourceforge) also works for unpacking tar.gz files for use
with Cygwin.

>   (Also, if you think that Eric's reply was "acrimonious", you're just having
> some sort of using-language-in-different-ways communication problem.  I saw
> it earlier and it seemed perfectly polite and proper to me, I honestly can't
> see what about it would offend you.)

+1

Lee

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Apparently solved! (Re: Sorry "people" (NOT MY taxonomy!!), but igncr IS flawed)

[manu0507]

Hi Dave,


Dave Korn-9 wrote:
> 
> On 17/12/2011 15:50, manu0507 wrote:
> 
>> - the script causing trouble comes in the GNU gdb distribution
>> gdb-7.3.1.tar.gz
> 
>   Hah!  I know what's happened: you used a windows program such as winzip
> or
> similar to unpack the archive, and it's gone and "helpfully" munged all
> the
> line endings for you.  Use 'tar xvfz gdb-7.3.1.tar.gz' at the Cygwin shell
> command-line and you'll get a correctly unpacked version.  (There
> definitely
> aren't CRLFs in the upstream release of the configure scripts.)  Windows
> GUI-based archivers are well known for causing this problem.
> 

My gosh, that's it! I Indeed extracted with WinZip... I WAS a bit
suspicious, and looked in WinZip for an option like "Output text files in
DOS format" or so that I could UNcheck, but having not found it, I thought
WinZip would not be so intrusive as to modify the files that were
compressed...

Well, I never said Win was perfect, did I?  ;-)

That being said, it's still sort of beyond me why all [CR][LF]s would be
converted ok... except the ones I reported.


Dave Korn-9 wrote:
> 
>   (Also, if you think that Eric's reply was "acrimonious", you're just
> having
> some sort of using-language-in-different-ways communication problem.  I
> saw it
> earlier and it seemed perfectly polite and proper to me, I honestly can't
> see
> what about it would offend you.)
> 
>     cheers,
>       DaveK
> 

Ok, maybe Eric actually likes being called "people"... my fault then.

Many thanks & Merry Christmas,

Emanuel

-- 
View this message in context: http://old.nabble.com/Sorry-%22people%22-%28NOT-MY-taxonomy%21%21%29%2C-but-igncr-IS-flawed-tp32989786p32995016.html
Sent from the Cygwin list mailing list archive at Nabble.com.


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Sorry "people" (NOT MY taxonomy!!), but igncr IS flawed

[Christopher Faylor]

On Sat, Dec 17, 2011 at 07:57:30AM -0800, manu0507 wrote:
>PLEASE read the three posts (the second one in particular) in the thread
>referenced below, and THEN decide who started the "rant". Thank you.

I did read Eric's reply (which is the only one which would have any
bearing).  I know Eric.  I know how he communicates in email and he is
rarely, if ever, acrimonious.  For instance, I have never seen him use
words like "idiotic" or "preposterous".

Your rant was uncalled for.

cgf

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Apparently solved! (Re: Sorry "people" (NOT MY taxonomy!!), but igncr IS flawed)

[Mike Brown]

On Sat, Dec 17, 2011 at 10:27:23AM -0800, manu0507 wrote:
> That being said, it's still sort of beyond me why all [CR][LF]s would be
> converted ok... except the ones I reported.

It isn't a smart comversion, it is a blind substitute all conversion.

MB
-- 
e-mail: vidiot@vidiot.com | vidiot@vidiot.net            /~\ The ASCII
        6082066843@email.uscc.net                        \ / Ribbon Campaign
Visit - URL: http://vidiot.com/                           X  Against
             http://vidiot.net/                          / \ HTML Email

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Sorry "people" (NOT MY taxonomy!!), but igncr IS flawed

[Andrey Repin]

Greetings, Dave Korn!

> Windows GUI-based archivers are well known for causing this problem.

To be fair, you should reduce your reference to "WinZIP is known for".
At the very least, WinRAR and 7-Zip, both won't try to hold your hand in this case.
I don't know about WinACE, though, and I'm not going to check it myself.


--
WBR,
Andrey Repin (anrdaemon@freemail.ru) 19.12.2011, <20:57>

Sorry for my terrible english...


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Sorry "people" (NOT MY taxonomy!!), but igncr IS flawed

[Dave Korn]

On 19/12/2011 17:35, Andrey Repin wrote:
> Greetings, Dave Korn!
> 
>> Windows GUI-based archivers are well known for causing this problem.
> 
> To be fair, you should reduce your reference to "WinZIP is known for".
> At the very least, WinRAR and 7-Zip, both won't try to hold your hand in this case.
> I don't know about WinACE, though, and I'm not going to check it myself.

  I haven't checked, but would bet WinRAR and 7-Zip don't accurately recreate
the posix permissions when they unpack a *nix tarball.  They probably don't
unpack device nodes right either.  They can probably get softlinks right but I
haven't tested that either.

  Rather than play guessing games about which gui-based unpackers will do a
good job of unpacking something into a Cygwin environment and have to
repeatedly re-solve the problems they may cause every time someone says "Oh,
well you said not to use WinZip, so I used WinRAR (or whatever) instead", I
just err on the side of caution and simply advise: for unpacking stuff into a
Cygwin environment, always use a Cygwin-aware tool.

    cheers,
      DaveK


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Sorry "people" (NOT MY taxonomy!!), but igncr IS flawed

[Andrey Repin]

Greetings, Dave Korn!

>>> Windows GUI-based archivers are well known for causing this problem.
>> 
>> To be fair, you should reduce your reference to "WinZIP is known for".
>> At the very least, WinRAR and 7-Zip, both won't try to hold your hand in this case.
>> I don't know about WinACE, though, and I'm not going to check it myself.

>   I haven't checked, but would bet WinRAR and 7-Zip don't accurately recreate
> the posix permissions when they unpack a *nix tarball.  They probably don't
> unpack device nodes right either.  They can probably get softlinks right but I
> haven't tested that either.

I doubt that so much. Neither of them is aware of Cygwin specifics.
But they don't have EOL issues for sure, as they treating files as binary at
all times.

>   Rather than play guessing games about which gui-based unpackers will do a
> good job of unpacking something into a Cygwin environment and have to
> repeatedly re-solve the problems they may cause every time someone says "Oh,
> well you said not to use WinZip, so I used WinRAR (or whatever) instead", I
> just err on the side of caution and simply advise: for unpacking stuff into a
> Cygwin environment, always use a Cygwin-aware tool.

That's right.


--
WBR,
Andrey Repin (anrdaemon@freemail.ru) 21.12.2011, <22:47>

Sorry for my terrible english...


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple