Join command fails to output fields on input file with DOS line endings.

Join command fails to output fields on input file with DOS line endings.

[Andy Hall]

This behavior of join surprised me: 

$ join -1 3 <(echo a b col3  c d | unix2dos) <(echo col3 f2 f3 f4 f5)
 f2 f3 f4 f5

Join parses the input line well enough to execute the join, but the presence of the DOS line endings suppresses the
output of fields from the first input.

Compare with

$ join -1 3 <(echo a b col3  c d) <(echo col3 f2 f3 f4 f5)
col3 a b c d f2 f3 f4 f5

which is correct.  

Here is a weirder example where the join field is missing and the output is reversed!

NOK
$ join <(echo col F1 | unix2dos) <(echo col F2)
 F2 F1

OK
$ join <(echo col F1) <(echo col F2)
col F1 F2






--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Join command fails to output fields on input file with DOS line endings.

[Eliot Moss]

On 7/22/2019 3:50 PM, Andy Hall wrote:
> This behavior of join surprised me:
> 
> $ join -1 3 <(echo a b col3  c d | unix2dos) <(echo col3 f2 f3 f4 f5)
>   f2 f3 f4 f5
> 
> Join parses the input line well enough to execute the join, but the presence of the DOS line endings suppresses the
> output of fields from the first input.
> 
> Compare with
> 
> $ join -1 3 <(echo a b col3  c d) <(echo col3 f2 f3 f4 f5)
> col3 a b c d f2 f3 f4 f5
> 
> which is correct.
> 
> Here is a weirder example where the join field is missing and the output is reversed!
> 
> NOK
> $ join <(echo col F1 | unix2dos) <(echo col F2)
>   F2 F1
> 
> OK
> $ join <(echo col F1) <(echo col F2)
> col F1 F2

The lesson is: You're better off not using DOS line endings with Cygwin.
It tends to do things the POSIX way, where the line ending is LF, not
CR LF.  Some things *may* work, but it's somewhat hit or miss ...  EM

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

RE: Join command fails to output fields on input file with DOS line endings.

[Andy Hall]

> On 7/22/2019 12:59 PM, Eliot Moss wrote:
> On 7/22/2019 12:50 PM, Andy Hall wrote:
> > This behavior of join surprised me:
> >
> > $ join -1 3 <(echo a b col3  c d | unix2dos) <(echo col3 f2 f3 f4 f5)
> >   f2 f3 f4 f5
> >
> > Join parses the input line well enough to execute the join, but the presence of the DOS line endings suppresses the
> > output of fields from the first input.
> >
> > Compare with
> >
> > $ join -1 3 <(echo a b col3  c d) <(echo col3 f2 f3 f4 f5)
> > col3 a b c d f2 f3 f4 f5
> >
> > which is correct.
> >
> > Here is a weirder example where the join field is missing and the output is reversed!
> >
> > NOK
> > $ join <(echo col F1 | unix2dos) <(echo col F2)
> >   F2 F1
> >
> > OK
> > $ join <(echo col F1) <(echo col F2)
> > col F1 F2
> 
> The lesson is: You're better off not using DOS line endings with Cygwin.
> It tends to do things the POSIX way, where the line ending is LF, not
> CR LF.  Some things *may* work, but it's somewhat hit or miss ...  EM
> 
Right.  But in this case,  the last field of the first file contains a \r which
completely mangles the output.  

But join appears to "work" if the lines of the second file are in DOS format.

E.g.
$ join <(echo col F1) <(echo col F2 | unix2dos)
col F1 F2


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Join command fails to output fields on input file with DOS line endings.

[Andrey Repin]

Greetings, Andy Hall!

> This behavior of join surprised me: 

> $ join -1 3 <(echo a b col3  c d | unix2dos) <(echo col3 f2 f3 f4 f5)
>  f2 f3 f4 f5

> Join parses the input line well enough to execute the join, but the
> presence of the DOS line endings suppresses the
> output of fields from the first input.

> Compare with

> $ join -1 3 <(echo a b col3  c d) <(echo col3 f2 f3 f4 f5)
> col3 a b c d f2 f3 f4 f5

> which is correct.  

> Here is a weirder example where the join field is missing and the output is reversed!

> NOK
> $ join <(echo col F1 | unix2dos) <(echo col F2)
>  F2 F1

This makes perfect sense and actually explains what you see.
1. ("col", "F1\r") & ("col", "F2")
2. ("col", "F1\r", "F2")
3. printing "col F1\r F2"
4. obvious result: "col F1", then \r place cursor at the beginning of the
line, then " F2" overprints the line beginning.
5. observed result: " F2 F1"

> OK
> $ join <(echo col F1) <(echo col F2)
> col F1 F2

As Eliot said, don't do that.


-- 
With best regards,
Andrey Repin
Tuesday, July 23, 2019 9:45:05

Sorry for my terrible english...


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

RE: Join command fails to output fields on input file with DOS line endings.

[Lemke, Michael ST/HZA-ZIC2]

On July 22, 2019 9:51 PM Andy Hall wrote:
>This behavior of join surprised me: 
>
>$ join -1 3 <(echo a b col3  c d | unix2dos) <(echo col3 f2 f3 f4 f5)
> f2 f3 f4 f5

See if this reduces your surprise somewhat:

$ join -1 3 <(echo a b col3  c d | unix2dos) <(echo col3 f2 f3 f4 f5) | cat -A

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple