public inbox for cygwin@cygwin.com
 help / color / mirror / Atom feed
* Join command fails to output fields on input file with DOS line endings.
@ 2019-07-22 19:50 Andy Hall
  2019-07-22 19:58 ` Eliot Moss
                   ` (2 more replies)
  0 siblings, 3 replies; 5+ messages in thread
From: Andy Hall @ 2019-07-22 19:50 UTC (permalink / raw)
  To: cygwin

This behavior of join surprised me: 

$ join -1 3 <(echo a b col3  c d | unix2dos) <(echo col3 f2 f3 f4 f5)
 f2 f3 f4 f5

Join parses the input line well enough to execute the join, but the presence of the DOS line endings suppresses the
output of fields from the first input.

Compare with

$ join -1 3 <(echo a b col3  c d) <(echo col3 f2 f3 f4 f5)
col3 a b c d f2 f3 f4 f5

which is correct.  

Here is a weirder example where the join field is missing and the output is reversed!

NOK
$ join <(echo col F1 | unix2dos) <(echo col F2)
 F2 F1

OK
$ join <(echo col F1) <(echo col F2)
col F1 F2






--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: Join command fails to output fields on input file with DOS line endings.
  2019-07-22 19:50 Join command fails to output fields on input file with DOS line endings Andy Hall
@ 2019-07-22 19:58 ` Eliot Moss
  2019-07-23 15:15   ` Andy Hall
  2019-07-23  6:50 ` Andrey Repin
  2019-07-23 12:18 ` Lemke, Michael  ST/HZA-ZIC2
  2 siblings, 1 reply; 5+ messages in thread
From: Eliot Moss @ 2019-07-22 19:58 UTC (permalink / raw)
  To: cygwin

On 7/22/2019 3:50 PM, Andy Hall wrote:
> This behavior of join surprised me:
> 
> $ join -1 3 <(echo a b col3  c d | unix2dos) <(echo col3 f2 f3 f4 f5)
>   f2 f3 f4 f5
> 
> Join parses the input line well enough to execute the join, but the presence of the DOS line endings suppresses the
> output of fields from the first input.
> 
> Compare with
> 
> $ join -1 3 <(echo a b col3  c d) <(echo col3 f2 f3 f4 f5)
> col3 a b c d f2 f3 f4 f5
> 
> which is correct.
> 
> Here is a weirder example where the join field is missing and the output is reversed!
> 
> NOK
> $ join <(echo col F1 | unix2dos) <(echo col F2)
>   F2 F1
> 
> OK
> $ join <(echo col F1) <(echo col F2)
> col F1 F2

The lesson is: You're better off not using DOS line endings with Cygwin.
It tends to do things the POSIX way, where the line ending is LF, not
CR LF.  Some things *may* work, but it's somewhat hit or miss ...  EM

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: Join command fails to output fields on input file with DOS line endings.
  2019-07-22 19:50 Join command fails to output fields on input file with DOS line endings Andy Hall
  2019-07-22 19:58 ` Eliot Moss
@ 2019-07-23  6:50 ` Andrey Repin
  2019-07-23 12:18 ` Lemke, Michael  ST/HZA-ZIC2
  2 siblings, 0 replies; 5+ messages in thread
From: Andrey Repin @ 2019-07-23  6:50 UTC (permalink / raw)
  To: Andy Hall, cygwin

Greetings, Andy Hall!

> This behavior of join surprised me: 

> $ join -1 3 <(echo a b col3  c d | unix2dos) <(echo col3 f2 f3 f4 f5)
>  f2 f3 f4 f5

> Join parses the input line well enough to execute the join, but the
> presence of the DOS line endings suppresses the
> output of fields from the first input.

> Compare with

> $ join -1 3 <(echo a b col3  c d) <(echo col3 f2 f3 f4 f5)
> col3 a b c d f2 f3 f4 f5

> which is correct.  

> Here is a weirder example where the join field is missing and the output is reversed!

> NOK
> $ join <(echo col F1 | unix2dos) <(echo col F2)
>  F2 F1

This makes perfect sense and actually explains what you see.
1. ("col", "F1\r") & ("col", "F2")
2. ("col", "F1\r", "F2")
3. printing "col F1\r F2"
4. obvious result: "col F1", then \r place cursor at the beginning of the
line, then " F2" overprints the line beginning.
5. observed result: " F2 F1"

> OK
> $ join <(echo col F1) <(echo col F2)
> col F1 F2

As Eliot said, don't do that.


-- 
With best regards,
Andrey Repin
Tuesday, July 23, 2019 9:45:05

Sorry for my terrible english...


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

^ permalink raw reply	[flat|nested] 5+ messages in thread

* RE: Join command fails to output fields on input file with DOS line endings.
  2019-07-22 19:50 Join command fails to output fields on input file with DOS line endings Andy Hall
  2019-07-22 19:58 ` Eliot Moss
  2019-07-23  6:50 ` Andrey Repin
@ 2019-07-23 12:18 ` Lemke, Michael  ST/HZA-ZIC2
  2 siblings, 0 replies; 5+ messages in thread
From: Lemke, Michael  ST/HZA-ZIC2 @ 2019-07-23 12:18 UTC (permalink / raw)
  To: Andy Hall, cygwin

On July 22, 2019 9:51 PM Andy Hall wrote:
>This behavior of join surprised me: 
>
>$ join -1 3 <(echo a b col3  c d | unix2dos) <(echo col3 f2 f3 f4 f5)
> f2 f3 f4 f5

See if this reduces your surprise somewhat:

$ join -1 3 <(echo a b col3  c d | unix2dos) <(echo col3 f2 f3 f4 f5) | cat -A

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

^ permalink raw reply	[flat|nested] 5+ messages in thread

* RE: Join command fails to output fields on input file with DOS line endings.
  2019-07-22 19:58 ` Eliot Moss
@ 2019-07-23 15:15   ` Andy Hall
  0 siblings, 0 replies; 5+ messages in thread
From: Andy Hall @ 2019-07-23 15:15 UTC (permalink / raw)
  To: moss, cygwin

> On 7/22/2019 12:59 PM, Eliot Moss wrote:
> On 7/22/2019 12:50 PM, Andy Hall wrote:
> > This behavior of join surprised me:
> >
> > $ join -1 3 <(echo a b col3  c d | unix2dos) <(echo col3 f2 f3 f4 f5)
> >   f2 f3 f4 f5
> >
> > Join parses the input line well enough to execute the join, but the presence of the DOS line endings suppresses the
> > output of fields from the first input.
> >
> > Compare with
> >
> > $ join -1 3 <(echo a b col3  c d) <(echo col3 f2 f3 f4 f5)
> > col3 a b c d f2 f3 f4 f5
> >
> > which is correct.
> >
> > Here is a weirder example where the join field is missing and the output is reversed!
> >
> > NOK
> > $ join <(echo col F1 | unix2dos) <(echo col F2)
> >   F2 F1
> >
> > OK
> > $ join <(echo col F1) <(echo col F2)
> > col F1 F2
> 
> The lesson is: You're better off not using DOS line endings with Cygwin.
> It tends to do things the POSIX way, where the line ending is LF, not
> CR LF.  Some things *may* work, but it's somewhat hit or miss ...  EM
> 
Right.  But in this case,  the last field of the first file contains a \r which
completely mangles the output.  

But join appears to "work" if the lines of the second file are in DOS format.

E.g.
$ join <(echo col F1) <(echo col F2 | unix2dos)
col F1 F2


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, other threads:[~2019-07-23 15:15 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-07-22 19:50 Join command fails to output fields on input file with DOS line endings Andy Hall
2019-07-22 19:58 ` Eliot Moss
2019-07-23 15:15   ` Andy Hall
2019-07-23  6:50 ` Andrey Repin
2019-07-23 12:18 ` Lemke, Michael  ST/HZA-ZIC2

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).