Problem with named pipes

Problem with named pipes

[Ken Brown]

I've noticed a difference between Cygwin and Linux involving named 
pipes.  I don't know if this a bug or simply a difference.

Consider the following two scripts:

$ cat fifo1.sh
#!/bin/sh
set -x
rm -f foo
mkfifo foo
exec 7>foo
echo blah > foo

$ cat fifo2.sh
#!/bin/sh
set -x
read bar < foo
echo $bar

I run fifo1.sh in Terminal 1 and get the following on both Cygwin and Linux:

[Terminal 1]
$ ./fifo1.sh
+ rm -f foo
+ mkfifo foo
+ exec

The call to echo in the next line blocks, because foo has not yet been 
opened for reading.  Now I run fifo2.sh in Terminal 2.  On Linux, the 
"read" in fifo2.sh unblocks fifo1.sh, and I see the following:

[Terminal 1, Linux]
$ ./fifo1.sh
+ rm -f foo
+ mkfifo foo
+ exec
+ echo blah

[Terminal 2, Linux]
$ ./fifo2.sh
+ read bar
+ echo blah
blah

On Cygwin, however, Terminal 1 remains blocked, and Terminal 2 is 
blocked when it tries to read:

[Terminal 2, Cygwin]
$ ./fifo2.sh
+ read bar

The problem disappears if I remove the line "exec 7>foo" from fifo1.sh. 
  The problem also disappears if I leave that line in, but change the 
last line to "echo blah >&7".

Ken

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Problem with named pipes

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1665 bytes --]

On Dec 25 16:39, Ken Brown wrote:
> I've noticed a difference between Cygwin and Linux involving named pipes.  I
> don't know if this a bug or simply a difference.
> 
> Consider the following two scripts:
> 
> $ cat fifo1.sh
> #!/bin/sh
> set -x
> rm -f foo
> mkfifo foo
> exec 7>foo
> echo blah > foo
> 
> $ cat fifo2.sh
> #!/bin/sh
> set -x
> read bar < foo
> echo $bar
> 
> I run fifo1.sh in Terminal 1 and get the following on both Cygwin and Linux:
> 
> [Terminal 1]
> $ ./fifo1.sh
> + rm -f foo
> + mkfifo foo
> + exec
> 
> The call to echo in the next line blocks, because foo has not yet been
> opened for reading.  Now I run fifo2.sh in Terminal 2.  On Linux, the "read"
> in fifo2.sh unblocks fifo1.sh, and I see the following:
> 
> [Terminal 1, Linux]
> $ ./fifo1.sh
> + rm -f foo
> + mkfifo foo
> + exec
> + echo blah
> 
> [Terminal 2, Linux]
> $ ./fifo2.sh
> + read bar
> + echo blah
> blah
> 
> On Cygwin, however, Terminal 1 remains blocked, and Terminal 2 is blocked
> when it tries to read:
> 
> [Terminal 2, Cygwin]
> $ ./fifo2.sh
> + read bar
> 
> The problem disappears if I remove the line "exec 7>foo" from fifo1.sh.  The
> problem also disappears if I leave that line in, but change the last line to
> "echo blah >&7".

Very tricky problem.  The FIFO code falls over its own feet trying to
handle more than one writer (exec 7> is the first, echo blah is the
second writer).  Sigh.  This code needs a thorough rewrite...


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: Problem with named pipes

[Ken Brown]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="utf-8", Size: 2447 bytes --]

On 1/8/2016 3:50 PM, Corinna Vinschen wrote:
> On Dec 25 16:39, Ken Brown wrote:
>> I've noticed a difference between Cygwin and Linux involving named pipes.  I
>> don't know if this a bug or simply a difference.
>>
>> Consider the following two scripts:
>>
>> $ cat fifo1.sh
>> #!/bin/sh
>> set -x
>> rm -f foo
>> mkfifo foo
>> exec 7>foo
>> echo blah > foo
>>
>> $ cat fifo2.sh
>> #!/bin/sh
>> set -x
>> read bar < foo
>> echo $bar
>>
>> I run fifo1.sh in Terminal 1 and get the following on both Cygwin and Linux:
>>
>> [Terminal 1]
>> $ ./fifo1.sh
>> + rm -f foo
>> + mkfifo foo
>> + exec
>>
>> The call to echo in the next line blocks, because foo has not yet been
>> opened for reading.  Now I run fifo2.sh in Terminal 2.  On Linux, the "read"
>> in fifo2.sh unblocks fifo1.sh, and I see the following:
>>
>> [Terminal 1, Linux]
>> $ ./fifo1.sh
>> + rm -f foo
>> + mkfifo foo
>> + exec
>> + echo blah
>>
>> [Terminal 2, Linux]
>> $ ./fifo2.sh
>> + read bar
>> + echo blah
>> blah
>>
>> On Cygwin, however, Terminal 1 remains blocked, and Terminal 2 is blocked
>> when it tries to read:
>>
>> [Terminal 2, Cygwin]
>> $ ./fifo2.sh
>> + read bar
>>
>> The problem disappears if I remove the line "exec 7>foo" from fifo1.sh.  The
>> problem also disappears if I leave that line in, but change the last line to
>> "echo blah >&7".
> 
> Very tricky problem.  The FIFO code falls over its own feet trying to
> handle more than one writer (exec 7> is the first, echo blah is the
> second writer).  Sigh.  This code needs a thorough rewrite...

Hi Corinna,

I'm coming back to this thread after three years.  I'd be willing to put in some 
time working on this, if you could point me in the right direction.  Do you have 
ideas about what a rewrite of the FIFO code might look like?

The one idea that I had was that a reader could open several instances of the 
Windows named pipe, allowing several writers to connect to it simultaneously.  I 
think that would solve the particular problem I reported (and the similar 
problem cited by Houder up-thread).  But it would just be a band-aid, and Cygwin 
FIFOs still wouldn't be POSIX compliant.

Ken
\x03B‹KCB”\x1c›Ø›\x19[H\x1c™\^[ܝ\x1cΈ\b\b\b\b\b\b\x1a\x1d\x1d\x1c\x0e‹ËØÞYÝÚ[‹˜ÛÛKÜ\x1c›Ø›\x19[\Ëš\x1d^[[\x03B‘TNˆ\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\x1a\x1d\x1d\x1c\x0e‹ËØÞYÝÚ[‹˜ÛÛKÙ˜\KÃB‘^[ØÝ[Y[\x18]\x1a[ÛŽˆ\b\b\b\b\b\b\b\b\x1a\x1d\x1d\x1c\x0e‹ËØÞYÝÚ[‹˜ÛÛKÙ^[ØÜËš\x1d^[[\x03B•[œÝXœØÜšX™H\x1a[™›Îˆ\b\b\b\b\b\x1a\x1d\x1d\x1c\x0e‹ËØÞYÝÚ[‹˜ÛÛKÛ[\vÈÝ[œÝXœØÜšX™K\Ú[\^[\x19CBƒB

Re: Problem with named pipes

[Dan Kegel]

On Tue, Dec 11, 2018 at 6:42 AM Ken Brown <kbrown@cornell.edu> wrote:
> > Very tricky problem.  The FIFO code falls over its own feet trying to
> > handle more than one writer (exec 7> is the first, echo blah is the
> > second writer).  Sigh.  This code needs a thorough rewrite...
>
> I'm coming back to this thread after three years.  I'd be willing to put in some
> time working on this, if you could point me in the right direction.

I'm no cygwin developer, but:
It might be worth spending a little quality time with
https://cygwin.com/git/gitweb.cgi?p=newlib-cygwin.git;a=tree;f=winsup/testsuite/winsup.api/ltp
to update it to handle the case you're interested in first...?
https://github.com/linux-test-project/ltp has come a long, long ways
since cygwin's copy was made.  It might be interesting to just
try building and running LTP on cygwin and see how far it is from
working at all... I would not be surprised if cygwin's copy is highly customized
to avoid spikes.
- Dan

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Problem with named pipes

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 2269 bytes --]

On Dec 11 14:42, Ken Brown wrote:
> On 1/8/2016 3:50 PM, Corinna Vinschen wrote:
> > On Dec 25 16:39, Ken Brown wrote:
> >> I've noticed a difference between Cygwin and Linux involving named pipes.  I
> >> don't know if this a bug or simply a difference.
> >>
> >> Consider the following two scripts:
> >> [...]
> >>
> >> The problem disappears if I remove the line "exec 7>foo" from fifo1.sh.  The
> >> problem also disappears if I leave that line in, but change the last line to
> >> "echo blah >&7".
> > 
> > Very tricky problem.  The FIFO code falls over its own feet trying to
> > handle more than one writer (exec 7> is the first, echo blah is the
> > second writer).  Sigh.  This code needs a thorough rewrite...
> 
> Hi Corinna,
> 
> I'm coming back to this thread after three years.  I'd be willing to put in some 
> time working on this, if you could point me in the right direction.  Do you have 
> ideas about what a rewrite of the FIFO code might look like?
> The one idea that I had was that a reader could open several instances of the 
> Windows named pipe, allowing several writers to connect to it simultaneously.  I 
> think that would solve the particular problem I reported (and the similar 
> problem cited by Houder up-thread).  But it would just be a band-aid, and Cygwin 

I think this is not required, as long as at least one instance of the
pipe is available at all times.  Perhaps the unfinished, new AF_UNIX
sockets code can be reused or partially duplicated to implement FIFOs.
The fhandler_socket_unix::accept4 function tries to emulate that
behaviour.  As soon as a client connects, it spawns another pipe
instance and uses that as listener handle, while the previous accepting
pipe handle is now connected with the client and is what gets used for
the new, accepted descriptor.

> FIFOs still wouldn't be POSIX compliant.

Yeah, this is really tricky to get with Windows pipe semantics.  I guess
if the most common scenarios are covered, we're good.

Thanks a lot for looking into that.  Feel free to discuss any and all
implementation details on the developers ML.  I guess it's time to pick
up the AF_UNIX pieces myself next year :}


Corinna

-- 
Corinna Vinschen
Cygwin Maintainer

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]