[Bug] Named Pipes (FIFO) / Bash

[Bug] Named Pipes (FIFO) / Bash

[Achim Gratz]

I'm having problems with some bash scripts that were developed (not be
me) and working OK in Cygwin 1.6.x versions, but not in 1.7.x (tested on
1.7.6, 1.7.7 and the latest snapshot).  After some gnashing of teeth and
pulling of hair I've whittled it down to a problem with named pipe
handling in bash.  Sifting through the archives of the mailing lists
didn't provide much clues.  I realize from earlier discussions that
named pipes aren't fully supported, but some examples of programs and
scripts seem to indicate that they should at least work on a basic
level.

I've tried both the current and the just announced experimental version
of bash and both Windows7/CoreDuo and WinXP/TurionX2 (I don't think the
dual cores have something to do with the problem, but if necessary I
should be able to at least boot the WinXP machine with only a single
core enabled).

$ mkfifo /tmp/pipe
$ ( sleep 10; echo -n "blafasel "; date; sleep 10; cat < /tmp/pipe )&
$ echo -n "dingdong "; date; exec 6> /tmp/pipe; echo -n "blumblum "; date
$ echo -n "hurrdurr " >&6; date

Running this on Linux outputs as expected:

dingdong Sa 22. Jan 20:41:45 CET 2011
blafasel Sa 22. Jan 20:41:55 CET 2011
blumblum Sa 22. Jan 20:42:05 CET 2011
hurrdurr Sa 22. Jan 20:42:05 CET 2011

On Cygwin, I get this:

dingdong Sa 22. Jan 20:41:45 CET 2011
blumblum Sa 22. Jan 20:41:45 CET 2011
blafasel Sa 22. Jan 20:41:55 CET 2011
cat: -: Communication error on send

In other words, opening the pipe is non-blocking, even though no
listener is present.  Listing /proc/self/fd lists fd 6 as opened to FIFO
/tmp/pipe, but any output to it blocks indefinitely.  I believe that the
FIFO has in fact never been opened as far as the system is concerned.
The output then blocks on the apparently not-open FIFO (I should be
getting a SIGPIPE instead when trying to write into a pipe that has no
listener).

Note: Looking at the problematic scripts I realize that the opening of
the FIFO must have been non-blocking in 1.6.x as well, but the output
was just blocked until the listener became available.

The last message goes away if I replace the "cat <" with simply "cat"
(this seems to be a separate bug in redirection handling of named pipes
which I could easily work around), but the shell still hangs up hard.  A
ps in another shell shows that bash is waiting for output and cat is
waiting for input.  I can then, however, write into the pipe from a
second shell and have the ouput appear in the tty of the first.

If I reverse the course of things and let the cat out first, so to
speak, then (and only then) everything works in a single shell.  This
however requires the very serialization that the named pipe was supposed
to provide.

I'd be grateful for any insights and/or ideas for possible workarounds.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Samples for the Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#BlofeldSamplesExtra


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: [Bug] Named Pipes (FIFO) / Bash

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 946 bytes --]

On 01/22/2011 01:57 PM, Achim Gratz wrote:
> 
> I'm having problems with some bash scripts that were developed (not be
> me) and working OK in Cygwin 1.6.x versions, but not in 1.7.x (tested on
> 1.7.6, 1.7.7 and the latest snapshot).  After some gnashing of teeth and
> pulling of hair I've whittled it down to a problem with named pipe
> handling in bash.

Not just in bash, but in cygwin in general.  Named pipes are still an
incomplete implementation, and until patches are written to get it
working more like posix, you can't expect them to work reliably.

> I'd be grateful for any insights and/or ideas for possible workarounds.

Sorry, I don't have any.  Autoconf is another client that would love to
use named pipes (for parallel testsuites), but which can't do so until
cygwin's implementation is completed.

-- 
Eric Blake   eblake@redhat.com    +1-801-349-2682
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 619 bytes --]

Re: [Bug] Named Pipes (FIFO) / Bash

[Achim Gratz]

Eric Blake <eblake@redhat.com> writes:
> Not just in bash, but in cygwin in general.  Named pipes are still an
> incomplete implementation, and until patches are written to get it
> working more like posix, you can't expect them to work reliably.

I said I knew that it wasn't working like POSIX pipes.  Now, there's
clearly a regression from 1.6->1.7.  Before they just weren't working
like POSIX pipes, but now they seem to be completely unuseable for
synchronisation purposes.

> Sorry, I don't have any.  Autoconf is another client that would love to
> use named pipes (for parallel testsuites), but which can't do so until
> cygwin's implementation is completed.

So, are you saying there's a development plan to get there, or not?
:-)


Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Wavetables for the Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#BlofeldUserWavetables


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: [Bug] Named Pipes (FIFO) / Bash

[Achim Gratz]

> I'd be grateful for any insights and/or ideas for possible workarounds.

Could anybody please comment if there are changes to the whole area of
(named) pipes in the upcoming 1.7.8 release?  Should I just grab the
latest snapshot and try my luck?  I don't mind if it's not POSIX
compliant if at least consistent synchronization can be achieved that
does not depend on the order of consumers and producers connecting to
the pipe (like it did on Cygwin 1.6).


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf Q+, Q and microQ:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: [Bug] Named Pipes (FIFO) / Bash

[Christopher Faylor]

On Sun, Jan 30, 2011 at 04:24:48PM +0100, Achim Gratz wrote:
>> I'd be grateful for any insights and/or ideas for possible workarounds.
>
>Could anybody please comment if there are changes to the whole area of
>(named) pipes in the upcoming 1.7.8 release?

There are no changes to named pipes in 1.7.8.  Unfortunately, they will
remain broken in this release.

cgf

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: [Bug] Named Pipes (FIFO) / Bash

[Achim Gratz]

Achim Gratz <Stromeko@nexgo.de> writes:
> In other words, opening the pipe is non-blocking, even though no
> listener is present.  Listing /proc/self/fd lists fd 6 as opened to FIFO
> /tmp/pipe, but any output to it blocks indefinitely.  I believe that the
> FIFO has in fact never been opened as far as the system is concerned.
> The output then blocks on the apparently not-open FIFO (I should be
> getting a SIGPIPE instead when trying to write into a pipe that has no
> listener).

I've checked again with the latest snapshot (2011-08-03), after doing a
full update and rebaseall.  While the FIFO implementation still does not
conform to POSIX w.r.t. blocking, it now gives consistent results again
(as it was in Cygwin 1.5), does not block listeners indefinitely anymore
and kills a listener with SIGPIPE when the producer goes away.  So if
the scripts are written to assume that behaviour, you can again use FIFO
to synchronize processes.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Wavetables for the Terratec KOMPLEXER:
http://Synth.Stromeko.net/Downloads.html#KomplexerWaves


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: [Bug] Named Pipes (FIFO) / Bash

[Achim Gratz]

Achim Gratz writes:
> I'm having problems with some bash scripts that were developed (not be
> me) and working OK in Cygwin 1.6.x versions, but not in 1.7.x (tested on
> 1.7.6, 1.7.7 and the latest snapshot).  After some gnashing of teeth and
> pulling of hair I've whittled it down to a problem with named pipe
> handling in bash.  Sifting through the archives of the mailing lists
> didn't provide much clues.  I realize from earlier discussions that
> named pipes aren't fully supported, but some examples of programs and
> scripts seem to indicate that they should at least work on a basic
> level.
>
> I've tried both the current and the just announced experimental version
> of bash and both Windows7/CoreDuo and WinXP/TurionX2 (I don't think the
> dual cores have something to do with the problem, but if necessary I
> should be able to at least boot the WinXP machine with only a single
> core enabled).
>
> $ mkfifo /tmp/pipe
> $ ( sleep 10; echo -n "blafasel "; date; sleep 10; cat < /tmp/pipe )&
> $ echo -n "dingdong "; date; exec 6> /tmp/pipe; echo -n "blumblum "; date
> $ echo -n "hurrdurr " >&6; date
>
> Running this on Linux outputs as expected:
>
> dingdong Sa 22. Jan 20:41:45 CET 2011
> blafasel Sa 22. Jan 20:41:55 CET 2011
> blumblum Sa 22. Jan 20:42:05 CET 2011
> hurrdurr Sa 22. Jan 20:42:05 CET 2011
>
> On Cygwin, I get this:
>
> dingdong Sa 22. Jan 20:41:45 CET 2011
> blumblum Sa 22. Jan 20:41:45 CET 2011
> blafasel Sa 22. Jan 20:41:55 CET 2011
> cat: -: Communication error on send

Blast from the past!  I've just tested this again with Cygwin 2.0 and it
finally works correctly.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptations for KORG EX-800 and Poly-800MkII V0.9:
http://Synth.Stromeko.net/Downloads.html#KorgSDada

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple