Re: needed feature in Cygwin fifos

Re: needed feature in Cygwin fifos

[bob 295]

>>>On Tue, Apr 26, 2011 at 01:34:16PM -0400, bob 295 wrote:
>>>>I recognize that Cygwin fifos are "buggy and not suitable for anything but 
>>>>simplest of applications",  however in the spirit of seeing if things can 
be 
>>>>improved here is some more test code which illustrates one of the "bugs".
>>
>>>Maybe I need to be clearer: I don't need demonstrations of bugs.  I know
>>>what doesn't work.
>>
>>>The fifo layer needs a total rewrite right now.  Sending this type of email
>>>is a waste of your time.
>>
>>>cgf
>>
>>Fair enough.   Is that total rewrite underway?  If so any idea when we'll 
see 
>>a workable version?  If not what would it take to expedite things?  More 
>>developer resources?  What type of skillset would you need?

>I'm not currently actively working on it.  I've just been contemplating
>various methods for dealing with the issues.  The bottom line is that I
>thought that Windows Named pipes would be the perfect way to implement
>Cygwin's fifos and it turns out they are actually not well suited for
>that task.

>This is a free software project so you can see for yourself what's
>needed.  The majority of the code is in fhandler_fifo.cc.

>cgf

Sounds like a perfect candidate for a Google Summer of Code project proposal.

bob

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: needed feature in Cygwin fifos

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 566 bytes --]

On 04/30/2011 05:31 AM, bob 295 wrote:
>> This is a free software project so you can see for yourself what's
>> needed.  The majority of the code is in fhandler_fifo.cc.
> 
>> cgf
> 
> Sounds like a perfect candidate for a Google Summer of Code project proposal.

Not this year; the deadline's already passed for 2011.  And I hope that
if someone wants to pitch in and help cgf with a rewrite that they would
do so before waiting for SoC 2012.

-- 
Eric Blake   eblake@redhat.com    +1-801-349-2682
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 619 bytes --]

Re: needed feature in Cygwin fifos

[Christopher Faylor]

On Tue, Apr 26, 2011 at 02:31:19PM -0400, bob 295 wrote:
>>On Tue, Apr 26, 2011 at 01:34:16PM -0400, bob 295 wrote:
>>>I recognize that Cygwin fifos are "buggy and not suitable for anything but 
>>>simplest of applications",  however in the spirit of seeing if things can be 
>>>improved here is some more test code which illustrates one of the "bugs".
>
>>Maybe I need to be clearer: I don't need demonstrations of bugs.  I know
>>what doesn't work.
>
>>The fifo layer needs a total rewrite right now.  Sending this type of email
>>is a waste of your time.
>
>>cgf
>
>Fair enough.   Is that total rewrite underway?  If so any idea when we'll see 
>a workable version?  If not what would it take to expedite things?  More 
>developer resources?  What type of skillset would you need?

I'm not currently actively working on it.  I've just been contemplating
various methods for dealing with the issues.  The bottom line is that I
thought that Windows Named pipes would be the perfect way to implement
Cygwin's fifos and it turns out they are actually not well suited for
that task.

This is a free software project so you can see for yourself what's
needed.  The majority of the code is in fhandler_fifo.cc.

cgf

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: needed feature in Cygwin fifos

[bob 295]

>On Tue, Apr 26, 2011 at 01:34:16PM -0400, bob 295 wrote:
>>I recognize that Cygwin fifos are "buggy and not suitable for anything but 
>>simplest of applications",  however in the spirit of seeing if things can be 
>>improved here is some more test code which illustrates one of the "bugs".

>Maybe I need to be clearer: I don't need demonstrations of bugs.  I know
>what doesn't work.

>The fifo layer needs a total rewrite right now.  Sending this type of email
>is a waste of your time.

>cgf

Fair enough.   Is that total rewrite underway?  If so any idea when we'll see 
a workable version?  If not what would it take to expedite things?  More 
developer resources?  What type of skillset would you need?

Thanks again.

bob


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: needed feature in Cygwin fifos

[Christopher Faylor]

On Tue, Apr 26, 2011 at 01:34:16PM -0400, bob 295 wrote:
>I recognize that Cygwin fifos are "buggy and not suitable for anything but 
>simplest of applications",  however in the spirit of seeing if things can be 
>improved here is some more test code which illustrates one of the "bugs".

Maybe I need to be clearer: I don't need demonstrations of bugs.  I know
what doesn't work.

The fifo layer needs a total rewrite right now.  Sending this type of email
is a waste of your time.

cgf

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

needed feature in Cygwin fifos

[bob 295]

I recognize that Cygwin fifos are "buggy and not suitable for anything but 
simplest of applications",  however in the spirit of seeing if things can be 
improved here is some more test code which illustrates one of the "bugs".

It seems that on Linux if you open a fifo for RDWR it will "trick" the system 
into supressing the EOF when a writer closes the other end of the fifo.  On 
the Cygwin system this EOF get generated regardless.  The Linux behavior is a 
desired feature for many fifo applications, including the one I'm trying to 
port to Cygwin, because many writers to a single fifo is a common 
configuration.  These writers can come and go (ie. close the fifo) at any 
time.   Generating an EOF in these conditions each time a writer closes its 
file descriptor is not logical.   It would appear that the Cygwin fifo gets 
knocked out of commission the first time an EOF gets generated.   ie. the 
first time a writer closes the other end of the pipe.   This strikes me as a 
bug rather than a feature.


========== begin sender code =========
#include <stdio.h>
#include <unistd.h>
#include <stdlib.h>
#include <stdarg.h>
#include <string.h>
#include <fcntl.h>
#include <errno.h>
#include <sys/select.h>

int blocked_read(int, char *, int);

int main(int argc, char *argv[])
{
char command[80];
char fifoname[80];
int fd=-1;
int rfd;
int i;
int j;
int *r;
int bytesToGo;
int numBytes;
char buf[5];
char *p;
int rc;

printf("fifo sender starting\n");

sprintf(fifoname,"/tmp/fsender");

mkfifo(fifoname, 0666);
chmod(fifoname, 0666);

printf("starting receiver\n");

sprintf(command,"./receiver&");
rc=system(command); 

printf("receiver started\n");

sleep(2);

fd=open(fifoname, O_RDWR|O_NONBLOCK);
//fd=open(fifoname, O_RDONLY|O_NONBLOCK);
rfd=open("/tmp/freceiver", O_WRONLY);

for(j=1; j<10; j++)
	{
printf("j=%d\n",j);

write(rfd,&j,sizeof(int));

numBytes=0;
memset(buf,0,4);
p=buf;
for(i=0; i< 10; i++)
	{
	bytesToGo=sizeof(int) - numBytes;

printf("bytesToGo=%d numBytes=%d\n",bytesToGo,numBytes);

	if(bytesToGo <= 0) break;

	rc=blocked_read(fd,p,bytesToGo);
printf("sender rc[%d]=%d\n",i,rc);
	if(rc == 0)
		{
printf("got eof\n");
//		close(fd);
//		fd=open(fifoname, O_RDONLY);
		}
	else
	if(rc == -1)
		{
		printf("%s\n",strerror(errno));
		}
	else
		{
		numBytes+=rc;
		p+=rc;
		}
	}

printf("buf: 0x%X-%X-%X-%X\n",buf[0],buf[1],buf[2],buf[3]);
r=(int *)buf;

printf("reply[%d]=%d\n",j,*r);
	}

sleep(1);

unlink(fifoname);
unlink("/tmp/freceiver");
	
exit(0);
}


int blocked_read(int fd, char *buff, int size)
{
int rc;
fd_set inset;

FD_ZERO(&inset);
FD_SET(fd, &inset);

printf("send: before select\n");
select(fd+1, &inset, NULL, NULL, NULL);
printf("send: before read\n");
rc=read(fd,buff,size);

printf("send: read rc=%d\n",rc);
return(rc);
}

========== end sender code ==========

========== begin receiver code =========
#include <stdio.h>
#include <unistd.h>
#include <stdlib.h>
#include <stdarg.h>
#include <string.h>
#include <fcntl.h>
#include <errno.h>
#include <sys/select.h>

int blocked_read(int, char *, int);

int main(int argc, char *argv[])
{
char command[80];
char fifoname[80];
int fd;
int rfd;
int dummy;
int i;
int j;
int k;
int *r;
int bytesToGo;
int numBytes;
char buf[4];
char *p;
int rc;

printf("fifo receiver starting\n");

sprintf(fifoname,"/tmp/freceiver");

mkfifo(fifoname, 0666);
chmod(fifoname, 0666);

fd=open(fifoname, O_RDWR | O_NONBLOCK);
//fd=open(fifoname, O_RDONLY | O_NONBLOCK);

for(k=0; k<10; k++)
	{

numBytes=0;
p=buf;
for(i=0; i< 10; i++)
	{
	bytesToGo=sizeof(int) - numBytes;

	if(bytesToGo <= 0) break;

//	rc=read(fd,p,bytesToGo);
	rc=blocked_read(fd,p,bytesToGo);
printf("recv rc[%d]=%d\n",i,rc);
	if(rc == 0)
		{
printf("recv eof\n");
		}
	else
	if(rc == -1)
		{
		printf("%s\n",strerror(errno));
		}
	else
		{
		numBytes+=rc;
		p+=rc;
		}
	}

r=(int *)buf;
j=*r;
printf("received[%d]=%d\n",k,j);
j++;

sleep(1);
	
rfd=open("/tmp/fsender", O_WRONLY);
write(rfd,&j,sizeof(int));
close(rfd);
	}

unlink(fifoname);

exit(0);
}

int blocked_read(int fd, char *buff, int size)
{
int rc;
fd_set inset;

FD_ZERO(&inset);
FD_SET(fd, &inset);

printf("recv: before select\n");
select(fd+1, &inset, NULL, NULL, NULL);
printf("recv: before read\n");
rc=read(fd,buff,size);

printf("recv: read rc=%d\n",rc);

return(rc);
}

========== end receiver code ==========

To run this test compile both sender.c and receiver.c and then run as
./sender

On my Linux system this is the result:

========== begin Linux result ============
fifo sender starting
starting receiver
fifo receiver starting
recv: before select
receiver started
j=1
bytesToGo=4 numBytes=0
send: before select
recv: before read
recv: read rc=4
recv rc[0]=4
received[0]=1
recv: before select
send: before read
send: read rc=4
sender rc[0]=4
bytesToGo=0 numBytes=4
buf: 0x2-0-0-0
reply[1]=2
j=2
bytesToGo=4 numBytes=0
send: before select
recv: before read
recv: read rc=4
recv rc[0]=4
received[1]=2
recv: before select
send: before read
send: read rc=4
sender rc[0]=4
bytesToGo=0 numBytes=4
buf: 0x3-0-0-0
reply[2]=3
j=3
bytesToGo=4 numBytes=0
send: before select
recv: before read
recv: read rc=4
recv rc[0]=4
received[2]=3
recv: before select
send: before read
send: read rc=4
sender rc[0]=4
bytesToGo=0 numBytes=4
buf: 0x4-0-0-0
reply[3]=4
j=4
bytesToGo=4 numBytes=0
send: before select
recv: before read
recv: read rc=4
recv rc[0]=4
received[3]=4
recv: before select
send: before read
send: read rc=4
sender rc[0]=4
bytesToGo=0 numBytes=4
buf: 0x5-0-0-0
reply[4]=5
j=5
bytesToGo=4 numBytes=0
send: before select
recv: before read
recv: read rc=4
recv rc[0]=4
received[4]=5
recv: before select
send: before read
send: read rc=4
sender rc[0]=4
bytesToGo=0 numBytes=4
buf: 0x6-0-0-0
reply[5]=6
j=6
bytesToGo=4 numBytes=0
send: before select
recv: before read
recv: read rc=4
recv rc[0]=4
received[5]=6
recv: before select
send: before read
send: read rc=4
sender rc[0]=4
bytesToGo=0 numBytes=4
buf: 0x7-0-0-0
reply[6]=7
j=7
bytesToGo=4 numBytes=0
send: before select
recv: before read
recv: read rc=4
recv rc[0]=4
received[6]=7
recv: before select
send: before read
send: read rc=4
sender rc[0]=4
bytesToGo=0 numBytes=4
buf: 0x8-0-0-0
reply[7]=8
j=8
bytesToGo=4 numBytes=0
send: before select
recv: before read
recv: read rc=4
recv rc[0]=4
received[7]=8
recv: before select
send: before read
send: read rc=4
sender rc[0]=4
bytesToGo=0 numBytes=4
buf: 0x9-0-0-0
reply[8]=9
j=9
bytesToGo=4 numBytes=0
send: before select
recv: before read
recv: read rc=4
recv rc[0]=4
received[8]=9
recv: before select
send: before read
send: read rc=4
sender rc[0]=4
bytesToGo=0 numBytes=4
buf: 0xA-0-0-0
reply[9]=10

========== end Linux result =============

On Cygwin the same code gives:

========== begin Cygwin result ============
fifo sender starting
starting receiver
receiver started
fifo receiver starting
recv: before select
j=1
bytesToGo=4 numBytes=0
recv: before read
send: before select
recv: read rc=4
recv rc[0]=4
received[0]=1
recv: before select
send: before read
send: read rc=4
sender rc[0]=4
bytesToGo=0 numBytes=4
buf: 0x2-0-0-0
reply[1]=2
j=2
bytesToGo=4 numBytes=0
recv: before read
send: before select
send: before read
send: read rc=0recv rc[0]=4
received[1]=20
got eof
bytesToGo=0 numBytes=4
send: before select
========== end Cygwin result =============

Other than the differences in sequencing of the printf's due to timing 
differences between the two environments,  the most obvious deviations 
between the Cygwin run and the Linux run are:

a) Cygwin run generates an EOF on the sender's fifo presumably because the 
receiver (the fifo writer) closes this fifo after writing a single integer 
response
b) the receiver gets a 20 instead of a 2 on second write sender -> receiver
c) the select call on the fifo hangs after the EOF is received

Interestingly on Linux if you switch the RDWR open for a RDONLY open in the 
sender code,  you generate the EOF but unlike Cygwin the select doesn't block 
and the EOF loops endlessly.   The only way you get the "desired" behavior on 
the Linux is to open the fifo RDWR.

Are there any plans in the Cygwin developer community to address this fifo 
issue in coming releases?   A quick Google reveals that similar fifo issues 
have persisted in Cygwin for at least a couple of years.

Thanks in advance for your responses.

bob

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple