1.7.8: write fails with EAGAIN

1.7.8: write fails with EAGAIN

[Robert Wruck]

[-- Attachment #1: Type: text/plain, Size: 1444 bytes --]

Hi,

recently, I found that cygwin-git was not able to 'cat-file' files that 
exceeded some size (in my case about 80MB).
I tracked this down to the cygwin implementation of write() that behaves 
quite odd in some cases.

I wrote a small program (source attached) that mmaps a given file and 
tries to write it to another file or stdout.

The results vary:

If the destination is a file (`writetest infile outfile` or `writetest 
infile > outfile`), the write succeeds in a single call.

If the destination is a pipe (`writetest infile | cat > outfile`), the 
write succeeds in most cases. BUT:

Under WinXP (XP Service Pack 2, 32bit), the call returns -1 and 
errno=EAGAIN. Nevertheless, SOME data is written to the pipe (in my case 
4096 byte for each call).
This breaks git since it does an infinite loop while errno=EAGAIN.

It happens only for big files. On one machine, 87MB failed, on another 
one all went well up to ~200MB. It's not a disk space issue. It 
shouldn't be a memory issue - the test machine has 4GB.
I wasn't able to reproduce this on Win2k (32bit) nor Win7 (64bit) but 
maybe I didn't try enough file sizes...

cygwin version is cygwin-1_7_8-release, actually a fresh install made today.

Any ideas where to investigate further? EAGAIN is set for example in 
fhandler.cc, fhandler_base_overlapped::has_ongoing_io but I don't know 
whether that function is involved in my case.

Please Cc me, I'm not subscribed.

-Robert

[-- Attachment #2: writetest.c --]
[-- Type: text/x-csrc, Size: 1193 bytes --]

#include <stdio.h>
#include <errno.h>
#include <unistd.h>
#include <fcntl.h>
#include <sys/mman.h>
#include <sys/stat.h>

int main(int argc, char** argv)
	{
	if (argc < 2)
		{
		fprintf(stderr, "wrong usage\n");
		return (1);
		}
	
	int fd;
	if ((fd = open(argv[1], O_RDONLY)) < 0)
		{
		perror("open");
		return (1);
		}
	
	struct stat st;
	if (fstat(fd, &st) < 0)
		{
		perror("fstat");
		close(fd);
		return (1);
		}
	size_t len = st.st_size;
	
	void* buf;
	if ((buf = mmap(NULL, len, PROT_READ, MAP_PRIVATE, fd, 0)) == MAP_FAILED)
		{
		perror("mmap");
		close(fd);
		return (1);
		}
	
	int fd2;
	if (argc > 2)
		{
		if ((fd2 = open(argv[2], O_WRONLY | O_CREAT | O_EXCL, 0644)) < 0)
			{
			perror("open");
			munmap(buf, len);
			close(fd);
			return (1);
			}
		}
	else
		fd2 = 1;
	
	void* writebuf = buf;
	size_t to_write = len;
	
	while (to_write)
		{
		fprintf(stderr, "writing %u bytes...\n", to_write);
		ssize_t written = write(fd2, writebuf, to_write);
		int err = errno;
		fprintf(stderr, "result is %i, errno is %i\n", written, err);
		if (written > 0)
			{
			to_write -= written;
			writebuf += written;
			}
		}
	
	close(fd2);
	munmap(buf, len);
	close(fd);
	return (0);
	}


[-- Attachment #3: Type: text/plain, Size: 218 bytes --]

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: 1.7.8: write fails with EAGAIN

[Corinna Vinschen]

On Mar  5 21:12, Robert Wruck wrote:
> Hi,
> 
> recently, I found that cygwin-git was not able to 'cat-file' files
> that exceeded some size (in my case about 80MB).
> I tracked this down to the cygwin implementation of write() that
> behaves quite odd in some cases.
> 
> I wrote a small program (source attached) that mmaps a given file
> and tries to write it to another file or stdout.
> 
> The results vary:
> 
> If the destination is a file (`writetest infile outfile` or
> `writetest infile > outfile`), the write succeeds in a single call.
> 
> If the destination is a pipe (`writetest infile | cat > outfile`),
> the write succeeds in most cases. BUT:
> 
> Under WinXP (XP Service Pack 2, 32bit), the call returns -1 and
> errno=EAGAIN. Nevertheless, SOME data is written to the pipe (in my
> case 4096 byte for each call).
> This breaks git since it does an infinite loop while errno=EAGAIN.

Hang on, you are saying that a *blocking* write(2) to a pipe returns
with EAGAIN?  Are you sure?  It would be quite a surprise if git would
actually do that.  EAGAIN is only an expected error for non-blocking
I/O, so applications which use blocking I/O usually only test for EINTR.

Where you able to reproduce the issue with the testcase you attached to
your mail?  If so, you forgot to add a hint how to use it and an example
of the expected outcome.  That would be quite helpful.

> Any ideas where to investigate further? EAGAIN is set for example in
> fhandler.cc, fhandler_base_overlapped::has_ongoing_io but I don't
> know whether that function is involved in my case.

It is.  The overlapped io stuff is only used for pipes and fifos right
now.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Project Co-Leader          cygwin AT cygwin DOT com
Red Hat

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: 1.7.8: write fails with EAGAIN

[Robert Wruck]

Hi,

> Hang on, you are saying that a *blocking* write(2) to a pipe returns
> with EAGAIN?  Are you sure?  It would be quite a surprise if git would
> actually do that.  EAGAIN is only an expected error for non-blocking
> I/O, so applications which use blocking I/O usually only test for EINTR.

since in the interesting case I'm writing to stdout (fd=1) I don't know 
whether that was initially opened with O_NONBLOCK - but I wouldn't think so.

I was able to reproduce this as follows:

./writetest infile | cat > outfile

where infile is a "large" file.
The output is:

$ ./writetest infile | cat > test.out
writing 78954543 bytes...
result is -1, errno is 11
writing 78954543 bytes...
result is -1, errno is 11
...

This does not happen for "small" files:

$ ./writetest infile2 | cat > test.out
writing 10978848 bytes...
result is 10978848, errno is 0

Also, I tried this with the same cygwin version on Win7/64-bit and this 
did not happen even for a 1GB file.


-Robert

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: 1.7.8: write fails with EAGAIN

[Christopher Faylor]

On Mon, Mar 07, 2011 at 11:39:51AM +0100, Corinna Vinschen wrote:
>On Mar  5 21:12, Robert Wruck wrote:
>> Hi,
>> 
>> recently, I found that cygwin-git was not able to 'cat-file' files
>> that exceeded some size (in my case about 80MB).
>> I tracked this down to the cygwin implementation of write() that
>> behaves quite odd in some cases.
>> 
>> I wrote a small program (source attached) that mmaps a given file
>> and tries to write it to another file or stdout.
>> 
>> The results vary:
>> 
>> If the destination is a file (`writetest infile outfile` or
>> `writetest infile > outfile`), the write succeeds in a single call.
>> 
>> If the destination is a pipe (`writetest infile | cat > outfile`),
>> the write succeeds in most cases. BUT:
>> 
>> Under WinXP (XP Service Pack 2, 32bit), the call returns -1 and
>> errno=EAGAIN. Nevertheless, SOME data is written to the pipe (in my
>> case 4096 byte for each call).
>> This breaks git since it does an infinite loop while errno=EAGAIN.
>
>Hang on, you are saying that a *blocking* write(2) to a pipe returns
>with EAGAIN?  Are you sure?  It would be quite a surprise if git would
>actually do that.  EAGAIN is only an expected error for non-blocking
>I/O, so applications which use blocking I/O usually only test for EINTR.

I can barely convince myself that there's a pathological case where an
EAGAIN could leak out.  I'm investigating now.

cgf

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: 1.7.8: write fails with EAGAIN

[Christopher Faylor]

On Mon, Mar 07, 2011 at 10:37:08AM -0500, Christopher Faylor wrote:
>On Mon, Mar 07, 2011 at 11:39:51AM +0100, Corinna Vinschen wrote:
>>On Mar  5 21:12, Robert Wruck wrote:
>>> Hi,
>>> 
>>> recently, I found that cygwin-git was not able to 'cat-file' files
>>> that exceeded some size (in my case about 80MB).
>>> I tracked this down to the cygwin implementation of write() that
>>> behaves quite odd in some cases.
>>> 
>>> I wrote a small program (source attached) that mmaps a given file
>>> and tries to write it to another file or stdout.
>>> 
>>> The results vary:
>>> 
>>> If the destination is a file (`writetest infile outfile` or
>>> `writetest infile > outfile`), the write succeeds in a single call.
>>> 
>>> If the destination is a pipe (`writetest infile | cat > outfile`),
>>> the write succeeds in most cases. BUT:
>>> 
>>> Under WinXP (XP Service Pack 2, 32bit), the call returns -1 and
>>> errno=EAGAIN. Nevertheless, SOME data is written to the pipe (in my
>>> case 4096 byte for each call).
>>> This breaks git since it does an infinite loop while errno=EAGAIN.
>>
>>Hang on, you are saying that a *blocking* write(2) to a pipe returns
>>with EAGAIN?  Are you sure?  It would be quite a surprise if git would
>>actually do that.  EAGAIN is only an expected error for non-blocking
>>I/O, so applications which use blocking I/O usually only test for EINTR.
>
>I can barely convince myself that there's a pathological case where an
>EAGAIN could leak out.  I'm investigating now.

Actually, in this case, it looks like the problem is that Windows
doesn't like sending a huge buffer to a pipe.  The errno in this case
should probably be something like EFBIG rather than EAGAIN.

Does git deal with this type of errno gracefully or does it just abort
if the write() fails for any reason?  I'd rather just fail and let the
caller deal with it than complicate Cygwin's code by trying to loop
writing smaller amount of data to the pipe so I'd prefer just changing
the errno if that solves the problem.

cgf

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Please try the latest snapshot [Re: 1.7.8: write fails with EAGAIN]

[Christopher Faylor]

On Mon, Mar 07, 2011 at 11:38:49AM -0500, Christopher Faylor wrote:
>On Mon, Mar 07, 2011 at 10:37:08AM -0500, Christopher Faylor wrote:
>>On Mon, Mar 07, 2011 at 11:39:51AM +0100, Corinna Vinschen wrote:
>>>On Mar  5 21:12, Robert Wruck wrote:
>>>> Hi,
>>>> 
>>>> recently, I found that cygwin-git was not able to 'cat-file' files
>>>> that exceeded some size (in my case about 80MB).
>>>> I tracked this down to the cygwin implementation of write() that
>>>> behaves quite odd in some cases.
>>>> 
>>>> I wrote a small program (source attached) that mmaps a given file
>>>> and tries to write it to another file or stdout.
>>>> 
>>>> The results vary:
>>>> 
>>>> If the destination is a file (`writetest infile outfile` or
>>>> `writetest infile > outfile`), the write succeeds in a single call.
>>>> 
>>>> If the destination is a pipe (`writetest infile | cat > outfile`),
>>>> the write succeeds in most cases. BUT:
>>>> 
>>>> Under WinXP (XP Service Pack 2, 32bit), the call returns -1 and
>>>> errno=EAGAIN. Nevertheless, SOME data is written to the pipe (in my
>>>> case 4096 byte for each call).
>>>> This breaks git since it does an infinite loop while errno=EAGAIN.
>>>
>>>Hang on, you are saying that a *blocking* write(2) to a pipe returns
>>>with EAGAIN?  Are you sure?  It would be quite a surprise if git would
>>>actually do that.  EAGAIN is only an expected error for non-blocking
>>>I/O, so applications which use blocking I/O usually only test for EINTR.
>>
>>I can barely convince myself that there's a pathological case where an
>>EAGAIN could leak out.  I'm investigating now.
>
>Actually, in this case, it looks like the problem is that Windows
>doesn't like sending a huge buffer to a pipe.  The errno in this case
>should probably be something like EFBIG rather than EAGAIN.
>
>Does git deal with this type of errno gracefully or does it just abort
>if the write() fails for any reason?  I'd rather just fail and let the
>caller deal with it than complicate Cygwin's code by trying to loop
>writing smaller amount of data to the pipe so I'd prefer just changing
>the errno if that solves the problem.

I've just uploaded a snapshot which attempts to work around this
problem.  I ended up making some fairly substantial changes to the
pipe handling so this needs some serious testing, especially on
different platforms, e.g., XP, XP64, Windows 2008, etc.

Please try the latest snapshot at:

http://cygwin.com/snapshots/

cgf

cgf

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: 1.7.8: write fails with EAGAIN

[Robert Wruck]

Hi,

> Hang on, you are saying that a *blocking* write(2) to a pipe returns
> with EAGAIN?  Are you sure?  It would be quite a surprise if git would
> actually do that.  EAGAIN is only an expected error for non-blocking
> I/O, so applications which use blocking I/O usually only test for EINTR.

I checked out the winsup sources and built a cygwin1.dll for debugging 
(btw: Could you update the FAQ to state that gcc4 is required instead of 
gcc?).

What I found out:

In fhandler.cc, write_overlapped, there is the call to WriteFile:

	bool res = WriteFile (get_output_handle (), ptr, len, &nbytes,
			      get_overlapped ());

Surprisingly, this returns FALSE / nbytes = 0 / GetLastError = 
ERROR_INVALID_HANDLE on WinXP if the number of bytes exceeds some 
maximum and the handle refers to a pipe.
Since you don't know what kind of handle you're writing to in 
write_overlapped, it seems reasonable to do the write in chunks.

The first comment to WriteFile 
(http://msdn.microsoft.com/en-us/library/aa365747%28v=vs.85%29.aspx) 
suggests to limit the size to less than 32MB per write - although I 
didn't have this problem on X64.

-Robert

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: 1.7.8: write fails with EAGAIN

[Christopher Faylor]

On Wed, Mar 09, 2011 at 07:24:11PM +0100, Robert Wruck wrote:
>On Mon, Mar 07, 2011 at 11:39:51AM +0100, Corinna Vinschen wrote
>>Hang on, you are saying that a *blocking* write(2) to a pipe returns
>>with EAGAIN?  Are you sure?  It would be quite a surprise if git would
>>actually do that.  EAGAIN is only an expected error for non-blocking
>>I/O, so applications which use blocking I/O usually only test for
>>EINTR.
>
>
>What I found out:
>
>In fhandler.cc, write_overlapped, there is the call to WriteFile:
>
>	bool res = WriteFile (get_output_handle (), ptr, len, &nbytes,
>			      get_overlapped ());
>
>Surprisingly, this returns FALSE / nbytes = 0 / GetLastError = 
>ERROR_INVALID_HANDLE on WinXP if the number of bytes exceeds some 
>maximum and the handle refers to a pipe.

Knock, knock.  Is this thing on?

http://cygwin.com/ml/cygwin/2011-03/msg00226.html
http://cygwin.com/ml/cygwin/2011-03/msg00280.html

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: 1.7.8: write fails with EAGAIN

[Larry Hall (Cygwin)]

On 3/9/2011 1:24 PM, Robert Wruck wrote:
> Hi,
>
>> Hang on, you are saying that a *blocking* write(2) to a pipe returns
>> with EAGAIN? Are you sure? It would be quite a surprise if git would
>> actually do that. EAGAIN is only an expected error for non-blocking
>> I/O, so applications which use blocking I/O usually only test for EINTR.
>
> I checked out the winsup sources and built a cygwin1.dll for debugging (btw:
>  Could you update the FAQ to state that gcc4 is required instead of gcc?).
>
> What I found out:
>
> In fhandler.cc, write_overlapped, there is the call to WriteFile:
>
> bool res = WriteFile (get_output_handle (), ptr, len, &nbytes, get_overlapped
> ());
>
> Surprisingly, this returns FALSE / nbytes = 0 / GetLastError =
> ERROR_INVALID_HANDLE on WinXP if the number of bytes exceeds some maximum and
>  the handle refers to a pipe. Since you don't know what kind of handle you're
> writing to in write_overlapped, it seems reasonable to do the write in
> chunks.

<http://cygwin.com/ml/cygwin/2011-03/msg00280.html>

-- 
Larry

_____________________________________________________________________

A: Yes.
> Q: Are you sure?
>> A: Because it reverses the logical flow of conversation.
>>> Q: Why is top posting annoying in email?

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: 1.7.8: write fails with EAGAIN

[Robert Wruck]

> Surprisingly, this returns FALSE / nbytes = 0 / GetLastError =
> ERROR_INVALID_HANDLE on WinXP if the number of bytes exceeds some
> maximum and the handle refers to a pipe.
> Since you don't know what kind of handle you're writing to in
> write_overlapped, it seems reasonable to do the write in chunks.

Sorry, I made a mistake.
The value returned by GetLastError() is 1450 / ERROR_NO_SYSTEM_RESOURCES.


-Robert

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: 1.7.8: write fails with EAGAIN

[Robert Wruck]

 > I've just uploaded a snapshot which attempts to work around this
 > problem.  I ended up making some fairly substantial changes to the
 > pipe handling so this needs some serious testing, especially on
 > different platforms, e.g., XP, XP64, Windows 2008, etc.

Oops..

I replaced the cygwin1.dll with cygwin1-20110309.dll and now a single 
write() with 78 MB never returns but seems to write repeatedly to the 
pipe (file was 5GB when I hit Ctrl-C).

-Robert

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: 1.7.8: write fails with EAGAIN

[Robert Wruck]

> I replaced the cygwin1.dll with cygwin1-20110309.dll and now a single
> write() with 78 MB never returns but seems to write repeatedly to the
> pipe (file was 5GB when I hit Ctrl-C).

... you don't want to keep_looping after calling 
write_overlapped_fallback ...

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: 1.7.8: write fails with EAGAIN

[Robert Wruck]

>> I replaced the cygwin1.dll with cygwin1-20110309.dll and now a single
>> write() with 78 MB never returns but seems to write repeatedly to the
>> pipe (file was 5GB when I hit Ctrl-C).

Just terminating the loop after write_overlapped_fallback is not enough.
When the size to write exceeds MAX_OVERLAPPED_WRITE_LEN and WriteFile 
fails for MAX_OVERLAPPED_WRITE_LEN, write() will always return 0.

In case of overlapped_error, nbytes should be set to -1.

I'd suggest to remove write_overlapped_fallback and instead decrement 
'len' on each failure. Since write is not guaranteed to write the whole 
data, this should be OK.

       do
	{
	  bool res = WriteFile (get_output_handle (), ptr, len, &nbytes,
				get_overlapped ());
	  switch (wait_overlapped (res, true, &nbytes, is_nonblocking (), 
(size_t) len))
	    {
	    case overlapped_fallback:
		  len /= 4; /* feel free to tune */
		  if (!len) {
		    set_errno (ENOSPC); /* Can't write anything */
		    nbytes = (DWORD) -1; /* Don't forget */
		    keep_looping = false;
		    break;
		  }
	      /* fall through intentionally */;
	    case overlapped_signal:
	      keep_looping = true;
	      break;
	    default:	/* Added to quiet gcc */
	    case overlapped_error:
	      nbytes = (DWORD) -1; /* Don't forget */
	      /* fall through intentionally */;
	    case overlapped_success:
	      keep_looping = false;
	      break;
	    }
	}
       while (keep_looping);

Result (from the application's perspective) is:

writing 78954543 bytes...
result is 19738635, errno is 0
writing 59215908 bytes...
result is 14803977, errno is 0
writing 44411931 bytes...
result is 11102982, errno is 0
writing 33308949 bytes...
result is 8327237, errno is 0
writing 24981712 bytes...
result is 24981712, errno is 0

-Robert

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: 1.7.8: write fails with EAGAIN

[Christopher Faylor]

On Wed, Mar 09, 2011 at 08:46:43PM +0100, Robert Wruck wrote:
>>> I replaced the cygwin1.dll with cygwin1-20110309.dll and now a single
>>> write() with 78 MB never returns but seems to write repeatedly to the
>>> pipe (file was 5GB when I hit Ctrl-C).
>
>Just terminating the loop after write_overlapped_fallback is not enough.
>When the size to write exceeds MAX_OVERLAPPED_WRITE_LEN and WriteFile 
>fails for MAX_OVERLAPPED_WRITE_LEN, write() will always return 0.
>
>In case of overlapped_error, nbytes should be set to -1.

In theory that's a situation that the caller would never see but I
will fix that.

>I'd suggest to remove write_overlapped_fallback and instead decrement 
>'len' on each failure. Since write is not guaranteed to write the whole 
>data, this should be OK.

The goal is to emulate what Linux does.  Linux succeeds when writing
huge blocks to a non-blocking pipe.  Since the code is already there
(modulo the bugs that you've noted) to do the same thing with Cygwin I'm
not going to back out what I've done and reduce the functionality.

I'm going to fix the bugs.

cgf

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: 1.7.8: write fails with EAGAIN

[Robert Wruck]

Hi,

tested with cvs HEAD:

> Just terminating the loop after write_overlapped_fallback is not enough.
> When the size to write exceeds MAX_OVERLAPPED_WRITE_LEN and WriteFile
> fails for MAX_OVERLAPPED_WRITE_LEN, write() will always return 0.

Now any write with length > MAX_OVERLAPPED_WRITE_LEN fails with -1 / 
EFBIG (on the machine that has the WriteFile limitation).

-Robert

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: 1.7.8: write fails with EAGAIN

[Christopher Faylor]

On Thu, Mar 10, 2011 at 09:11:52PM +0100, Robert Wruck wrote:
>Hi,
>
>tested with cvs HEAD:
>
>> Just terminating the loop after write_overlapped_fallback is not enough.
>> When the size to write exceeds MAX_OVERLAPPED_WRITE_LEN and WriteFile
>> fails for MAX_OVERLAPPED_WRITE_LEN, write() will always return 0.
>
>Now any write with length > MAX_OVERLAPPED_WRITE_LEN fails with -1 / 
>EFBIG (on the machine that has the WriteFile limitation).

How about if you try a snapshot rather than your own home-built version?

cgf

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: 1.7.8: write fails with EAGAIN

[Christopher Faylor]

On Thu, Mar 10, 2011 at 04:11:41PM -0500, Christopher Faylor wrote:
>On Thu, Mar 10, 2011 at 09:11:52PM +0100, Robert Wruck wrote:
>>Hi,
>>
>>tested with cvs HEAD:
>>
>>> Just terminating the loop after write_overlapped_fallback is not enough.
>>> When the size to write exceeds MAX_OVERLAPPED_WRITE_LEN and WriteFile
>>> fails for MAX_OVERLAPPED_WRITE_LEN, write() will always return 0.
>>
>>Now any write with length > MAX_OVERLAPPED_WRITE_LEN fails with -1 / 
>>EFBIG (on the machine that has the WriteFile limitation).
>
>How about if you try a snapshot rather than your own home-built version?

I've just run your test script a number of times and saw a failure a couple
of times.  It's a race condition that is fixable.

The irritating thing that I noticed is that, despite returning an error,
WriteFile nevertheless still writes 64K of the data.  That's pretty
annoying and is something that I have to fix.

cgf

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: 1.7.8: write fails with EAGAIN

[Robert Wruck]

> How about if you try a snapshot rather than your own home-built
> version?

There is no snapshot dated 03-10 on http://cygwin.com/snapshots/.

Unfortunately, there is no "hard limit" of 64MB.
On the machine where I first encountered the error, the limit seems to 
be about 16MB. On another machine, it's higher, about 200MB.
When the real limit is less than MAX_OVERLAPPED_WRITE_LEN, 
write_overlapped_fallback cannot succeed.

-Robert

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: 1.7.8: write fails with EAGAIN

[Robert Wruck]

Hi,

just re-tested this with cygwin1-20110313.dll.
Here is the result (output of my "writetest" program):

writing 78954543 bytes...
result is 32505856, errno is 27
writing 46448687 bytes...
result is -1, errno is 27
writing 46448687 bytes...
result is -1, errno is 27
[ repeating a few hundred times ]
writing 46448687 bytes...
result is 46448687, errno is 27

The total number of write attempts varies, on my test machine it's 
between 400 and 600.
The resulting output file is correct.

I also had _one_ case where the program seemed to be stuck but I was not 
able to reproduce this one:

writing 78954543 bytes...
result is 32505856, errno is 27
writing 46448687 bytes...
result is -1, errno is 27
writing 46448687 bytes...
result is -1, errno is 27
[ repeating a few hundred times ]
writing 46448687 bytes...
result is 32505856, errno is 27
writing 13942831 bytes...
result is 0, errno is 27
writing 13942831 bytes...
result is 0, errno is 27
[ repeating until Ctrl-C ]


-Robert

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple