Cygwin on Wine: changing buffer content immediately after aync write() to a pipe corrupts written result

[Qian Hong <fracting@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 2379 bytes --]

Dear list,

Recently we are investigating a Wine (Wine Staging) bug encountered by
Cygwin/MSYS2, which also suggest a potential bug in Cygwin.

When running xz on Cygwin/MSYS2 on Wine to decompress data and write
to a pipe, we found data corruption in the final result. This is
reported as Wine Staging bug 394:

https://bugs.wine-staging.com/show_bug.cgi?id=394

The command line to reproduce the bug is below:
$ cd /usr/share/gettext
$ xz -d -c < archive.dir.tar.xz | sha1sum # output result is random on Wine

Tested version:

Cygwin version
2.1.0, 2015/07/14 C:\cygwin\bin\cygwin1.dll

XZ version
$ xz --version
xz (XZ Utils) 5.2.1
liblzma 5.2.1

Wine Staging:
$ wine --version
wine-1.7.49-717-g3404140 (Staging)

In case anyone interesting, we also tested on msys2:
MSYS_NT-5.1  2.2.1(0.289/5/3) 2015-08-09 14:43 i686 Msys

***

With help from Lazhu (xz developer), Corrina and Sebastian, the
problem become a bit clearer. For archive, here are some testing
results we got:


1. This problem can't be reproduce on WinXP or Win7 according to my tests.

2. This problem doesn't happen with xz 5.0.8. Lazhu pointed out a
relate issue which doesn't happen in xz 5.0.8 but discovered by 5.2.1:
https://cygwin.com/ml/cygwin/2015-02/msg00575.html

That's very helpful information, finally we made sure the two problem
are different but get a clear idea what might be wrong.

3. Modified from Lazhu's previous test case, we wrote a new test case
to simulate the new problem, as attachment writer.c:
$ gcc writer.c -o writer
$ ./writer | sha1sum # the result is randomly on Wine but constant on Windows

The test case reproduce the problem on Wine, which generate randomly sha1sum.

If pipe is not used, then there is no data corruption:
$ ./write > good.txt && sha1sume good.txt # always good on Wine and Windows

However, the same test case doesn't reproduce anything wrong on WinXP
and Win7, sha1sum is always constant.

I tried modified buffer size, loop counts, etc inside the test case, I
also made sure EAGAIN is trigger on both Wine and Windows. I also
tried something like below to slow down reader speed:
$ ./write | pv -L512k | sha1sume

In any case, it's 100% reproduce on Wine, but never reproduce on Windows.

***

We are going to do some more test, will update this thread when we
have something interesting.

Thanks.


-- 
Regards,
Qian Hong

-
http://www.winehq.org

[-- Attachment #2: writer.c --]
[-- Type: text/x-csrc, Size: 1123 bytes --]

// gcc -std=gnu99 -Wall -Wextra writer.c -o writer

#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <unistd.h>
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include <errno.h>

static void
fail(const char *str)
{
	fprintf(stderr, "W: %s\n", str);
	exit(EXIT_FAILURE);
}

int
main(void)
{
	int flags = fcntl(STDOUT_FILENO, F_GETFL);
	if (flags == -1)
		fail("Cannot get stdout file status flags");

	if (fcntl(STDOUT_FILENO, F_SETFL, flags | O_NONBLOCK) == -1)
		fail("Setting stdout to non-blocking mode failed");

	static unsigned char buf[64 << 10];
	int blocking = 0;
	int i = 0;

	while (i < 256)
	{
		int ret = write(STDOUT_FILENO, buf, sizeof(buf));
		if (ret == -1 && errno == EAGAIN)
		{
			blocking++;
			continue;
		}

		if (ret != sizeof(buf))
			fail("Unexpected number of bytes");

		memset(buf, ++i, sizeof(buf));
	}

	if (!blocking)
		fail("Was never blocking, read was too fast");

	if (fcntl(STDOUT_FILENO, F_SETFL, flags) == -1)
		fail("Restoring stdout file status flags failed");

	if (close(STDOUT_FILENO))
		fail("Error closing stdout");

	return EXIT_SUCCESS;
}

[-- Attachment #3: Type: text/plain, Size: 218 bytes --]

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple