Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?

[Takashi Yano <takashi.yano@nifty.ne.jp>]

[-- Attachment #1: Type: text/plain, Size: 3634 bytes --]

On Tue, 31 Aug 2021 12:18:57 +0200
Corinna Vinschen wrote:
> On Aug 31 12:05, Corinna Vinschen wrote:
> > On Aug 31 18:25, Takashi Yano wrote:
> > > On Tue, 31 Aug 2021 11:08:42 +0200
> > > Corinna Vinschenwrote:
> > > > On Aug 31 17:55, Takashi Yano wrote:
> > > > > On Mon, 30 Aug 2021 22:14:15 +0200
> > > > > Corinna Vinschen wrote:
> > > > > > Hi Ken, Hi Takashi,
> > > > > > 
> > > > > > On Aug 30 19:00, Corinna Vinschen wrote:
> > > > > > Well, what about keeping a duplicate of the read side handle on the 
> > > > > > write side just for calling NtQueryInformationFile?
> > > > > > 
> > > > > > Attached is an untested patch, can you have a look if that makes sense?
> > > > > > 
> > > > > > Btw., I think I found a bug in the new fhandler_pipe::create.  If the
> > > > > > function fails to create the write side fhandler, it deletes the read
> > > > > > side fhandler, but neglects to close the read handle.  My patch fixes
> > > > > > that.
> > > > > > 
> > > > > > While looking into this I found a problem in fhandler_disk_file in
> > > > > > terms of handle inheritance of the special handle for pread/pwrite.
> > > > > > I already force pushed this onto topic/pipe.
> > > > > 
> > > > > I tested your patch attached. Unfortunately, select() does not work
> > > > > as expected for write pipe. Even if the select reports write pipe
> > > > > is available, writing to pipe fails. It seems that your patch fails
> > > > > to detect pipe full.
> > > > 
> > > > Bummer.  Is that with byte mode pipes or with message mode pipes?  If
> > > > the latter, if you try to write more data than available in the buffer,
> > > > it's bound to fail.
> > > 
> > > Both message pipe and byte pipe.
> > > 
> > > > Did you add debug output to pipe_data_available to see how the
> > > > information looks like?  Or do you have a simple, self-contained
> > > > testcase in plain C?
> > > 
> > > The test case is attached. If select() works as expected, the program
> > > does not show "r" or "w". However, with your patch, the program prints
> > > many "w" (means write() fails with EAGAIN).
> > 
> > Thanks!  I found th culprit, but we have another problem.  Even if
> > select returns correct info,  A write, trying to write more bytes
> > than are available in the buffer, hangs.  This shouldn't happen.
> > Still digging...
> 
> That's, of course, correct behaviour for pipes in blocking mode.  D'oh! 
> 
> Please try the attached patch on top of topic/pipe.

Thanks for the new patch. I have confirmed that above issue
is fixed and select() for write pipe seems to work as expected.


BTW, I found one minor difference between Linux and this pipe
implementation.

The test case is attached. The test case uses non-bloking I/O.
If this STC runs on Linux, the result is:

1024/1024
1740/1740
2958/2958
5028/5028
8547/8547
14529/14529
24699/24699
41988/41988
22227/71379
65536/121344
65536/206284
Total: 247KB in 0.000612 second, 403517.628166KB/s

On cygwin 3.2.0, the result is similar to Linux.

1024/1024
1740/1740
2957/2957
5026/5026
8544/8544
14524/14524
24690/24690
41972/41972
65536/71352
65536/121298
65536/206206
Total: 290KB in 0.062653 second, 4628.669018KB/s


However, on topic/pipe implementation, the result is

1024/1024
1740/1740
2957/2957
5026/5026
8544/8544
14524/14524
24690/24690
-1/41972
w-1/71352
w-1/121298
w-1/206206
wTotal: 57KB in 0.000330 second, 172989.377845KB/s

In non-blocking mode, writing more than pipe space will fail with
EAGAIN in this implementation.

In Linux and cygwin 3.2.0, it seems to write as much as writable.

Is this difficult to be fixed?

-- 
Takashi Yano <takashi.yano@nifty.ne.jp>

[-- Attachment #2: stc3.c --]
[-- Type: text/x-csrc, Size: 1786 bytes --]

#include <unistd.h>
#include <sys/wait.h>
#include <stdio.h>
#include <time.h>
#include <sys/select.h>
#include <fcntl.h>
#include <errno.h>

#define BLKSIZ (65536*4)
#define NBLK (100*(1<<20)/BLKSIZ)

int main()
{
	int fd[2];
	pid_t pid;
	int nonblock = 1;

	pipe(fd);

	if (!(pid = fork ())) {
		int i;
		char buf[BLKSIZ] = {0,};
		close(fd[0]);
		if (nonblock) {
			int flags;
			flags = fcntl(fd[1], F_GETFL);
			flags |= O_NONBLOCK;
			fcntl(fd[1], F_SETFL, flags);
		}
		fd_set wfds;
		for (int wlen=1024; wlen<=BLKSIZ; wlen *= 1.7) {
			FD_ZERO(&wfds);
			FD_SET(fd[1], &wfds);
			if (select(fd[1]+1, NULL, &wfds, NULL, NULL) > 0
					&& FD_ISSET(fd[1], &wfds)) {
				ssize_t len = write(fd[1], buf, wlen);
				printf("%d/%d\n", len, wlen);
				if (len < 0 && errno == EAGAIN) printf("w", i);
				if (len < 0 && errno == EAGAIN) i --;
			}
		}
		close(fd[1]);
	} else {
		char buf[BLKSIZ] = {0,};
		int total = 0;
		fd_set rfds;
		struct timespec tv0, tv1;
		double elasped;
		close(fd[1]);
		if (nonblock) {
			int flags;
			flags = fcntl(fd[0], F_GETFL);
			flags |= O_NONBLOCK;
			fcntl(fd[0], F_SETFL, flags);
		}
		usleep(1000000); /* Delay to start reader */
		clock_gettime(CLOCK_MONOTONIC, &tv0);
		for (;;) {
			FD_ZERO(&rfds);
			FD_SET(fd[0], &rfds);
			if (select(fd[0]+1, &rfds, NULL, NULL, NULL) > 0
					&& FD_ISSET(fd[0], &rfds)) {
				ssize_t len = read(fd[0], buf, sizeof(buf));
				if (len < 0 && errno == EAGAIN) printf("r");
				else if (len <= 0) break;
				else total += len;
			}
		}
		clock_gettime(CLOCK_MONOTONIC, &tv1);
		close(fd[0]);
		elasped = (tv1.tv_sec - tv0.tv_sec) + (tv1.tv_nsec - tv0.tv_nsec)*1e-9;
		printf("Total: %dKB in %f second, %fKB/s\n",
			total/(1<<10), elasped, total/(1<<10)/elasped);
		waitpid(pid, NULL, 0);
	}

	return 0;
}