Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?

[Takashi Yano <takashi.yano@nifty.ne.jp>]

[-- Attachment #1: Type: text/plain, Size: 1750 bytes --]

On Tue, 31 Aug 2021 11:08:42 +0200
Corinna Vinschenwrote:
> On Aug 31 17:55, Takashi Yano wrote:
> > On Mon, 30 Aug 2021 22:14:15 +0200
> > Corinna Vinschen wrote:
> > > Hi Ken, Hi Takashi,
> > > 
> > > On Aug 30 19:00, Corinna Vinschen wrote:
> > > Well, what about keeping a duplicate of the read side handle on the 
> > > write side just for calling NtQueryInformationFile?
> > > 
> > > Attached is an untested patch, can you have a look if that makes sense?
> > > 
> > > Btw., I think I found a bug in the new fhandler_pipe::create.  If the
> > > function fails to create the write side fhandler, it deletes the read
> > > side fhandler, but neglects to close the read handle.  My patch fixes
> > > that.
> > > 
> > > While looking into this I found a problem in fhandler_disk_file in
> > > terms of handle inheritance of the special handle for pread/pwrite.
> > > I already force pushed this onto topic/pipe.
> > 
> > I tested your patch attached. Unfortunately, select() does not work
> > as expected for write pipe. Even if the select reports write pipe
> > is available, writing to pipe fails. It seems that your patch fails
> > to detect pipe full.
> 
> Bummer.  Is that with byte mode pipes or with message mode pipes?  If
> the latter, if you try to write more data than available in the buffer,
> it's bound to fail.

Both message pipe and byte pipe.

> Did you add debug output to pipe_data_available to see how the
> information looks like?  Or do you have a simple, self-contained
> testcase in plain C?

The test case is attached. If select() works as expected, the program
does not show "r" or "w". However, with your patch, the program prints
many "w" (means write() fails with EAGAIN).

-- 
Takashi Yano <takashi.yano@nifty.ne.jp>

[-- Attachment #2: stc.c --]
[-- Type: text/x-csrc, Size: 1797 bytes --]

#include <unistd.h>
#include <sys/wait.h>
#include <stdio.h>
#include <time.h>
#include <sys/select.h>
#include <fcntl.h>
#include <errno.h>

#define BLKSIZ 16384
#define NBLK (100*(1<<20)/BLKSIZ)

int main()
{
	int fd[2];
	pid_t pid;
	int nonblock = 1;

	pipe(fd);

	if (!(pid = fork ())) {
		int i;
		char buf[BLKSIZ] = {0,};
		close(fd[0]);
		if (nonblock) {
			int flags;
			flags = fcntl(fd[1], F_GETFL);
			flags |= O_NONBLOCK;
			fcntl(fd[1], F_SETFL, flags);
		}
		fd_set wfds;
		for (i=0; i<NBLK; i++) {
			FD_ZERO(&wfds);
			FD_SET(fd[1], &wfds);
			if (select(fd[1]+1, NULL, &wfds, NULL, NULL) > 0
					&& FD_ISSET(fd[1], &wfds)) {
				ssize_t len = write(fd[1], buf, sizeof(buf));
				if (len <= 0 && errno == EAGAIN) printf("w", i);
				if (len <= 0 && errno == EAGAIN) i --;
			}
		}
		close(fd[1]);
	} else {
		int i;
		char buf[BLKSIZ] = {0,};
		int total = 0;
		fd_set rfds;
		struct timespec tv0, tv1;
		double elasped;
		close(fd[1]);
		if (nonblock) {
			int flags;
			flags = fcntl(fd[0], F_GETFL);
			flags |= O_NONBLOCK;
			fcntl(fd[0], F_SETFL, flags);
		}
		usleep(100000); /* Delay to start reader */
		clock_gettime(CLOCK_MONOTONIC, &tv0);
		for (i=0; i<NBLK; i++) {
			FD_ZERO(&rfds);
			FD_SET(fd[0], &rfds);
			if (select(fd[0]+1, &rfds, NULL, NULL, NULL) > 0
					&& FD_ISSET(fd[0], &rfds)) {
				ssize_t len = read(fd[0], buf, sizeof(buf));
				if (len <= 0 && errno == EAGAIN) printf("r");
				if (len <= 0 && errno == EAGAIN) i --;
				else if(len < 0) break;
				else total += len;
			}
		}
		clock_gettime(CLOCK_MONOTONIC, &tv1);
		close(fd[0]);
		elasped = (tv1.tv_sec - tv0.tv_sec) + (tv1.tv_nsec - tv0.tv_nsec)*1e-9;
		printf("Total: %dMB in %f second, %fMB/s\n",
			total/(1<<20), elasped, total/(1<<20)/elasped);
		waitpid(pid, NULL, 0);
	}

	return 0;
}