* cygrunsrv + sshd + rsync = 20 times too slow -- throttled? @ 2021-08-24 19:49 Chris Roehrig 2021-08-24 23:43 ` Mark Geisert ` (3 more replies) 0 siblings, 4 replies; 86+ messages in thread From: Chris Roehrig @ 2021-08-24 19:49 UTC (permalink / raw) To: cygwin I have a network of Windows, Linux and Mac machines and I use rsync to synchronize various directories between them. I'm trying to figure out why my rsync transfers are so slow (<4 MB/s) only when the remote endpoint is Cygwin rsync over sshd (with both a Linux or Cygwin rsync client). In all other scenarios, I get the full 100MB/s as expected from gigabit ethernet. This has been an ongoing problem for me for a couple of years over several Windows and Cygwin versions, and I'd like to try to fix it. If I run rsync --daemon --no-detach under mintty in the foreground on the remote Windows endpoint, I get the full 100 MB/s transfers, so it seems like it has something to do with rsync.exe running in the background under the cygrunsrv+sshd service (which was installed normally using ssh-host-config). If I do: pv /dev/zero | ssh $WINHOST "cat > /dev/null" or even pv /dev/urandom | ssh $WINHOST md5sum I also get the full 100 MB/s transfers, so it doesn't look like sshd itself is being throttled by bandwidth or CPU. The machines have less than 15% CPU utilization while transferring, with each of the 4 cores less than 30%, so it doesn't look to be CPU issue. In Task Manager, sshd.exe and rsync.exe seem to be running normally using only few percent CPU, and show Power Throttling=Disabled, Priority=Normal. Setting their Priority to High doesn't seem to change things. Looking in Resource Monitor on the remote endpoint, the network usage is pretty much a flat horizontal line at about 18 Mbps (2.5 MB/s), so it sure looks to me as if rsync is somehow being bandwidth-throttled\b when run in the background under cygsshd. It's almost as if rsync has an implicit --bwlimit override when it is run from cygrunsrv+sshd (I've tried --bwlimit=0 on the client which makes no difference). Any ideas? Not sure where to go from here. Thanks, -- Chris Windows 10 v2004 (64-bit) CYGWIN_NT-10.0 .... 3.2.0(0.340/5/3) 2021-03-29 08:42 x86_64 Cygwin rsync version 3.2.4dev protocol version 31 Linux Mint 20 ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? 2021-08-24 19:49 cygrunsrv + sshd + rsync = 20 times too slow -- throttled? Chris Roehrig @ 2021-08-24 23:43 ` Mark Geisert 2021-08-25 0:35 ` Chris Roehrig 2021-08-25 2:02 ` NightStrike ` (2 subsequent siblings) 3 siblings, 1 reply; 86+ messages in thread From: Mark Geisert @ 2021-08-24 23:43 UTC (permalink / raw) To: cygwin Chris Roehrig wrote: > I have a network of Windows, Linux and Mac machines and I use rsync to synchronize various directories between them. > > I'm trying to figure out why my rsync transfers are so slow (<4 MB/s) only when the remote endpoint is Cygwin rsync over sshd (with both a Linux or Cygwin rsync client). In all other scenarios, I get the full 100MB/s as expected from gigabit ethernet. This has been an ongoing problem for me for a couple of years over several Windows and Cygwin versions, and I'd like to try to fix it. > > If I run rsync --daemon --no-detach under mintty in the foreground on the remote Windows endpoint, I get the full 100 MB/s transfers, so it seems like it has something to do with rsync.exe running in the background under the cygrunsrv+sshd service (which was installed normally using ssh-host-config). > > If I do: > pv /dev/zero | ssh $WINHOST "cat > /dev/null" > or even > pv /dev/urandom | ssh $WINHOST md5sum > I also get the full 100 MB/s transfers, so it doesn't look like sshd itself is being throttled by bandwidth or CPU. > > The machines have less than 15% CPU utilization while transferring, with each of the 4 cores less than 30%, so it doesn't look to be CPU issue. > In Task Manager, sshd.exe and rsync.exe seem to be running normally using only few percent CPU, and show Power Throttling=Disabled, Priority=Normal. Setting their Priority to High doesn't seem to change things. > > Looking in Resource Monitor on the remote endpoint, the network usage is pretty much a flat horizontal line at about 18 Mbps (2.5 MB/s), so it sure looks to me as if rsync is somehow being bandwidth-throttled\b when run in the background under cygsshd. > > It's almost as if rsync has an implicit --bwlimit override when it is run from cygrunsrv+sshd (I've tried --bwlimit=0 on the client which makes no difference). > > > Any ideas? Not sure where to go from here. You're not the first to report this. I don't have any quick answer. But can you give one or two simple examples of commands that give slow transfers in your environment? Simple like your 'pv' examples, if possible, using whatever method that works. ..mark ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? 2021-08-24 23:43 ` Mark Geisert @ 2021-08-25 0:35 ` Chris Roehrig 2021-08-25 2:12 ` Chris Roehrig 0 siblings, 1 reply; 86+ messages in thread From: Chris Roehrig @ 2021-08-25 0:35 UTC (permalink / raw) To: Mark Geisert; +Cc: cygwin Here's my test in a nutshell: # ON WINDOWS: install Cygwin and enable cygsshd ssh-host-config -y # set up authorized keys, etc to make things easier # LINUX: Create a 2GB random file on Linux: dd if=/dev/urandom of=/tmp/bigfile.bin bs=1M count=2000 # WINDOWS: rsync "pull" to cygwin LINUXHOST=mylinuxhost rsync -Pva $LINUXHOST:/tmp/bigfile.bin /tmp/ # 100MB/s full speed rm /tmp/bigfile.bin # LINUX: rsync "push" to cygwin (from Linux machine) WINHOST=mywindowshost rsync -Pva /tmp/bigfile.bin $WINHOST:/tmp/ # slow, dropping to 4MB/s I get the same results transferring the other direction (WIN -> LINUX): The Cygwin remote endpoint is always slow. On Tue Aug 24 2021, at 4:43 PM, Mark Geisert <mark@maxrnd.com> wrote: > Chris Roehrig wrote: >> I have a network of Windows, Linux and Mac machines and I use rsync to synchronize various directories between them. >> I'm trying to figure out why my rsync transfers are so slow (<4 MB/s) only when the remote endpoint is Cygwin rsync over sshd (with both a Linux or Cygwin rsync client). In all other scenarios, I get the full 100MB/s as expected from gigabit ethernet. This has been an ongoing problem for me for a couple of years over several Windows and Cygwin versions, and I'd like to try to fix it. >> If I run rsync --daemon --no-detach under mintty in the foreground on the remote Windows endpoint, I get the full 100 MB/s transfers, so it seems like it has something to do with rsync.exe running in the background under the cygrunsrv+sshd service (which was installed normally using ssh-host-config). >> If I do: >> pv /dev/zero | ssh $WINHOST "cat > /dev/null" >> or even >> pv /dev/urandom | ssh $WINHOST md5sum >> I also get the full 100 MB/s transfers, so it doesn't look like sshd itself is being throttled by bandwidth or CPU. >> The machines have less than 15% CPU utilization while transferring, with each of the 4 cores less than 30%, so it doesn't look to be CPU issue. >> In Task Manager, sshd.exe and rsync.exe seem to be running normally using only few percent CPU, and show Power Throttling=Disabled, Priority=Normal. Setting their Priority to High doesn't seem to change things. >> Looking in Resource Monitor on the remote endpoint, the network usage is pretty much a flat horizontal line at about 18 Mbps (2.5 MB/s), so it sure looks to me as if rsync is somehow being bandwidth-throttled\b when run in the background under cygsshd. >> It's almost as if rsync has an implicit --bwlimit override when it is run from cygrunsrv+sshd (I've tried --bwlimit=0 on the client which makes no difference). >> Any ideas? Not sure where to go from here. > > You're not the first to report this. I don't have any quick answer. But can you give one or two simple examples of commands that give slow transfers in your environment? Simple like your 'pv' examples, if possible, using whatever method that works. > > ..mark > > -- > Problem reports: https://cygwin.com/problems.html > FAQ: https://cygwin.com/faq/ > Documentation: https://cygwin.com/docs.html > Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? 2021-08-25 0:35 ` Chris Roehrig @ 2021-08-25 2:12 ` Chris Roehrig 0 siblings, 0 replies; 86+ messages in thread From: Chris Roehrig @ 2021-08-25 2:12 UTC (permalink / raw) To: Mark Geisert; +Cc: cygwin Here's the other direction with rsync also in --daemon mode: # ON WINDOWS (from a mintty terminal): dd if=/dev/urandom of=/tmp/bigfile.bin bs=1M count=2000 # create 2GB file printf "[TMP]\npath = /tmp\n" > /etc/rsyncd.conf # create rsyncd.conf rsync --daemon --no-detach # run rsync in daemon mode # ON LINUX: rm -f /tmp/bigfile.bin; rsync -Pva $WINHOST:/tmp/bigfile.bin /tmp/ # 4MB/s rm -f /tmp/bigfile.bin; rsync -Pva rsync://$WINHOST/TMP/bigfile.bin /tmp/ # 100MB/s I was thinking that there might be some Windows Policy that rate-limits some syscalls for background process, but I also tried: # ON WINDOWS (Administrator mintty): cygrunsrv -E cygsshd # exit sshd service /usr/sbin/sshd -D # run sshd in "foreground" (GUI app) and I still get 4MB/s. Maybe some IPC rate-limit issue between sshd and rsync (but that isn't there between e.g. sshd and md5sum... ?) On Tue Aug 24 2021, at 7:02 PM, NightStrike <nightstrike@gmail.com> wrote: > Older versions of windows had a setting to optimize the OS for either background services or foreground applications. One of the things this did was throttle network usage. I don't know if windows 10 has the same setting though. I read about a NetworkThrottlingIndex that limits "non-multimedia" packets to 10 per millisecond which can be disabled by setting: [HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Multimedia\SystemProfile] "NetworkThrottlingIndex"=dword:ffffffff but this made no difference to my tests. On Tue Aug 24 2021, at 5:35 PM, Chris Roehrig <croehrig@house.org> wrote: > Here's my test in a nutshell: > > # ON WINDOWS: install Cygwin and enable cygsshd > ssh-host-config -y > # set up authorized keys, etc to make things easier > > # LINUX: Create a 2GB random file on Linux: > dd if=/dev/urandom of=/tmp/bigfile.bin bs=1M count=2000 > > # WINDOWS: rsync "pull" to cygwin > LINUXHOST=mylinuxhost > rsync -Pva $LINUXHOST:/tmp/bigfile.bin /tmp/ # 100MB/s full speed > rm /tmp/bigfile.bin > > # LINUX: rsync "push" to cygwin (from Linux machine) > WINHOST=mywindowshost > rsync -Pva /tmp/bigfile.bin $WINHOST:/tmp/ # slow, dropping to 4MB/s > > > I get the same results transferring the other direction (WIN -> LINUX): The Cygwin remote endpoint is always slow. > > > > > > On Tue Aug 24 2021, at 4:43 PM, Mark Geisert <mark@maxrnd.com> wrote: > >> Chris Roehrig wrote: >>> I have a network of Windows, Linux and Mac machines and I use rsync to synchronize various directories between them. >>> I'm trying to figure out why my rsync transfers are so slow (<4 MB/s) only when the remote endpoint is Cygwin rsync over sshd (with both a Linux or Cygwin rsync client). In all other scenarios, I get the full 100MB/s as expected from gigabit ethernet. This has been an ongoing problem for me for a couple of years over several Windows and Cygwin versions, and I'd like to try to fix it. >>> If I run rsync --daemon --no-detach under mintty in the foreground on the remote Windows endpoint, I get the full 100 MB/s transfers, so it seems like it has something to do with rsync.exe running in the background under the cygrunsrv+sshd service (which was installed normally using ssh-host-config). >>> If I do: >>> pv /dev/zero | ssh $WINHOST "cat > /dev/null" >>> or even >>> pv /dev/urandom | ssh $WINHOST md5sum >>> I also get the full 100 MB/s transfers, so it doesn't look like sshd itself is being throttled by bandwidth or CPU. >>> The machines have less than 15% CPU utilization while transferring, with each of the 4 cores less than 30%, so it doesn't look to be CPU issue. >>> In Task Manager, sshd.exe and rsync.exe seem to be running normally using only few percent CPU, and show Power Throttling=Disabled, Priority=Normal. Setting their Priority to High doesn't seem to change things. >>> Looking in Resource Monitor on the remote endpoint, the network usage is pretty much a flat horizontal line at about 18 Mbps (2.5 MB/s), so it sure looks to me as if rsync is somehow being bandwidth-throttled\b when run in the background under cygsshd. >>> It's almost as if rsync has an implicit --bwlimit override when it is run from cygrunsrv+sshd (I've tried --bwlimit=0 on the client which makes no difference). >>> Any ideas? Not sure where to go from here. >> >> You're not the first to report this. I don't have any quick answer. But can you give one or two simple examples of commands that give slow transfers in your environment? Simple like your 'pv' examples, if possible, using whatever method that works. >> >> ..mark >> >> -- >> Problem reports: https://cygwin.com/problems.html >> FAQ: https://cygwin.com/faq/ >> Documentation: https://cygwin.com/docs.html >> Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple > > > -- > Problem reports: https://cygwin.com/problems.html > FAQ: https://cygwin.com/faq/ > Documentation: https://cygwin.com/docs.html > Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? 2021-08-24 19:49 cygrunsrv + sshd + rsync = 20 times too slow -- throttled? Chris Roehrig 2021-08-24 23:43 ` Mark Geisert @ 2021-08-25 2:02 ` NightStrike 2021-08-25 3:20 ` Mark Geisert 2021-08-25 11:18 ` Takashi Yano 2021-09-16 22:00 ` Keith Christian 3 siblings, 1 reply; 86+ messages in thread From: NightStrike @ 2021-08-25 2:02 UTC (permalink / raw) To: Chris Roehrig; +Cc: cygwin On Tue, Aug 24, 2021, 09:50 Chris Roehrig <croehrig@house.org> wrote: > I have a network of Windows, Linux and Mac machines and I use rsync to > synchronize various directories between them. > > I'm trying to figure out why my rsync transfers are so slow (<4 MB/s) only > when the remote endpoint is Cygwin rsync over sshd (with both a Linux or > Cygwin rsync client). In all other scenarios, I get the full 100MB/s as > expected from gigabit ethernet. This has been an ongoing problem for me > for a couple of years over several Windows and Cygwin versions, and I'd > like to try to fix it. > > If I run rsync --daemon --no-detach under mintty in the foreground on the > remote Windows endpoint, I get the full 100 MB/s transfers, so it seems > like it has something to do with rsync.exe running in the background under > the cygrunsrv+sshd service (which was installed normally using > ssh-host-config). > > If I do: > pv /dev/zero | ssh $WINHOST "cat > /dev/null" > or even > pv /dev/urandom | ssh $WINHOST md5sum > I also get the full 100 MB/s transfers, so it doesn't look like sshd > itself is being throttled by bandwidth or CPU. > > The machines have less than 15% CPU utilization while transferring, with > each of the 4 cores less than 30%, so it doesn't look to be CPU issue. > In Task Manager, sshd.exe and rsync.exe seem to be running normally using > only few percent CPU, and show Power Throttling=Disabled, Priority=Normal. > Setting their Priority to High doesn't seem to change things. > > Looking in Resource Monitor on the remote endpoint, the network usage is > pretty much a flat horizontal line at about 18 Mbps (2.5 MB/s), so it sure > looks to me as if rsync is somehow being bandwidth-throttled when run in > the background under cygsshd. > > It's almost as if rsync has an implicit --bwlimit override when it is run > from cygrunsrv+sshd (I've tried --bwlimit=0 on the client which makes no > difference). > > > Any ideas? Not sure where to go from here. > > Thanks, > -- Chris > > > Windows 10 v2004 (64-bit) > CYGWIN_NT-10.0 .... 3.2.0(0.340/5/3) 2021-03-29 08:42 x86_64 Cygwin > rsync version 3.2.4dev protocol version 31 > Linux Mint 20 > Older versions of windows had a setting to optimize the OS for either background services or foreground applications. One of the things this did was throttle network usage. I don't know if windows 10 has the same setting though. > ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? 2021-08-25 2:02 ` NightStrike @ 2021-08-25 3:20 ` Mark Geisert 2021-08-25 17:24 ` Chris Roehrig 0 siblings, 1 reply; 86+ messages in thread From: Mark Geisert @ 2021-08-25 3:20 UTC (permalink / raw) Cc: cygwin NightStrike via Cygwin wrote: > Older versions of windows had a setting to optimize the OS for either > background services or foreground applications. One of the things this did > was throttle network usage. I don't know if windows 10 has the same setting > though. Yes, it does. Getting to it is a pain. Search the "Settings" thingy for "Advanced system settings". That will bring up the System Properties dialog you used to be able to get to from the Control Panel. Click the Advanced tab. Under Performance, click Settings... . Click the Advanced tab there. Right in that dialog there's a Processor Scheduling box where you select for best performance of "Programs" or "Background services". It would be interesting to hear what it's currently set to before you try changing it. Try to keep all your testcases the same so we're dealing with apples to apples. ..mark ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? 2021-08-25 3:20 ` Mark Geisert @ 2021-08-25 17:24 ` Chris Roehrig 0 siblings, 0 replies; 86+ messages in thread From: Chris Roehrig @ 2021-08-25 17:24 UTC (permalink / raw) To: cygwin It was set to "Programs". Changing it to "Background services" didn't make a difference. TCPOptimizer can adjust 2 registry entries that I think are related to that Windows Setting: [HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Multimedia\SystemProfile] "NetworkThrottlingIndex"=dword:0000000a "SystemResponsiveness"=dword:00000014 The first rate-limits IP packets; the second reserves a percent (0-100) of CPU scheduling for background tasks. Changing either of those values made no difference. I think the CPU scheduling reservation only applies when there's CPU contention (not on an idle system) so I wouldn't expect it to make a difference when CPU/core usage is low. On Tue Aug 24 2021, at 8:20 PM, Mark Geisert <mark@maxrnd.com> wrote: > NightStrike via Cygwin wrote: >> Older versions of windows had a setting to optimize the OS for either >> background services or foreground applications. One of the things this did >> was throttle network usage. I don't know if windows 10 has the same setting >> though. > > Yes, it does. Getting to it is a pain. Search the "Settings" thingy for "Advanced system settings". That will bring up the System Properties dialog you used to be able to get to from the Control Panel. Click the Advanced tab. Under Performance, click Settings... . Click the Advanced tab there. Right in that dialog there's a Processor Scheduling box where you select for best performance of "Programs" or "Background services". > > It would be interesting to hear what it's currently set to before you try changing it. Try to keep all your testcases the same so we're dealing with apples to apples. > > ..mark > > -- > Problem reports: https://cygwin.com/problems.html > FAQ: https://cygwin.com/faq/ > Documentation: https://cygwin.com/docs.html > Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? 2021-08-24 19:49 cygrunsrv + sshd + rsync = 20 times too slow -- throttled? Chris Roehrig 2021-08-24 23:43 ` Mark Geisert 2021-08-25 2:02 ` NightStrike @ 2021-08-25 11:18 ` Takashi Yano 2021-08-25 17:31 ` Chris Roehrig 2021-08-25 17:52 ` Ken Brown 2021-09-16 22:00 ` Keith Christian 3 siblings, 2 replies; 86+ messages in thread From: Takashi Yano @ 2021-08-25 11:18 UTC (permalink / raw) To: cygwin On Tue, 24 Aug 2021 12:49:52 -0700 Chris Roehrig wrote: > I have a network of Windows, Linux and Mac machines and I use rsync to synchronize various directories between them. > > I'm trying to figure out why my rsync transfers are so slow (<4 MB/s) only when the remote endpoint is Cygwin rsync over sshd (with both a Linux or Cygwin rsync client). In all other scenarios, I get the full 100MB/s as expected from gigabit ethernet. This has been an ongoing problem for me for a couple of years over several Windows and Cygwin versions, and I'd like to try to fix it. > > If I run rsync --daemon --no-detach under mintty in the foreground on the remote Windows endpoint, I get the full 100 MB/s transfers, so it seems like it has something to do with rsync.exe running in the background under the cygrunsrv+sshd service (which was installed normally using ssh-host-config). > > If I do: > pv /dev/zero | ssh $WINHOST "cat > /dev/null" > or even > pv /dev/urandom | ssh $WINHOST md5sum > I also get the full 100 MB/s transfers, so it doesn't look like sshd itself is being throttled by bandwidth or CPU. > > The machines have less than 15% CPU utilization while transferring, with each of the 4 cores less than 30%, so it doesn't look to be CPU issue. > In Task Manager, sshd.exe and rsync.exe seem to be running normally using only few percent CPU, and show Power Throttling=Disabled, Priority=Normal. Setting their Priority to High doesn't seem to change things. > > Looking in Resource Monitor on the remote endpoint, the network usage is pretty much a flat horizontal line at about 18 Mbps (2.5 MB/s), so it sure looks to me as if rsync is somehow being bandwidth-throttled\b when run in the background under cygsshd. > > It's almost as if rsync has an implicit --bwlimit override when it is run from cygrunsrv+sshd (I've tried --bwlimit=0 on the client which makes no difference). > > > Any ideas? Not sure where to go from here. In cygwin, just scp is very slow. The transfer speed in my environment is as follows. The tests were done with 100MB of test.dat file. (1-1) From cygwin-PC, [yano@cygwin-PC ~]$ scp test.dat yano@linux-server:. yano@linux-server's password: test.dat 100% 100MB 4.0MB/s 00:24 [yano@cygwin-PC ~]$ scp yano@linux-server:test.dat . yano@linux-server's password: test.dat 100% 100MB 8.0MB/s 00:12 (1-2) From linux-server, yano@linux-server:~$ scp yano@cygwin-PC:test.dat . yano@cygwin-PC's password: test.dat 100% 100MB 4.0MB/s 00:24 yano@linux-server:~$ scp test.dat yano@cygwin-PC:. yano@cygwin-PC's password: test.dat 100% 100MB 4.1MB/s 00:24 I looked into this problem, and noticed that this is caused by cygwin pipe implementation. Pipe in cygwin is configured with FILE_FLAG_OVERLAPPED. If the pipe is configured without FILE_FLAG_OVERLAPPED, the transfer speed is much improved as follows. (2-1) From cygwin-PC, [yano@cygwin-PC ~]$ scp test.dat yano@linux-server:. yano@linux-server's password: test.dat 100% 100MB 85.5MB/s 00:01 [yano@cygwin-PC ~]$ scp yano@linux-server:test.dat . yano@linux-server's password: test.dat 100% 100MB 69.7MB/s 00:01 (2-2) From linux-server, yano@linux-server:~$ scp yano@cygwin-PC:test.dat . yano@cygwin-PC's password: test.dat 100% 100MB 80.1MB/s 00:01 yano@linux-server:~$ scp test.dat yano@cygwin-PC:. yano@cygwin-PC's password: test.dat 100% 100MB 57.7MB/s 00:01 I am not sure why this happens and how to fix this. Any idea? -- Takashi Yano <takashi.yano@nifty.ne.jp> ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? 2021-08-25 11:18 ` Takashi Yano @ 2021-08-25 17:31 ` Chris Roehrig 2021-08-25 17:52 ` Ken Brown 1 sibling, 0 replies; 86+ messages in thread From: Chris Roehrig @ 2021-08-25 17:31 UTC (permalink / raw) To: Takashi Yano; +Cc: cygwin The FILE_FLAG_OVERLAPPED avenue looks promising. I get exactly the same results as you using 'scp': 4MB/s in either direction (with either remote endpoint.) I guess the next step is to install a build environment and build rsync and sshd ... On Wed Aug 25 2021, at 4:18 AM, Takashi Yano via Cygwin <cygwin@cygwin.com> wrote: > On Tue, 24 Aug 2021 12:49:52 -0700 > Chris Roehrig wrote: >> I have a network of Windows, Linux and Mac machines and I use rsync to synchronize various directories between them. >> >> I'm trying to figure out why my rsync transfers are so slow (<4 MB/s) only when the remote endpoint is Cygwin rsync over sshd (with both a Linux or Cygwin rsync client). In all other scenarios, I get the full 100MB/s as expected from gigabit ethernet. This has been an ongoing problem for me for a couple of years over several Windows and Cygwin versions, and I'd like to try to fix it. >> >> If I run rsync --daemon --no-detach under mintty in the foreground on the remote Windows endpoint, I get the full 100 MB/s transfers, so it seems like it has something to do with rsync.exe running in the background under the cygrunsrv+sshd service (which was installed normally using ssh-host-config). >> >> If I do: >> pv /dev/zero | ssh $WINHOST "cat > /dev/null" >> or even >> pv /dev/urandom | ssh $WINHOST md5sum >> I also get the full 100 MB/s transfers, so it doesn't look like sshd itself is being throttled by bandwidth or CPU. >> >> The machines have less than 15% CPU utilization while transferring, with each of the 4 cores less than 30%, so it doesn't look to be CPU issue. >> In Task Manager, sshd.exe and rsync.exe seem to be running normally using only few percent CPU, and show Power Throttling=Disabled, Priority=Normal. Setting their Priority to High doesn't seem to change things. >> >> Looking in Resource Monitor on the remote endpoint, the network usage is pretty much a flat horizontal line at about 18 Mbps (2.5 MB/s), so it sure looks to me as if rsync is somehow being bandwidth-throttled\b when run in the background under cygsshd. >> >> It's almost as if rsync has an implicit --bwlimit override when it is run from cygrunsrv+sshd (I've tried --bwlimit=0 on the client which makes no difference). >> >> >> Any ideas? Not sure where to go from here. > > In cygwin, just scp is very slow. > > The transfer speed in my environment is as follows. > The tests were done with 100MB of test.dat file. > > (1-1) From cygwin-PC, > [yano@cygwin-PC ~]$ scp test.dat yano@linux-server:. > yano@linux-server's password: > test.dat 100% 100MB 4.0MB/s 00:24 > [yano@cygwin-PC ~]$ scp yano@linux-server:test.dat . > yano@linux-server's password: > test.dat 100% 100MB 8.0MB/s 00:12 > > (1-2) From linux-server, > yano@linux-server:~$ scp yano@cygwin-PC:test.dat . > yano@cygwin-PC's password: > test.dat 100% 100MB 4.0MB/s 00:24 > yano@linux-server:~$ scp test.dat yano@cygwin-PC:. > yano@cygwin-PC's password: > test.dat 100% 100MB 4.1MB/s 00:24 > > > I looked into this problem, and noticed that this is caused > by cygwin pipe implementation. Pipe in cygwin is configured > with FILE_FLAG_OVERLAPPED. > > If the pipe is configured without FILE_FLAG_OVERLAPPED, > the transfer speed is much improved as follows. > > > (2-1) From cygwin-PC, > [yano@cygwin-PC ~]$ scp test.dat yano@linux-server:. > yano@linux-server's password: > test.dat 100% 100MB 85.5MB/s 00:01 > [yano@cygwin-PC ~]$ scp yano@linux-server:test.dat . > yano@linux-server's password: > test.dat 100% 100MB 69.7MB/s 00:01 > > (2-2) From linux-server, > yano@linux-server:~$ scp yano@cygwin-PC:test.dat . > yano@cygwin-PC's password: > test.dat 100% 100MB 80.1MB/s 00:01 > yano@linux-server:~$ scp test.dat yano@cygwin-PC:. > yano@cygwin-PC's password: > test.dat 100% 100MB 57.7MB/s 00:01 > > I am not sure why this happens and how to fix this. > Any idea? > > -- > Takashi Yano <takashi.yano@nifty.ne.jp> > > -- > Problem reports: https://cygwin.com/problems.html > FAQ: https://cygwin.com/faq/ > Documentation: https://cygwin.com/docs.html > Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? 2021-08-25 11:18 ` Takashi Yano 2021-08-25 17:31 ` Chris Roehrig @ 2021-08-25 17:52 ` Ken Brown 2021-08-25 18:18 ` Chris Roehrig ` (2 more replies) 1 sibling, 3 replies; 86+ messages in thread From: Ken Brown @ 2021-08-25 17:52 UTC (permalink / raw) To: cygwin On 8/25/2021 7:18 AM, Takashi Yano via Cygwin wrote: > On Tue, 24 Aug 2021 12:49:52 -0700 > Chris Roehrig wrote: >> I have a network of Windows, Linux and Mac machines and I use rsync to synchronize various directories between them. >> >> I'm trying to figure out why my rsync transfers are so slow (<4 MB/s) only when the remote endpoint is Cygwin rsync over sshd (with both a Linux or Cygwin rsync client). In all other scenarios, I get the full 100MB/s as expected from gigabit ethernet. This has been an ongoing problem for me for a couple of years over several Windows and Cygwin versions, and I'd like to try to fix it. >> >> If I run rsync --daemon --no-detach under mintty in the foreground on the remote Windows endpoint, I get the full 100 MB/s transfers, so it seems like it has something to do with rsync.exe running in the background under the cygrunsrv+sshd service (which was installed normally using ssh-host-config). >> >> If I do: >> pv /dev/zero | ssh $WINHOST "cat > /dev/null" >> or even >> pv /dev/urandom | ssh $WINHOST md5sum >> I also get the full 100 MB/s transfers, so it doesn't look like sshd itself is being throttled by bandwidth or CPU. >> >> The machines have less than 15% CPU utilization while transferring, with each of the 4 cores less than 30%, so it doesn't look to be CPU issue. >> In Task Manager, sshd.exe and rsync.exe seem to be running normally using only few percent CPU, and show Power Throttling=Disabled, Priority=Normal. Setting their Priority to High doesn't seem to change things. >> >> Looking in Resource Monitor on the remote endpoint, the network usage is pretty much a flat horizontal line at about 18 Mbps (2.5 MB/s), so it sure looks to me as if rsync is somehow being bandwidth-throttled\b when run in the background under cygsshd. >> >> It's almost as if rsync has an implicit --bwlimit override when it is run from cygrunsrv+sshd (I've tried --bwlimit=0 on the client which makes no difference). >> >> >> Any ideas? Not sure where to go from here. > > In cygwin, just scp is very slow. > > The transfer speed in my environment is as follows. > The tests were done with 100MB of test.dat file. > > (1-1) From cygwin-PC, > [yano@cygwin-PC ~]$ scp test.dat yano@linux-server:. > yano@linux-server's password: > test.dat 100% 100MB 4.0MB/s 00:24 > [yano@cygwin-PC ~]$ scp yano@linux-server:test.dat . > yano@linux-server's password: > test.dat 100% 100MB 8.0MB/s 00:12 > > (1-2) From linux-server, > yano@linux-server:~$ scp yano@cygwin-PC:test.dat . > yano@cygwin-PC's password: > test.dat 100% 100MB 4.0MB/s 00:24 > yano@linux-server:~$ scp test.dat yano@cygwin-PC:. > yano@cygwin-PC's password: > test.dat 100% 100MB 4.1MB/s 00:24 > > > I looked into this problem, and noticed that this is caused > by cygwin pipe implementation. Pipe in cygwin is configured > with FILE_FLAG_OVERLAPPED. > > If the pipe is configured without FILE_FLAG_OVERLAPPED, > the transfer speed is much improved as follows. > > > (2-1) From cygwin-PC, > [yano@cygwin-PC ~]$ scp test.dat yano@linux-server:. > yano@linux-server's password: > test.dat 100% 100MB 85.5MB/s 00:01 > [yano@cygwin-PC ~]$ scp yano@linux-server:test.dat . > yano@linux-server's password: > test.dat 100% 100MB 69.7MB/s 00:01 > > (2-2) From linux-server, > yano@linux-server:~$ scp yano@cygwin-PC:test.dat . > yano@cygwin-PC's password: > test.dat 100% 100MB 80.1MB/s 00:01 > yano@linux-server:~$ scp test.dat yano@cygwin-PC:. > yano@cygwin-PC's password: > test.dat 100% 100MB 57.7MB/s 00:01 > > I am not sure why this happens and how to fix this. A couple years ago I had an idea for changing the pipe implementation to avoid overlapped I/O: https://cygwin.com/pipermail/cygwin-patches/2019q2/009393.html https://cygwin.com/pipermail/cygwin-patches/2019q2/009423.html I never followed up on it. But if you think it might help with this problem, I could dust it off and try to finish it. Ken ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? 2021-08-25 17:52 ` Ken Brown @ 2021-08-25 18:18 ` Chris Roehrig 2021-08-25 18:51 ` Chris Roehrig 2021-08-26 15:44 ` Ken Brown 2021-08-25 20:33 ` Mario Emmenlauer 2021-08-25 21:29 ` Takashi Yano 2 siblings, 2 replies; 86+ messages in thread From: Chris Roehrig @ 2021-08-25 18:18 UTC (permalink / raw) To: cygwin On Wed Aug 25 2021, at 10:52 AM, Ken Brown via Cygwin <cygwin@cygwin.com> wrote: > A couple years ago I had an idea for changing the pipe implementation to avoid overlapped I/O: > > https://cygwin.com/pipermail/cygwin-patches/2019q2/009393.html > https://cygwin.com/pipermail/cygwin-patches/2019q2/009423.html > > I never followed up on it. But if you think it might help with this problem, I could dust it off and try to finish it. > > Ken I'm not familiar enough with the innards of rsync, sshd or cygwin to know how this would work. Is it possible to have a new CYGWIN environment option to switch the pipe behaviour without requiring changes to the ssh or rsync source code (and without breaking any existing stuff)? - Chris > On 8/25/2021 7:18 AM, Takashi Yano via Cygwin wrote: >> On Tue, 24 Aug 2021 12:49:52 -0700 >> Chris Roehrig wrote: >>> I have a network of Windows, Linux and Mac machines and I use rsync to synchronize various directories between them. >>> >>> I'm trying to figure out why my rsync transfers are so slow (<4 MB/s) only when the remote endpoint is Cygwin rsync over sshd (with both a Linux or Cygwin rsync client). In all other scenarios, I get the full 100MB/s as expected from gigabit ethernet. This has been an ongoing problem for me for a couple of years over several Windows and Cygwin versions, and I'd like to try to fix it. >>> >>> If I run rsync --daemon --no-detach under mintty in the foreground on the remote Windows endpoint, I get the full 100 MB/s transfers, so it seems like it has something to do with rsync.exe running in the background under the cygrunsrv+sshd service (which was installed normally using ssh-host-config). >>> >>> If I do: >>> pv /dev/zero | ssh $WINHOST "cat > /dev/null" >>> or even >>> pv /dev/urandom | ssh $WINHOST md5sum >>> I also get the full 100 MB/s transfers, so it doesn't look like sshd itself is being throttled by bandwidth or CPU. >>> >>> The machines have less than 15% CPU utilization while transferring, with each of the 4 cores less than 30%, so it doesn't look to be CPU issue. >>> In Task Manager, sshd.exe and rsync.exe seem to be running normally using only few percent CPU, and show Power Throttling=Disabled, Priority=Normal. Setting their Priority to High doesn't seem to change things. >>> >>> Looking in Resource Monitor on the remote endpoint, the network usage is pretty much a flat horizontal line at about 18 Mbps (2.5 MB/s), so it sure looks to me as if rsync is somehow being bandwidth-throttled\b when run in the background under cygsshd. >>> >>> It's almost as if rsync has an implicit --bwlimit override when it is run from cygrunsrv+sshd (I've tried --bwlimit=0 on the client which makes no difference). >>> >>> >>> Any ideas? Not sure where to go from here. >> In cygwin, just scp is very slow. >> The transfer speed in my environment is as follows. >> The tests were done with 100MB of test.dat file. >> (1-1) From cygwin-PC, >> [yano@cygwin-PC ~]$ scp test.dat yano@linux-server:. >> yano@linux-server's password: >> test.dat 100% 100MB 4.0MB/s 00:24 >> [yano@cygwin-PC ~]$ scp yano@linux-server:test.dat . >> yano@linux-server's password: >> test.dat 100% 100MB 8.0MB/s 00:12 >> (1-2) From linux-server, >> yano@linux-server:~$ scp yano@cygwin-PC:test.dat . >> yano@cygwin-PC's password: >> test.dat 100% 100MB 4.0MB/s 00:24 >> yano@linux-server:~$ scp test.dat yano@cygwin-PC:. >> yano@cygwin-PC's password: >> test.dat 100% 100MB 4.1MB/s 00:24 >> I looked into this problem, and noticed that this is caused >> by cygwin pipe implementation. Pipe in cygwin is configured >> with FILE_FLAG_OVERLAPPED. >> If the pipe is configured without FILE_FLAG_OVERLAPPED, >> the transfer speed is much improved as follows. >> (2-1) From cygwin-PC, >> [yano@cygwin-PC ~]$ scp test.dat yano@linux-server:. >> yano@linux-server's password: >> test.dat 100% 100MB 85.5MB/s 00:01 >> [yano@cygwin-PC ~]$ scp yano@linux-server:test.dat . >> yano@linux-server's password: >> test.dat 100% 100MB 69.7MB/s 00:01 >> (2-2) From linux-server, >> yano@linux-server:~$ scp yano@cygwin-PC:test.dat . >> yano@cygwin-PC's password: >> test.dat 100% 100MB 80.1MB/s 00:01 >> yano@linux-server:~$ scp test.dat yano@cygwin-PC:. >> yano@cygwin-PC's password: >> test.dat 100% 100MB 57.7MB/s 00:01 >> I am not sure why this happens and how to fix this. > > > > -- > Problem reports: https://cygwin.com/problems.html > FAQ: https://cygwin.com/faq/ > Documentation: https://cygwin.com/docs.html > Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? 2021-08-25 18:18 ` Chris Roehrig @ 2021-08-25 18:51 ` Chris Roehrig 2021-08-26 15:44 ` Ken Brown 1 sibling, 0 replies; 86+ messages in thread From: Chris Roehrig @ 2021-08-25 18:51 UTC (permalink / raw) To: cygwin It looks like there was a previous (2013) patch and attempt to add a pipe_nooverlap CYGWIN option that was rejected by the maintainers: https://stackoverflow.com/questions/10385424/good-alternatives-to-cygwin-cygwin-doesnt-support-natively-support-win32-app Is this something that can be revisited? It seems to be affecting pure-UNIX situations (in my case) which is Cygwin's core audience. - Chris On Wed Aug 25 2021, at 11:18 AM, Chris Roehrig <croehrig@house.org> wrote: > On Wed Aug 25 2021, at 10:52 AM, Ken Brown via Cygwin <cygwin@cygwin.com> wrote: >> A couple years ago I had an idea for changing the pipe implementation to avoid overlapped I/O: >> >> https://cygwin.com/pipermail/cygwin-patches/2019q2/009393.html >> https://cygwin.com/pipermail/cygwin-patches/2019q2/009423.html >> >> I never followed up on it. But if you think it might help with this problem, I could dust it off and try to finish it. >> >> Ken > > I'm not familiar enough with the innards of rsync, sshd or cygwin to know how this would work. > Is it possible to have a new CYGWIN environment option to switch the pipe behaviour without requiring changes to the ssh or rsync source code (and without breaking any existing stuff)? > > - Chris > > > >> On 8/25/2021 7:18 AM, Takashi Yano via Cygwin wrote: >>> On Tue, 24 Aug 2021 12:49:52 -0700 >>> Chris Roehrig wrote: >>>> I have a network of Windows, Linux and Mac machines and I use rsync to synchronize various directories between them. >>>> >>>> I'm trying to figure out why my rsync transfers are so slow (<4 MB/s) only when the remote endpoint is Cygwin rsync over sshd (with both a Linux or Cygwin rsync client). In all other scenarios, I get the full 100MB/s as expected from gigabit ethernet. This has been an ongoing problem for me for a couple of years over several Windows and Cygwin versions, and I'd like to try to fix it. >>>> >>>> If I run rsync --daemon --no-detach under mintty in the foreground on the remote Windows endpoint, I get the full 100 MB/s transfers, so it seems like it has something to do with rsync.exe running in the background under the cygrunsrv+sshd service (which was installed normally using ssh-host-config). >>>> >>>> If I do: >>>> pv /dev/zero | ssh $WINHOST "cat > /dev/null" >>>> or even >>>> pv /dev/urandom | ssh $WINHOST md5sum >>>> I also get the full 100 MB/s transfers, so it doesn't look like sshd itself is being throttled by bandwidth or CPU. >>>> >>>> The machines have less than 15% CPU utilization while transferring, with each of the 4 cores less than 30%, so it doesn't look to be CPU issue. >>>> In Task Manager, sshd.exe and rsync.exe seem to be running normally using only few percent CPU, and show Power Throttling=Disabled, Priority=Normal. Setting their Priority to High doesn't seem to change things. >>>> >>>> Looking in Resource Monitor on the remote endpoint, the network usage is pretty much a flat horizontal line at about 18 Mbps (2.5 MB/s), so it sure looks to me as if rsync is somehow being bandwidth-throttled\b when run in the background under cygsshd. >>>> >>>> It's almost as if rsync has an implicit --bwlimit override when it is run from cygrunsrv+sshd (I've tried --bwlimit=0 on the client which makes no difference). >>>> >>>> >>>> Any ideas? Not sure where to go from here. >>> In cygwin, just scp is very slow. >>> The transfer speed in my environment is as follows. >>> The tests were done with 100MB of test.dat file. >>> (1-1) From cygwin-PC, >>> [yano@cygwin-PC ~]$ scp test.dat yano@linux-server:. >>> yano@linux-server's password: >>> test.dat 100% 100MB 4.0MB/s 00:24 >>> [yano@cygwin-PC ~]$ scp yano@linux-server:test.dat . >>> yano@linux-server's password: >>> test.dat 100% 100MB 8.0MB/s 00:12 >>> (1-2) From linux-server, >>> yano@linux-server:~$ scp yano@cygwin-PC:test.dat . >>> yano@cygwin-PC's password: >>> test.dat 100% 100MB 4.0MB/s 00:24 >>> yano@linux-server:~$ scp test.dat yano@cygwin-PC:. >>> yano@cygwin-PC's password: >>> test.dat 100% 100MB 4.1MB/s 00:24 >>> I looked into this problem, and noticed that this is caused >>> by cygwin pipe implementation. Pipe in cygwin is configured >>> with FILE_FLAG_OVERLAPPED. >>> If the pipe is configured without FILE_FLAG_OVERLAPPED, >>> the transfer speed is much improved as follows. >>> (2-1) From cygwin-PC, >>> [yano@cygwin-PC ~]$ scp test.dat yano@linux-server:. >>> yano@linux-server's password: >>> test.dat 100% 100MB 85.5MB/s 00:01 >>> [yano@cygwin-PC ~]$ scp yano@linux-server:test.dat . >>> yano@linux-server's password: >>> test.dat 100% 100MB 69.7MB/s 00:01 >>> (2-2) From linux-server, >>> yano@linux-server:~$ scp yano@cygwin-PC:test.dat . >>> yano@cygwin-PC's password: >>> test.dat 100% 100MB 80.1MB/s 00:01 >>> yano@linux-server:~$ scp test.dat yano@cygwin-PC:. >>> yano@cygwin-PC's password: >>> test.dat 100% 100MB 57.7MB/s 00:01 >>> I am not sure why this happens and how to fix this. >> >> >> >> -- >> Problem reports: https://cygwin.com/problems.html >> FAQ: https://cygwin.com/faq/ >> Documentation: https://cygwin.com/docs.html >> Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple > > > -- > Problem reports: https://cygwin.com/problems.html > FAQ: https://cygwin.com/faq/ > Documentation: https://cygwin.com/docs.html > Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? 2021-08-25 18:18 ` Chris Roehrig 2021-08-25 18:51 ` Chris Roehrig @ 2021-08-26 15:44 ` Ken Brown 1 sibling, 0 replies; 86+ messages in thread From: Ken Brown @ 2021-08-26 15:44 UTC (permalink / raw) To: cygwin On 8/25/2021 2:18 PM, Chris Roehrig wrote: > On Wed Aug 25 2021, at 10:52 AM, Ken Brown via Cygwin <cygwin@cygwin.com> wrote: >> A couple years ago I had an idea for changing the pipe implementation to avoid overlapped I/O: >> >> https://cygwin.com/pipermail/cygwin-patches/2019q2/009393.html >> https://cygwin.com/pipermail/cygwin-patches/2019q2/009423.html >> >> I never followed up on it. But if you think it might help with this problem, I could dust it off and try to finish it. >> >> Ken > > I'm not familiar enough with the innards of rsync, sshd or cygwin to know how this would work. > Is it possible to have a new CYGWIN environment option to switch the pipe behaviour without requiring changes to the ssh or rsync source code (and without breaking any existing stuff)? My proposed change would be purely internal to Cygwin. There should be no user-visible change in behavior except, perhaps, improved performance. (And we don't know yet whether there would be improved performance.) So I don't see a need for a new CYGWIN option. Ken ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? 2021-08-25 17:52 ` Ken Brown 2021-08-25 18:18 ` Chris Roehrig @ 2021-08-25 20:33 ` Mario Emmenlauer 2021-08-26 15:47 ` Ken Brown 2021-08-25 21:29 ` Takashi Yano 2 siblings, 1 reply; 86+ messages in thread From: Mario Emmenlauer @ 2021-08-25 20:33 UTC (permalink / raw) To: Ken Brown, cygwin On 25.08.21 19:52, Ken Brown via Cygwin wrote: > A couple years ago I had an idea for changing the pipe implementation to avoid overlapped I/O: > > https://cygwin.com/pipermail/cygwin-patches/2019q2/009393.html > https://cygwin.com/pipermail/cygwin-patches/2019q2/009423.html > > I never followed up on it. But if you think it might help with this problem, I could dust it off and try to finish it. This sounds quite interesting! Do I assume correctly that these changes may likely lead to a general improvement also in other situations if the impact of the pipe is so big for this specific case? Pipes are most likely used in quite a lot of scenarios?! All the best, Mario Emmenlauer ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? 2021-08-25 20:33 ` Mario Emmenlauer @ 2021-08-26 15:47 ` Ken Brown 0 siblings, 0 replies; 86+ messages in thread From: Ken Brown @ 2021-08-26 15:47 UTC (permalink / raw) To: cygwin On 8/25/2021 4:33 PM, Mario Emmenlauer wrote: > On 25.08.21 19:52, Ken Brown via Cygwin wrote: >> A couple years ago I had an idea for changing the pipe implementation to avoid >> overlapped I/O: >> >> https://cygwin.com/pipermail/cygwin-patches/2019q2/009393.html >> https://cygwin.com/pipermail/cygwin-patches/2019q2/009423.html >> >> I never followed up on it. But if you think it might help with this problem, >> I could dust it off and try to finish it. > > This sounds quite interesting! Do I assume correctly that these > changes may likely lead to a general improvement also in other > situations if the impact of the pipe is so big for this specific > case? Pipes are most likely used in quite a lot of scenarios?! I'm not prepared to make any predictions. We won't really know without testing. Ken ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? 2021-08-25 17:52 ` Ken Brown 2021-08-25 18:18 ` Chris Roehrig 2021-08-25 20:33 ` Mario Emmenlauer @ 2021-08-25 21:29 ` Takashi Yano 2021-08-26 15:56 ` Ken Brown 2 siblings, 1 reply; 86+ messages in thread From: Takashi Yano @ 2021-08-25 21:29 UTC (permalink / raw) To: cygwin On Wed, 25 Aug 2021 13:52:19 -0400 Ken Brown wrote: > On 8/25/2021 7:18 AM, Takashi Yano via Cygwin wrote: > > On Tue, 24 Aug 2021 12:49:52 -0700 > > Chris Roehrig wrote: > >> I have a network of Windows, Linux and Mac machines and I use rsync to synchronize various directories between them. > >> > >> I'm trying to figure out why my rsync transfers are so slow (<4 MB/s) only when the remote endpoint is Cygwin rsync over sshd (with both a Linux or Cygwin rsync client). In all other scenarios, I get the full 100MB/s as expected from gigabit ethernet. This has been an ongoing problem for me for a couple of years over several Windows and Cygwin versions, and I'd like to try to fix it. > >> > >> If I run rsync --daemon --no-detach under mintty in the foreground on the remote Windows endpoint, I get the full 100 MB/s transfers, so it seems like it has something to do with rsync.exe running in the background under the cygrunsrv+sshd service (which was installed normally using ssh-host-config). > >> > >> If I do: > >> pv /dev/zero | ssh $WINHOST "cat > /dev/null" > >> or even > >> pv /dev/urandom | ssh $WINHOST md5sum > >> I also get the full 100 MB/s transfers, so it doesn't look like sshd itself is being throttled by bandwidth or CPU. > >> > >> The machines have less than 15% CPU utilization while transferring, with each of the 4 cores less than 30%, so it doesn't look to be CPU issue. > >> In Task Manager, sshd.exe and rsync.exe seem to be running normally using only few percent CPU, and show Power Throttling=Disabled, Priority=Normal. Setting their Priority to High doesn't seem to change things. > >> > >> Looking in Resource Monitor on the remote endpoint, the network usage is pretty much a flat horizontal line at about 18 Mbps (2.5 MB/s), so it sure looks to me as if rsync is somehow being bandwidth-throttled\b when run in the background under cygsshd. > >> > >> It's almost as if rsync has an implicit --bwlimit override when it is run from cygrunsrv+sshd (I've tried --bwlimit=0 on the client which makes no difference). > >> > >> > >> Any ideas? Not sure where to go from here. > > > > In cygwin, just scp is very slow. > > > > The transfer speed in my environment is as follows. > > The tests were done with 100MB of test.dat file. > > > > (1-1) From cygwin-PC, > > [yano@cygwin-PC ~]$ scp test.dat yano@linux-server:. > > yano@linux-server's password: > > test.dat 100% 100MB 4.0MB/s 00:24 > > [yano@cygwin-PC ~]$ scp yano@linux-server:test.dat . > > yano@linux-server's password: > > test.dat 100% 100MB 8.0MB/s 00:12 > > > > (1-2) From linux-server, > > yano@linux-server:~$ scp yano@cygwin-PC:test.dat . > > yano@cygwin-PC's password: > > test.dat 100% 100MB 4.0MB/s 00:24 > > yano@linux-server:~$ scp test.dat yano@cygwin-PC:. > > yano@cygwin-PC's password: > > test.dat 100% 100MB 4.1MB/s 00:24 > > > > > > I looked into this problem, and noticed that this is caused > > by cygwin pipe implementation. Pipe in cygwin is configured > > with FILE_FLAG_OVERLAPPED. > > > > If the pipe is configured without FILE_FLAG_OVERLAPPED, > > the transfer speed is much improved as follows. > > > > > > (2-1) From cygwin-PC, > > [yano@cygwin-PC ~]$ scp test.dat yano@linux-server:. > > yano@linux-server's password: > > test.dat 100% 100MB 85.5MB/s 00:01 > > [yano@cygwin-PC ~]$ scp yano@linux-server:test.dat . > > yano@linux-server's password: > > test.dat 100% 100MB 69.7MB/s 00:01 > > > > (2-2) From linux-server, > > yano@linux-server:~$ scp yano@cygwin-PC:test.dat . > > yano@cygwin-PC's password: > > test.dat 100% 100MB 80.1MB/s 00:01 > > yano@linux-server:~$ scp test.dat yano@cygwin-PC:. > > yano@cygwin-PC's password: > > test.dat 100% 100MB 57.7MB/s 00:01 > > > > I am not sure why this happens and how to fix this. > > A couple years ago I had an idea for changing the pipe implementation to avoid > overlapped I/O: > > https://cygwin.com/pipermail/cygwin-patches/2019q2/009393.html > https://cygwin.com/pipermail/cygwin-patches/2019q2/009423.html > > I never followed up on it. But if you think it might help with this problem, I > could dust it off and try to finish it. Interesting. It will be also helpfull for: https://cygwin.com/pipermail/cygwin/2021-March/247987.html which seems to be the same issue with https://stackoverflow.com/questions/10385424/good-alternatives-to-cygwin-cygwin-doesnt-support-natively-support-win32-app However, I wonder why scp dislikes overlapped I/O. -- Takashi Yano <takashi.yano@nifty.ne.jp> ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? 2021-08-25 21:29 ` Takashi Yano @ 2021-08-26 15:56 ` Ken Brown 2021-08-26 22:18 ` Ken Brown 0 siblings, 1 reply; 86+ messages in thread From: Ken Brown @ 2021-08-26 15:56 UTC (permalink / raw) To: cygwin On 8/25/2021 5:29 PM, Takashi Yano wrote: > On Wed, 25 Aug 2021 13:52:19 -0400 > Ken Brown wrote: >> On 8/25/2021 7:18 AM, Takashi Yano via Cygwin wrote: >>> On Tue, 24 Aug 2021 12:49:52 -0700 >>> Chris Roehrig wrote: >>>> I have a network of Windows, Linux and Mac machines and I use rsync to synchronize various directories between them. >>>> >>>> I'm trying to figure out why my rsync transfers are so slow (<4 MB/s) only when the remote endpoint is Cygwin rsync over sshd (with both a Linux or Cygwin rsync client). In all other scenarios, I get the full 100MB/s as expected from gigabit ethernet. This has been an ongoing problem for me for a couple of years over several Windows and Cygwin versions, and I'd like to try to fix it. >>>> >>>> If I run rsync --daemon --no-detach under mintty in the foreground on the remote Windows endpoint, I get the full 100 MB/s transfers, so it seems like it has something to do with rsync.exe running in the background under the cygrunsrv+sshd service (which was installed normally using ssh-host-config). >>>> >>>> If I do: >>>> pv /dev/zero | ssh $WINHOST "cat > /dev/null" >>>> or even >>>> pv /dev/urandom | ssh $WINHOST md5sum >>>> I also get the full 100 MB/s transfers, so it doesn't look like sshd itself is being throttled by bandwidth or CPU. >>>> >>>> The machines have less than 15% CPU utilization while transferring, with each of the 4 cores less than 30%, so it doesn't look to be CPU issue. >>>> In Task Manager, sshd.exe and rsync.exe seem to be running normally using only few percent CPU, and show Power Throttling=Disabled, Priority=Normal. Setting their Priority to High doesn't seem to change things. >>>> >>>> Looking in Resource Monitor on the remote endpoint, the network usage is pretty much a flat horizontal line at about 18 Mbps (2.5 MB/s), so it sure looks to me as if rsync is somehow being bandwidth-throttled\b when run in the background under cygsshd. >>>> >>>> It's almost as if rsync has an implicit --bwlimit override when it is run from cygrunsrv+sshd (I've tried --bwlimit=0 on the client which makes no difference). >>>> >>>> >>>> Any ideas? Not sure where to go from here. >>> >>> In cygwin, just scp is very slow. >>> >>> The transfer speed in my environment is as follows. >>> The tests were done with 100MB of test.dat file. >>> >>> (1-1) From cygwin-PC, >>> [yano@cygwin-PC ~]$ scp test.dat yano@linux-server:. >>> yano@linux-server's password: >>> test.dat 100% 100MB 4.0MB/s 00:24 >>> [yano@cygwin-PC ~]$ scp yano@linux-server:test.dat . >>> yano@linux-server's password: >>> test.dat 100% 100MB 8.0MB/s 00:12 >>> >>> (1-2) From linux-server, >>> yano@linux-server:~$ scp yano@cygwin-PC:test.dat . >>> yano@cygwin-PC's password: >>> test.dat 100% 100MB 4.0MB/s 00:24 >>> yano@linux-server:~$ scp test.dat yano@cygwin-PC:. >>> yano@cygwin-PC's password: >>> test.dat 100% 100MB 4.1MB/s 00:24 >>> >>> >>> I looked into this problem, and noticed that this is caused >>> by cygwin pipe implementation. Pipe in cygwin is configured >>> with FILE_FLAG_OVERLAPPED. >>> >>> If the pipe is configured without FILE_FLAG_OVERLAPPED, >>> the transfer speed is much improved as follows. >>> >>> >>> (2-1) From cygwin-PC, >>> [yano@cygwin-PC ~]$ scp test.dat yano@linux-server:. >>> yano@linux-server's password: >>> test.dat 100% 100MB 85.5MB/s 00:01 >>> [yano@cygwin-PC ~]$ scp yano@linux-server:test.dat . >>> yano@linux-server's password: >>> test.dat 100% 100MB 69.7MB/s 00:01 >>> >>> (2-2) From linux-server, >>> yano@linux-server:~$ scp yano@cygwin-PC:test.dat . >>> yano@cygwin-PC's password: >>> test.dat 100% 100MB 80.1MB/s 00:01 >>> yano@linux-server:~$ scp test.dat yano@cygwin-PC:. >>> yano@cygwin-PC's password: >>> test.dat 100% 100MB 57.7MB/s 00:01 >>> >>> I am not sure why this happens and how to fix this. >> >> A couple years ago I had an idea for changing the pipe implementation to avoid >> overlapped I/O: >> >> https://cygwin.com/pipermail/cygwin-patches/2019q2/009393.html >> https://cygwin.com/pipermail/cygwin-patches/2019q2/009423.html >> >> I never followed up on it. But if you think it might help with this problem, I >> could dust it off and try to finish it. > > Interesting. > > It will be also helpfull for: > https://cygwin.com/pipermail/cygwin/2021-March/247987.html > which seems to be the same issue with > https://stackoverflow.com/questions/10385424/good-alternatives-to-cygwin-cygwin-doesnt-support-natively-support-win32-app > > However, I wonder why scp dislikes overlapped I/O. I agree that it would be good to understand this. When I first proposed the change, I was thinking in terms of code simplification. If it also improves performance (which we don't know yet), it becomes a higher priority, but in that case it would be nice to understand why it improves performance. Ken ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? 2021-08-26 15:56 ` Ken Brown @ 2021-08-26 22:18 ` Ken Brown 2021-08-27 11:24 ` Takashi Yano 0 siblings, 1 reply; 86+ messages in thread From: Ken Brown @ 2021-08-26 22:18 UTC (permalink / raw) To: cygwin On 8/26/2021 11:56 AM, Ken Brown via Cygwin wrote: > On 8/25/2021 5:29 PM, Takashi Yano wrote: >> On Wed, 25 Aug 2021 13:52:19 -0400 >> Ken Brown wrote: >>> On 8/25/2021 7:18 AM, Takashi Yano via Cygwin wrote: >>>> On Tue, 24 Aug 2021 12:49:52 -0700 >>>> Chris Roehrig wrote: >>>>> I have a network of Windows, Linux and Mac machines and I use rsync to >>>>> synchronize various directories between them. >>>>> >>>>> I'm trying to figure out why my rsync transfers are so slow (<4 MB/s) only >>>>> when the remote endpoint is Cygwin rsync over sshd (with both a Linux or >>>>> Cygwin rsync client). In all other scenarios, I get the full 100MB/s as >>>>> expected from gigabit ethernet. This has been an ongoing problem for me >>>>> for a couple of years over several Windows and Cygwin versions, and I'd >>>>> like to try to fix it. >>>>> >>>>> If I run rsync --daemon --no-detach under mintty in the foreground on the >>>>> remote Windows endpoint, I get the full 100 MB/s transfers, so it seems >>>>> like it has something to do with rsync.exe running in the background under >>>>> the cygrunsrv+sshd service (which was installed normally using >>>>> ssh-host-config). >>>>> >>>>> If I do: >>>>> pv /dev/zero | ssh $WINHOST "cat > /dev/null" >>>>> or even >>>>> pv /dev/urandom | ssh $WINHOST md5sum >>>>> I also get the full 100 MB/s transfers, so it doesn't look like sshd itself >>>>> is being throttled by bandwidth or CPU. >>>>> >>>>> The machines have less than 15% CPU utilization while transferring, with >>>>> each of the 4 cores less than 30%, so it doesn't look to be CPU issue. >>>>> In Task Manager, sshd.exe and rsync.exe seem to be running normally using >>>>> only few percent CPU, and show Power Throttling=Disabled, >>>>> Priority=Normal. Setting their Priority to High doesn't seem to change >>>>> things. >>>>> >>>>> Looking in Resource Monitor on the remote endpoint, the network usage is >>>>> pretty much a flat horizontal line at about 18 Mbps (2.5 MB/s), so it sure >>>>> looks to me as if rsync is somehow being bandwidth-throttled\b when run in >>>>> the background under cygsshd. >>>>> >>>>> It's almost as if rsync has an implicit --bwlimit override when it is run >>>>> from cygrunsrv+sshd (I've tried --bwlimit=0 on the client which makes no >>>>> difference). >>>>> >>>>> >>>>> Any ideas? Not sure where to go from here. >>>> >>>> In cygwin, just scp is very slow. >>>> >>>> The transfer speed in my environment is as follows. >>>> The tests were done with 100MB of test.dat file. >>>> >>>> (1-1) From cygwin-PC, >>>> [yano@cygwin-PC ~]$ scp test.dat yano@linux-server:. >>>> yano@linux-server's password: >>>> test.dat 100% 100MB 4.0MB/s 00:24 >>>> [yano@cygwin-PC ~]$ scp yano@linux-server:test.dat . >>>> yano@linux-server's password: >>>> test.dat 100% 100MB 8.0MB/s 00:12 >>>> >>>> (1-2) From linux-server, >>>> yano@linux-server:~$ scp yano@cygwin-PC:test.dat . >>>> yano@cygwin-PC's password: >>>> test.dat 100% 100MB 4.0MB/s 00:24 >>>> yano@linux-server:~$ scp test.dat yano@cygwin-PC:. >>>> yano@cygwin-PC's password: >>>> test.dat 100% 100MB 4.1MB/s 00:24 >>>> >>>> >>>> I looked into this problem, and noticed that this is caused >>>> by cygwin pipe implementation. Pipe in cygwin is configured >>>> with FILE_FLAG_OVERLAPPED. >>>> >>>> If the pipe is configured without FILE_FLAG_OVERLAPPED, >>>> the transfer speed is much improved as follows. >>>> >>>> >>>> (2-1) From cygwin-PC, >>>> [yano@cygwin-PC ~]$ scp test.dat yano@linux-server:. >>>> yano@linux-server's password: >>>> test.dat 100% 100MB 85.5MB/s 00:01 >>>> [yano@cygwin-PC ~]$ scp yano@linux-server:test.dat . >>>> yano@linux-server's password: >>>> test.dat 100% 100MB 69.7MB/s 00:01 >>>> >>>> (2-2) From linux-server, >>>> yano@linux-server:~$ scp yano@cygwin-PC:test.dat . >>>> yano@cygwin-PC's password: >>>> test.dat 100% 100MB 80.1MB/s 00:01 >>>> yano@linux-server:~$ scp test.dat yano@cygwin-PC:. >>>> yano@cygwin-PC's password: >>>> test.dat 100% 100MB 57.7MB/s 00:01 >>>> >>>> I am not sure why this happens and how to fix this. >>> >>> A couple years ago I had an idea for changing the pipe implementation to avoid >>> overlapped I/O: >>> >>> https://cygwin.com/pipermail/cygwin-patches/2019q2/009393.html >>> https://cygwin.com/pipermail/cygwin-patches/2019q2/009423.html >>> >>> I never followed up on it. But if you think it might help with this problem, I >>> could dust it off and try to finish it. >> >> Interesting. >> >> It will be also helpfull for: >> https://cygwin.com/pipermail/cygwin/2021-March/247987.html >> which seems to be the same issue with >> https://stackoverflow.com/questions/10385424/good-alternatives-to-cygwin-cygwin-doesnt-support-natively-support-win32-app >> >> >> However, I wonder why scp dislikes overlapped I/O. > > I agree that it would be good to understand this. When I first proposed the > change, I was thinking in terms of code simplification. If it also improves > performance (which we don't know yet), it becomes a higher priority, but in that > case it would be nice to understand why it improves performance. Hi Takashi, In case you want to try out my proposed change, I've just rebased the patches to the current master and pushed them to a new topic/pipe branch. Ken ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? 2021-08-26 22:18 ` Ken Brown @ 2021-08-27 11:24 ` Takashi Yano 2021-08-27 16:00 ` Ken Brown 0 siblings, 1 reply; 86+ messages in thread From: Takashi Yano @ 2021-08-27 11:24 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 8082 bytes --] On Thu, 26 Aug 2021 18:18:29 -0400 Ken Brown wrote: > On 8/26/2021 11:56 AM, Ken Brown via Cygwin wrote: > > On 8/25/2021 5:29 PM, Takashi Yano wrote: > >> On Wed, 25 Aug 2021 13:52:19 -0400 > >> Ken Brown wrote: > >>> On 8/25/2021 7:18 AM, Takashi Yano via Cygwin wrote: > >>>> On Tue, 24 Aug 2021 12:49:52 -0700 > >>>> Chris Roehrig wrote: > >>>>> I have a network of Windows, Linux and Mac machines and I use rsync to > >>>>> synchronize various directories between them. > >>>>> > >>>>> I'm trying to figure out why my rsync transfers are so slow (<4 MB/s) only > >>>>> when the remote endpoint is Cygwin rsync over sshd (with both a Linux or > >>>>> Cygwin rsync client). In all other scenarios, I get the full 100MB/s as > >>>>> expected from gigabit ethernet. This has been an ongoing problem for me > >>>>> for a couple of years over several Windows and Cygwin versions, and I'd > >>>>> like to try to fix it. > >>>>> > >>>>> If I run rsync --daemon --no-detach under mintty in the foreground on the > >>>>> remote Windows endpoint, I get the full 100 MB/s transfers, so it seems > >>>>> like it has something to do with rsync.exe running in the background under > >>>>> the cygrunsrv+sshd service (which was installed normally using > >>>>> ssh-host-config). > >>>>> > >>>>> If I do: > >>>>> pv /dev/zero | ssh $WINHOST "cat > /dev/null" > >>>>> or even > >>>>> pv /dev/urandom | ssh $WINHOST md5sum > >>>>> I also get the full 100 MB/s transfers, so it doesn't look like sshd itself > >>>>> is being throttled by bandwidth or CPU. > >>>>> > >>>>> The machines have less than 15% CPU utilization while transferring, with > >>>>> each of the 4 cores less than 30%, so it doesn't look to be CPU issue. > >>>>> In Task Manager, sshd.exe and rsync.exe seem to be running normally using > >>>>> only few percent CPU, and show Power Throttling=Disabled, > >>>>> Priority=Normal. Setting their Priority to High doesn't seem to change > >>>>> things. > >>>>> > >>>>> Looking in Resource Monitor on the remote endpoint, the network usage is > >>>>> pretty much a flat horizontal line at about 18 Mbps (2.5 MB/s), so it sure > >>>>> looks to me as if rsync is somehow being bandwidth-throttled\b when run in > >>>>> the background under cygsshd. > >>>>> > >>>>> It's almost as if rsync has an implicit --bwlimit override when it is run > >>>>> from cygrunsrv+sshd (I've tried --bwlimit=0 on the client which makes no > >>>>> difference). > >>>>> > >>>>> > >>>>> Any ideas? Not sure where to go from here. > >>>> > >>>> In cygwin, just scp is very slow. > >>>> > >>>> The transfer speed in my environment is as follows. > >>>> The tests were done with 100MB of test.dat file. > >>>> > >>>> (1-1) From cygwin-PC, > >>>> [yano@cygwin-PC ~]$ scp test.dat yano@linux-server:. > >>>> yano@linux-server's password: > >>>> test.dat 100% 100MB 4.0MB/s 00:24 > >>>> [yano@cygwin-PC ~]$ scp yano@linux-server:test.dat . > >>>> yano@linux-server's password: > >>>> test.dat 100% 100MB 8.0MB/s 00:12 > >>>> > >>>> (1-2) From linux-server, > >>>> yano@linux-server:~$ scp yano@cygwin-PC:test.dat . > >>>> yano@cygwin-PC's password: > >>>> test.dat 100% 100MB 4.0MB/s 00:24 > >>>> yano@linux-server:~$ scp test.dat yano@cygwin-PC:. > >>>> yano@cygwin-PC's password: > >>>> test.dat 100% 100MB 4.1MB/s 00:24 > >>>> > >>>> > >>>> I looked into this problem, and noticed that this is caused > >>>> by cygwin pipe implementation. Pipe in cygwin is configured > >>>> with FILE_FLAG_OVERLAPPED. > >>>> > >>>> If the pipe is configured without FILE_FLAG_OVERLAPPED, > >>>> the transfer speed is much improved as follows. > >>>> > >>>> > >>>> (2-1) From cygwin-PC, > >>>> [yano@cygwin-PC ~]$ scp test.dat yano@linux-server:. > >>>> yano@linux-server's password: > >>>> test.dat 100% 100MB 85.5MB/s 00:01 > >>>> [yano@cygwin-PC ~]$ scp yano@linux-server:test.dat . > >>>> yano@linux-server's password: > >>>> test.dat 100% 100MB 69.7MB/s 00:01 > >>>> > >>>> (2-2) From linux-server, > >>>> yano@linux-server:~$ scp yano@cygwin-PC:test.dat . > >>>> yano@cygwin-PC's password: > >>>> test.dat 100% 100MB 80.1MB/s 00:01 > >>>> yano@linux-server:~$ scp test.dat yano@cygwin-PC:. > >>>> yano@cygwin-PC's password: > >>>> test.dat 100% 100MB 57.7MB/s 00:01 > >>>> > >>>> I am not sure why this happens and how to fix this. > >>> > >>> A couple years ago I had an idea for changing the pipe implementation to avoid > >>> overlapped I/O: > >>> > >>> https://cygwin.com/pipermail/cygwin-patches/2019q2/009393.html > >>> https://cygwin.com/pipermail/cygwin-patches/2019q2/009423.html > >>> > >>> I never followed up on it. But if you think it might help with this problem, I > >>> could dust it off and try to finish it. > >> > >> Interesting. > >> > >> It will be also helpfull for: > >> https://cygwin.com/pipermail/cygwin/2021-March/247987.html > >> which seems to be the same issue with > >> https://stackoverflow.com/questions/10385424/good-alternatives-to-cygwin-cygwin-doesnt-support-natively-support-win32-app > >> > >> > >> However, I wonder why scp dislikes overlapped I/O. > > > > I agree that it would be good to understand this. When I first proposed the > > change, I was thinking in terms of code simplification. If it also improves > > performance (which we don't know yet), it becomes a higher priority, but in that > > case it would be nice to understand why it improves performance. > > Hi Takashi, > > In case you want to try out my proposed change, I've just rebased the patches to > the current master and pushed them to a new topic/pipe branch. Hi Ken, Thanks much! I tested topic/pipe branch. [yano@cygwin-PC ~]$ scp test.dat yano@linux-server:. yano@linux-server's password: test.dat 100% 100MB 95.9MB/s 00:01 [yano@cygwin-PC ~]$ scp yano@linux-server:test.dat . yano@linux-server's password: test.dat 100% 100MB 8.0MB/s 00:12 yano@linux-server:~$ scp yano@cygwin-PC:test.dat . yano@cygwin-PC's password: test.dat 100% 100MB 109.7MB/s 00:00 yano@linux-server:~$ scp test.dat yano@cygwin-PC:. yano@cygwin-PC's password: test.dat 100% 100MB 31.4MB/s 00:03 As shown above, outgoing transfer-rate has been improved upto near theoretical limit. However, incoming transfer-rate is not improved much. I digged further and found the first patch attached solves the issue as follows. [yano@cygwin-PC ~]$ scp yano@linux-server:test.dat . yano@linux-server's password: test.dat 100% 100MB 112.8MB/s 00:00 yano@linux-server2:~$ scp test.dat yano@cygwin-PC:. yano@cygwin-PC's password: test.dat 100% 100MB 102.5MB/s 00:00 I also tested the case: > >> https://cygwin.com/pipermail/cygwin/2021-March/247987.html > >> which seems to be the same issue with > >> https://stackoverflow.com/questions/10385424/good-alternatives-to-cygwin-cygwin-doesnt-support-natively-support-win32-app Unfortunately, topic/pipe does not help. I confirmed that applying the second patch attached, which reverts to create() rather than nt_create(), and setting CYGWIN=pipe_byte fixes the problem. What do you think of this alternative implementation which does not use nt_create()? -- Takashi Yano <takashi.yano@nifty.ne.jp> [-- Attachment #2: 0001-Cygwin-select-Improve-select-poll-response.patch --] [-- Type: application/octet-stream, Size: 2460 bytes --] From c2ececea731fb633a9e10666d7546e1c61c8c4f7 Mon Sep 17 00:00:00 2001 From: Takashi Yano <takashi.yano@nifty.ne.jp> Date: Fri, 27 Aug 2021 19:54:41 +0900 Subject: [PATCH 1/2] Cygwin: select: Improve select/poll response. --- winsup/cygwin/select.cc | 32 ++++++++++++++++++++++++++++---- 1 file changed, 28 insertions(+), 4 deletions(-) diff --git a/winsup/cygwin/select.cc b/winsup/cygwin/select.cc index 8ad982c12..83e1c00e0 100644 --- a/winsup/cygwin/select.cc +++ b/winsup/cygwin/select.cc @@ -735,6 +735,7 @@ thread_pipe (void *arg) select_pipe_info *pi = (select_pipe_info *) arg; DWORD sleep_time = 0; bool looping = true; + DWORD t0 = GetTickCount (); while (looping) { @@ -754,7 +755,12 @@ thread_pipe (void *arg) break; cygwait (pi->bye, sleep_time >> 3); if (sleep_time < 80) - ++sleep_time; + { + DWORD t1 = GetTickCount (); + if (t0 != t1) + ++sleep_time; + t0 = t1; + } if (pi->stop_thread) break; } @@ -930,6 +936,7 @@ thread_fifo (void *arg) select_fifo_info *pi = (select_fifo_info *) arg; DWORD sleep_time = 0; bool looping = true; + DWORD t0 = GetTickCount (); while (looping) { @@ -949,7 +956,12 @@ thread_fifo (void *arg) break; cygwait (pi->bye, sleep_time >> 3); if (sleep_time < 80) - ++sleep_time; + { + DWORD t1 = GetTickCount (); + if (t0 != t1) + ++sleep_time; + t0 = t1; + } if (pi->stop_thread) break; } @@ -1125,6 +1137,7 @@ thread_console (void *arg) select_console_info *ci = (select_console_info *) arg; DWORD sleep_time = 0; bool looping = true; + DWORD t0 = GetTickCount (); while (looping) { @@ -1144,7 +1157,12 @@ thread_console (void *arg) break; cygwait (ci->bye, sleep_time >> 3); if (sleep_time < 80) - ++sleep_time; + { + DWORD t1 = GetTickCount (); + if (t0 != t1) + ++sleep_time; + t0 = t1; + } if (ci->stop_thread) break; } @@ -1364,6 +1382,7 @@ thread_pty_slave (void *arg) select_pipe_info *pi = (select_pipe_info *) arg; DWORD sleep_time = 0; bool looping = true; + DWORD t0 = GetTickCount (); while (looping) { @@ -1383,7 +1402,12 @@ thread_pty_slave (void *arg) break; cygwait (pi->bye, sleep_time >> 3); if (sleep_time < 80) - ++sleep_time; + { + DWORD t1 = GetTickCount (); + if (t0 != t1) + ++sleep_time; + t0 = t1; + } if (pi->stop_thread) break; } -- 2.33.0 [-- Attachment #3: 0002-Cygwin-pipe-Revert-to-create-rather-than-nt_create.patch --] [-- Type: application/octet-stream, Size: 2368 bytes --] From 784892dc272a6b7899de5f89c65bb8cfab79591d Mon Sep 17 00:00:00 2001 From: Takashi Yano <takashi.yano@nifty.ne.jp> Date: Fri, 27 Aug 2021 20:06:04 +0900 Subject: [PATCH 2/2] Cygwin: pipe: Revert to create() rather than nt_create(). --- winsup/cygwin/fhandler_pipe.cc | 24 +++++++++++++++++++++++- 1 file changed, 23 insertions(+), 1 deletion(-) diff --git a/winsup/cygwin/fhandler_pipe.cc b/winsup/cygwin/fhandler_pipe.cc index 2dec0a848..b3af4a0a0 100644 --- a/winsup/cygwin/fhandler_pipe.cc +++ b/winsup/cygwin/fhandler_pipe.cc @@ -234,6 +234,24 @@ fhandler_pipe::raw_read (void *ptr, size_t& len) return; } + if (is_nonblocking ()) + { + DWORD n; + BOOL ret = PeekNamedPipe (get_handle (), NULL, 0, NULL, &n, NULL); + if (!ret) + { + __seterrno (); + len = (size_t) -1; + return; + } + if (n == 0) + { + set_errno (EAGAIN); + len = (size_t) -1; + return; + } + } + do { len = orig_len; @@ -571,6 +589,7 @@ fhandler_pipe::create (LPSECURITY_ATTRIBUTES sa_ptr, PHANDLE r, PHANDLE w, return 0; } +#if 0 /* The next version of fhandler_pipe::create used to call the previous version. But the read handle created by the latter doesn't have FILE_WRITE_ATTRIBUTES access unless the pipe is created with @@ -588,6 +607,7 @@ fhandler_pipe::create (LPSECURITY_ATTRIBUTES sa_ptr, PHANDLE r, PHANDLE w, static int nt_create (LPSECURITY_ATTRIBUTES, PHANDLE, PHANDLE, DWORD, int64_t *); +#endif int fhandler_pipe::create (fhandler_pipe *fhs[2], unsigned psize, int mode) @@ -597,7 +617,7 @@ fhandler_pipe::create (fhandler_pipe *fhs[2], unsigned psize, int mode) int res = -1; int64_t unique_id; - int ret = nt_create (sa, &r, &w, psize, &unique_id); + int ret = create (sa, &r, &w, psize, NULL, 0, &unique_id); if (ret) __seterrno_from_win_error (ret); else if ((fhs[0] = (fhandler_pipe *) build_fh_dev (*piper_dev)) == NULL) @@ -624,6 +644,7 @@ fhandler_pipe::create (fhandler_pipe *fhs[2], unsigned psize, int mode) return res; } +#if 0 static int nt_create (LPSECURITY_ATTRIBUTES sa_ptr, PHANDLE r, PHANDLE w, DWORD psize, int64_t *unique_id) @@ -755,6 +776,7 @@ nt_create (LPSECURITY_ATTRIBUTES sa_ptr, PHANDLE r, PHANDLE w, /* Success. */ return 0; } +#endif int fhandler_pipe::ioctl (unsigned int cmd, void *p) -- 2.33.0 ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? 2021-08-27 11:24 ` Takashi Yano @ 2021-08-27 16:00 ` Ken Brown 2021-08-27 17:21 ` Takashi Yano 0 siblings, 1 reply; 86+ messages in thread From: Ken Brown @ 2021-08-27 16:00 UTC (permalink / raw) To: Takashi Yano, cygwin On 8/27/2021 7:24 AM, Takashi Yano wrote: > On Thu, 26 Aug 2021 18:18:29 -0400 > Ken Brown wrote: >> On 8/26/2021 11:56 AM, Ken Brown via Cygwin wrote: [...] >> In case you want to try out my proposed change, I've just rebased the patches to >> the current master and pushed them to a new topic/pipe branch. > > Hi Ken, > > Thanks much! I tested topic/pipe branch. > > [yano@cygwin-PC ~]$ scp test.dat yano@linux-server:. > yano@linux-server's password: > test.dat 100% 100MB 95.9MB/s 00:01 > [yano@cygwin-PC ~]$ scp yano@linux-server:test.dat . > yano@linux-server's password: > test.dat 100% 100MB 8.0MB/s 00:12 > > yano@linux-server:~$ scp yano@cygwin-PC:test.dat . > yano@cygwin-PC's password: > test.dat 100% 100MB 109.7MB/s 00:00 > yano@linux-server:~$ scp test.dat yano@cygwin-PC:. > yano@cygwin-PC's password: > test.dat 100% 100MB 31.4MB/s 00:03 > > As shown above, outgoing transfer-rate has been improved upto near > theoretical limit. However, incoming transfer-rate is not improved > much. > > I digged further and found the first patch attached solves the issue > as follows. > > [yano@cygwin-PC ~]$ scp yano@linux-server:test.dat . > yano@linux-server's password: > test.dat 100% 100MB 112.8MB/s 00:00 > > yano@linux-server2:~$ scp test.dat yano@cygwin-PC:. > yano@cygwin-PC's password: > test.dat 100% 100MB 102.5MB/s 00:00 Great! > I also tested the case: >>>> https://cygwin.com/pipermail/cygwin/2021-March/247987.html >>>> which seems to be the same issue with >>>> https://stackoverflow.com/questions/10385424/good-alternatives-to-cygwin-cygwin-doesnt-support-natively-support-win32-app > > Unfortunately, topic/pipe does not help. > > I confirmed that applying the second patch attached, which reverts > to create() rather than nt_create(), and setting CYGWIN=pipe_byte > fixes the problem. > > What do you think of this alternative implementation which does > not use nt_create()? Two years ago I thought I needed nt_create to avoid problems when calling set_pipe_non_blocking. Are you saying that's not an issue? Is set_pipe_non_blocking unnecessary? Is that the point of your modification to raw_read? Ken ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? 2021-08-27 16:00 ` Ken Brown @ 2021-08-27 17:21 ` Takashi Yano 2021-08-28 2:00 ` Takashi Yano 2021-08-28 8:43 ` Corinna Vinschen 0 siblings, 2 replies; 86+ messages in thread From: Takashi Yano @ 2021-08-27 17:21 UTC (permalink / raw) To: cygwin On Fri, 27 Aug 2021 12:00:50 -0400 Ken Brown wrote: > On 8/27/2021 7:24 AM, Takashi Yano wrote: > > On Thu, 26 Aug 2021 18:18:29 -0400 > > Ken Brown wrote: > >> On 8/26/2021 11:56 AM, Ken Brown via Cygwin wrote: > [...] > >> In case you want to try out my proposed change, I've just rebased the patches to > >> the current master and pushed them to a new topic/pipe branch. > > > > Hi Ken, > > > > Thanks much! I tested topic/pipe branch. > > > > [yano@cygwin-PC ~]$ scp test.dat yano@linux-server:. > > yano@linux-server's password: > > test.dat 100% 100MB 95.9MB/s 00:01 > > [yano@cygwin-PC ~]$ scp yano@linux-server:test.dat . > > yano@linux-server's password: > > test.dat 100% 100MB 8.0MB/s 00:12 > > > > yano@linux-server:~$ scp yano@cygwin-PC:test.dat . > > yano@cygwin-PC's password: > > test.dat 100% 100MB 109.7MB/s 00:00 > > yano@linux-server:~$ scp test.dat yano@cygwin-PC:. > > yano@cygwin-PC's password: > > test.dat 100% 100MB 31.4MB/s 00:03 > > > > As shown above, outgoing transfer-rate has been improved upto near > > theoretical limit. However, incoming transfer-rate is not improved > > much. > > > > I digged further and found the first patch attached solves the issue > > as follows. > > > > [yano@cygwin-PC ~]$ scp yano@linux-server:test.dat . > > yano@linux-server's password: > > test.dat 100% 100MB 112.8MB/s 00:00 > > > > yano@linux-server2:~$ scp test.dat yano@cygwin-PC:. > > yano@cygwin-PC's password: > > test.dat 100% 100MB 102.5MB/s 00:00 > > Great! > > > I also tested the case: > >>>> https://cygwin.com/pipermail/cygwin/2021-March/247987.html > >>>> which seems to be the same issue with > >>>> https://stackoverflow.com/questions/10385424/good-alternatives-to-cygwin-cygwin-doesnt-support-natively-support-win32-app > > > > Unfortunately, topic/pipe does not help. > > > > I confirmed that applying the second patch attached, which reverts > > to create() rather than nt_create(), and setting CYGWIN=pipe_byte > > fixes the problem. > > > > What do you think of this alternative implementation which does > > not use nt_create()? > > Two years ago I thought I needed nt_create to avoid problems when calling > set_pipe_non_blocking. Are you saying that's not an issue? Is > set_pipe_non_blocking unnecessary? Is that the point of your modification to > raw_read? Yes. Instead of making windows read function itself non-blocking, it is possible to check if the pipe can be read before read using PeekNamedPipe(). If the pipe cannot be read right now, EAGAIN is returned. -- Takashi Yano <takashi.yano@nifty.ne.jp> ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? 2021-08-27 17:21 ` Takashi Yano @ 2021-08-28 2:00 ` Takashi Yano 2021-08-28 3:03 ` Takashi Yano 2021-08-28 8:43 ` Corinna Vinschen 1 sibling, 1 reply; 86+ messages in thread From: Takashi Yano @ 2021-08-28 2:00 UTC (permalink / raw) To: cygwin On Sat, 28 Aug 2021 02:21:11 +0900 Takashi Yano wrote: > On Fri, 27 Aug 2021 12:00:50 -0400 > Ken Brown wrote: > > > On 8/27/2021 7:24 AM, Takashi Yano wrote: > > > On Thu, 26 Aug 2021 18:18:29 -0400 > > > Ken Brown wrote: > > >> On 8/26/2021 11:56 AM, Ken Brown via Cygwin wrote: > > [...] > > >> In case you want to try out my proposed change, I've just rebased the patches to > > >> the current master and pushed them to a new topic/pipe branch. > > > > > > Hi Ken, > > > > > > Thanks much! I tested topic/pipe branch. > > > > > > [yano@cygwin-PC ~]$ scp test.dat yano@linux-server:. > > > yano@linux-server's password: > > > test.dat 100% 100MB 95.9MB/s 00:01 > > > [yano@cygwin-PC ~]$ scp yano@linux-server:test.dat . > > > yano@linux-server's password: > > > test.dat 100% 100MB 8.0MB/s 00:12 > > > > > > yano@linux-server:~$ scp yano@cygwin-PC:test.dat . > > > yano@cygwin-PC's password: > > > test.dat 100% 100MB 109.7MB/s 00:00 > > > yano@linux-server:~$ scp test.dat yano@cygwin-PC:. > > > yano@cygwin-PC's password: > > > test.dat 100% 100MB 31.4MB/s 00:03 > > > > > > As shown above, outgoing transfer-rate has been improved upto near > > > theoretical limit. However, incoming transfer-rate is not improved > > > much. > > > > > > I digged further and found the first patch attached solves the issue > > > as follows. > > > > > > [yano@cygwin-PC ~]$ scp yano@linux-server:test.dat . > > > yano@linux-server's password: > > > test.dat 100% 100MB 112.8MB/s 00:00 > > > > > > yano@linux-server2:~$ scp test.dat yano@cygwin-PC:. > > > yano@cygwin-PC's password: > > > test.dat 100% 100MB 102.5MB/s 00:00 > > > > Great! > > > > > I also tested the case: > > >>>> https://cygwin.com/pipermail/cygwin/2021-March/247987.html > > >>>> which seems to be the same issue with > > >>>> https://stackoverflow.com/questions/10385424/good-alternatives-to-cygwin-cygwin-doesnt-support-natively-support-win32-app > > > > > > Unfortunately, topic/pipe does not help. > > > > > > I confirmed that applying the second patch attached, which reverts > > > to create() rather than nt_create(), and setting CYGWIN=pipe_byte > > > fixes the problem. > > > > > > What do you think of this alternative implementation which does > > > not use nt_create()? > > > > Two years ago I thought I needed nt_create to avoid problems when calling > > set_pipe_non_blocking. Are you saying that's not an issue? Is > > set_pipe_non_blocking unnecessary? Is that the point of your modification to > > raw_read? > > Yes. Instead of making windows read function itself non-blocking, > it is possible to check if the pipe can be read before read using > PeekNamedPipe(). If the pipe cannot be read right now, EAGAIN is > returned. As for writint to pipe, set_pipe_non_blocking seems to take effect and be necessary. -- Takashi Yano <takashi.yano@nifty.ne.jp> ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? 2021-08-28 2:00 ` Takashi Yano @ 2021-08-28 3:03 ` Takashi Yano 0 siblings, 0 replies; 86+ messages in thread From: Takashi Yano @ 2021-08-28 3:03 UTC (permalink / raw) To: cygwin On Sat, 28 Aug 2021 11:00:24 +0900 Takashi Yano wrote: > On Sat, 28 Aug 2021 02:21:11 +0900 > Takashi Yano wrote: > > > On Fri, 27 Aug 2021 12:00:50 -0400 > > Ken Brown wrote: > > > > > On 8/27/2021 7:24 AM, Takashi Yano wrote: > > > > On Thu, 26 Aug 2021 18:18:29 -0400 > > > > Ken Brown wrote: > > > >> On 8/26/2021 11:56 AM, Ken Brown via Cygwin wrote: > > > [...] > > > >> In case you want to try out my proposed change, I've just rebased the patches to > > > >> the current master and pushed them to a new topic/pipe branch. > > > > > > > > Hi Ken, > > > > > > > > Thanks much! I tested topic/pipe branch. > > > > > > > > [yano@cygwin-PC ~]$ scp test.dat yano@linux-server:. > > > > yano@linux-server's password: > > > > test.dat 100% 100MB 95.9MB/s 00:01 > > > > [yano@cygwin-PC ~]$ scp yano@linux-server:test.dat . > > > > yano@linux-server's password: > > > > test.dat 100% 100MB 8.0MB/s 00:12 > > > > > > > > yano@linux-server:~$ scp yano@cygwin-PC:test.dat . > > > > yano@cygwin-PC's password: > > > > test.dat 100% 100MB 109.7MB/s 00:00 > > > > yano@linux-server:~$ scp test.dat yano@cygwin-PC:. > > > > yano@cygwin-PC's password: > > > > test.dat 100% 100MB 31.4MB/s 00:03 > > > > > > > > As shown above, outgoing transfer-rate has been improved upto near > > > > theoretical limit. However, incoming transfer-rate is not improved > > > > much. > > > > > > > > I digged further and found the first patch attached solves the issue > > > > as follows. > > > > > > > > [yano@cygwin-PC ~]$ scp yano@linux-server:test.dat . > > > > yano@linux-server's password: > > > > test.dat 100% 100MB 112.8MB/s 00:00 > > > > > > > > yano@linux-server2:~$ scp test.dat yano@cygwin-PC:. > > > > yano@cygwin-PC's password: > > > > test.dat 100% 100MB 102.5MB/s 00:00 > > > > > > Great! > > > > > > > I also tested the case: > > > >>>> https://cygwin.com/pipermail/cygwin/2021-March/247987.html > > > >>>> which seems to be the same issue with > > > >>>> https://stackoverflow.com/questions/10385424/good-alternatives-to-cygwin-cygwin-doesnt-support-natively-support-win32-app > > > > > > > > Unfortunately, topic/pipe does not help. > > > > > > > > I confirmed that applying the second patch attached, which reverts > > > > to create() rather than nt_create(), and setting CYGWIN=pipe_byte > > > > fixes the problem. > > > > > > > > What do you think of this alternative implementation which does > > > > not use nt_create()? > > > > > > Two years ago I thought I needed nt_create to avoid problems when calling > > > set_pipe_non_blocking. Are you saying that's not an issue? Is > > > set_pipe_non_blocking unnecessary? Is that the point of your modification to > > > raw_read? > > > > Yes. Instead of making windows read function itself non-blocking, > > it is possible to check if the pipe can be read before read using > > PeekNamedPipe(). If the pipe cannot be read right now, EAGAIN is > > returned. > > As for writint to pipe, set_pipe_non_blocking seems to take effect > and be necessary. So, the following patch seems to be appropriate. diff --git a/winsup/cygwin/fhandler_pipe.cc b/winsup/cygwin/fhandler_pipe.cc index b3af4a0a0..82f5de3f4 100644 --- a/winsup/cygwin/fhandler_pipe.cc +++ b/winsup/cygwin/fhandler_pipe.cc @@ -48,6 +48,8 @@ fhandler_pipe::set_pipe_non_blocking (bool nonblocking) IO_STATUS_BLOCK io; FILE_PIPE_INFORMATION fpi; + if ((openflags & O_ACCMODE) == O_RDONLY) + return; /* Do nothing for read pipe */ fpi.ReadMode = FILE_PIPE_BYTE_STREAM_MODE; fpi.CompletionMode = nonblocking ? FILE_PIPE_COMPLETE_OPERATION : FILE_PIPE_QUEUE_OPERATION; -- Takashi Yano <takashi.yano@nifty.ne.jp> ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? 2021-08-27 17:21 ` Takashi Yano 2021-08-28 2:00 ` Takashi Yano @ 2021-08-28 8:43 ` Corinna Vinschen 2021-08-28 9:41 ` Takashi Yano 2021-08-28 15:17 ` Ken Brown 1 sibling, 2 replies; 86+ messages in thread From: Corinna Vinschen @ 2021-08-28 8:43 UTC (permalink / raw) To: cygwin On Aug 28 02:21, Takashi Yano via Cygwin wrote: > On Fri, 27 Aug 2021 12:00:50 -0400 > Ken Brown wrote: > > Two years ago I thought I needed nt_create to avoid problems when calling > > set_pipe_non_blocking. Are you saying that's not an issue? Is > > set_pipe_non_blocking unnecessary? Is that the point of your modification to > > raw_read? > > Yes. Instead of making windows read function itself non-blocking, > it is possible to check if the pipe can be read before read using > PeekNamedPipe(). If the pipe cannot be read right now, EAGAIN is > returned. The problem is this: if (PeekNamedPipe()) ReadFile(blocking); is not atomic. I. e., if PeekNamedPipe succeeds, nothing keeps another thread from draining the pipe between the PeekNamedPipe and the ReadFile call. And as soon as ReadFile runs, it hangs indefinitely and we can't stop it via a signal. Is a blocking ReadFile actually faster than a non-blocking read? Or does it mainly depend on BYTE vs. MESSAGE mode? What if the pipe is created non-blocking and stays non-blocking all the time and uses BYTE mode all the time? Just as sockets, it would always only emulate blocking mode. Wouldn't that drop code size a lot and fix most problems? Corinna ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? 2021-08-28 8:43 ` Corinna Vinschen @ 2021-08-28 9:41 ` Takashi Yano 2021-08-28 11:58 ` Corinna Vinschen 2021-08-29 9:07 ` Takashi Yano 2021-08-28 15:17 ` Ken Brown 1 sibling, 2 replies; 86+ messages in thread From: Takashi Yano @ 2021-08-28 9:41 UTC (permalink / raw) To: cygwin On Sat, 28 Aug 2021 10:43:27 +0200 Corinna Vinschen wrote: > On Aug 28 02:21, Takashi Yano via Cygwin wrote: > > On Fri, 27 Aug 2021 12:00:50 -0400 > > Ken Brown wrote: > > > Two years ago I thought I needed nt_create to avoid problems when calling > > > set_pipe_non_blocking. Are you saying that's not an issue? Is > > > set_pipe_non_blocking unnecessary? Is that the point of your modification to > > > raw_read? > > > > Yes. Instead of making windows read function itself non-blocking, > > it is possible to check if the pipe can be read before read using > > PeekNamedPipe(). If the pipe cannot be read right now, EAGAIN is > > returned. > > The problem is this: > > if (PeekNamedPipe()) > ReadFile(blocking); > > is not atomic. I. e., if PeekNamedPipe succeeds, nothing keeps another > thread from draining the pipe between the PeekNamedPipe and the ReadFile > call. And as soon as ReadFile runs, it hangs indefinitely and we can't > stop it via a signal. Hmm, you are right. Mutex guard seems to be necessary like pty code if we go this way. > Is a blocking ReadFile actually faster than a non-blocking read? Or > does it mainly depend on BYTE vs. MESSAGE mode? Actually, I don't think so. Perhaps it is not essential problem of overlapped I/O but something is wrong with current pipe code. > What if the pipe is created non-blocking and stays non-blocking all the > time and uses BYTE mode all the time? Just as sockets, it would always > only emulate blocking mode. Wouldn't that drop code size a lot and fix > most problems? If 'non-blocking' means overlapped I/O, only the problem will be: https://cygwin.com/pipermail/cygwin/2021-March/247987.html -- Takashi Yano <takashi.yano@nifty.ne.jp> ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? 2021-08-28 9:41 ` Takashi Yano @ 2021-08-28 11:58 ` Corinna Vinschen 2021-08-28 15:43 ` Takashi Yano 2021-08-29 9:07 ` Takashi Yano 1 sibling, 1 reply; 86+ messages in thread From: Corinna Vinschen @ 2021-08-28 11:58 UTC (permalink / raw) To: cygwin On Aug 28 18:41, Takashi Yano via Cygwin wrote: > On Sat, 28 Aug 2021 10:43:27 +0200 > Corinna Vinschen wrote: > > On Aug 28 02:21, Takashi Yano via Cygwin wrote: > > > On Fri, 27 Aug 2021 12:00:50 -0400 > > > Ken Brown wrote: > > > > Two years ago I thought I needed nt_create to avoid problems when calling > > > > set_pipe_non_blocking. Are you saying that's not an issue? Is > > > > set_pipe_non_blocking unnecessary? Is that the point of your modification to > > > > raw_read? > > > > > > Yes. Instead of making windows read function itself non-blocking, > > > it is possible to check if the pipe can be read before read using > > > PeekNamedPipe(). If the pipe cannot be read right now, EAGAIN is > > > returned. > > > > The problem is this: > > > > if (PeekNamedPipe()) > > ReadFile(blocking); > > > > is not atomic. I. e., if PeekNamedPipe succeeds, nothing keeps another > > thread from draining the pipe between the PeekNamedPipe and the ReadFile > > call. And as soon as ReadFile runs, it hangs indefinitely and we can't > > stop it via a signal. > > Hmm, you are right. Mutex guard seems to be necessary like pty code > if we go this way. > > > Is a blocking ReadFile actually faster than a non-blocking read? Or > > does it mainly depend on BYTE vs. MESSAGE mode? > > Actually, I don't think so. Perhaps it is not essential problem of > overlapped I/O but something is wrong with current pipe code. > > > What if the pipe is created non-blocking and stays non-blocking all the > > time and uses BYTE mode all the time? Just as sockets, it would always > > only emulate blocking mode. Wouldn't that drop code size a lot and fix > > most problems? > > If 'non-blocking' means overlapped I/O, only the problem will be: > https://cygwin.com/pipermail/cygwin/2021-March/247987.html Sorry if that wasn't clear, but I was not talking about overlapped I/O, which we should get rid off, but of real non-blocking mode, which Windows pipes are fortunately capable of. Corinna ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? 2021-08-28 11:58 ` Corinna Vinschen @ 2021-08-28 15:43 ` Takashi Yano 2021-08-28 20:55 ` Ken Brown 2021-08-29 10:27 ` Corinna Vinschen 0 siblings, 2 replies; 86+ messages in thread From: Takashi Yano @ 2021-08-28 15:43 UTC (permalink / raw) To: cygwin On Sat, 28 Aug 2021 13:58:08 +0200 Corinna Vinschen wrote: > On Aug 28 18:41, Takashi Yano via Cygwin wrote: > > On Sat, 28 Aug 2021 10:43:27 +0200 > > Corinna Vinschen wrote: > > > On Aug 28 02:21, Takashi Yano via Cygwin wrote: > > > > On Fri, 27 Aug 2021 12:00:50 -0400 > > > > Ken Brown wrote: > > > > > Two years ago I thought I needed nt_create to avoid problems when calling > > > > > set_pipe_non_blocking. Are you saying that's not an issue? Is > > > > > set_pipe_non_blocking unnecessary? Is that the point of your modification to > > > > > raw_read? > > > > > > > > Yes. Instead of making windows read function itself non-blocking, > > > > it is possible to check if the pipe can be read before read using > > > > PeekNamedPipe(). If the pipe cannot be read right now, EAGAIN is > > > > returned. > > > > > > The problem is this: > > > > > > if (PeekNamedPipe()) > > > ReadFile(blocking); > > > > > > is not atomic. I. e., if PeekNamedPipe succeeds, nothing keeps another > > > thread from draining the pipe between the PeekNamedPipe and the ReadFile > > > call. And as soon as ReadFile runs, it hangs indefinitely and we can't > > > stop it via a signal. > > > > Hmm, you are right. Mutex guard seems to be necessary like pty code > > if we go this way. > > > > > Is a blocking ReadFile actually faster than a non-blocking read? Or > > > does it mainly depend on BYTE vs. MESSAGE mode? > > > > Actually, I don't think so. Perhaps it is not essential problem of > > overlapped I/O but something is wrong with current pipe code. > > > > > What if the pipe is created non-blocking and stays non-blocking all the > > > time and uses BYTE mode all the time? Just as sockets, it would always > > > only emulate blocking mode. Wouldn't that drop code size a lot and fix > > > most problems? > > > > If 'non-blocking' means overlapped I/O, only the problem will be: > > https://cygwin.com/pipermail/cygwin/2021-March/247987.html > > Sorry if that wasn't clear, but I was not talking about overlapped I/O, > which we should get rid off, but of real non-blocking mode, which > Windows pipes are fortunately capable of. Do you mean, PIPE_NOWAIT flag? If this flags is specified in the read pipe, non-cygwin apps cannot read the pipe correctly. -- Takashi Yano <takashi.yano@nifty.ne.jp> ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? 2021-08-28 15:43 ` Takashi Yano @ 2021-08-28 20:55 ` Ken Brown 2021-08-29 8:41 ` Takashi Yano 2021-08-29 10:27 ` Corinna Vinschen 1 sibling, 1 reply; 86+ messages in thread From: Ken Brown @ 2021-08-28 20:55 UTC (permalink / raw) To: cygwin On 8/28/2021 11:43 AM, Takashi Yano via Cygwin wrote: > On Sat, 28 Aug 2021 13:58:08 +0200 > Corinna Vinschen wrote: >> On Aug 28 18:41, Takashi Yano via Cygwin wrote: >>> On Sat, 28 Aug 2021 10:43:27 +0200 >>> Corinna Vinschen wrote: >>>> On Aug 28 02:21, Takashi Yano via Cygwin wrote: >>>>> On Fri, 27 Aug 2021 12:00:50 -0400 >>>>> Ken Brown wrote: >>>>>> Two years ago I thought I needed nt_create to avoid problems when calling >>>>>> set_pipe_non_blocking. Are you saying that's not an issue? Is >>>>>> set_pipe_non_blocking unnecessary? Is that the point of your modification to >>>>>> raw_read? >>>>> >>>>> Yes. Instead of making windows read function itself non-blocking, >>>>> it is possible to check if the pipe can be read before read using >>>>> PeekNamedPipe(). If the pipe cannot be read right now, EAGAIN is >>>>> returned. >>>> >>>> The problem is this: >>>> >>>> if (PeekNamedPipe()) >>>> ReadFile(blocking); >>>> >>>> is not atomic. I. e., if PeekNamedPipe succeeds, nothing keeps another >>>> thread from draining the pipe between the PeekNamedPipe and the ReadFile >>>> call. And as soon as ReadFile runs, it hangs indefinitely and we can't >>>> stop it via a signal. >>> >>> Hmm, you are right. Mutex guard seems to be necessary like pty code >>> if we go this way. >>> >>>> Is a blocking ReadFile actually faster than a non-blocking read? Or >>>> does it mainly depend on BYTE vs. MESSAGE mode? >>> >>> Actually, I don't think so. Perhaps it is not essential problem of >>> overlapped I/O but something is wrong with current pipe code. >>> >>>> What if the pipe is created non-blocking and stays non-blocking all the >>>> time and uses BYTE mode all the time? Just as sockets, it would always >>>> only emulate blocking mode. Wouldn't that drop code size a lot and fix >>>> most problems? >>> >>> If 'non-blocking' means overlapped I/O, only the problem will be: >>> https://cygwin.com/pipermail/cygwin/2021-March/247987.html >> >> Sorry if that wasn't clear, but I was not talking about overlapped I/O, >> which we should get rid off, but of real non-blocking mode, which >> Windows pipes are fortunately capable of. > > Do you mean, PIPE_NOWAIT flag? If this flags is specified in > the read pipe, non-cygwin apps cannot read the pipe correctly. While waiting for Corinna's response to this, I have one more question. Do you understand why nt_create() failed and you had to revert to create()? Was it an access problem because nt_create requested FILE_WRITE_ATTRIBUTES? Or did I make some careless mistake in writing nt_create? Ken ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? 2021-08-28 20:55 ` Ken Brown @ 2021-08-29 8:41 ` Takashi Yano 2021-08-29 22:42 ` Ken Brown 0 siblings, 1 reply; 86+ messages in thread From: Takashi Yano @ 2021-08-29 8:41 UTC (permalink / raw) To: cygwin Hi Ken, On Sat, 28 Aug 2021 16:55:52 -0400 Ken Brown wrote: > On 8/28/2021 11:43 AM, Takashi Yano via Cygwin wrote: > > On Sat, 28 Aug 2021 13:58:08 +0200 > > Corinna Vinschen wrote: > >> On Aug 28 18:41, Takashi Yano via Cygwin wrote: > >>> On Sat, 28 Aug 2021 10:43:27 +0200 > >>> Corinna Vinschen wrote: > >>>> On Aug 28 02:21, Takashi Yano via Cygwin wrote: > >>>>> On Fri, 27 Aug 2021 12:00:50 -0400 > >>>>> Ken Brown wrote: > >>>>>> Two years ago I thought I needed nt_create to avoid problems when calling > >>>>>> set_pipe_non_blocking. Are you saying that's not an issue? Is > >>>>>> set_pipe_non_blocking unnecessary? Is that the point of your modification to > >>>>>> raw_read? > >>>>> > >>>>> Yes. Instead of making windows read function itself non-blocking, > >>>>> it is possible to check if the pipe can be read before read using > >>>>> PeekNamedPipe(). If the pipe cannot be read right now, EAGAIN is > >>>>> returned. > >>>> > >>>> The problem is this: > >>>> > >>>> if (PeekNamedPipe()) > >>>> ReadFile(blocking); > >>>> > >>>> is not atomic. I. e., if PeekNamedPipe succeeds, nothing keeps another > >>>> thread from draining the pipe between the PeekNamedPipe and the ReadFile > >>>> call. And as soon as ReadFile runs, it hangs indefinitely and we can't > >>>> stop it via a signal. > >>> > >>> Hmm, you are right. Mutex guard seems to be necessary like pty code > >>> if we go this way. > >>> > >>>> Is a blocking ReadFile actually faster than a non-blocking read? Or > >>>> does it mainly depend on BYTE vs. MESSAGE mode? > >>> > >>> Actually, I don't think so. Perhaps it is not essential problem of > >>> overlapped I/O but something is wrong with current pipe code. > >>> > >>>> What if the pipe is created non-blocking and stays non-blocking all the > >>>> time and uses BYTE mode all the time? Just as sockets, it would always > >>>> only emulate blocking mode. Wouldn't that drop code size a lot and fix > >>>> most problems? > >>> > >>> If 'non-blocking' means overlapped I/O, only the problem will be: > >>> https://cygwin.com/pipermail/cygwin/2021-March/247987.html > >> > >> Sorry if that wasn't clear, but I was not talking about overlapped I/O, > >> which we should get rid off, but of real non-blocking mode, which > >> Windows pipes are fortunately capable of. > > > > Do you mean, PIPE_NOWAIT flag? If this flags is specified in > > the read pipe, non-cygwin apps cannot read the pipe correctly. > > While waiting for Corinna's response to this, I have one more question. Do you > understand why nt_create() failed and you had to revert to create()? Was it an > access problem because nt_create requested FILE_WRITE_ATTRIBUTES? Or did I make > some careless mistake in writing nt_create? I am sorry but no. I don't understand why piping C# program via the pipe created by nt_create() has the issue. I tried to change setup parameters in nt_create(), however, I did not succeed it to work. I also couldn't find any mistake in nt_create() so far. Win32 programs which use ReadFile() and WriteFile() work even with the pipe created by nt_create() as well as overlapped I/O. What does C# program differ from legacy win32 program at all? -- Takashi Yano <takashi.yano@nifty.ne.jp> ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? 2021-08-29 8:41 ` Takashi Yano @ 2021-08-29 22:42 ` Ken Brown 0 siblings, 0 replies; 86+ messages in thread From: Ken Brown @ 2021-08-29 22:42 UTC (permalink / raw) To: cygwin On 8/29/2021 4:41 AM, Takashi Yano wrote: > Hi Ken, > > On Sat, 28 Aug 2021 16:55:52 -0400 > Ken Brown wrote: >> On 8/28/2021 11:43 AM, Takashi Yano via Cygwin wrote: >>> On Sat, 28 Aug 2021 13:58:08 +0200 >>> Corinna Vinschen wrote: >>>> On Aug 28 18:41, Takashi Yano via Cygwin wrote: >>>>> On Sat, 28 Aug 2021 10:43:27 +0200 >>>>> Corinna Vinschen wrote: >>>>>> On Aug 28 02:21, Takashi Yano via Cygwin wrote: >>>>>>> On Fri, 27 Aug 2021 12:00:50 -0400 >>>>>>> Ken Brown wrote: >>>>>>>> Two years ago I thought I needed nt_create to avoid problems when calling >>>>>>>> set_pipe_non_blocking. Are you saying that's not an issue? Is >>>>>>>> set_pipe_non_blocking unnecessary? Is that the point of your modification to >>>>>>>> raw_read? >>>>>>> >>>>>>> Yes. Instead of making windows read function itself non-blocking, >>>>>>> it is possible to check if the pipe can be read before read using >>>>>>> PeekNamedPipe(). If the pipe cannot be read right now, EAGAIN is >>>>>>> returned. >>>>>> >>>>>> The problem is this: >>>>>> >>>>>> if (PeekNamedPipe()) >>>>>> ReadFile(blocking); >>>>>> >>>>>> is not atomic. I. e., if PeekNamedPipe succeeds, nothing keeps another >>>>>> thread from draining the pipe between the PeekNamedPipe and the ReadFile >>>>>> call. And as soon as ReadFile runs, it hangs indefinitely and we can't >>>>>> stop it via a signal. >>>>> >>>>> Hmm, you are right. Mutex guard seems to be necessary like pty code >>>>> if we go this way. >>>>> >>>>>> Is a blocking ReadFile actually faster than a non-blocking read? Or >>>>>> does it mainly depend on BYTE vs. MESSAGE mode? >>>>> >>>>> Actually, I don't think so. Perhaps it is not essential problem of >>>>> overlapped I/O but something is wrong with current pipe code. >>>>> >>>>>> What if the pipe is created non-blocking and stays non-blocking all the >>>>>> time and uses BYTE mode all the time? Just as sockets, it would always >>>>>> only emulate blocking mode. Wouldn't that drop code size a lot and fix >>>>>> most problems? >>>>> >>>>> If 'non-blocking' means overlapped I/O, only the problem will be: >>>>> https://cygwin.com/pipermail/cygwin/2021-March/247987.html >>>> >>>> Sorry if that wasn't clear, but I was not talking about overlapped I/O, >>>> which we should get rid off, but of real non-blocking mode, which >>>> Windows pipes are fortunately capable of. >>> >>> Do you mean, PIPE_NOWAIT flag? If this flags is specified in >>> the read pipe, non-cygwin apps cannot read the pipe correctly. >> >> While waiting for Corinna's response to this, I have one more question. Do you >> understand why nt_create() failed and you had to revert to create()? Was it an >> access problem because nt_create requested FILE_WRITE_ATTRIBUTES? Or did I make >> some careless mistake in writing nt_create? > > I am sorry but no. I don't understand why piping C# program via > the pipe created by nt_create() has the issue. I tried to change > setup parameters in nt_create(), however, I did not succeed it to > work. I also couldn't find any mistake in nt_create() so far. > > Win32 programs which use ReadFile() and WriteFile() work even > with the pipe created by nt_create() as well as overlapped I/O. > > What does C# program differ from legacy win32 program at all? I don't know. By the way, when I introduced nt_create(), my preference would have been to simply change create() to use the NT API, but I was afraid to do that because I didn't want to take a chance on breaking something. That's still my preference, if we can find a way to work around this problem with C# programs. Ken ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? 2021-08-28 15:43 ` Takashi Yano 2021-08-28 20:55 ` Ken Brown @ 2021-08-29 10:27 ` Corinna Vinschen 1 sibling, 0 replies; 86+ messages in thread From: Corinna Vinschen @ 2021-08-29 10:27 UTC (permalink / raw) To: cygwin On Aug 29 00:43, Takashi Yano via Cygwin wrote: > On Sat, 28 Aug 2021 13:58:08 +0200 > Corinna Vinschen wrote: > > On Aug 28 18:41, Takashi Yano via Cygwin wrote: > > > On Sat, 28 Aug 2021 10:43:27 +0200 > > > Corinna Vinschen wrote: > > > > On Aug 28 02:21, Takashi Yano via Cygwin wrote: > > > > > On Fri, 27 Aug 2021 12:00:50 -0400 > > > > > Ken Brown wrote: > > > > > > Two years ago I thought I needed nt_create to avoid problems when calling > > > > > > set_pipe_non_blocking. Are you saying that's not an issue? Is > > > > > > set_pipe_non_blocking unnecessary? Is that the point of your modification to > > > > > > raw_read? > > > > > > > > > > Yes. Instead of making windows read function itself non-blocking, > > > > > it is possible to check if the pipe can be read before read using > > > > > PeekNamedPipe(). If the pipe cannot be read right now, EAGAIN is > > > > > returned. > > > > > > > > The problem is this: > > > > > > > > if (PeekNamedPipe()) > > > > ReadFile(blocking); > > > > > > > > is not atomic. I. e., if PeekNamedPipe succeeds, nothing keeps another > > > > thread from draining the pipe between the PeekNamedPipe and the ReadFile > > > > call. And as soon as ReadFile runs, it hangs indefinitely and we can't > > > > stop it via a signal. > > > > > > Hmm, you are right. Mutex guard seems to be necessary like pty code > > > if we go this way. > > > > > > > Is a blocking ReadFile actually faster than a non-blocking read? Or > > > > does it mainly depend on BYTE vs. MESSAGE mode? > > > > > > Actually, I don't think so. Perhaps it is not essential problem of > > > overlapped I/O but something is wrong with current pipe code. > > > > > > > What if the pipe is created non-blocking and stays non-blocking all the > > > > time and uses BYTE mode all the time? Just as sockets, it would always > > > > only emulate blocking mode. Wouldn't that drop code size a lot and fix > > > > most problems? > > > > > > If 'non-blocking' means overlapped I/O, only the problem will be: > > > https://cygwin.com/pipermail/cygwin/2021-March/247987.html > > > > Sorry if that wasn't clear, but I was not talking about overlapped I/O, > > which we should get rid off, but of real non-blocking mode, which > > Windows pipes are fortunately capable of. > > Do you mean, PIPE_NOWAIT flag? If this flags is specified in > the read pipe, non-cygwin apps cannot read the pipe correctly. Do you mean after execve? If the child is a non-Cygwin process, we could run a loop over the stdio descriptors in child_info_spawn::worker setting pipes to blocking. But that's a corner problem, easily solved. It's more important that it works nicely for Cygwin processes. Corinna ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? 2021-08-28 9:41 ` Takashi Yano 2021-08-28 11:58 ` Corinna Vinschen @ 2021-08-29 9:07 ` Takashi Yano 2021-08-29 15:57 ` Ken Brown 2021-08-29 21:04 ` Ken Brown 1 sibling, 2 replies; 86+ messages in thread From: Takashi Yano @ 2021-08-29 9:07 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 1555 bytes --] On Sat, 28 Aug 2021 18:41:02 +0900 Takashi Yano wrote: > On Sat, 28 Aug 2021 10:43:27 +0200 > Corinna Vinschen wrote: > > On Aug 28 02:21, Takashi Yano via Cygwin wrote: > > > On Fri, 27 Aug 2021 12:00:50 -0400 > > > Ken Brown wrote: > > > > Two years ago I thought I needed nt_create to avoid problems when calling > > > > set_pipe_non_blocking. Are you saying that's not an issue? Is > > > > set_pipe_non_blocking unnecessary? Is that the point of your modification to > > > > raw_read? > > > > > > Yes. Instead of making windows read function itself non-blocking, > > > it is possible to check if the pipe can be read before read using > > > PeekNamedPipe(). If the pipe cannot be read right now, EAGAIN is > > > returned. > > > > The problem is this: > > > > if (PeekNamedPipe()) > > ReadFile(blocking); > > > > is not atomic. I. e., if PeekNamedPipe succeeds, nothing keeps another > > thread from draining the pipe between the PeekNamedPipe and the ReadFile > > call. And as soon as ReadFile runs, it hangs indefinitely and we can't > > stop it via a signal. > > Hmm, you are right. Mutex guard seems to be necessary like pty code > if we go this way. I have found that set_pipe_non_blocking() succeeds for both read and write pipes if the write pipe is created by CreateNamedPipe() and the read pipe is created by CreateFile() contrary to the current create() code. Therefore, not only nt_create() but also PeekNamedPipe() become unnecessary. Please see the revised patch attached. -- Takashi Yano <takashi.yano@nifty.ne.jp> [-- Attachment #2: v2-0002-Cygwin-pipe-Revert-to-create-rather-than-nt_creat.patch --] [-- Type: application/octet-stream, Size: 8881 bytes --] From 3f4ed10f86c88e1194a5f11201a8e088ce53d127 Mon Sep 17 00:00:00 2001 From: Takashi Yano <takashi.yano@nifty.ne.jp> Date: Sun, 29 Aug 2021 17:23:57 +0900 Subject: [PATCH v2 2/2] Cygwin: pipe: Revert to create() rather than nt_create(). --- winsup/cygwin/fhandler_pipe.cc | 192 ++++----------------------------- 1 file changed, 21 insertions(+), 171 deletions(-) diff --git a/winsup/cygwin/fhandler_pipe.cc b/winsup/cygwin/fhandler_pipe.cc index 2dec0a848..a6ece17a6 100644 --- a/winsup/cygwin/fhandler_pipe.cc +++ b/winsup/cygwin/fhandler_pipe.cc @@ -463,13 +463,13 @@ fhandler_pipe::create (LPSECURITY_ATTRIBUTES sa_ptr, PHANDLE r, PHANDLE w, if (name) len += __small_sprintf (pipename + len, "%s", name); - open_mode |= PIPE_ACCESS_INBOUND | FILE_FLAG_FIRST_PIPE_INSTANCE; + open_mode |= PIPE_ACCESS_OUTBOUND | FILE_FLAG_FIRST_PIPE_INSTANCE; /* Retry CreateNamedPipe as long as the pipe name is in use. Retrying will probably never be necessary, but we want to be as robust as possible. */ DWORD err = 0; - while (r && !*r) + while (w && !*w) { static volatile ULONG pipe_unique_id; if (!name) @@ -498,12 +498,12 @@ fhandler_pipe::create (LPSECURITY_ATTRIBUTES sa_ptr, PHANDLE r, PHANDLE w, definitely required for pty handling since fhandler_pty_master writes to the pipe in chunks, terminated by newline when CANON mode is specified. */ - *r = CreateNamedPipe (pipename, open_mode, pipe_mode, 1, psize, + *w = CreateNamedPipe (pipename, open_mode, pipe_mode, 1, psize, psize, NMPWAIT_USE_DEFAULT_WAIT, sa_ptr); - if (*r != INVALID_HANDLE_VALUE) + if (*w != INVALID_HANDLE_VALUE) { - debug_printf ("pipe read handle %p", *r); + debug_printf ("pipe write handle %p", *w); err = 0; break; } @@ -516,14 +516,14 @@ fhandler_pipe::create (LPSECURITY_ATTRIBUTES sa_ptr, PHANDLE r, PHANDLE w, Pick a new name and retry. */ debug_printf ("pipe busy", !name ? ", retrying" : ""); if (!name) - *r = NULL; + *w = NULL; break; case ERROR_ACCESS_DENIED: /* The pipe is already open with incompatible parameters. Pick a new name and retry. */ debug_printf ("pipe access denied%s", !name ? ", retrying" : ""); if (!name) - *r = NULL; + *w = NULL; break; default: { @@ -535,60 +535,42 @@ fhandler_pipe::create (LPSECURITY_ATTRIBUTES sa_ptr, PHANDLE r, PHANDLE w, if (err) { - *r = NULL; + *w = NULL; return err; } - if (!w) - debug_printf ("pipe write handle NULL"); + if (!r) + debug_printf ("pipe read handle NULL"); else { debug_printf ("CreateFile: name %s", pipename); - /* Open the named pipe for writing. - Be sure to permit FILE_READ_ATTRIBUTES access. */ - DWORD access = GENERIC_WRITE | FILE_READ_ATTRIBUTES; + /* Open the named pipe for reading. + Be sure to permit FILE_WRITE_ATTRIBUTES access. */ + DWORD access = GENERIC_READ | FILE_WRITE_ATTRIBUTES; if ((open_mode & PIPE_ACCESS_DUPLEX) == PIPE_ACCESS_DUPLEX) - access |= GENERIC_READ | FILE_WRITE_ATTRIBUTES; - *w = CreateFile (pipename, access, 0, sa_ptr, OPEN_EXISTING, + access |= GENERIC_WRITE | FILE_READ_ATTRIBUTES; + *r = CreateFile (pipename, access, 0, sa_ptr, OPEN_EXISTING, open_mode & FILE_FLAG_OVERLAPPED, 0); - if (!*w || *w == INVALID_HANDLE_VALUE) + if (!*r || *r == INVALID_HANDLE_VALUE) { /* Failure. */ DWORD err = GetLastError (); debug_printf ("CreateFile failed, r %p, %E", r); - if (r) - CloseHandle (*r); - *w = NULL; + if (w) + CloseHandle (*w); + *r = NULL; return err; } - debug_printf ("pipe write handle %p", *w); + debug_printf ("pipe read handle %p", *r); } /* Success. */ return 0; } -/* The next version of fhandler_pipe::create used to call the previous - version. But the read handle created by the latter doesn't have - FILE_WRITE_ATTRIBUTES access unless the pipe is created with - PIPE_ACCESS_DUPLEX, and it doesn't seem possible to add that access - right. This causes set_pipe_non_blocking to fail. - - To fix this we will define a helper function 'nt_create' that is - similar to the above fhandler_pipe::create but uses - NtCreateNamedPipeFile instead of CreateNamedPipe; this gives more - flexibility in setting the access rights. We will use this helper - function in the version of fhandler_pipe::create below, which - suffices for all of our uses of set_pipe_non_blocking. For - simplicity, nt_create will omit the 'open_mode' and 'name' - parameters, which aren't needed for our purposes. */ - -static int nt_create (LPSECURITY_ATTRIBUTES, PHANDLE, PHANDLE, DWORD, - int64_t *); - int fhandler_pipe::create (fhandler_pipe *fhs[2], unsigned psize, int mode) { @@ -597,7 +579,7 @@ fhandler_pipe::create (fhandler_pipe *fhs[2], unsigned psize, int mode) int res = -1; int64_t unique_id; - int ret = nt_create (sa, &r, &w, psize, &unique_id); + int ret = create (sa, &r, &w, psize, NULL, 0, &unique_id); if (ret) __seterrno_from_win_error (ret); else if ((fhs[0] = (fhandler_pipe *) build_fh_dev (*piper_dev)) == NULL) @@ -624,138 +606,6 @@ fhandler_pipe::create (fhandler_pipe *fhs[2], unsigned psize, int mode) return res; } -static int -nt_create (LPSECURITY_ATTRIBUTES sa_ptr, PHANDLE r, PHANDLE w, - DWORD psize, int64_t *unique_id) -{ - NTSTATUS status; - HANDLE npfsh; - ACCESS_MASK access; - OBJECT_ATTRIBUTES attr; - IO_STATUS_BLOCK io; - LARGE_INTEGER timeout; - - /* Default to error. */ - if (r) - *r = NULL; - if (w) - *w = NULL; - - status = fhandler_base::npfs_handle (npfsh); - if (!NT_SUCCESS (status)) - { - __seterrno_from_nt_status (status); - return GetLastError (); - } - - /* Ensure that there is enough pipe buffer space for atomic writes. */ - if (!psize) - psize = DEFAULT_PIPEBUFSIZE; - - UNICODE_STRING pipename; - WCHAR pipename_buf[MAX_PATH]; - size_t len = __small_swprintf (pipename_buf, L"%S-%u-", - &cygheap->installation_key, - GetCurrentProcessId ()); - - access = GENERIC_READ | FILE_WRITE_ATTRIBUTES; - - ULONG pipe_type = pipe_byte ? FILE_PIPE_BYTE_STREAM_TYPE - : FILE_PIPE_MESSAGE_TYPE; - - /* Retry NtCreateNamedPipeFile as long as the pipe name is in use. - Retrying will probably never be necessary, but we want - to be as robust as possible. */ - DWORD err = 0; - while (r && !*r) - { - static volatile ULONG pipe_unique_id; - LONG id = InterlockedIncrement ((LONG *) &pipe_unique_id); - __small_swprintf (pipename_buf + len, L"pipe-nt-%p", id); - if (unique_id) - *unique_id = ((int64_t) id << 32 | GetCurrentProcessId ()); - - debug_printf ("name %W, size %u, mode %s", pipename_buf, psize, - (pipe_type & FILE_PIPE_MESSAGE_TYPE) - ? "PIPE_TYPE_MESSAGE" : "PIPE_TYPE_BYTE"); - - RtlInitUnicodeString (&pipename, pipename_buf); - - InitializeObjectAttributes (&attr, &pipename, - sa_ptr->bInheritHandle ? OBJ_INHERIT : 0, - npfsh, sa_ptr->lpSecurityDescriptor); - - timeout.QuadPart = -500000; - status = NtCreateNamedPipeFile (r, access, &attr, &io, - FILE_SHARE_READ | FILE_SHARE_WRITE, - FILE_CREATE, 0, pipe_type, - FILE_PIPE_BYTE_STREAM_MODE, - 0, 1, psize, psize, &timeout); - - if (NT_SUCCESS (status)) - { - debug_printf ("pipe read handle %p", *r); - err = 0; - break; - } - - switch (status) - { - case STATUS_PIPE_BUSY: - case STATUS_INSTANCE_NOT_AVAILABLE: - case STATUS_PIPE_NOT_AVAILABLE: - /* The pipe is already open with compatible parameters. - Pick a new name and retry. */ - debug_printf ("pipe busy, retrying"); - *r = NULL; - break; - case STATUS_ACCESS_DENIED: - /* The pipe is already open with incompatible parameters. - Pick a new name and retry. */ - debug_printf ("pipe access denied, retrying"); - *r = NULL; - break; - default: - { - __seterrno_from_nt_status (status); - err = GetLastError (); - debug_printf ("failed, %E"); - *r = INVALID_HANDLE_VALUE; - } - } - } - - if (err) - { - *r = NULL; - return err; - } - - if (!w) - debug_printf ("pipe write handle NULL"); - else - { - debug_printf ("NtOpenFile: name %S", &pipename); - - access = GENERIC_WRITE | FILE_READ_ATTRIBUTES; - status = NtOpenFile (w, access, &attr, &io, 0, 0); - if (!NT_SUCCESS (status)) - { - DWORD err = GetLastError (); - debug_printf ("NtOpenFile failed, r %p, %E", r); - if (r) - CloseHandle (*r); - *w = NULL; - return err; - } - - debug_printf ("pipe write handle %p", *w); - } - - /* Success. */ - return 0; -} - int fhandler_pipe::ioctl (unsigned int cmd, void *p) { -- 2.33.0 ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? 2021-08-29 9:07 ` Takashi Yano @ 2021-08-29 15:57 ` Ken Brown 2021-08-29 19:24 ` Chris Roehrig 2021-08-29 19:37 ` cygrunsrv + sshd + rsync = 20 times too slow -- throttled? Takashi Yano 2021-08-29 21:04 ` Ken Brown 1 sibling, 2 replies; 86+ messages in thread From: Ken Brown @ 2021-08-29 15:57 UTC (permalink / raw) To: cygwin On 8/29/2021 5:07 AM, Takashi Yano via Cygwin wrote: > On Sat, 28 Aug 2021 18:41:02 +0900 > Takashi Yano wrote: >> On Sat, 28 Aug 2021 10:43:27 +0200 >> Corinna Vinschen wrote: >>> On Aug 28 02:21, Takashi Yano via Cygwin wrote: >>>> On Fri, 27 Aug 2021 12:00:50 -0400 >>>> Ken Brown wrote: >>>>> Two years ago I thought I needed nt_create to avoid problems when calling >>>>> set_pipe_non_blocking. Are you saying that's not an issue? Is >>>>> set_pipe_non_blocking unnecessary? Is that the point of your modification to >>>>> raw_read? >>>> >>>> Yes. Instead of making windows read function itself non-blocking, >>>> it is possible to check if the pipe can be read before read using >>>> PeekNamedPipe(). If the pipe cannot be read right now, EAGAIN is >>>> returned. >>> >>> The problem is this: >>> >>> if (PeekNamedPipe()) >>> ReadFile(blocking); >>> >>> is not atomic. I. e., if PeekNamedPipe succeeds, nothing keeps another >>> thread from draining the pipe between the PeekNamedPipe and the ReadFile >>> call. And as soon as ReadFile runs, it hangs indefinitely and we can't >>> stop it via a signal. >> >> Hmm, you are right. Mutex guard seems to be necessary like pty code >> if we go this way. > > I have found that set_pipe_non_blocking() succeeds for both read and > write pipes if the write pipe is created by CreateNamedPipe() and the > read pipe is created by CreateFile() contrary to the current create() > code. Therefore, not only nt_create() but also PeekNamedPipe() become > unnecessary. > > Please see the revised patch attached. That's a great idea. I've applied your two patches to the topic/pipe branch. I also rebased it and did a forced push in order to bring in Corinna's loader script fix. So you'll have to do 'git fetch' and 'git rebase --hard origin/topic/pipe'. Does this now fix all known problems with pipes? Corinna, do you still see any benefit to switching to PIPE_NOWAIT? AFAICT, it wouldn't decrease the code size at this point, so the only question is whether it might improve performance. If you think it's worth trying, I'd be glad to code it up on a new branch, and we could compare the two. Aside from that, I'm wondering how and when to merge the new pipe implementation to master. It obviously needs much more widespread testing than it's gotten so far. I'm a little nervous about it because I haven't thought about the details for two years, and no one other than me has tested it until a few days ago. Ken ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? 2021-08-29 15:57 ` Ken Brown @ 2021-08-29 19:24 ` Chris Roehrig 2021-08-29 22:24 ` Ken Brown 2021-08-29 19:37 ` cygrunsrv + sshd + rsync = 20 times too slow -- throttled? Takashi Yano 1 sibling, 1 reply; 86+ messages in thread From: Chris Roehrig @ 2021-08-29 19:24 UTC (permalink / raw) To: Ken Brown via Cygwin I'd be happy to test it out if you can give me some commands to git it and build it to replace my existing stock Cygwin installation. (Or it is just the cygwin.dll I'd need to replace?) My daily backup scripts use a lot of pipes and named pipes. -- Chris On Sun Aug 29 2021, at 8:57 AM, Ken Brown via Cygwin <cygwin@cygwin.com> wrote: > > Aside from that, I'm wondering how and when to merge the new pipe implementation to master. It obviously needs much more widespread testing than it's gotten so far. I'm a little nervous about it because I haven't thought about the details for two years, and no one other than me has tested it until a few days ago. > > Ken > > -- > Problem reports: https://cygwin.com/problems.html > FAQ: https://cygwin.com/faq/ > Documentation: https://cygwin.com/docs.html > Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? 2021-08-29 19:24 ` Chris Roehrig @ 2021-08-29 22:24 ` Ken Brown 2021-08-30 23:58 ` Chris Roehrig 0 siblings, 1 reply; 86+ messages in thread From: Ken Brown @ 2021-08-29 22:24 UTC (permalink / raw) To: cygwin On 8/29/2021 3:24 PM, Chris Roehrig wrote: > I'd be happy to test it out if you can give me some commands to git it and build it to replace my existing stock Cygwin installation. > (Or it is just the cygwin.dll I'd need to replace?) > > My daily backup scripts use a lot of pipes and named pipes. It's just cygwin1.dll that you'd need to replace. Instructions for building the latter can be found at https://www.cygwin.com/faq.html#faq.programming.building-cygwin but they're out of date in two respects: 1. Currently the build does *not* ignore errors in building the documentation. 2. In addition to the packages listed in the FAQ, you'll need the following: texlive-collection-latexrecommended texlive-collection-fontsrecommended texlive-collection-pictures python38-lxml python38-ply These instructions will produce cygwin1.dll in the bin subdirectory of your install directory, built from the git master branch. If you want to try the topic/pipe branch, just go to the newlib-cygwin git repo that you cloned, issue the command 'git checkout topic/pipe', and rebuild. I'll be glad to help if you run into problems. Ken ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? 2021-08-29 22:24 ` Ken Brown @ 2021-08-30 23:58 ` Chris Roehrig 2021-08-31 19:05 ` Ken Brown 0 siblings, 1 reply; 86+ messages in thread From: Chris Roehrig @ 2021-08-30 23:58 UTC (permalink / raw) To: Ken Brown via Cygwin I got it to build and tried out the topic/pipe branch (checked out on Monday around 4:30pm PDT): 1. I didn't see any improvement in my sshd+rsync time, still 3-4 MB/sec. 2. I get the following error from procps: procps:ps/output.c:2195: please report this bug (I also get this using the main branch build). I first updated my cygwin normally using setup-x86_64.exe, then built and copied only the cygwin1.dll into /bin from the install/bin directory (quitting all cygwin tasks first). On Sun Aug 29 2021, at 3:24 PM, Ken Brown via Cygwin <cygwin@cygwin.com> wrote: > On 8/29/2021 3:24 PM, Chris Roehrig wrote: >> I'd be happy to test it out if you can give me some commands to git it and build it to replace my existing stock Cygwin installation. >> (Or it is just the cygwin.dll I'd need to replace?) >> My daily backup scripts use a lot of pipes and named pipes. > > It's just cygwin1.dll that you'd need to replace. Instructions for building the latter can be found at > > https://www.cygwin.com/faq.html#faq.programming.building-cygwin > > but they're out of date in two respects: > > 1. Currently the build does *not* ignore errors in building the documentation. > > 2. In addition to the packages listed in the FAQ, you'll need the following: > > texlive-collection-latexrecommended > texlive-collection-fontsrecommended > texlive-collection-pictures > python38-lxml > python38-ply > > These instructions will produce cygwin1.dll in the bin subdirectory of your install directory, built from the git master branch. > > If you want to try the topic/pipe branch, just go to the newlib-cygwin git repo that you cloned, issue the command 'git checkout topic/pipe', and rebuild. > > I'll be glad to help if you run into problems. > > Ken > > -- > Problem reports: https://cygwin.com/problems.html > FAQ: https://cygwin.com/faq/ > Documentation: https://cygwin.com/docs.html > Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? 2021-08-30 23:58 ` Chris Roehrig @ 2021-08-31 19:05 ` Ken Brown 2021-08-31 19:53 ` Chris Roehrig 0 siblings, 1 reply; 86+ messages in thread From: Ken Brown @ 2021-08-31 19:05 UTC (permalink / raw) To: cygwin On 8/30/2021 7:58 PM, Chris Roehrig wrote: > I got it to build and tried out the topic/pipe branch (checked out on Monday around 4:30pm PDT): > > 1. I didn't see any improvement in my sshd+rsync time, still 3-4 MB/sec. > > 2. I get the following error from procps: procps:ps/output.c:2195: please report this bug > (I also get this using the main branch build). > > I first updated my cygwin normally using setup-x86_64.exe, then built and copied only the cygwin1.dll into /bin from the install/bin directory (quitting all cygwin tasks first). Thanks for testing. We're still working on this. (The discussion has moved to the cygwin-developers mailing list.) I'll let you know when it's stable, and maybe you can try again. As to the procps error, can you give details so that someone can try to reproduce it and debug it? Ken ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? 2021-08-31 19:05 ` Ken Brown @ 2021-08-31 19:53 ` Chris Roehrig 2021-08-31 20:23 ` Chris Roehrig 0 siblings, 1 reply; 86+ messages in thread From: Chris Roehrig @ 2021-08-31 19:53 UTC (permalink / raw) To: Ken Brown via Cygwin Thanks, I did some more tests: scp also shows no improvement with topic/pipe. I tried running cygsshd with CYGWIN=pipe_byte as well as empty (in the registry HKLM/SYSTEM/CurrentControlSet/Services/cygsshd/Parameters/Environment/), using net stop cygsshd + net start cygsshd to restart it. Not sure if that CYGWIN only applies to cygsshd or to all cygwin tasks. I get the procps error just running 'procps' command with no args from mintty (/usr/bin/procps). Looking at the procps-ng source (procps-ng 3.3.17-1 is what setup_x86_64 reports as my installed package), it appears to be a mmap() failure. I'm wondering if I built/installed cygwin1.dll correctly because I don't get the procps error using the latest stock cygwin1.dll as installed by setup_x86_64. -- Chris On Tue Aug 31 2021, at 12:05 PM, Ken Brown via Cygwin <cygwin@cygwin.com> wrote: > On 8/30/2021 7:58 PM, Chris Roehrig wrote: >> I got it to build and tried out the topic/pipe branch (checked out on Monday around 4:30pm PDT): >> 1. I didn't see any improvement in my sshd+rsync time, still 3-4 MB/sec. >> 2. I get the following error from procps: procps:ps/output.c:2195: please report this bug >> (I also get this using the main branch build). >> I first updated my cygwin normally using setup-x86_64.exe, then built and copied only the cygwin1.dll into /bin from the install/bin directory (quitting all cygwin tasks first). > > Thanks for testing. > > We're still working on this. (The discussion has moved to the cygwin-developers mailing list.) I'll let you know when it's stable, and maybe you can try again. > > As to the procps error, can you give details so that someone can try to reproduce it and debug it? > > Ken > > -- > Problem reports: https://cygwin.com/problems.html > FAQ: https://cygwin.com/faq/ > Documentation: https://cygwin.com/docs.html > Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? 2021-08-31 19:53 ` Chris Roehrig @ 2021-08-31 20:23 ` Chris Roehrig 2021-08-31 21:29 ` Brian Inglis 2021-09-01 21:11 ` Chris Roehrig 0 siblings, 2 replies; 86+ messages in thread From: Chris Roehrig @ 2021-08-31 20:23 UTC (permalink / raw) To: Ken Brown via Cygwin I did a 'git pull' of the latest topic/pipe and rebuilt and I now do indeed get 100MB/s transfers using both rsync and scp (without pipe_byte). (It turns out last time I forgot 'make install' -- Doh!) I still get the procps error however. On Tue Aug 31 2021, at 12:53 PM, Chris Roehrig <croehrig@house.org> wrote: > Thanks, I did some more tests: > scp also shows no improvement with topic/pipe. I tried running cygsshd with CYGWIN=pipe_byte as well as empty (in the registry HKLM/SYSTEM/CurrentControlSet/Services/cygsshd/Parameters/Environment/), using net stop cygsshd + net start cygsshd to restart it. Not sure if that CYGWIN only applies to cygsshd or to all cygwin tasks. > > I get the procps error just running 'procps' command with no args from mintty (/usr/bin/procps). Looking at the procps-ng source (procps-ng 3.3.17-1 is what setup_x86_64 reports as my installed package), it appears to be a mmap() failure. > > I'm wondering if I built/installed cygwin1.dll correctly because I don't get the procps error using the latest stock cygwin1.dll as installed by setup_x86_64. > > -- Chris > > > > On Tue Aug 31 2021, at 12:05 PM, Ken Brown via Cygwin <cygwin@cygwin.com> wrote: > >> On 8/30/2021 7:58 PM, Chris Roehrig wrote: >>> I got it to build and tried out the topic/pipe branch (checked out on Monday around 4:30pm PDT): >>> 1. I didn't see any improvement in my sshd+rsync time, still 3-4 MB/sec. >>> 2. I get the following error from procps: procps:ps/output.c:2195: please report this bug >>> (I also get this using the main branch build). >>> I first updated my cygwin normally using setup-x86_64.exe, then built and copied only the cygwin1.dll into /bin from the install/bin directory (quitting all cygwin tasks first). >> >> Thanks for testing. >> >> We're still working on this. (The discussion has moved to the cygwin-developers mailing list.) I'll let you know when it's stable, and maybe you can try again. >> >> As to the procps error, can you give details so that someone can try to reproduce it and debug it? >> >> Ken >> >> -- >> Problem reports: https://cygwin.com/problems.html >> FAQ: https://cygwin.com/faq/ >> Documentation: https://cygwin.com/docs.html >> Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple > > > -- > Problem reports: https://cygwin.com/problems.html > FAQ: https://cygwin.com/faq/ > Documentation: https://cygwin.com/docs.html > Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? 2021-08-31 20:23 ` Chris Roehrig @ 2021-08-31 21:29 ` Brian Inglis 2021-09-01 21:11 ` Chris Roehrig 1 sibling, 0 replies; 86+ messages in thread From: Brian Inglis @ 2021-08-31 21:29 UTC (permalink / raw) To: cygwin I've found you must also copy the matching cygwin-console-helper.exe for everything to work correctly! On 2021-08-31 14:23, Chris Roehrig wrote: > I did a 'git pull' of the latest topic/pipe and rebuilt and I now do indeed get 100MB/s transfers using both rsync and scp (without pipe_byte). > (It turns out last time I forgot 'make install' -- Doh!) > > I still get the procps error however. > On Tue Aug 31 2021, at 12:53 PM, Chris Roehrig wrote: >> Thanks, I did some more tests: >> scp also shows no improvement with topic/pipe. I tried running cygsshd with CYGWIN=pipe_byte as well as empty (in the registry HKLM/SYSTEM/CurrentControlSet/Services/cygsshd/Parameters/Environment/), using net stop cygsshd + net start cygsshd to restart it. Not sure if that CYGWIN only applies to cygsshd or to all cygwin tasks. >> >> I get the procps error just running 'procps' command with no args from mintty (/usr/bin/procps). Looking at the procps-ng source (procps-ng 3.3.17-1 is what setup_x86_64 reports as my installed package), it appears to be a mmap() failure. >> >> I'm wondering if I built/installed cygwin1.dll correctly because I don't get the procps error using the latest stock cygwin1.dll as installed by setup_ >> On Tue Aug 31 2021, at 12:05 PM, Ken Brown via Cygwin wrote: >>> On 8/30/2021 7:58 PM, Chris Roehrig wrote: >>>> I got it to build and tried out the topic/pipe branch (checked out on Monday around 4:30pm PDT): >>>> 1. I didn't see any improvement in my sshd+rsync time, still 3-4 MB/sec. >>>> 2. I get the following error from procps: procps:ps/output.c:2195: please report this bug >>>> (I also get this using the main branch build). >>>> I first updated my cygwin normally using setup-x86_64.exe, then built and copied only the cygwin1.dll into /bin from the install/bin directory (quitting all cygwin tasks first). >>> >>> Thanks for testing. >>> >>> We're still working on this. (The discussion has moved to the cygwin-developers mailing list.) I'll let you know when it's stable, and maybe you can try again. >>> >>> As to the procps error, can you give details so that someone can try to reproduce it and debug it? -- Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada This email may be disturbing to some readers as it contains too much technical detail. Reader discretion is advised. [Data in binary units and prefixes, physical quantities in SI.] ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? 2021-08-31 20:23 ` Chris Roehrig 2021-08-31 21:29 ` Brian Inglis @ 2021-09-01 21:11 ` Chris Roehrig 2021-09-02 15:25 ` Ken Brown 1 sibling, 1 reply; 86+ messages in thread From: Chris Roehrig @ 2021-09-01 21:11 UTC (permalink / raw) To: Ken Brown via Cygwin I rebuild procps 3.3.17.29-2480 from source and it appears to work, so maybe the stock procps package is incompatibility with the current master branch. (However, I built it against the stock /usr/include, not the current branch...) I first needed to 'make /proc/libprocps.la', and there was an undefined siginfo_t.si_int in ./lib/test_process.c which I just commented-out to get it to build. -- Chris On Tue Aug 31 2021, at 1:23 PM, Chris Roehrig <croehrig@house.org> wrote: > I did a 'git pull' of the latest topic/pipe and rebuilt and I now do indeed get 100MB/s transfers using both rsync and scp (without pipe_byte). > (It turns out last time I forgot 'make install' -- Doh!) > > I still get the procps error however. > > > On Tue Aug 31 2021, at 12:53 PM, Chris Roehrig <croehrig@house.org> wrote: > >> Thanks, I did some more tests: >> scp also shows no improvement with topic/pipe. I tried running cygsshd with CYGWIN=pipe_byte as well as empty (in the registry HKLM/SYSTEM/CurrentControlSet/Services/cygsshd/Parameters/Environment/), using net stop cygsshd + net start cygsshd to restart it. Not sure if that CYGWIN only applies to cygsshd or to all cygwin tasks. >> >> I get the procps error just running 'procps' command with no args from mintty (/usr/bin/procps). Looking at the procps-ng source (procps-ng 3.3.17-1 is what setup_x86_64 reports as my installed package), it appears to be a mmap() failure. >> >> I'm wondering if I built/installed cygwin1.dll correctly because I don't get the procps error using the latest stock cygwin1.dll as installed by setup_x86_64. >> >> -- Chris >> >> >> >> On Tue Aug 31 2021, at 12:05 PM, Ken Brown via Cygwin <cygwin@cygwin.com> wrote: >> >>> On 8/30/2021 7:58 PM, Chris Roehrig wrote: >>>> I got it to build and tried out the topic/pipe branch (checked out on Monday around 4:30pm PDT): >>>> 1. I didn't see any improvement in my sshd+rsync time, still 3-4 MB/sec. >>>> 2. I get the following error from procps: procps:ps/output.c:2195: please report this bug >>>> (I also get this using the main branch build). >>>> I first updated my cygwin normally using setup-x86_64.exe, then built and copied only the cygwin1.dll into /bin from the install/bin directory (quitting all cygwin tasks first). >>> >>> Thanks for testing. >>> >>> We're still working on this. (The discussion has moved to the cygwin-developers mailing list.) I'll let you know when it's stable, and maybe you can try again. >>> >>> As to the procps error, can you give details so that someone can try to reproduce it and debug it? >>> >>> Ken >>> >>> -- >>> Problem reports: https://cygwin.com/problems.html >>> FAQ: https://cygwin.com/faq/ >>> Documentation: https://cygwin.com/docs.html >>> Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple >> >> >> -- >> Problem reports: https://cygwin.com/problems.html >> FAQ: https://cygwin.com/faq/ >> Documentation: https://cygwin.com/docs.html >> Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple > > > -- > Problem reports: https://cygwin.com/problems.html > FAQ: https://cygwin.com/faq/ > Documentation: https://cygwin.com/docs.html > Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? 2021-09-01 21:11 ` Chris Roehrig @ 2021-09-02 15:25 ` Ken Brown 2021-09-02 19:03 ` Chris Roehrig 0 siblings, 1 reply; 86+ messages in thread From: Ken Brown @ 2021-09-02 15:25 UTC (permalink / raw) To: cygwin On 9/1/2021 5:11 PM, Chris Roehrig wrote: > I rebuild procps 3.3.17.29-2480 from source and it appears to work, so maybe the stock procps package is incompatibility with the current master branch. Maybe, but it could also be a Cygwin bug. I'll do a bisection of the Cygwin sources to see if I can track it down. Ken P.S. The custom on this list is not to top-post. Thanks. ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? 2021-09-02 15:25 ` Ken Brown @ 2021-09-02 19:03 ` Chris Roehrig 2021-09-03 17:26 ` Ken Brown 2021-09-03 19:55 ` Corinna Vinschen 0 siblings, 2 replies; 86+ messages in thread From: Chris Roehrig @ 2021-09-02 19:03 UTC (permalink / raw) To: Ken Brown via Cygwin On Thu Sep 2 2021, at 8:25 AM, Ken Brown via Cygwin <cygwin@cygwin.com> wrote: > On 9/1/2021 5:11 PM, Chris Roehrig wrote: >> I rebuild procps 3.3.17.29-2480 from source and it appears to work, so maybe the stock procps package is incompatible with the current master branch. > > Maybe, but it could also be a Cygwin bug. I'll do a bisection of the Cygwin sources to see if I can track it down. I did some more tests and it still doesn't completely work: procps -ef # works procps -eo user,stime,tty,time,args # works procps -eo pid # fails with that same mmap() error I also rebuilt it all using cygport and it gives the same error (pscommand.exe with no args). > P.S. The custom on this list is not to top-post. Thanks. Sorry, I didn't realize there was an etiquette. Fixed. -- Chris ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? 2021-09-02 19:03 ` Chris Roehrig @ 2021-09-03 17:26 ` Ken Brown 2021-09-03 19:55 ` Corinna Vinschen 1 sibling, 0 replies; 86+ messages in thread From: Ken Brown @ 2021-09-03 17:26 UTC (permalink / raw) To: cygwin On 9/2/2021 3:03 PM, Chris Roehrig wrote: > > On Thu Sep 2 2021, at 8:25 AM, Ken Brown via Cygwin <cygwin@cygwin.com> wrote: >> On 9/1/2021 5:11 PM, Chris Roehrig wrote: >>> I rebuild procps 3.3.17.29-2480 from source and it appears to work, so maybe the stock procps package is incompatible with the current master branch. >> >> Maybe, but it could also be a Cygwin bug. I'll do a bisection of the Cygwin sources to see if I can track it down. > > I did some more tests and it still doesn't completely work: > procps -ef # works > procps -eo user,stime,tty,time,args # works > procps -eo pid # fails with that same mmap() error > > I also rebuilt it all using cygport and it gives the same error (pscommand.exe with no args). I'm still seeing the error too, and I'm even seeing it if I rebuild Cygwin 3.2.0 with the current gcc. I don't know what the problem is. >> P.S. The custom on this list is not to top-post. Thanks. > > Sorry, I didn't realize there was an etiquette. Fixed. Thanks! Ken ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? 2021-09-02 19:03 ` Chris Roehrig 2021-09-03 17:26 ` Ken Brown @ 2021-09-03 19:55 ` Corinna Vinschen 2021-09-03 20:59 ` Chris Roehrig 2021-09-04 22:37 ` mmap failure [was: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?] Ken Brown 1 sibling, 2 replies; 86+ messages in thread From: Corinna Vinschen @ 2021-09-03 19:55 UTC (permalink / raw) To: cygwin [resent, this time with the ML in To] On Sep 2 12:03, Chris Roehrig wrote: > > On Thu Sep 2 2021, at 8:25 AM, Ken Brown via Cygwin <cygwin@cygwin.com> wrote: > > On 9/1/2021 5:11 PM, Chris Roehrig wrote: > >> I rebuild procps 3.3.17.29-2480 from source and it appears to work, so maybe the stock procps package is incompatible with the current master branch. > > > > Maybe, but it could also be a Cygwin bug. I'll do a bisection of the Cygwin sources to see if I can track it down. > > I did some more tests and it still doesn't completely work: > procps -ef # works > procps -eo user,stime,tty,time,args # works > procps -eo pid # fails with that same mmap() error > > I also rebuilt it all using cygport and it gives the same error (pscommand.exe with no args). Since you're building Cygwin by yourself anyway, can you do me a favor and try this: $ git revert 2f05de4dbf9c and see if that fixes your issue? Thanks, Corinna ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? 2021-09-03 19:55 ` Corinna Vinschen @ 2021-09-03 20:59 ` Chris Roehrig 2021-09-04 8:32 ` Achim Gratz 2021-09-04 16:45 ` Brian Inglis 2021-09-04 22:37 ` mmap failure [was: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?] Ken Brown 1 sibling, 2 replies; 86+ messages in thread From: Chris Roehrig @ 2021-09-03 20:59 UTC (permalink / raw) To: cygwin On Fri Sep 3 2021, at 12:55 PM, Corinna Vinschen <corinna-cygwin@cygwin.com> wrote: > [resent, this time with the ML in To] > > On Sep 2 12:03, Chris Roehrig wrote: >> >> On Thu Sep 2 2021, at 8:25 AM, Ken Brown via Cygwin <cygwin@cygwin.com> wrote: >>> On 9/1/2021 5:11 PM, Chris Roehrig wrote: >>>> I rebuild procps 3.3.17.29-2480 from source and it appears to work, so maybe the stock procps package is incompatible with the current master branch. >>> >>> Maybe, but it could also be a Cygwin bug. I'll do a bisection of the Cygwin sources to see if I can track it down. >> >> I did some more tests and it still doesn't completely work: >> procps -ef # works >> procps -eo user,stime,tty,time,args # works >> procps -eo pid # fails with that same mmap() error >> >> I also rebuilt it all using cygport and it gives the same error (pscommand.exe with no args). > > Since you're building Cygwin by yourself anyway, can you do me a favor > and try this: > > $ git revert 2f05de4dbf9c > > and see if that fixes your issue? I got procps working I think (both with and without the revert). I think the problem might just be that I wasn't also copying the rebuilt /bin/cygprocps-8.dll to match the /bin/procps.exe. There's some tricky renaming that make install does, so I did 'make install DESTDIR=/tmp/install' and copied just those two files. I'm guessing it would all work on a properly fully installed build. -- Chris ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? 2021-09-03 20:59 ` Chris Roehrig @ 2021-09-04 8:32 ` Achim Gratz 2021-09-04 16:45 ` Brian Inglis 1 sibling, 0 replies; 86+ messages in thread From: Achim Gratz @ 2021-09-04 8:32 UTC (permalink / raw) To: cygwin Am 03.09.2021 um 22:59 schrieb Chris Roehrig: > I got procps working I think (both with and without the revert). That likely wasn't what Corinna wanted to know, though. Please re-install the procps-ng, cygwin and cygwin-devel packages from setup (and revert any other alterations you may have made). Then check that the error has gone away, rebuild and install the new cygwin1.dll with and without the revert mentioned by Corinna and check for the error under these two conditions as well. -- Achim. (on the road :-) ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? 2021-09-03 20:59 ` Chris Roehrig 2021-09-04 8:32 ` Achim Gratz @ 2021-09-04 16:45 ` Brian Inglis 2021-09-05 8:18 ` Achim Gratz 1 sibling, 1 reply; 86+ messages in thread From: Brian Inglis @ 2021-09-04 16:45 UTC (permalink / raw) To: cygwin On 2021-09-03 14:59, Chris Roehrig wrote: > On Fri Sep 3 2021, at 12:55 PM, Corinna Vinschen wrote: >> On Sep 2 12:03, Chris Roehrig wrote: >>> On Thu Sep 2 2021, at 8:25 AM, Ken Brown via Cygwin wrote: >>>> On 9/1/2021 5:11 PM, Chris Roehrig wrote: >>>>> I rebuild procps 3.3.17.29-2480 from source and it appears to >>>>> work, so maybe the stock procps package is incompatible with >>>>> the current master branch. >>>> Maybe, but it could also be a Cygwin bug. I'll do a bisection >>>> of the Cygwin sources to see if I can track it down. >>> I did some more tests and it still doesn't completely work: >>> procps -ef # works >>> procps -eo user,stime,tty,time,args # works >>> procps -eo pid # fails with that same mmap() error >>> I also rebuilt it all using cygport and it gives the same error >>> (pscommand.exe with no args). >> Since you're building Cygwin by yourself anyway, can you do me a >> favor and try this: >> $ git revert 2f05de4dbf9c >> and see if that fixes your issue? > I got procps working I think (both with and without the revert). I > think the problem might just be that I wasn't also copying the > rebuilt /bin/cygprocps-8.dll to match the /bin/procps.exe. There's > some tricky renaming that make install does, so I did 'make install > DESTDIR=/tmp/install' and copied just those two files. I'm guessing > it would all work on a properly fully installed build. If you are building packages in the same way as scallywag with: $ cygport PKG.cygport download all check then to install all binary packages for dogfooding: $ for tar in PKG-VER-1.ARCH/dist/PKG/**/*PKG*-VER.tar.*z* do tar -x -C / -f $tar # * -C / installs usr/... under /usr/ done for example, I just did: $ for tar in \ dash-0.5.11.5-1.x86_64/dist/dash/**/*dash*-0.5.11.5-1.tar.*z* do tar -x -C / -f $tar done -- Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada This email may be disturbing to some readers as it contains too much technical detail. Reader discretion is advised. [Data in binary units and prefixes, physical quantities in SI.] ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? 2021-09-04 16:45 ` Brian Inglis @ 2021-09-05 8:18 ` Achim Gratz 2021-09-05 15:11 ` Brian Inglis 0 siblings, 1 reply; 86+ messages in thread From: Achim Gratz @ 2021-09-05 8:18 UTC (permalink / raw) To: cygwin 04.09.2021 18:45, Brian Inglis: [...] > then to install all binary packages for dogfooding: Would you please stop telling folks to do things that potentially breaks their systems? There are quite a few more steps to take if you want to emulate what setup does. Then again you cannot do that on a live Cygwin system unless you _really_ know what exactly it is you're doing and which packages can or cannot be touched that way. -- Achim. (on the road :-) ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? 2021-09-05 8:18 ` Achim Gratz @ 2021-09-05 15:11 ` Brian Inglis 2021-09-06 10:42 ` Achim Gratz 0 siblings, 1 reply; 86+ messages in thread From: Brian Inglis @ 2021-09-05 15:11 UTC (permalink / raw) To: cygwin On 2021-09-05 02:18, Achim Gratz wrote: > 04.09.2021 18:45, Brian Inglis: > [...] >> then to install all binary packages for dogfooding: > > Would you please stop telling folks to do things that potentially breaks > their systems? > > There are quite a few more steps to take if you want to emulate what > setup does. Then again you cannot do that on a live Cygwin system > unless you _really_ know what exactly it is you're doing and which > packages can or cannot be touched that way. This is a downside of having only a general and not also a "technical" mailing list to discuss and diagnose "technical" issues and approaches. The suggestion was intended as a tip to ensure *complete* locally rebuilt package contents are installed, by folks who are already having issues with their installation, rebuilding Cygwin and packages, and replacing the Cygwin dll etc. and packages to diagnose those issues. So hopefully they are also aware of preremove and postinstall scripts, in packages where they are provided, and the /etc/setup/PKG.lst.gz manifests for files to be removed before installation, and provided after installation. For the latter I would expect technical folks to make use of them but leave them to Cygwin Setup programs for officially released package upgrades. [Also some of us do not have the appropriate technical background and expertise to get a Cygwin overlay mirror working reliably from the available instructions (if it was easy enough, someone would have provided a local-overlay-mirror package to do it years ago), nor systems fast enough to run Cygwin Setup for every package install. Due to Windows "improvements", my system upgraded a few years ago is just as "fast" as the 10 year old system it replaced. ;^> Cygwin Setup upgrades can take as long as Windows Update installing the latest patches. Some of my approaches, suggestions, and participation here are coloured by those limitations (also by expensive, slow, uncompetitive networks).] -- Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada This email may be disturbing to some readers as it contains too much technical detail. Reader discretion is advised. [Data in binary units and prefixes, physical quantities in SI.] ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? 2021-09-05 15:11 ` Brian Inglis @ 2021-09-06 10:42 ` Achim Gratz 0 siblings, 0 replies; 86+ messages in thread From: Achim Gratz @ 2021-09-06 10:42 UTC (permalink / raw) To: cygwin 05.09.2021 17:11, Brian Inglis: > The suggestion was intended as a tip to ensure *complete* locally > rebuilt package contents are installed, Setup has its "from_cwd" installation mode for precisely that reason: installing a local package without the need to create a full install hierarchy and setup.ini files. > Due to Windows "improvements", my system upgraded a few years ago is > just as "fast" as the 10 year old system it replaced. ;^> > Cygwin Setup upgrades can take as long as Windows Update installing the > latest patches. If you haven't already, put your Cygwin installation on SSD and exclude the Cygwin root (and local mirror directory) from real-time virus scans. I do monthly updates on slow machines (CPU is a 1.8GHz IvyBridge Celeron) and that takes around two to five minutes depending on how large the update is in any given month (the man-db update happens in the background, so is not part of these figures). -- Achim. (on the road :-) ^ permalink raw reply [flat|nested] 86+ messages in thread
* mmap failure [was: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?] 2021-09-03 19:55 ` Corinna Vinschen 2021-09-03 20:59 ` Chris Roehrig @ 2021-09-04 22:37 ` Ken Brown 2021-09-04 22:54 ` Ken Brown 2021-09-04 23:18 ` Brian Inglis 1 sibling, 2 replies; 86+ messages in thread From: Ken Brown @ 2021-09-04 22:37 UTC (permalink / raw) To: cygwin I've reduced the procps failure to the following test case: $ cat mmap_test.c #include <unistd.h> #include <stdio.h> #include <sys/mman.h> int main () { void *addr; int page_size = getpagesize (); addr = mmap (0, page_size, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); if (addr == MAP_FAILED) perror ("mmap"); } $ gcc mmap_test.c $ ./a mmap: Invalid argument This happens if I use cygwin1.dll built from the current git master. But it also happens if I build from cygwin-3_2_0-release with the recent patches applied that make Cygwin build without error: 0e12b4939 Cygwin: C++17: register keyword is deprecated a7340e0c0 Cygwin: dumper: fix up GCC pragma for g++ 11.2 2a212c086 Cygwin: workaround a g++ 11.2 initialization bug 9e3f1737e Cygwin: loader script: add DWARF 5 sections bdfd2b004 Cygwin: testsuite: avoid "conflicting types" gcc warning 6fc498e2e strstr: avoid warnings 26da270b2 ldexp/ldexpf: avoid assembler warning edce2a557 Cygwin: fix declaration of RtlInitEmptyUnicodeString So there appears to be something wrong with cygwin1.dll built with the current build tools (gcc 11.2.0, binutils 2.37, not sure what else is relevant). Ken ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: mmap failure [was: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?] 2021-09-04 22:37 ` mmap failure [was: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?] Ken Brown @ 2021-09-04 22:54 ` Ken Brown 2021-09-04 22:58 ` Ken Brown 2021-09-04 23:18 ` Brian Inglis 1 sibling, 1 reply; 86+ messages in thread From: Ken Brown @ 2021-09-04 22:54 UTC (permalink / raw) To: cygwin On 9/4/2021 6:37 PM, Ken Brown via Cygwin wrote: > I've reduced the procps failure to the following test case: > > $ cat mmap_test.c > #include <unistd.h> > #include <stdio.h> > #include <sys/mman.h> > > int > main () > { > void *addr; > int page_size = getpagesize (); > > addr = mmap (0, page_size, PROT_READ | PROT_WRITE, > MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); > if (addr == MAP_FAILED) > perror ("mmap"); > } > > $ gcc mmap_test.c > > $ ./a > mmap: Invalid argument > > This happens if I use cygwin1.dll built from the current git master. But it > also happens if I build from cygwin-3_2_0-release with the recent patches > applied that make Cygwin build without error: > > 0e12b4939 Cygwin: C++17: register keyword is deprecated > a7340e0c0 Cygwin: dumper: fix up GCC pragma for g++ 11.2 > 2a212c086 Cygwin: workaround a g++ 11.2 initialization bug > 9e3f1737e Cygwin: loader script: add DWARF 5 sections > bdfd2b004 Cygwin: testsuite: avoid "conflicting types" gcc warning > 6fc498e2e strstr: avoid warnings > 26da270b2 ldexp/ldexpf: avoid assembler warning > edce2a557 Cygwin: fix declaration of RtlInitEmptyUnicodeString Sorry, those commit ids are wrong. They're what I got after rebasing the master branch so that those commits come first. > So there appears to be something wrong with cygwin1.dll built with the current > build tools (gcc 11.2.0, binutils 2.37, not sure what else is relevant). > > Ken > ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: mmap failure [was: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?] 2021-09-04 22:54 ` Ken Brown @ 2021-09-04 22:58 ` Ken Brown 2021-09-05 0:04 ` Ken Brown 0 siblings, 1 reply; 86+ messages in thread From: Ken Brown @ 2021-09-04 22:58 UTC (permalink / raw) To: cygwin On 9/4/2021 6:54 PM, Ken Brown via Cygwin wrote: > On 9/4/2021 6:37 PM, Ken Brown via Cygwin wrote: >> I've reduced the procps failure to the following test case: >> >> $ cat mmap_test.c >> #include <unistd.h> >> #include <stdio.h> >> #include <sys/mman.h> >> >> int >> main () >> { >> void *addr; >> int page_size = getpagesize (); >> >> addr = mmap (0, page_size, PROT_READ | PROT_WRITE, >> MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); >> if (addr == MAP_FAILED) >> perror ("mmap"); >> } >> >> $ gcc mmap_test.c >> >> $ ./a >> mmap: Invalid argument >> >> This happens if I use cygwin1.dll built from the current git master. But it >> also happens if I build from cygwin-3_2_0-release with the recent patches >> applied that make Cygwin build without error: >> >> 0e12b4939 Cygwin: C++17: register keyword is deprecated >> a7340e0c0 Cygwin: dumper: fix up GCC pragma for g++ 11.2 >> 2a212c086 Cygwin: workaround a g++ 11.2 initialization bug >> 9e3f1737e Cygwin: loader script: add DWARF 5 sections >> bdfd2b004 Cygwin: testsuite: avoid "conflicting types" gcc warning >> 6fc498e2e strstr: avoid warnings >> 26da270b2 ldexp/ldexpf: avoid assembler warning >> edce2a557 Cygwin: fix declaration of RtlInitEmptyUnicodeString > > Sorry, those commit ids are wrong. They're what I got after rebasing the master > branch so that those commits come first. [Pressed Send too soon.] Here are the correct commits: 8169e39ab Cygwin: C++17: register keyword is deprecated 3ca80b360 Cygwin: dumper: fix up GCC pragma for g++ 11.2 bdb7991db Cygwin: workaround a g++ 11.2 initialization bug 801120c1f Cygwin: loader script: add DWARF 5 sections d5cc66426 Cygwin: testsuite: avoid "conflicting types" gcc warning c2fe205b5 strstr: avoid warnings 76c2c7a89 ldexp/ldexpf: avoid assembler warning eeeb5650c Cygwin: fix declaration of RtlInitEmptyUnicodeString > >> So there appears to be something wrong with cygwin1.dll built with the current >> build tools (gcc 11.2.0, binutils 2.37, not sure what else is relevant). >> >> Ken >> > ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: mmap failure [was: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?] 2021-09-04 22:58 ` Ken Brown @ 2021-09-05 0:04 ` Ken Brown 2021-09-05 13:24 ` Ken Brown 0 siblings, 1 reply; 86+ messages in thread From: Ken Brown @ 2021-09-05 0:04 UTC (permalink / raw) To: cygwin On 9/4/2021 6:58 PM, Ken Brown via Cygwin wrote: > On 9/4/2021 6:54 PM, Ken Brown via Cygwin wrote: >> On 9/4/2021 6:37 PM, Ken Brown via Cygwin wrote: >>> I've reduced the procps failure to the following test case: >>> >>> $ cat mmap_test.c >>> #include <unistd.h> >>> #include <stdio.h> >>> #include <sys/mman.h> >>> >>> int >>> main () >>> { >>> void *addr; >>> int page_size = getpagesize (); >>> >>> addr = mmap (0, page_size, PROT_READ | PROT_WRITE, >>> MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); >>> if (addr == MAP_FAILED) >>> perror ("mmap"); >>> } >>> >>> $ gcc mmap_test.c >>> >>> $ ./a >>> mmap: Invalid argument >>> >>> This happens if I use cygwin1.dll built from the current git master. But it >>> also happens if I build from cygwin-3_2_0-release with the recent patches >>> applied that make Cygwin build without error: >>> >>> 0e12b4939 Cygwin: C++17: register keyword is deprecated >>> a7340e0c0 Cygwin: dumper: fix up GCC pragma for g++ 11.2 >>> 2a212c086 Cygwin: workaround a g++ 11.2 initialization bug >>> 9e3f1737e Cygwin: loader script: add DWARF 5 sections >>> bdfd2b004 Cygwin: testsuite: avoid "conflicting types" gcc warning >>> 6fc498e2e strstr: avoid warnings >>> 26da270b2 ldexp/ldexpf: avoid assembler warning >>> edce2a557 Cygwin: fix declaration of RtlInitEmptyUnicodeString >> >> Sorry, those commit ids are wrong. They're what I got after rebasing the >> master branch so that those commits come first. > > [Pressed Send too soon.] > > Here are the correct commits: > > 8169e39ab Cygwin: C++17: register keyword is deprecated > 3ca80b360 Cygwin: dumper: fix up GCC pragma for g++ 11.2 > bdb7991db Cygwin: workaround a g++ 11.2 initialization bug > 801120c1f Cygwin: loader script: add DWARF 5 sections > d5cc66426 Cygwin: testsuite: avoid "conflicting types" gcc warning > c2fe205b5 strstr: avoid warnings > 76c2c7a89 ldexp/ldexpf: avoid assembler warning > eeeb5650c Cygwin: fix declaration of RtlInitEmptyUnicodeString > >> >>> So there appears to be something wrong with cygwin1.dll built with the >>> current build tools (gcc 11.2.0, binutils 2.37, not sure what else is relevant). Wait a minute...I'll bet this is related to the MEM_EXTENDED_PARAMETER initialization problem that was dealt with in commit bdb7991db. Ken ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: mmap failure [was: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?] 2021-09-05 0:04 ` Ken Brown @ 2021-09-05 13:24 ` Ken Brown 2021-09-06 15:32 ` Corinna Vinschen 0 siblings, 1 reply; 86+ messages in thread From: Ken Brown @ 2021-09-05 13:24 UTC (permalink / raw) To: cygwin On 9/4/2021 8:04 PM, Ken Brown via Cygwin wrote: > On 9/4/2021 6:58 PM, Ken Brown via Cygwin wrote: >> On 9/4/2021 6:54 PM, Ken Brown via Cygwin wrote: >>> On 9/4/2021 6:37 PM, Ken Brown via Cygwin wrote: >>>> I've reduced the procps failure to the following test case: >>>> >>>> $ cat mmap_test.c >>>> #include <unistd.h> >>>> #include <stdio.h> >>>> #include <sys/mman.h> >>>> >>>> int >>>> main () >>>> { >>>> void *addr; >>>> int page_size = getpagesize (); >>>> >>>> addr = mmap (0, page_size, PROT_READ | PROT_WRITE, >>>> MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); >>>> if (addr == MAP_FAILED) >>>> perror ("mmap"); >>>> } >>>> >>>> $ gcc mmap_test.c >>>> >>>> $ ./a >>>> mmap: Invalid argument >>>> >>>> This happens if I use cygwin1.dll built from the current git master. But it >>>> also happens if I build from cygwin-3_2_0-release with the recent patches >>>> applied that make Cygwin build without error: >>>> >>>> 0e12b4939 Cygwin: C++17: register keyword is deprecated >>>> a7340e0c0 Cygwin: dumper: fix up GCC pragma for g++ 11.2 >>>> 2a212c086 Cygwin: workaround a g++ 11.2 initialization bug >>>> 9e3f1737e Cygwin: loader script: add DWARF 5 sections >>>> bdfd2b004 Cygwin: testsuite: avoid "conflicting types" gcc warning >>>> 6fc498e2e strstr: avoid warnings >>>> 26da270b2 ldexp/ldexpf: avoid assembler warning >>>> edce2a557 Cygwin: fix declaration of RtlInitEmptyUnicodeString >>> >>> Sorry, those commit ids are wrong. They're what I got after rebasing the >>> master branch so that those commits come first. >> >> [Pressed Send too soon.] >> >> Here are the correct commits: >> >> 8169e39ab Cygwin: C++17: register keyword is deprecated >> 3ca80b360 Cygwin: dumper: fix up GCC pragma for g++ 11.2 >> bdb7991db Cygwin: workaround a g++ 11.2 initialization bug >> 801120c1f Cygwin: loader script: add DWARF 5 sections >> d5cc66426 Cygwin: testsuite: avoid "conflicting types" gcc warning >> c2fe205b5 strstr: avoid warnings >> 76c2c7a89 ldexp/ldexpf: avoid assembler warning >> eeeb5650c Cygwin: fix declaration of RtlInitEmptyUnicodeString >> >>> >>>> So there appears to be something wrong with cygwin1.dll built with the >>>> current build tools (gcc 11.2.0, binutils 2.37, not sure what else is >>>> relevant). > > Wait a minute...I'll bet this is related to the MEM_EXTENDED_PARAMETER > initialization problem that was dealt with in commit bdb7991db. More data: When I run the test case under gdb, it succeeds. When I run it under strace, I see VirtualAlloc2 in fhandler_dev_zero::mmap failing with windows error 87. Ken ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: mmap failure [was: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?] 2021-09-05 13:24 ` Ken Brown @ 2021-09-06 15:32 ` Corinna Vinschen 2021-09-06 17:12 ` Ken Brown 0 siblings, 1 reply; 86+ messages in thread From: Corinna Vinschen @ 2021-09-06 15:32 UTC (permalink / raw) To: cygwin On Sep 5 09:24, Ken Brown via Cygwin wrote: > On 9/4/2021 8:04 PM, Ken Brown via Cygwin wrote: > > On 9/4/2021 6:58 PM, Ken Brown via Cygwin wrote: > > > Here are the correct commits: > > > > > > 8169e39ab Cygwin: C++17: register keyword is deprecated > > > 3ca80b360 Cygwin: dumper: fix up GCC pragma for g++ 11.2 > > > bdb7991db Cygwin: workaround a g++ 11.2 initialization bug > > > 801120c1f Cygwin: loader script: add DWARF 5 sections > > > d5cc66426 Cygwin: testsuite: avoid "conflicting types" gcc warning > > > c2fe205b5 strstr: avoid warnings > > > 76c2c7a89 ldexp/ldexpf: avoid assembler warning > > > eeeb5650c Cygwin: fix declaration of RtlInitEmptyUnicodeString > > > > > > > > > > > > So there appears to be something wrong with cygwin1.dll > > > > > built with the current build tools (gcc 11.2.0, binutils > > > > > 2.37, not sure what else is relevant). > > > > Wait a minute...I'll bet this is related to the MEM_EXTENDED_PARAMETER > > initialization problem that was dealt with in commit bdb7991db. > > More data: When I run the test case under gdb, it succeeds. When I run it > under strace, I see VirtualAlloc2 in fhandler_dev_zero::mmap failing with > windows error 87. Are the const's I added to the MEM_EXTENDED_PARAMETER data invalid, perhaps? Corinna ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: mmap failure [was: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?] 2021-09-06 15:32 ` Corinna Vinschen @ 2021-09-06 17:12 ` Ken Brown 2021-09-06 17:38 ` Ken Brown 0 siblings, 1 reply; 86+ messages in thread From: Ken Brown @ 2021-09-06 17:12 UTC (permalink / raw) To: cygwin On 9/6/2021 11:32 AM, Corinna Vinschen via Cygwin wrote: > On Sep 5 09:24, Ken Brown via Cygwin wrote: >> On 9/4/2021 8:04 PM, Ken Brown via Cygwin wrote: >>> On 9/4/2021 6:58 PM, Ken Brown via Cygwin wrote: >>>> Here are the correct commits: >>>> >>>> 8169e39ab Cygwin: C++17: register keyword is deprecated >>>> 3ca80b360 Cygwin: dumper: fix up GCC pragma for g++ 11.2 >>>> bdb7991db Cygwin: workaround a g++ 11.2 initialization bug >>>> 801120c1f Cygwin: loader script: add DWARF 5 sections >>>> d5cc66426 Cygwin: testsuite: avoid "conflicting types" gcc warning >>>> c2fe205b5 strstr: avoid warnings >>>> 76c2c7a89 ldexp/ldexpf: avoid assembler warning >>>> eeeb5650c Cygwin: fix declaration of RtlInitEmptyUnicodeString >>>> >>>>> >>>>>> So there appears to be something wrong with cygwin1.dll >>>>>> built with the current build tools (gcc 11.2.0, binutils >>>>>> 2.37, not sure what else is relevant). >>> >>> Wait a minute...I'll bet this is related to the MEM_EXTENDED_PARAMETER >>> initialization problem that was dealt with in commit bdb7991db. >> >> More data: When I run the test case under gdb, it succeeds. When I run it >> under strace, I see VirtualAlloc2 in fhandler_dev_zero::mmap failing with >> windows error 87. > > Are the const's I added to the MEM_EXTENDED_PARAMETER data invalid, > perhaps? I tried removing them, and I got the same error. I also tried removing static, and I tried removing both static and const. Ken ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: mmap failure [was: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?] 2021-09-06 17:12 ` Ken Brown @ 2021-09-06 17:38 ` Ken Brown 2021-09-06 17:43 ` Eliot Moss 2021-09-06 17:59 ` Corinna Vinschen 0 siblings, 2 replies; 86+ messages in thread From: Ken Brown @ 2021-09-06 17:38 UTC (permalink / raw) To: cygwin On 9/6/2021 1:12 PM, Ken Brown via Cygwin wrote: > On 9/6/2021 11:32 AM, Corinna Vinschen via Cygwin wrote: >> On Sep 5 09:24, Ken Brown via Cygwin wrote: >>> On 9/4/2021 8:04 PM, Ken Brown via Cygwin wrote: >>>> On 9/4/2021 6:58 PM, Ken Brown via Cygwin wrote: >>>>> Here are the correct commits: >>>>> >>>>> 8169e39ab Cygwin: C++17: register keyword is deprecated >>>>> 3ca80b360 Cygwin: dumper: fix up GCC pragma for g++ 11.2 >>>>> bdb7991db Cygwin: workaround a g++ 11.2 initialization bug >>>>> 801120c1f Cygwin: loader script: add DWARF 5 sections >>>>> d5cc66426 Cygwin: testsuite: avoid "conflicting types" gcc warning >>>>> c2fe205b5 strstr: avoid warnings >>>>> 76c2c7a89 ldexp/ldexpf: avoid assembler warning >>>>> eeeb5650c Cygwin: fix declaration of RtlInitEmptyUnicodeString >>>>> >>>>>> >>>>>>> So there appears to be something wrong with cygwin1.dll >>>>>>> built with the current build tools (gcc 11.2.0, binutils >>>>>>> 2.37, not sure what else is relevant). >>>> >>>> Wait a minute...I'll bet this is related to the MEM_EXTENDED_PARAMETER >>>> initialization problem that was dealt with in commit bdb7991db. >>> >>> More data: When I run the test case under gdb, it succeeds. When I run it >>> under strace, I see VirtualAlloc2 in fhandler_dev_zero::mmap failing with >>> windows error 87. >> >> Are the const's I added to the MEM_EXTENDED_PARAMETER data invalid, >> perhaps? > > I tried removing them, and I got the same error. I also tried removing static, > and I tried removing both static and const. BTW, when I reported that the test case succeeds under gdb, that only happens when I build the test case without optimization. If I build with -O2, it fails under gdb also. [In all my tests, I built cygwin1.dll without optimization.] This makes no sense to me at all. Ken ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: mmap failure [was: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?] 2021-09-06 17:38 ` Ken Brown @ 2021-09-06 17:43 ` Eliot Moss 2021-09-06 17:59 ` Corinna Vinschen 1 sibling, 0 replies; 86+ messages in thread From: Eliot Moss @ 2021-09-06 17:43 UTC (permalink / raw) To: cygwin On 9/6/2021 1:38 PM, Ken Brown via Cygwin wrote: > On 9/6/2021 1:12 PM, Ken Brown via Cygwin wrote: >> On 9/6/2021 11:32 AM, Corinna Vinschen via Cygwin wrote: >>> On Sep 5 09:24, Ken Brown via Cygwin wrote: >>>> On 9/4/2021 8:04 PM, Ken Brown via Cygwin wrote: >>>>> On 9/4/2021 6:58 PM, Ken Brown via Cygwin wrote: >>>>>> Here are the correct commits: >>>>>> >>>>>> 8169e39ab Cygwin: C++17: register keyword is deprecated >>>>>> 3ca80b360 Cygwin: dumper: fix up GCC pragma for g++ 11.2 >>>>>> bdb7991db Cygwin: workaround a g++ 11.2 initialization bug >>>>>> 801120c1f Cygwin: loader script: add DWARF 5 sections >>>>>> d5cc66426 Cygwin: testsuite: avoid "conflicting types" gcc warning >>>>>> c2fe205b5 strstr: avoid warnings >>>>>> 76c2c7a89 ldexp/ldexpf: avoid assembler warning >>>>>> eeeb5650c Cygwin: fix declaration of RtlInitEmptyUnicodeString >>>>>> >>>>>>> >>>>>>>> So there appears to be something wrong with cygwin1.dll >>>>>>>> built with the current build tools (gcc 11.2.0, binutils >>>>>>>> 2.37, not sure what else is relevant). >>>>> >>>>> Wait a minute...I'll bet this is related to the MEM_EXTENDED_PARAMETER >>>>> initialization problem that was dealt with in commit bdb7991db. >>>> >>>> More data: When I run the test case under gdb, it succeeds. When I run it >>>> under strace, I see VirtualAlloc2 in fhandler_dev_zero::mmap failing with >>>> windows error 87. >>> >>> Are the const's I added to the MEM_EXTENDED_PARAMETER data invalid, >>> perhaps? >> >> I tried removing them, and I got the same error. I also tried removing static, and I tried >> removing both static and const. > > BTW, when I reported that the test case succeeds under gdb, that only happens when I build the test > case without optimization. If I build with -O2, it fails under gdb also. [In all my tests, I built > cygwin1.dll without optimization.] This makes no sense to me at all. There can be a number of possibilities, but I wonder about a variable uninitialized along some path. By accident, the contents may have a non-failing value with -O0 by the situation can be different for -O2. If you're dealing with concurrency and such, then -O0 and -O2 can act differently with respect to races. Just some thoughts ... Best - Eliot Moss ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: mmap failure [was: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?] 2021-09-06 17:38 ` Ken Brown 2021-09-06 17:43 ` Eliot Moss @ 2021-09-06 17:59 ` Corinna Vinschen 2021-09-06 18:07 ` Corinna Vinschen 1 sibling, 1 reply; 86+ messages in thread From: Corinna Vinschen @ 2021-09-06 17:59 UTC (permalink / raw) To: cygwin On Sep 6 13:38, Ken Brown via Cygwin wrote: > On 9/6/2021 1:12 PM, Ken Brown via Cygwin wrote: > > On 9/6/2021 11:32 AM, Corinna Vinschen via Cygwin wrote: > > > On Sep 5 09:24, Ken Brown via Cygwin wrote: > > > > On 9/4/2021 8:04 PM, Ken Brown via Cygwin wrote: > > > > > On 9/4/2021 6:58 PM, Ken Brown via Cygwin wrote: > > > > > > Here are the correct commits: > > > > > > > > > > > > 8169e39ab Cygwin: C++17: register keyword is deprecated > > > > > > 3ca80b360 Cygwin: dumper: fix up GCC pragma for g++ 11.2 > > > > > > bdb7991db Cygwin: workaround a g++ 11.2 initialization bug > > > > > > 801120c1f Cygwin: loader script: add DWARF 5 sections > > > > > > d5cc66426 Cygwin: testsuite: avoid "conflicting types" gcc warning > > > > > > c2fe205b5 strstr: avoid warnings > > > > > > 76c2c7a89 ldexp/ldexpf: avoid assembler warning > > > > > > eeeb5650c Cygwin: fix declaration of RtlInitEmptyUnicodeString > > > > > > > > > > > > > > > > > > > > > So there appears to be something wrong with cygwin1.dll > > > > > > > > built with the current build tools (gcc 11.2.0, binutils > > > > > > > > 2.37, not sure what else is relevant). > > > > > > > > > > Wait a minute...I'll bet this is related to the MEM_EXTENDED_PARAMETER > > > > > initialization problem that was dealt with in commit bdb7991db. > > > > > > > > More data: When I run the test case under gdb, it succeeds. When I run it > > > > under strace, I see VirtualAlloc2 in fhandler_dev_zero::mmap failing with > > > > windows error 87. > > > > > > Are the const's I added to the MEM_EXTENDED_PARAMETER data invalid, > > > perhaps? > > > > I tried removing them, and I got the same error. I also tried removing > > static, and I tried removing both static and const. > > BTW, when I reported that the test case succeeds under gdb, that only > happens when I build the test case without optimization. If I build with > -O2, it fails under gdb also. [In all my tests, I built cygwin1.dll without > optimization.] This makes no sense to me at all. Good hint. I found the culprit. With optimization, the code doesn't set the "Reserved" bits in the first struct of MEM_EXTENDED_PARAMETER to 0. This is at least required by the VirtualAlloc2 function, though. Needless to say that this behaviour isn't documented... I'll push a patch shortly. Thanks, Corinna ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: mmap failure [was: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?] 2021-09-06 17:59 ` Corinna Vinschen @ 2021-09-06 18:07 ` Corinna Vinschen 2021-09-06 18:40 ` Ken Brown 0 siblings, 1 reply; 86+ messages in thread From: Corinna Vinschen @ 2021-09-06 18:07 UTC (permalink / raw) To: cygwin On Sep 6 19:59, Corinna Vinschen via Cygwin wrote: > On Sep 6 13:38, Ken Brown via Cygwin wrote: > > On 9/6/2021 1:12 PM, Ken Brown via Cygwin wrote: > > > On 9/6/2021 11:32 AM, Corinna Vinschen via Cygwin wrote: > > > > On Sep 5 09:24, Ken Brown via Cygwin wrote: > > > > > On 9/4/2021 8:04 PM, Ken Brown via Cygwin wrote: > > > > > > On 9/4/2021 6:58 PM, Ken Brown via Cygwin wrote: > > > > > > > Here are the correct commits: > > > > > > > > > > > > > > 8169e39ab Cygwin: C++17: register keyword is deprecated > > > > > > > 3ca80b360 Cygwin: dumper: fix up GCC pragma for g++ 11.2 > > > > > > > bdb7991db Cygwin: workaround a g++ 11.2 initialization bug > > > > > > > 801120c1f Cygwin: loader script: add DWARF 5 sections > > > > > > > d5cc66426 Cygwin: testsuite: avoid "conflicting types" gcc warning > > > > > > > c2fe205b5 strstr: avoid warnings > > > > > > > 76c2c7a89 ldexp/ldexpf: avoid assembler warning > > > > > > > eeeb5650c Cygwin: fix declaration of RtlInitEmptyUnicodeString > > > > > > > > > > > > > > > > > > > > > > > > So there appears to be something wrong with cygwin1.dll > > > > > > > > > built with the current build tools (gcc 11.2.0, binutils > > > > > > > > > 2.37, not sure what else is relevant). > > > > > > > > > > > > Wait a minute...I'll bet this is related to the MEM_EXTENDED_PARAMETER > > > > > > initialization problem that was dealt with in commit bdb7991db. > > > > > > > > > > More data: When I run the test case under gdb, it succeeds. When I run it > > > > > under strace, I see VirtualAlloc2 in fhandler_dev_zero::mmap failing with > > > > > windows error 87. > > > > > > > > Are the const's I added to the MEM_EXTENDED_PARAMETER data invalid, > > > > perhaps? > > > > > > I tried removing them, and I got the same error. I also tried removing > > > static, and I tried removing both static and const. > > > > BTW, when I reported that the test case succeeds under gdb, that only > > happens when I build the test case without optimization. If I build with > > -O2, it fails under gdb also. [In all my tests, I built cygwin1.dll without > > optimization.] This makes no sense to me at all. > > Good hint. I found the culprit. With optimization, the code doesn't > set the "Reserved" bits in the first struct of MEM_EXTENDED_PARAMETER > to 0. No, wait. I get what you say. The optimzation settings of the test case should have no influence on the code inside the DLL. That doesn't make sense for sure. However, I ran the testcase under GDB, I could reproduce the issue, and I could fix it by setting mmap_ext.Reserved = 0; Go figure! Corinna ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: mmap failure [was: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?] 2021-09-06 18:07 ` Corinna Vinschen @ 2021-09-06 18:40 ` Ken Brown 2021-09-06 20:52 ` Peter Dons Tychsen 2021-09-06 20:54 ` Peter Dons Tychsen 0 siblings, 2 replies; 86+ messages in thread From: Ken Brown @ 2021-09-06 18:40 UTC (permalink / raw) To: cygwin On 9/6/2021 2:07 PM, Corinna Vinschen via Cygwin wrote: > On Sep 6 19:59, Corinna Vinschen via Cygwin wrote: >> On Sep 6 13:38, Ken Brown via Cygwin wrote: >>> On 9/6/2021 1:12 PM, Ken Brown via Cygwin wrote: >>>> On 9/6/2021 11:32 AM, Corinna Vinschen via Cygwin wrote: >>>>> On Sep 5 09:24, Ken Brown via Cygwin wrote: >>>>>> On 9/4/2021 8:04 PM, Ken Brown via Cygwin wrote: >>>>>>> On 9/4/2021 6:58 PM, Ken Brown via Cygwin wrote: >>>>>>>> Here are the correct commits: >>>>>>>> >>>>>>>> 8169e39ab Cygwin: C++17: register keyword is deprecated >>>>>>>> 3ca80b360 Cygwin: dumper: fix up GCC pragma for g++ 11.2 >>>>>>>> bdb7991db Cygwin: workaround a g++ 11.2 initialization bug >>>>>>>> 801120c1f Cygwin: loader script: add DWARF 5 sections >>>>>>>> d5cc66426 Cygwin: testsuite: avoid "conflicting types" gcc warning >>>>>>>> c2fe205b5 strstr: avoid warnings >>>>>>>> 76c2c7a89 ldexp/ldexpf: avoid assembler warning >>>>>>>> eeeb5650c Cygwin: fix declaration of RtlInitEmptyUnicodeString >>>>>>>> >>>>>>>>> >>>>>>>>>> So there appears to be something wrong with cygwin1.dll >>>>>>>>>> built with the current build tools (gcc 11.2.0, binutils >>>>>>>>>> 2.37, not sure what else is relevant). >>>>>>> >>>>>>> Wait a minute...I'll bet this is related to the MEM_EXTENDED_PARAMETER >>>>>>> initialization problem that was dealt with in commit bdb7991db. >>>>>> >>>>>> More data: When I run the test case under gdb, it succeeds. When I run it >>>>>> under strace, I see VirtualAlloc2 in fhandler_dev_zero::mmap failing with >>>>>> windows error 87. >>>>> >>>>> Are the const's I added to the MEM_EXTENDED_PARAMETER data invalid, >>>>> perhaps? >>>> >>>> I tried removing them, and I got the same error. I also tried removing >>>> static, and I tried removing both static and const. >>> >>> BTW, when I reported that the test case succeeds under gdb, that only >>> happens when I build the test case without optimization. If I build with >>> -O2, it fails under gdb also. [In all my tests, I built cygwin1.dll without >>> optimization.] This makes no sense to me at all. >> >> Good hint. I found the culprit. With optimization, the code doesn't >> set the "Reserved" bits in the first struct of MEM_EXTENDED_PARAMETER >> to 0. > > No, wait. I get what you say. The optimzation settings of the test > case should have no influence on the code inside the DLL. That doesn't > make sense for sure. However, I ran the testcase under GDB, I could > reproduce the issue, and I could fix it by setting mmap_ext.Reserved = 0; > Go figure! I don't get it, but I can confirm that the problem is fixed. Thanks. Ken ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: mmap failure [was: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?] 2021-09-06 18:40 ` Ken Brown @ 2021-09-06 20:52 ` Peter Dons Tychsen 2021-09-06 20:54 ` Peter Dons Tychsen 1 sibling, 0 replies; 86+ messages in thread From: Peter Dons Tychsen @ 2021-09-06 20:52 UTC (permalink / raw) To: Ken Brown, cygwin Hi there, On Mon, 2021-09-06 at 14:40 -0400, Ken Brown via Cygwin wrote: > > No, wait. I get what you say. The optimzation settings of the test > > case should have no influence on the code inside the DLL. That > > doesn't > > make sense for sure. However, I ran the testcase under GDB, I could > > reproduce the issue, and I could fix it by setting mmap_ext.Reserved > > = 0; > > Go figure! > > I don't get it, but I can confirm that the problem is fixed. That sounds a bit like a voodoo fix, that could quickly regress again. Here is my 2 cents: Currently the mmap_ext structure is setup like this: 215 MEM_EXTENDED_PARAMETER mmap_ext = { 216 .Type = MemExtendedParameterAddressRequirements, 217 .Pointer = (PVOID) &mmap_req 218 }; This means that all other entries in the struct are zero at initialization as described here: https://en.cppreference.com/w/c/language/struct_initialization So if you set "mmap_ext.Reserved = 0" again after that its a double failure. 1) The compiler should ignore it as it is redundent. 2) It makes no sense, as it is already zero :-) Since it is not ignored, the compiler clearly puts in code to to reinitialize the variable (which some compilers do not optimize for). The reason this makes it "work" it probably because of some other stack curruption that is going on which then disappears because of the code moving around a bit due to the new line. My best guess would be that other fun stuff in the same location would also "fix" the problem. These are not the droids you are looking for. The real problem is elsewhere, and is likely due to some stack-smashing going on. This is also likely why recompiling the test program makes a difference as that changes what goes on the variable stack. When the code moves around again (e.g. new compiler version), it could break again. Looking at the test program -02 vs -O0: pushq %rbp .seh_pushreg %rbp movq %rsp, %rbp .seh_setframe %rbp, 0 subq $64, %rsp .seh_stackalloc 64 .seh_endprologue vs subq $56, %rsp .seh_stackalloc 56 Which gives a different stack layout. I think the problem must be in the start of mmap() or subsequent calls like CreateMapping() and MapView(). Something smashes or affects the stack. Thanks, /pedro ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: mmap failure [was: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?] 2021-09-06 18:40 ` Ken Brown 2021-09-06 20:52 ` Peter Dons Tychsen @ 2021-09-06 20:54 ` Peter Dons Tychsen 2021-09-06 21:24 ` Ken Brown 1 sibling, 1 reply; 86+ messages in thread From: Peter Dons Tychsen @ 2021-09-06 20:54 UTC (permalink / raw) To: Ken Brown, cygwin Hi there, On Mon, 2021-09-06 at 14:40 -0400, Ken Brown via Cygwin wrote: > > No, wait. I get what you say. The optimzation settings of the test > > case should have no influence on the code inside the DLL. That > > doesn't > > make sense for sure. However, I ran the testcase under GDB, I could > > reproduce the issue, and I could fix it by setting mmap_ext.Reserved > > = 0; > > Go figure! > > I don't get it, but I can confirm that the problem is fixed. That sounds a bit like a voodoo fix, that could quickly regress again. Here is my 2 cents: Currently the mmap_ext structure is setup like this: 215 MEM_EXTENDED_PARAMETER mmap_ext = { 216 .Type = MemExtendedParameterAddressRequirements, 217 .Pointer = (PVOID) &mmap_req 218 }; This means that all other entries in the struct are zero at initialization as described here: https://en.cppreference.com/w/c/language/struct_initialization So if you set "mmap_ext.Reserved = 0" again after that its a double failure. 1) The compiler should ignore it as it is redundent. 2) It makes no sense, as it is already zero :-) Since it is not ignored, the compiler clearly puts in code to to reinitialize the variable (which some compilers do not optimize for). The reason this makes it "work" it probably because of some other stack curruption that is going on which then disappears because of the code moving around a bit due to the new line. My best guess would be that other fun stuff in the same location would also "fix" the problem. These are not the droids you are looking for. The real problem is elsewhere, and is likely due to some stack-smashing going on. This is also likely why recompiling the test program makes a difference as that changes what goes on the variable stack. When the code moves around again (e.g. new compiler version), it could break again. Looking at the test program -02 vs -O0: pushq %rbp .seh_pushreg %rbp movq %rsp, %rbp .seh_setframe %rbp, 0 subq $64, %rsp .seh_stackalloc 64 .seh_endprologue vs subq $56, %rsp .seh_stackalloc 56 Which gives a different stack layout. I think the problem must be in the start of mmap() or subsequent calls like CreateMapping() and MapView(). Something smashes or affects the stack. Thanks, /pedro ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: mmap failure [was: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?] 2021-09-06 20:54 ` Peter Dons Tychsen @ 2021-09-06 21:24 ` Ken Brown 2021-09-06 21:31 ` Ken Brown ` (2 more replies) 0 siblings, 3 replies; 86+ messages in thread From: Ken Brown @ 2021-09-06 21:24 UTC (permalink / raw) To: Peter Dons Tychsen, cygwin On 9/6/2021 4:54 PM, Peter Dons Tychsen wrote: > Hi there, > > On Mon, 2021-09-06 at 14:40 -0400, Ken Brown via Cygwin wrote: >>> No, wait. I get what you say. The optimzation settings of the test >>> case should have no influence on the code inside the DLL. That >>> doesn't >>> make sense for sure. However, I ran the testcase under GDB, I could >>> reproduce the issue, and I could fix it by setting mmap_ext.Reserved >>> = 0; >>> Go figure! >> >> I don't get it, but I can confirm that the problem is fixed. > > That sounds a bit like a voodoo fix, that could quickly regress again. > > Here is my 2 cents: > > Currently the mmap_ext structure is setup like this: > > 215 MEM_EXTENDED_PARAMETER mmap_ext = { > 216 .Type = MemExtendedParameterAddressRequirements, > 217 .Pointer = (PVOID) &mmap_req > 218 }; > > This means that all other entries in the struct are zero at > initialization as described here: > https://en.cppreference.com/w/c/language/struct_initialization > > So if you set "mmap_ext.Reserved = 0" again after that its a double > failure. You're looking at the wrong source code. The bug didn't occur until the code was changed to do the following: /* g++ 11.2 workaround: don't use initializer */ MEM_EXTENDED_PARAMETER mmap_ext; mmap_ext.Type = MemExtendedParameterAddressRequirements; mmap_ext.Pointer = (PVOID) &mmap_req; This left mmap_ext.Reserved uninitialized, which Corinna has now fixed. Ken ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: mmap failure [was: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?] 2021-09-06 21:24 ` Ken Brown @ 2021-09-06 21:31 ` Ken Brown 2021-09-07 3:34 ` Brian Inglis 2021-09-07 21:41 ` Peter Dons Tychsen 2 siblings, 0 replies; 86+ messages in thread From: Ken Brown @ 2021-09-06 21:31 UTC (permalink / raw) To: Peter Dons Tychsen, cygwin On 9/6/2021 5:24 PM, Ken Brown via Cygwin wrote: > On 9/6/2021 4:54 PM, Peter Dons Tychsen wrote: >> Hi there, >> >> On Mon, 2021-09-06 at 14:40 -0400, Ken Brown via Cygwin wrote: >>>> No, wait. I get what you say. The optimzation settings of the test >>>> case should have no influence on the code inside the DLL. That >>>> doesn't >>>> make sense for sure. However, I ran the testcase under GDB, I could >>>> reproduce the issue, and I could fix it by setting mmap_ext.Reserved >>>> = 0; >>>> Go figure! >>> >>> I don't get it, but I can confirm that the problem is fixed. >> >> That sounds a bit like a voodoo fix, that could quickly regress again. >> >> Here is my 2 cents: >> >> Currently the mmap_ext structure is setup like this: >> >> 215 MEM_EXTENDED_PARAMETER mmap_ext = { >> 216 .Type = MemExtendedParameterAddressRequirements, >> 217 .Pointer = (PVOID) &mmap_req >> 218 }; >> >> This means that all other entries in the struct are zero at >> initialization as described here: >> https://en.cppreference.com/w/c/language/struct_initialization >> >> So if you set "mmap_ext.Reserved = 0" again after that its a double >> failure. > > You're looking at the wrong source code. The bug didn't occur until the code > was changed to do the following: > > /* g++ 11.2 workaround: don't use initializer */ > MEM_EXTENDED_PARAMETER mmap_ext; > mmap_ext.Type = MemExtendedParameterAddressRequirements; > mmap_ext.Pointer = (PVOID) &mmap_req; > > This left mmap_ext.Reserved uninitialized, which Corinna has now fixed. I should add that when I said "I don't get it", I wasn't referring to Corinna's fix. I was referring to the fact that the bug did *not* occur when the unoptimized build was run under gdb. Ken ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: mmap failure [was: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?] 2021-09-06 21:24 ` Ken Brown 2021-09-06 21:31 ` Ken Brown @ 2021-09-07 3:34 ` Brian Inglis 2021-09-07 16:28 ` Ken Brown 2021-09-08 8:18 ` Corinna Vinschen 2021-09-07 21:41 ` Peter Dons Tychsen 2 siblings, 2 replies; 86+ messages in thread From: Brian Inglis @ 2021-09-07 3:34 UTC (permalink / raw) To: cygwin On 2021-09-06 15:24, Ken Brown via Cygwin wrote: > On 9/6/2021 4:54 PM, Peter Dons Tychsen wrote: >> Hi there, >> >> On Mon, 2021-09-06 at 14:40 -0400, Ken Brown via Cygwin wrote: >>>> No, wait. I get what you say. The optimzation settings of the test >>>> case should have no influence on the code inside the DLL. That >>>> doesn't >>>> make sense for sure. However, I ran the testcase under GDB, I could >>>> reproduce the issue, and I could fix it by setting mmap_ext.Reserved >>>> = 0; >>>> Go figure! >>> >>> I don't get it, but I can confirm that the problem is fixed. >> >> That sounds a bit like a voodoo fix, that could quickly regress again. >> >> Here is my 2 cents: >> >> Currently the mmap_ext structure is setup like this: >> >> 215 MEM_EXTENDED_PARAMETER mmap_ext = { >> 216 .Type = MemExtendedParameterAddressRequirements, >> 217 .Pointer = (PVOID) &mmap_req >> 218 }; >> >> This means that all other entries in the struct are zero at >> initialization as described here: >> https://en.cppreference.com/w/c/language/struct_initialization >> >> So if you set "mmap_ext.Reserved = 0" again after that its a double >> failure. > > You're looking at the wrong source code. The bug didn't occur until the > code was changed to do the following: > > /* g++ 11.2 workaround: don't use initializer */ > MEM_EXTENDED_PARAMETER mmap_ext; > mmap_ext.Type = MemExtendedParameterAddressRequirements; > mmap_ext.Pointer = (PVOID) &mmap_req; > > This left mmap_ext.Reserved uninitialized, which Corinna has now fixed. With undocumented structure member initialization an issue, maybe better to future proof using e.g. MEM_EXTENDED_PARAMETER mmap_ext = { 0 }; // or memset or bzero ... -- Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada This email may be disturbing to some readers as it contains too much technical detail. Reader discretion is advised. [Data in binary units and prefixes, physical quantities in SI.] ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: mmap failure [was: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?] 2021-09-07 3:34 ` Brian Inglis @ 2021-09-07 16:28 ` Ken Brown 2021-09-07 21:52 ` Lavrentiev, Anton (NIH/NLM/NCBI) [C] 2021-09-08 8:18 ` Corinna Vinschen 1 sibling, 1 reply; 86+ messages in thread From: Ken Brown @ 2021-09-07 16:28 UTC (permalink / raw) To: cygwin On 9/6/2021 11:34 PM, Brian Inglis wrote: > On 2021-09-06 15:24, Ken Brown via Cygwin wrote: >> On 9/6/2021 4:54 PM, Peter Dons Tychsen wrote: >>> Hi there, >>> >>> On Mon, 2021-09-06 at 14:40 -0400, Ken Brown via Cygwin wrote: >>>>> No, wait. I get what you say. The optimzation settings of the test >>>>> case should have no influence on the code inside the DLL. That >>>>> doesn't >>>>> make sense for sure. However, I ran the testcase under GDB, I could >>>>> reproduce the issue, and I could fix it by setting mmap_ext.Reserved >>>>> = 0; >>>>> Go figure! >>>> >>>> I don't get it, but I can confirm that the problem is fixed. >>> >>> That sounds a bit like a voodoo fix, that could quickly regress again. >>> >>> Here is my 2 cents: >>> >>> Currently the mmap_ext structure is setup like this: >>> >>> 215 MEM_EXTENDED_PARAMETER mmap_ext = { >>> 216 .Type = MemExtendedParameterAddressRequirements, >>> 217 .Pointer = (PVOID) &mmap_req >>> 218 }; >>> >>> This means that all other entries in the struct are zero at >>> initialization as described here: >>> https://en.cppreference.com/w/c/language/struct_initialization >>> >>> So if you set "mmap_ext.Reserved = 0" again after that its a double >>> failure. >> >> You're looking at the wrong source code. The bug didn't occur until the code >> was changed to do the following: >> >> /* g++ 11.2 workaround: don't use initializer */ >> MEM_EXTENDED_PARAMETER mmap_ext; >> mmap_ext.Type = MemExtendedParameterAddressRequirements; >> mmap_ext.Pointer = (PVOID) &mmap_req; >> >> This left mmap_ext.Reserved uninitialized, which Corinna has now fixed. > > With undocumented structure member initialization an issue, maybe better to > future proof using e.g. > > MEM_EXTENDED_PARAMETER mmap_ext = { 0 }; // or memset or bzero I don't see what this would accomplish. We're already initializing every member after Corinna's last patch. Ken ^ permalink raw reply [flat|nested] 86+ messages in thread
* RE: mmap failure [was: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?] 2021-09-07 16:28 ` Ken Brown @ 2021-09-07 21:52 ` Lavrentiev, Anton (NIH/NLM/NCBI) [C] 2021-09-07 22:44 ` Ken Brown 0 siblings, 1 reply; 86+ messages in thread From: Lavrentiev, Anton (NIH/NLM/NCBI) [C] @ 2021-09-07 21:52 UTC (permalink / raw) To: Ken Brown; +Cc: 'cygwin@cygwin.com' > > > > With undocumented structure member initialization an issue, maybe better to > > future proof using e.g. > > > > MEM_EXTENDED_PARAMETER mmap_ext = { 0 }; // or memset or bzero > > I don't see what this would accomplish. We're already initializing every member > after Corinna's last patch. > Well, if one day MS decides to use some of the Reserved field by splinting off a new named field out of that bulk of bits, the code will be in trouble again... While with memset, bzero or C-style initializer, everything is guaranteed to be zapped with 0. Anton Lavrentiev Contractor NIH/NLM/NCBI ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: mmap failure [was: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?] 2021-09-07 21:52 ` Lavrentiev, Anton (NIH/NLM/NCBI) [C] @ 2021-09-07 22:44 ` Ken Brown 2021-09-08 6:14 ` Sam Edge 0 siblings, 1 reply; 86+ messages in thread From: Ken Brown @ 2021-09-07 22:44 UTC (permalink / raw) To: Lavrentiev, Anton (NIH/NLM/NCBI) [C]; +Cc: 'cygwin@cygwin.com' On 9/7/2021 5:52 PM, Lavrentiev, Anton (NIH/NLM/NCBI) [C] wrote: >>> >>> With undocumented structure member initialization an issue, maybe better to >>> future proof using e.g. >>> >>> MEM_EXTENDED_PARAMETER mmap_ext = { 0 }; // or memset or bzero >> >> I don't see what this would accomplish. We're already initializing every member >> after Corinna's last patch. >> > > Well, if one day MS decides to use some of the Reserved field by splinting off a new > named field out of that bulk of bits, the code will be in trouble again... > While with memset, bzero or C-style initializer, everything is guaranteed to be zapped with 0. MS can't add a new named field to a documented struct without breaking a lot of code. I think it's extremely unlikely that they would do that. On the other hand, I think it's very likely that a reader of the Cygwin code would be confused by code that initializes a struct to 0 prior to assigning a value to every member. Ken ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: mmap failure [was: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?] 2021-09-07 22:44 ` Ken Brown @ 2021-09-08 6:14 ` Sam Edge 0 siblings, 0 replies; 86+ messages in thread From: Sam Edge @ 2021-09-08 6:14 UTC (permalink / raw) To: cygwin On 07/09/2021 23:44, Ken Brown via Cygwin wrote: > > MS can't add a new named field to a documented struct without breaking a lot of code. I think it's extremely unlikely that they would do that. On the other hand, I think it's very likely that a reader of the Cygwin code would be confused by code that initializes a struct to 0 prior to assigning a value to every member. Hi Ken. It is common practice when following a coding standard (especially those for safety or security sensitive code) to initialize each and every automatic C/C++ variable at the point of definition even if it is later assigned a value. The '= {0}' mechanism for structures is a common way of ensuring this is the case for all fields in a structure and should be familiar to most developers. This ensures deterministic behaviour instead of bugs that come and go depending upon what was on the stack or in the register before the variable is used. It also helps when reviewing code. A compiler or static analyser cannot always determine whether a variable is used before it's assigned so getting into the habit of doing this saves a lot of grief down the line, as demonstrated by this thread! -- Sam "Here Endeth The Lesson" Edge ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: mmap failure [was: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?] 2021-09-07 3:34 ` Brian Inglis 2021-09-07 16:28 ` Ken Brown @ 2021-09-08 8:18 ` Corinna Vinschen 1 sibling, 0 replies; 86+ messages in thread From: Corinna Vinschen @ 2021-09-08 8:18 UTC (permalink / raw) To: cygwin On Sep 6 21:34, Brian Inglis wrote: > On 2021-09-06 15:24, Ken Brown via Cygwin wrote: > > On 9/6/2021 4:54 PM, Peter Dons Tychsen wrote: > > > Hi there, > > > > > > On Mon, 2021-09-06 at 14:40 -0400, Ken Brown via Cygwin wrote: > > > > > No, wait. I get what you say. The optimzation settings of the test > > > > > case should have no influence on the code inside the DLL. That > > > > > doesn't > > > > > make sense for sure. However, I ran the testcase under GDB, I could > > > > > reproduce the issue, and I could fix it by setting mmap_ext.Reserved > > > > > = 0; > > > > > Go figure! > > > > > > > > I don't get it, but I can confirm that the problem is fixed. > > > > > > That sounds a bit like a voodoo fix, that could quickly regress again. > > > > > > Here is my 2 cents: > > > > > > Currently the mmap_ext structure is setup like this: > > > > > > 215 MEM_EXTENDED_PARAMETER mmap_ext = { > > > 216 .Type = MemExtendedParameterAddressRequirements, > > > 217 .Pointer = (PVOID) &mmap_req > > > 218 }; > > > > > > This means that all other entries in the struct are zero at > > > initialization as described here: > > > https://en.cppreference.com/w/c/language/struct_initialization > > > > > > So if you set "mmap_ext.Reserved = 0" again after that its a double > > > failure. > > > > You're looking at the wrong source code. The bug didn't occur until the > > code was changed to do the following: > > > > /* g++ 11.2 workaround: don't use initializer */ > > MEM_EXTENDED_PARAMETER mmap_ext; > > mmap_ext.Type = MemExtendedParameterAddressRequirements; > > mmap_ext.Pointer = (PVOID) &mmap_req; > > > > This left mmap_ext.Reserved uninitialized, which Corinna has now fixed. > > With undocumented structure member initialization an issue, maybe better to > future proof using e.g. > > MEM_EXTENDED_PARAMETER mmap_ext = { 0 }; // or memset or bzero > ... You're right. The bits in Reserved are just that, reserved. So MSFT may decide to use some of the bits for other purposes, should the need arise. I've fixed that in git master. Thanks, Corinna ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: mmap failure [was: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?] 2021-09-06 21:24 ` Ken Brown 2021-09-06 21:31 ` Ken Brown 2021-09-07 3:34 ` Brian Inglis @ 2021-09-07 21:41 ` Peter Dons Tychsen 2 siblings, 0 replies; 86+ messages in thread From: Peter Dons Tychsen @ 2021-09-07 21:41 UTC (permalink / raw) To: Ken Brown, cygwin Hi Ken, On Mon, 2021-09-06 at 17:24 -0400, Ken Brown via Cygwin wrote: > You're looking at the wrong source code. The bug didn't occur until > the code > was changed to do the following: You are right. I do not know why i looked at an old checkout of the code. Shame on me! Sorry for wasting your time. Maybe i should stop looking at code after 24:00. Bad idea! :-). So with my new shiny view, the fix makes perfectly sense. Great! /pedro ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: mmap failure [was: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?] 2021-09-04 22:37 ` mmap failure [was: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?] Ken Brown 2021-09-04 22:54 ` Ken Brown @ 2021-09-04 23:18 ` Brian Inglis 1 sibling, 0 replies; 86+ messages in thread From: Brian Inglis @ 2021-09-04 23:18 UTC (permalink / raw) To: cygwin On 2021-09-04 16:37, Ken Brown via Cygwin wrote: > I've reduced the procps failure to the following test case: > > $ cat mmap_test.c > #include <unistd.h> > #include <stdio.h> > #include <sys/mman.h> > > int > main () > { > void *addr; > int page_size = getpagesize (); > > addr = mmap (0, page_size, PROT_READ | PROT_WRITE, > MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); > if (addr == MAP_FAILED) > perror ("mmap"); > } > > $ gcc mmap_test.c > > $ ./a > mmap: Invalid argument Check on Linux and retry on Cygwin without MAP_PRIVATE? MAP_PRIVATE implies an fd but MAP_ANON does not, and MAP_ANON is in Linux(2)/Unix? not POSIX(3p). -- Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada This email may be disturbing to some readers as it contains too much technical detail. Reader discretion is advised. [Data in binary units and prefixes, physical quantities in SI.] ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? 2021-08-29 15:57 ` Ken Brown 2021-08-29 19:24 ` Chris Roehrig @ 2021-08-29 19:37 ` Takashi Yano 2021-08-29 21:09 ` Ken Brown 1 sibling, 1 reply; 86+ messages in thread From: Takashi Yano @ 2021-08-29 19:37 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 2804 bytes --] On Sun, 29 Aug 2021 11:57:04 -0400 Ken Brown wrote: > On 8/29/2021 5:07 AM, Takashi Yano via Cygwin wrote: > > On Sat, 28 Aug 2021 18:41:02 +0900 > > Takashi Yano wrote: > >> On Sat, 28 Aug 2021 10:43:27 +0200 > >> Corinna Vinschen wrote: > >>> On Aug 28 02:21, Takashi Yano via Cygwin wrote: > >>>> On Fri, 27 Aug 2021 12:00:50 -0400 > >>>> Ken Brown wrote: > >>>>> Two years ago I thought I needed nt_create to avoid problems when calling > >>>>> set_pipe_non_blocking. Are you saying that's not an issue? Is > >>>>> set_pipe_non_blocking unnecessary? Is that the point of your modification to > >>>>> raw_read? > >>>> > >>>> Yes. Instead of making windows read function itself non-blocking, > >>>> it is possible to check if the pipe can be read before read using > >>>> PeekNamedPipe(). If the pipe cannot be read right now, EAGAIN is > >>>> returned. > >>> > >>> The problem is this: > >>> > >>> if (PeekNamedPipe()) > >>> ReadFile(blocking); > >>> > >>> is not atomic. I. e., if PeekNamedPipe succeeds, nothing keeps another > >>> thread from draining the pipe between the PeekNamedPipe and the ReadFile > >>> call. And as soon as ReadFile runs, it hangs indefinitely and we can't > >>> stop it via a signal. > >> > >> Hmm, you are right. Mutex guard seems to be necessary like pty code > >> if we go this way. > > > > I have found that set_pipe_non_blocking() succeeds for both read and > > write pipes if the write pipe is created by CreateNamedPipe() and the > > read pipe is created by CreateFile() contrary to the current create() > > code. Therefore, not only nt_create() but also PeekNamedPipe() become > > unnecessary. > > > > Please see the revised patch attached. > > That's a great idea. > > I've applied your two patches to the topic/pipe branch. I also rebased it and > did a forced push in order to bring in Corinna's loader script fix. So you'll > have to do 'git fetch' and 'git rebase --hard origin/topic/pipe'. > > Does this now fix all known problems with pipes? Only the small thing remaining is pipe mode. If the pipe mode is changed to byte mode, the following issue will be solved. https://cygwin.com/pipermail/cygwin/2021-March/247987.html How about the simple patch attached? The comment in pipe code says: Note that the write side of the pipe is opened as PIPE_TYPE_MESSAGE. This *seems* to more closely mimic Linux pipe behavior and is definitely required for pty handling since fhandler_pty_master writes to the pipe in chunks, terminated by newline when CANON mode is specified. This mentions about pty behaiviour in canonical mode, however, the pty pipe is created as message mode even with this patch. Are there any other reasons that message mode is preferred for pipe? -- Takashi Yano <takashi.yano@nifty.ne.jp> [-- Attachment #2: 0003-Cygwin-pipe-Set-byte-mode-as-default-rather-than-mes.patch --] [-- Type: application/octet-stream, Size: 1091 bytes --] From ec2d43f25f7cf4fc3b01bd2e15e3b48e3c9461d1 Mon Sep 17 00:00:00 2001 From: Takashi Yano <takashi.yano@nifty.ne.jp> Date: Mon, 30 Aug 2021 04:12:56 +0900 Subject: [PATCH 3/3] Cygwin: pipe: Set byte mode as default rather than message mode. Currently, the pipe mode is message by default. This causes a issue with C# (.NET) programs if they are piped. With this patch, default pipe mode is changed to byte mode. The pipe mode can be revert to message mode by setting CYGWIN=nopipe_byte. Addresses: https://cygwin.com/pipermail/cygwin/2021-March/247987.html --- winsup/cygwin/globals.cc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/winsup/cygwin/globals.cc b/winsup/cygwin/globals.cc index 48fb312de..3534381ec 100644 --- a/winsup/cygwin/globals.cc +++ b/winsup/cygwin/globals.cc @@ -69,7 +69,7 @@ int NO_COPY dynamically_loaded; /* Some CYGWIN environment variable variables. */ bool allow_glob = true; bool ignore_case_with_glob; -bool pipe_byte; +bool pipe_byte = true; bool reset_com; bool wincmdln; winsym_t allow_winsymlinks = WSYM_default; -- 2.33.0 ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? 2021-08-29 19:37 ` cygrunsrv + sshd + rsync = 20 times too slow -- throttled? Takashi Yano @ 2021-08-29 21:09 ` Ken Brown 0 siblings, 0 replies; 86+ messages in thread From: Ken Brown @ 2021-08-29 21:09 UTC (permalink / raw) To: cygwin On 8/29/2021 3:37 PM, Takashi Yano wrote: > On Sun, 29 Aug 2021 11:57:04 -0400 > Ken Brown wrote: >> On 8/29/2021 5:07 AM, Takashi Yano via Cygwin wrote: >>> On Sat, 28 Aug 2021 18:41:02 +0900 >>> Takashi Yano wrote: >>>> On Sat, 28 Aug 2021 10:43:27 +0200 >>>> Corinna Vinschen wrote: >>>>> On Aug 28 02:21, Takashi Yano via Cygwin wrote: >>>>>> On Fri, 27 Aug 2021 12:00:50 -0400 >>>>>> Ken Brown wrote: >>>>>>> Two years ago I thought I needed nt_create to avoid problems when calling >>>>>>> set_pipe_non_blocking. Are you saying that's not an issue? Is >>>>>>> set_pipe_non_blocking unnecessary? Is that the point of your modification to >>>>>>> raw_read? >>>>>> >>>>>> Yes. Instead of making windows read function itself non-blocking, >>>>>> it is possible to check if the pipe can be read before read using >>>>>> PeekNamedPipe(). If the pipe cannot be read right now, EAGAIN is >>>>>> returned. >>>>> >>>>> The problem is this: >>>>> >>>>> if (PeekNamedPipe()) >>>>> ReadFile(blocking); >>>>> >>>>> is not atomic. I. e., if PeekNamedPipe succeeds, nothing keeps another >>>>> thread from draining the pipe between the PeekNamedPipe and the ReadFile >>>>> call. And as soon as ReadFile runs, it hangs indefinitely and we can't >>>>> stop it via a signal. >>>> >>>> Hmm, you are right. Mutex guard seems to be necessary like pty code >>>> if we go this way. >>> >>> I have found that set_pipe_non_blocking() succeeds for both read and >>> write pipes if the write pipe is created by CreateNamedPipe() and the >>> read pipe is created by CreateFile() contrary to the current create() >>> code. Therefore, not only nt_create() but also PeekNamedPipe() become >>> unnecessary. >>> >>> Please see the revised patch attached. >> >> That's a great idea. >> >> I've applied your two patches to the topic/pipe branch. I also rebased it and >> did a forced push in order to bring in Corinna's loader script fix. So you'll >> have to do 'git fetch' and 'git rebase --hard origin/topic/pipe'. >> >> Does this now fix all known problems with pipes? > > Only the small thing remaining is pipe mode. If the pipe mode > is changed to byte mode, the following issue will be solved. > https://cygwin.com/pipermail/cygwin/2021-March/247987.html > > How about the simple patch attached? > > The comment in pipe code says: > Note that the write side of the pipe is opened as PIPE_TYPE_MESSAGE. > This *seems* to more closely mimic Linux pipe behavior and is > definitely required for pty handling since fhandler_pty_master > writes to the pipe in chunks, terminated by newline when CANON mode > is specified. > > This mentions about pty behaiviour in canonical mode, however, the > pty pipe is created as message mode even with this patch. Are there > any other reasons that message mode is preferred for pipe? No idea. All I remember is that there was a lot of discussion around the time that it was decided to use PIPE_TYPE_MESSAGE by default. Corinna probably remembers the reasons. Ken ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? 2021-08-29 9:07 ` Takashi Yano 2021-08-29 15:57 ` Ken Brown @ 2021-08-29 21:04 ` Ken Brown 2021-08-30 0:13 ` Takashi Yano 1 sibling, 1 reply; 86+ messages in thread From: Ken Brown @ 2021-08-29 21:04 UTC (permalink / raw) To: cygwin On 8/29/2021 5:07 AM, Takashi Yano via Cygwin wrote: > On Sat, 28 Aug 2021 18:41:02 +0900 > Takashi Yano wrote: >> On Sat, 28 Aug 2021 10:43:27 +0200 >> Corinna Vinschen wrote: >>> On Aug 28 02:21, Takashi Yano via Cygwin wrote: >>>> On Fri, 27 Aug 2021 12:00:50 -0400 >>>> Ken Brown wrote: >>>>> Two years ago I thought I needed nt_create to avoid problems when calling >>>>> set_pipe_non_blocking. Are you saying that's not an issue? Is >>>>> set_pipe_non_blocking unnecessary? Is that the point of your modification to >>>>> raw_read? >>>> >>>> Yes. Instead of making windows read function itself non-blocking, >>>> it is possible to check if the pipe can be read before read using >>>> PeekNamedPipe(). If the pipe cannot be read right now, EAGAIN is >>>> returned. >>> >>> The problem is this: >>> >>> if (PeekNamedPipe()) >>> ReadFile(blocking); >>> >>> is not atomic. I. e., if PeekNamedPipe succeeds, nothing keeps another >>> thread from draining the pipe between the PeekNamedPipe and the ReadFile >>> call. And as soon as ReadFile runs, it hangs indefinitely and we can't >>> stop it via a signal. >> >> Hmm, you are right. Mutex guard seems to be necessary like pty code >> if we go this way. > > I have found that set_pipe_non_blocking() succeeds for both read and > write pipes if the write pipe is created by CreateNamedPipe() and the > read pipe is created by CreateFile() contrary to the current create() > code. Therefore, not only nt_create() but also PeekNamedPipe() become > unnecessary. > > Please see the revised patch attached. I haven't had a chance to test this myself yet, but occurs to me that we might have a different problem after this patch: Does the write handle that we get from CreateNamedPipe() have FILE_READ_ATTRIBUTES access? Ken ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? 2021-08-29 21:04 ` Ken Brown @ 2021-08-30 0:13 ` Takashi Yano 2021-08-30 0:22 ` Takashi Yano 0 siblings, 1 reply; 86+ messages in thread From: Takashi Yano @ 2021-08-30 0:13 UTC (permalink / raw) To: cygwin On Sun, 29 Aug 2021 17:04:56 -0400 Ken Brown wrote: > On 8/29/2021 5:07 AM, Takashi Yano via Cygwin wrote: > > On Sat, 28 Aug 2021 18:41:02 +0900 > > Takashi Yano wrote: > >> On Sat, 28 Aug 2021 10:43:27 +0200 > >> Corinna Vinschen wrote: > >>> On Aug 28 02:21, Takashi Yano via Cygwin wrote: > >>>> On Fri, 27 Aug 2021 12:00:50 -0400 > >>>> Ken Brown wrote: > >>>>> Two years ago I thought I needed nt_create to avoid problems when calling > >>>>> set_pipe_non_blocking. Are you saying that's not an issue? Is > >>>>> set_pipe_non_blocking unnecessary? Is that the point of your modification to > >>>>> raw_read? > >>>> > >>>> Yes. Instead of making windows read function itself non-blocking, > >>>> it is possible to check if the pipe can be read before read using > >>>> PeekNamedPipe(). If the pipe cannot be read right now, EAGAIN is > >>>> returned. > >>> > >>> The problem is this: > >>> > >>> if (PeekNamedPipe()) > >>> ReadFile(blocking); > >>> > >>> is not atomic. I. e., if PeekNamedPipe succeeds, nothing keeps another > >>> thread from draining the pipe between the PeekNamedPipe and the ReadFile > >>> call. And as soon as ReadFile runs, it hangs indefinitely and we can't > >>> stop it via a signal. > >> > >> Hmm, you are right. Mutex guard seems to be necessary like pty code > >> if we go this way. > > > > I have found that set_pipe_non_blocking() succeeds for both read and > > write pipes if the write pipe is created by CreateNamedPipe() and the > > read pipe is created by CreateFile() contrary to the current create() > > code. Therefore, not only nt_create() but also PeekNamedPipe() become > > unnecessary. > > > > Please see the revised patch attached. > > I haven't had a chance to test this myself yet, but occurs to me that we might > have a different problem after this patch: Does the write handle that we get > from CreateNamedPipe() have FILE_READ_ATTRIBUTES access? I have just checked this, and the answer is "No". Due to this problem, NtQueryInformationFile() call in select() fails on the write pipe. It seems that we need more consideration... -- Takashi Yano <takashi.yano@nifty.ne.jp> ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? 2021-08-30 0:13 ` Takashi Yano @ 2021-08-30 0:22 ` Takashi Yano 2021-08-30 2:15 ` Ken Brown 0 siblings, 1 reply; 86+ messages in thread From: Takashi Yano @ 2021-08-30 0:22 UTC (permalink / raw) To: cygwin On Mon, 30 Aug 2021 09:13:14 +0900 Takashi Yano wrote: > On Sun, 29 Aug 2021 17:04:56 -0400 > Ken Brown wrote: > > On 8/29/2021 5:07 AM, Takashi Yano via Cygwin wrote: > > > On Sat, 28 Aug 2021 18:41:02 +0900 > > > Takashi Yano wrote: > > >> On Sat, 28 Aug 2021 10:43:27 +0200 > > >> Corinna Vinschen wrote: > > >>> On Aug 28 02:21, Takashi Yano via Cygwin wrote: > > >>>> On Fri, 27 Aug 2021 12:00:50 -0400 > > >>>> Ken Brown wrote: > > >>>>> Two years ago I thought I needed nt_create to avoid problems when calling > > >>>>> set_pipe_non_blocking. Are you saying that's not an issue? Is > > >>>>> set_pipe_non_blocking unnecessary? Is that the point of your modification to > > >>>>> raw_read? > > >>>> > > >>>> Yes. Instead of making windows read function itself non-blocking, > > >>>> it is possible to check if the pipe can be read before read using > > >>>> PeekNamedPipe(). If the pipe cannot be read right now, EAGAIN is > > >>>> returned. > > >>> > > >>> The problem is this: > > >>> > > >>> if (PeekNamedPipe()) > > >>> ReadFile(blocking); > > >>> > > >>> is not atomic. I. e., if PeekNamedPipe succeeds, nothing keeps another > > >>> thread from draining the pipe between the PeekNamedPipe and the ReadFile > > >>> call. And as soon as ReadFile runs, it hangs indefinitely and we can't > > >>> stop it via a signal. > > >> > > >> Hmm, you are right. Mutex guard seems to be necessary like pty code > > >> if we go this way. > > > > > > I have found that set_pipe_non_blocking() succeeds for both read and > > > write pipes if the write pipe is created by CreateNamedPipe() and the > > > read pipe is created by CreateFile() contrary to the current create() > > > code. Therefore, not only nt_create() but also PeekNamedPipe() become > > > unnecessary. > > > > > > Please see the revised patch attached. > > > > I haven't had a chance to test this myself yet, but occurs to me that we might > > have a different problem after this patch: Does the write handle that we get > > from CreateNamedPipe() have FILE_READ_ATTRIBUTES access? > > I have just checked this, and the answer is "No". Due to this problem, > NtQueryInformationFile() call in select() fails on the write pipe. > > It seems that we need more consideration... We have two easy options: 1) Configure the pipe with PIPE_ACCESS_DUPLEX. 2) Use nt_create() again and forget C# program issue. Even without this problem, select() for writing pipe has a bug and does not wrok as expected. The following patch seems to be needed. diff --git a/winsup/cygwin/select.cc b/winsup/cygwin/select.cc index 83e1c00e0..ac2fd227e 100644 --- a/winsup/cygwin/select.cc +++ b/winsup/cygwin/select.cc @@ -612,7 +612,6 @@ pipe_data_available (int fd, fhandler_base *fh, HANDLE h, bool writing) that. This means that a pipe could still block since you could be trying to write more to the pipe than is available in the buffer but that is the hazard of select(). */ - fpli.WriteQuotaAvailable = fpli.OutboundQuota - fpli.ReadDataAvailable; if (fpli.WriteQuotaAvailable > 0) { paranoid_printf ("fd %d, %s, write: size %u, avail %u", fd, -- Takashi Yano <takashi.yano@nifty.ne.jp> ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? 2021-08-30 0:22 ` Takashi Yano @ 2021-08-30 2:15 ` Ken Brown 2021-08-30 8:02 ` Takashi Yano 0 siblings, 1 reply; 86+ messages in thread From: Ken Brown @ 2021-08-30 2:15 UTC (permalink / raw) To: cygwin On 8/29/2021 8:22 PM, Takashi Yano via Cygwin wrote: > On Mon, 30 Aug 2021 09:13:14 +0900 > Takashi Yano wrote: >> On Sun, 29 Aug 2021 17:04:56 -0400 >> Ken Brown wrote: >>> On 8/29/2021 5:07 AM, Takashi Yano via Cygwin wrote: >>>> On Sat, 28 Aug 2021 18:41:02 +0900 >>>> Takashi Yano wrote: >>>>> On Sat, 28 Aug 2021 10:43:27 +0200 >>>>> Corinna Vinschen wrote: >>>>>> On Aug 28 02:21, Takashi Yano via Cygwin wrote: >>>>>>> On Fri, 27 Aug 2021 12:00:50 -0400 >>>>>>> Ken Brown wrote: >>>>>>>> Two years ago I thought I needed nt_create to avoid problems when calling >>>>>>>> set_pipe_non_blocking. Are you saying that's not an issue? Is >>>>>>>> set_pipe_non_blocking unnecessary? Is that the point of your modification to >>>>>>>> raw_read? >>>>>>> >>>>>>> Yes. Instead of making windows read function itself non-blocking, >>>>>>> it is possible to check if the pipe can be read before read using >>>>>>> PeekNamedPipe(). If the pipe cannot be read right now, EAGAIN is >>>>>>> returned. >>>>>> >>>>>> The problem is this: >>>>>> >>>>>> if (PeekNamedPipe()) >>>>>> ReadFile(blocking); >>>>>> >>>>>> is not atomic. I. e., if PeekNamedPipe succeeds, nothing keeps another >>>>>> thread from draining the pipe between the PeekNamedPipe and the ReadFile >>>>>> call. And as soon as ReadFile runs, it hangs indefinitely and we can't >>>>>> stop it via a signal. >>>>> >>>>> Hmm, you are right. Mutex guard seems to be necessary like pty code >>>>> if we go this way. >>>> >>>> I have found that set_pipe_non_blocking() succeeds for both read and >>>> write pipes if the write pipe is created by CreateNamedPipe() and the >>>> read pipe is created by CreateFile() contrary to the current create() >>>> code. Therefore, not only nt_create() but also PeekNamedPipe() become >>>> unnecessary. >>>> >>>> Please see the revised patch attached. >>> >>> I haven't had a chance to test this myself yet, but occurs to me that we might >>> have a different problem after this patch: Does the write handle that we get >>> from CreateNamedPipe() have FILE_READ_ATTRIBUTES access? >> >> I have just checked this, and the answer is "No". Due to this problem, >> NtQueryInformationFile() call in select() fails on the write pipe. >> >> It seems that we need more consideration... > > We have two easy options: > 1) Configure the pipe with PIPE_ACCESS_DUPLEX. > 2) Use nt_create() again and forget C# program issue. I vote for 2), but let's see what Corinna thinks. > Even without this problem, select() for writing pipe has a bug > and does not wrok as expected. The following patch seems to be > needed. > > diff --git a/winsup/cygwin/select.cc b/winsup/cygwin/select.cc > index 83e1c00e0..ac2fd227e 100644 > --- a/winsup/cygwin/select.cc > +++ b/winsup/cygwin/select.cc > @@ -612,7 +612,6 @@ pipe_data_available (int fd, fhandler_base *fh, HANDLE h, bool writing) > that. This means that a pipe could still block since you could > be trying to write more to the pipe than is available in the > buffer but that is the hazard of select(). */ > - fpli.WriteQuotaAvailable = fpli.OutboundQuota - fpli.ReadDataAvailable; > if (fpli.WriteQuotaAvailable > 0) > { > paranoid_printf ("fd %d, %s, write: size %u, avail %u", fd, > I agree. Ken ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? 2021-08-30 2:15 ` Ken Brown @ 2021-08-30 8:02 ` Takashi Yano 0 siblings, 0 replies; 86+ messages in thread From: Takashi Yano @ 2021-08-30 8:02 UTC (permalink / raw) To: cygwin On Sun, 29 Aug 2021 22:15:29 -0400 Ken Brown wrote: > On 8/29/2021 8:22 PM, Takashi Yano via Cygwin wrote: > > On Mon, 30 Aug 2021 09:13:14 +0900 > > Takashi Yano wrote: > >> On Sun, 29 Aug 2021 17:04:56 -0400 > >> Ken Brown wrote: > >>> On 8/29/2021 5:07 AM, Takashi Yano via Cygwin wrote: > >>>> On Sat, 28 Aug 2021 18:41:02 +0900 > >>>> Takashi Yano wrote: > >>>>> On Sat, 28 Aug 2021 10:43:27 +0200 > >>>>> Corinna Vinschen wrote: > >>>>>> On Aug 28 02:21, Takashi Yano via Cygwin wrote: > >>>>>>> On Fri, 27 Aug 2021 12:00:50 -0400 > >>>>>>> Ken Brown wrote: > >>>>>>>> Two years ago I thought I needed nt_create to avoid problems when calling > >>>>>>>> set_pipe_non_blocking. Are you saying that's not an issue? Is > >>>>>>>> set_pipe_non_blocking unnecessary? Is that the point of your modification to > >>>>>>>> raw_read? > >>>>>>> > >>>>>>> Yes. Instead of making windows read function itself non-blocking, > >>>>>>> it is possible to check if the pipe can be read before read using > >>>>>>> PeekNamedPipe(). If the pipe cannot be read right now, EAGAIN is > >>>>>>> returned. > >>>>>> > >>>>>> The problem is this: > >>>>>> > >>>>>> if (PeekNamedPipe()) > >>>>>> ReadFile(blocking); > >>>>>> > >>>>>> is not atomic. I. e., if PeekNamedPipe succeeds, nothing keeps another > >>>>>> thread from draining the pipe between the PeekNamedPipe and the ReadFile > >>>>>> call. And as soon as ReadFile runs, it hangs indefinitely and we can't > >>>>>> stop it via a signal. > >>>>> > >>>>> Hmm, you are right. Mutex guard seems to be necessary like pty code > >>>>> if we go this way. > >>>> > >>>> I have found that set_pipe_non_blocking() succeeds for both read and > >>>> write pipes if the write pipe is created by CreateNamedPipe() and the > >>>> read pipe is created by CreateFile() contrary to the current create() > >>>> code. Therefore, not only nt_create() but also PeekNamedPipe() become > >>>> unnecessary. > >>>> > >>>> Please see the revised patch attached. > >>> > >>> I haven't had a chance to test this myself yet, but occurs to me that we might > >>> have a different problem after this patch: Does the write handle that we get > >>> from CreateNamedPipe() have FILE_READ_ATTRIBUTES access? > >> > >> I have just checked this, and the answer is "No". Due to this problem, > >> NtQueryInformationFile() call in select() fails on the write pipe. > >> > >> It seems that we need more consideration... > > > > We have two easy options: > > 1) Configure the pipe with PIPE_ACCESS_DUPLEX. > > 2) Use nt_create() again and forget C# program issue. > > I vote for 2), but let's see what Corinna thinks. BTW. what's wrong if just: static int nt_create (LPSECURITY_ATTRIBUTES sa_ptr, PHANDLE r, PHANDLE w, DWORD psize, int64_t *unique_id) { if (r && w) { static volatile ULONG pipe_unique_id; LONG id = InterlockedIncrement ((LONG *) &pipe_unique_id); if (unique_id) *unique_id = ((int64_t) id << 32 | GetCurrentProcessId ()); if (!CreatePipe (r, w, sa_ptr, psize)) { *r = *w = NULL; return GetLastError (); } } return 0; } ? In my environment, I cannot find any defects. - No performance degradation. - set_pipe_non_blocking() works for both read and write pipes. - NtQueryInformationFile() in select() works for both r/w pipes. - Piping C# program works. Is naming the pipe really necessary? -- Takashi Yano <takashi.yano@nifty.ne.jp> ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? 2021-08-28 8:43 ` Corinna Vinschen 2021-08-28 9:41 ` Takashi Yano @ 2021-08-28 15:17 ` Ken Brown 1 sibling, 0 replies; 86+ messages in thread From: Ken Brown @ 2021-08-28 15:17 UTC (permalink / raw) To: cygwin On 8/28/2021 4:43 AM, Corinna Vinschen via Cygwin wrote: > On Aug 28 02:21, Takashi Yano via Cygwin wrote: >> On Fri, 27 Aug 2021 12:00:50 -0400 >> Ken Brown wrote: >>> Two years ago I thought I needed nt_create to avoid problems when calling >>> set_pipe_non_blocking. Are you saying that's not an issue? Is >>> set_pipe_non_blocking unnecessary? Is that the point of your modification to >>> raw_read? >> >> Yes. Instead of making windows read function itself non-blocking, >> it is possible to check if the pipe can be read before read using >> PeekNamedPipe(). If the pipe cannot be read right now, EAGAIN is >> returned. > > The problem is this: > > if (PeekNamedPipe()) > ReadFile(blocking); > > is not atomic. I. e., if PeekNamedPipe succeeds, nothing keeps another > thread from draining the pipe between the PeekNamedPipe and the ReadFile > call. And as soon as ReadFile runs, it hangs indefinitely and we can't > stop it via a signal. > > Is a blocking ReadFile actually faster than a non-blocking read? Or > does it mainly depend on BYTE vs. MESSAGE mode? > > What if the pipe is created non-blocking and stays non-blocking all the > time and uses BYTE mode all the time? Just as sockets, it would always > only emulate blocking mode. Wouldn't that drop code size a lot and fix > most problems? It would certainly reduce the code size. I think it's worth a try, and we can see if any problems remain. Ken ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? 2021-08-24 19:49 cygrunsrv + sshd + rsync = 20 times too slow -- throttled? Chris Roehrig ` (2 preceding siblings ...) 2021-08-25 11:18 ` Takashi Yano @ 2021-09-16 22:00 ` Keith Christian 2021-09-16 22:48 ` Ken Brown 3 siblings, 1 reply; 86+ messages in thread From: Keith Christian @ 2021-09-16 22:00 UTC (permalink / raw) To: cygwin I've been following his thread with interest both here and on the Cygwin developers list. I, too rsync between Cygwin and Linux machines. Lots of good debate showing the genius of the folks that support the cygwin infrastructure. Looks like the lively discussion on both lists has stopped. What is the final consensus? Should we look for a new updates to the cygwin dll, cygrunsrv, ssh, rsync, etc? ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? 2021-09-16 22:00 ` Keith Christian @ 2021-09-16 22:48 ` Ken Brown 2021-09-16 22:58 ` Keith Christian 0 siblings, 1 reply; 86+ messages in thread From: Ken Brown @ 2021-09-16 22:48 UTC (permalink / raw) To: cygwin On 9/16/2021 6:00 PM, Keith Christian via Cygwin wrote: > I've been following his thread with interest both here and on the Cygwin > developers list. I, too rsync between Cygwin and Linux machines. > > Lots of good debate showing the genius of the folks that support the cygwin > infrastructure. > > Looks like the lively discussion on both lists has stopped. What is the > final consensus? > > Should we look for a new updates to the cygwin dll, cygrunsrv, ssh, rsync, > etc? All that's needed is an update to the cygwin package. There should be a test release coming very soon, possibly tomorrow if nothing unexpected shows up. Ken ^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? 2021-09-16 22:48 ` Ken Brown @ 2021-09-16 22:58 ` Keith Christian 0 siblings, 0 replies; 86+ messages in thread From: Keith Christian @ 2021-09-16 22:58 UTC (permalink / raw) To: cygwin Ken, Thank you for the reply, this is great news! Looking forward to speedier rsyncs with Cygwin. ------------ Keith On Thu, Sep 16, 2021, 16:49 Ken Brown via Cygwin <cygwin@cygwin.com> wrote: > On 9/16/2021 6:00 PM, Keith Christian via Cygwin wrote: > > I've been following his thread with interest both here and on the Cygwin > > developers list. I, too rsync between Cygwin and Linux machines. > > > > Lots of good debate showing the genius of the folks that support the > cygwin > > infrastructure. > > > > Looks like the lively discussion on both lists has stopped. What is the > > final consensus? > > > > Should we look for a new updates to the cygwin dll, cygrunsrv, ssh, > rsync, > > etc? > > All that's needed is an update to the cygwin package. There should be a > test > release coming very soon, possibly tomorrow if nothing unexpected shows up. > > Ken > > -- > Problem reports: https://cygwin.com/problems.html > FAQ: https://cygwin.com/faq/ > Documentation: https://cygwin.com/docs.html > Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple > ^ permalink raw reply [flat|nested] 86+ messages in thread
end of thread, other threads:[~2021-09-16 22:59 UTC | newest] Thread overview: 86+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2021-08-24 19:49 cygrunsrv + sshd + rsync = 20 times too slow -- throttled? Chris Roehrig 2021-08-24 23:43 ` Mark Geisert 2021-08-25 0:35 ` Chris Roehrig 2021-08-25 2:12 ` Chris Roehrig 2021-08-25 2:02 ` NightStrike 2021-08-25 3:20 ` Mark Geisert 2021-08-25 17:24 ` Chris Roehrig 2021-08-25 11:18 ` Takashi Yano 2021-08-25 17:31 ` Chris Roehrig 2021-08-25 17:52 ` Ken Brown 2021-08-25 18:18 ` Chris Roehrig 2021-08-25 18:51 ` Chris Roehrig 2021-08-26 15:44 ` Ken Brown 2021-08-25 20:33 ` Mario Emmenlauer 2021-08-26 15:47 ` Ken Brown 2021-08-25 21:29 ` Takashi Yano 2021-08-26 15:56 ` Ken Brown 2021-08-26 22:18 ` Ken Brown 2021-08-27 11:24 ` Takashi Yano 2021-08-27 16:00 ` Ken Brown 2021-08-27 17:21 ` Takashi Yano 2021-08-28 2:00 ` Takashi Yano 2021-08-28 3:03 ` Takashi Yano 2021-08-28 8:43 ` Corinna Vinschen 2021-08-28 9:41 ` Takashi Yano 2021-08-28 11:58 ` Corinna Vinschen 2021-08-28 15:43 ` Takashi Yano 2021-08-28 20:55 ` Ken Brown 2021-08-29 8:41 ` Takashi Yano 2021-08-29 22:42 ` Ken Brown 2021-08-29 10:27 ` Corinna Vinschen 2021-08-29 9:07 ` Takashi Yano 2021-08-29 15:57 ` Ken Brown 2021-08-29 19:24 ` Chris Roehrig 2021-08-29 22:24 ` Ken Brown 2021-08-30 23:58 ` Chris Roehrig 2021-08-31 19:05 ` Ken Brown 2021-08-31 19:53 ` Chris Roehrig 2021-08-31 20:23 ` Chris Roehrig 2021-08-31 21:29 ` Brian Inglis 2021-09-01 21:11 ` Chris Roehrig 2021-09-02 15:25 ` Ken Brown 2021-09-02 19:03 ` Chris Roehrig 2021-09-03 17:26 ` Ken Brown 2021-09-03 19:55 ` Corinna Vinschen 2021-09-03 20:59 ` Chris Roehrig 2021-09-04 8:32 ` Achim Gratz 2021-09-04 16:45 ` Brian Inglis 2021-09-05 8:18 ` Achim Gratz 2021-09-05 15:11 ` Brian Inglis 2021-09-06 10:42 ` Achim Gratz 2021-09-04 22:37 ` mmap failure [was: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?] Ken Brown 2021-09-04 22:54 ` Ken Brown 2021-09-04 22:58 ` Ken Brown 2021-09-05 0:04 ` Ken Brown 2021-09-05 13:24 ` Ken Brown 2021-09-06 15:32 ` Corinna Vinschen 2021-09-06 17:12 ` Ken Brown 2021-09-06 17:38 ` Ken Brown 2021-09-06 17:43 ` Eliot Moss 2021-09-06 17:59 ` Corinna Vinschen 2021-09-06 18:07 ` Corinna Vinschen 2021-09-06 18:40 ` Ken Brown 2021-09-06 20:52 ` Peter Dons Tychsen 2021-09-06 20:54 ` Peter Dons Tychsen 2021-09-06 21:24 ` Ken Brown 2021-09-06 21:31 ` Ken Brown 2021-09-07 3:34 ` Brian Inglis 2021-09-07 16:28 ` Ken Brown 2021-09-07 21:52 ` Lavrentiev, Anton (NIH/NLM/NCBI) [C] 2021-09-07 22:44 ` Ken Brown 2021-09-08 6:14 ` Sam Edge 2021-09-08 8:18 ` Corinna Vinschen 2021-09-07 21:41 ` Peter Dons Tychsen 2021-09-04 23:18 ` Brian Inglis 2021-08-29 19:37 ` cygrunsrv + sshd + rsync = 20 times too slow -- throttled? Takashi Yano 2021-08-29 21:09 ` Ken Brown 2021-08-29 21:04 ` Ken Brown 2021-08-30 0:13 ` Takashi Yano 2021-08-30 0:22 ` Takashi Yano 2021-08-30 2:15 ` Ken Brown 2021-08-30 8:02 ` Takashi Yano 2021-08-28 15:17 ` Ken Brown 2021-09-16 22:00 ` Keith Christian 2021-09-16 22:48 ` Ken Brown 2021-09-16 22:58 ` Keith Christian
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).