Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?

[Ken Brown <kbrown@cornell.edu>]

On 9/1/2021 9:52 AM, Corinna Vinschen wrote:
> On Sep  1 08:56, Ken Brown wrote:
>> On 9/1/2021 4:46 AM, Corinna Vinschen wrote:
>>> On Sep  1 17:23, Takashi Yano wrote:
>>>> On Wed, 1 Sep 2021 10:07:48 +0200
>>>> Corinna Vinschen wrote:
>>>>> On Sep  1 09:16, Takashi Yano wrote:
>>>>>> On Wed, 1 Sep 2021 08:02:20 +0900
>>>>>> Takashi Yano wrote:
>>>>>>> On Tue, 31 Aug 2021 17:50:14 +0200
>>>>>>> Corinna Vinschen wrote:
>>>>>>>> So for the time being I suggest the below patch on top of topic/pipe.
>>>>>>>> It contains everything we discussed so far.
>>>>>>>
>>>>>>> One more thing. 'git log' cannot stop normally with 'q' with your patch.
>>>>>>
>>>>>> The same happes with 'yes |less'.
>>>>>>
>>>>>> The cause is that write side cannot detect closing read side because
>>>>>> query_hdl (read handle) is still opened.
>>>>>
>>>>> Oh
>>>>>
>>>>> my
>>>>>
>>>>> god.
>>>>>
>>>>>
>>>>> That kills the entire idea of keeping the read handle :(
>>>>
>>>> One idea is:
>>>>
>>>> Count read handle and write handle opned using NtQueryObject().
>>>> If the numbers of opened handle are equal each other, only
>>>> the write side (pair of write handle and query_hdl) is alive.
>>>> In this case, write() returns error.
>>>> If read side is alive, number of read handles is greater than
>>>> number of write handles.
>>>
>>> Interesting idea.  But where do you do the count?  The event object
>>> will not get signalled, so WFMO will not return when performing a
>>> blocking write.
>>
>> What if we create an event that we signal every time a reader closes, and we
>> add that to the events that WFMO is waiting for?
>>
>> If this doesn't work for some reason, a different (but more complicated)
>> idea is to keep a count of the number of open readers in shared memory.
>> When this is 0, write returns an error.  I'm thinking of shared memory as in
>> topic/af_unix (which I copied in the fifo implementation), but maybe
>> something simpler would work since we only have a single variable to keep
>> track of.
> 
> Great idea that.  What we need would be some semaphore upside down.
> One that can be used to count items and which is signalled if it's
> down to zero.

Here's an idea (untested), based on 
https://stackoverflow.com/questions/6559854/is-there-something-opposite-to-semaphore:

We create an ordinary Windows semaphore and use it to count the readers: It 
starts at 0, we increment it by calling ReleaseSemaphore when a reader is opened 
(by fhandler_pipe::create, fork/exec, dup), and we decrement it by calling WFSO 
when a reader closes.  When we decrement it, we test whether it's been reduced 
to 0.  We do this by calling ReleaseSemaphore and using its lpPreviousCount 
argument.

We also create an event that we can use to make WFMO return during a blocking 
write.  We signal this event if a reader closes and we've discovered that there 
are no more readers.  In this case we cancel the pending write [*] and return an 
error.

I'm sure I've overlooked something, but does this seem feasible?

Ken

[*] I don't know offhand if Windows provides a way to cancel a pending write. 
If not, we could use query_hdl to drain the pipe.