From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mout.kundenserver.de (mout.kundenserver.de [212.227.126.134]) by sourceware.org (Postfix) with ESMTPS id 3F51F385800E for ; Mon, 6 Sep 2021 08:13:14 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 3F51F385800E Authentication-Results: sourceware.org; dmarc=fail (p=none dis=none) header.from=cygwin.com Authentication-Results: sourceware.org; spf=fail smtp.mailfrom=cygwin.com Received: from calimero.vinschen.de ([24.134.7.25]) by mrelayeu.kundenserver.de (mreue010 [212.227.15.167]) with ESMTPSA (Nemesis) id 1MNtCi-1mcUm72Nw0-00OJs9 for ; Mon, 06 Sep 2021 10:13:11 +0200 Received: by calimero.vinschen.de (Postfix, from userid 500) id 8E8B2A80DAA; Mon, 6 Sep 2021 10:13:10 +0200 (CEST) Date: Mon, 6 Sep 2021 10:13:10 +0200 From: Corinna Vinschen To: cygwin-developers@cygwin.com Subject: Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? Message-ID: Reply-To: cygwin-developers@cygwin.com Mail-Followup-To: cygwin-developers@cygwin.com References: <20210903212205.acc2fc68cc4ffce9c1db3dd9@nifty.ne.jp> <20210904210258.342eb795ac53f1d5352ea512@nifty.ne.jp> <20210904213713.8760e7ba3a4d68fbb78d677e@nifty.ne.jp> <51cb0cef-c3fd-1320-c2dd-a868bf1ffaae@cornell.edu> <20210905081523.0db04d9402abf87635066eb7@nifty.ne.jp> <20210905224059.cfdc8f23d3eeaa1ea16ecf2e@nifty.ne.jp> <20210905225037.c625ee0bcd479181848763f8@nifty.ne.jp> <20210906050950.56b397be7c5eb3da848691e9@nifty.ne.jp> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <20210906050950.56b397be7c5eb3da848691e9@nifty.ne.jp> X-Provags-ID: V03:K1:7BWHkbmD4EN2MCkFiXlySR3Y9x9tOsxbi1kIDDjRDWTyZtp0tjW bf3NBr8YvuN+Q/O/1nWCTzVONyATxO1P4rCwNNoGR8I+HlNf7xZ/l/Jl1X/s/4WUwFUgHtt SruGMK/fiKYt3ZgE+6vJvFrWEL5BhrawHKamHoOhZCEQgC3gO4FBFVGDcHpOCrO9iCwj5KN mAyNgk085OAfnDkmkLgnw== X-UI-Out-Filterresults: notjunk:1;V03:K0:I+3sWbIWxpE=:yyVV4l/Hqpd/4RyzeUbDm6 udzACuQYVbQaHyVT0hwEE7uIdlqeySI9pTdNWh5e0y2SvPAxDnzJXNmae2Or0DxyuQ7UU79QR CFgMg6g4x64gA+PH7bVTIb4OKoea1aFjZD/mf+pYG6T38gNbilLo3QXiEF/U678gHb2NZsctb Al583kBLZ4T9q5uHDy2K58f4o/SXZ/n6Y/KHAZkSH11OiHO+N99SrgoVxiq2zIvotrLgzJG5+ hwqpr9KPYv8bXg7FgKe+Sqnc+tHKFuKzW/fxafomanW4GrBgjvlvTpaCa8HvTuzwMspjQgMYJ ffPZnrxM9+9uWACPrhyDLsKbazeT1hCcFOK2HZrdfewzYsRTAgKYjvMqLpMlMqvz/x+v0zHQe crGXCkPyggKbueJjH2FcEuDedn4/b/lBlr8n/wTcTQ5qEMWJc0qFy/AfCmER2AQEbAODFFG+M ttjLUV34AK9Q1BfbRLWXhv4v00TFCzkk0nFQ3yCtbWkKB/2tBBma6GZEQGO1/F4hD/7enRQcu UejPBCnLyxLUPVWaNdEp1iholZntEzmmh3v/QKJyzdfn1ESCQ4AXdwggb0ZikufmEp16AN12O p4Ji92gEaP6qrnJhT54WGgHrfj9HoYCkGvKUGEJ9tRR/jATXAIhaZSeElkQuj/pON6MmBJe28 YWbUJvuXkFI4b/OjFCKgeIfFeu6X+Psng5GLxmdiRHlK6FPiKjG/Wox06levZ7Bd6pN5nkrMr XqjOzCjAFBDzi/1ISA03mkxNTDMZ+mPJy6fPwTtecUoeFfYWFpetPrDiJNM= X-Spam-Status: No, score=-100.0 required=5.0 tests=BAYES_00, GOOD_FROM_CORINNA_CYGWIN, KAM_DMARC_NONE, KAM_DMARC_STATUS, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_NEUTRAL, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: cygwin-developers@cygwin.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Cygwin core component developers mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Sep 2021 08:13:16 -0000 On Sep 6 05:09, Takashi Yano wrote: > On Sun, 5 Sep 2021 14:47:26 -0400 > Ken Brown wrote: > > On 9/5/2021 9:50 AM, Takashi Yano wrote: > > > On Sun, 5 Sep 2021 22:40:59 +0900 > > > Takashi Yano wrote: > > >> On Sun, 5 Sep 2021 08:15:23 +0900 > > >> Takashi Yano wrote: > > >>> On Sat, 4 Sep 2021 10:04:12 -0400 > > >>> Ken Brown wrote: > > >>>> On 9/4/2021 8:37 AM, Takashi Yano wrote: > > >>>>> On Sat, 4 Sep 2021 21:02:58 +0900 > > >>>>> Takashi Yano wrote: > > >>>>>> On Fri, 3 Sep 2021 09:27:37 -0400 > > >>>>>> Ken Brown wrote: > > >>>>>>> On 9/3/2021 8:22 AM, Takashi Yano wrote: > > >>>>>>>> POSIX says: > > >>>>>>>> The value returned may be less than nbyte if the number of bytes left > > >>>>>>>> in the file is less than nbyte, if the read() request was interrupted > > >>>>>>>> by a signal, or if the file is a pipe or FIFO or special file and has > > >>>>>>>> ~~~ > > >>>>>>>> fewer than nbyte bytes immediately available for reading. > > >>>>>>>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > >>>>>>>> > > >>>>>>>> https://pubs.opengroup.org/onlinepubs/009604599/functions/read.html > > >>>>>>>> > > >>>>>>>> If it is turned over, read() should read all data immediately available, > > >>>>>>>> I think. > > >>>>>>> [...] > > >>>>>> Corinna Vinschen wrote: > > >>>>>>> We can't keep the mutex while waiting on the pending read, and there > > >>>>>>> could be more than one pending read running at the time. if so, > > >>>>>>> chances are extremly high, that the data written to the buffer gets > > >>>>>>> split like this: > > >>>>>>> [...] > > >>>>> Takashi Yano wrote: > > >>>>>> Why can't we keep the mutex while waiting on the pending read? > > >>>>>> If we can keep the mutex, the issue above mentioned does not > > >>>>>> happen, right? > [...] > @@ -239,43 +237,51 @@ fhandler_pipe::raw_read (void *ptr, size_t& len) > return; > } > > - do > + DWORD timeout = is_nonblocking () ? 0 : INFINITE; > + if (WAIT_TIMEOUT == WaitForSingleObject (read_mtx, timeout)) My code was originally not supposed to serialise the readers. The mutex block should be short lived and only create an atomic block for the two calls NtQueryInformationFile and NtReadFile. If you have multiple readers, all but one of them will hang in this WFSO. They will block here without a chance to kill or Ctrl-C them and thread cancellation won't work. To fix that you have to use cygwait and handle signals and thread cancellation the same way as in the below code following the NtReadFile. > /* If the pipe is empty, don't request more bytes than half the > - pipe buffer size. Every pending read lowers WriteQuotaAvailable > - on the write side and thus affects select's ability to return > - more or less reliable info whether a write succeeds or not. > - > - Let the size of the request depend on the number of readers > - at the time. */ > + pipe buffer size. Pending read lowers WriteQuotaAvailable on > + the write side and thus affects select's ability to return > + more or less reliable info whether a write succeeds or not. */ > + ULONG chunk = max_atomic_write / 2; > status = NtQueryInformationFile (get_handle (), &io, > &fpli, sizeof (fpli), > FilePipeLocalInformation); > - if (NT_SUCCESS (status) && fpli.ReadDataAvailable == 0) If the readers are serialized anyway, why fetch only half the remaining buffer size? In that case fetching fpli.InboundQuota - 1 is as good as fetching just the half of it, isn't it? Corinna