From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mout.kundenserver.de (mout.kundenserver.de [217.72.192.73]) by sourceware.org (Postfix) with ESMTPS id EC25F38515F4 for ; Thu, 2 Sep 2021 19:00:32 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org EC25F38515F4 Authentication-Results: sourceware.org; dmarc=fail (p=none dis=none) header.from=cygwin.com Authentication-Results: sourceware.org; spf=fail smtp.mailfrom=cygwin.com Received: from calimero.vinschen.de ([24.134.7.25]) by mrelayeu.kundenserver.de (mreue107 [212.227.15.183]) with ESMTPSA (Nemesis) id 1N6szR-1n3FVQ1y8d-018Hb3 for ; Thu, 02 Sep 2021 21:00:31 +0200 Received: by calimero.vinschen.de (Postfix, from userid 500) id 7C95EA80D84; Thu, 2 Sep 2021 21:00:30 +0200 (CEST) Date: Thu, 2 Sep 2021 21:00:30 +0200 From: Corinna Vinschen To: cygwin-developers@cygwin.com Subject: Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? Message-ID: Reply-To: cygwin-developers@cygwin.com Mail-Followup-To: cygwin-developers@cygwin.com References: <20210901080220.ee4a5bfbea62cc1ae0a9598e@nifty.ne.jp> <20210901091652.6bf3cccbcaed4a22f6ffa6b0@nifty.ne.jp> <20210901172339.1039604b7067e0492534a20f@nifty.ne.jp> <24138e20-aa97-cfea-bf48-198fc67755ea@cornell.edu> <9ba687eb-f4a0-18f8-b10b-76e7e51e123e@cornell.edu> <152bfc0c-2f72-c684-6fc5-aa7c36c136b8@cornell.edu> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <152bfc0c-2f72-c684-6fc5-aa7c36c136b8@cornell.edu> X-Provags-ID: V03:K1:AnphkJPqOORt3anCBDvBJc98630BytEYTO+52H+BU96sV1QpQou B9tg9sHbYcTz2xSxPBPXOFIng0GkLtwkJtNtLXPY7iu6xvHxtpIi7f4qTfhZaY/3Y//vVfb /A6oH4c52+9P4dHw0uWcBzXs1r2eZip6P9piNECbAl8kFW+mt+doHajNdZbJtXPmmJrvcx+ a/JN6Ofqb1B2eeu7pBJkQ== X-UI-Out-Filterresults: notjunk:1;V03:K0:nO4MCx0keFI=:rNvIi46rUCvs5CFMtfoS9u Ou6IjSE8rCDBKsZoalgPs4cTgGFz+8o6JTiqqTTU0tcrnmCPrTIwBExMx8QnwemVWd26kP8nX 4oSnZZ2IqIgMTA8wNP/rleQMGn1homSDxAkGL8D3yI9XgRacXalwSZ8fqGyuENV5rM5pFEW9j ebd+dQUhiIymbywC1GPImVpFYs+4XUZ69G2eU4bZDm5UbsuHqb5JZ6fBBSrw6JMLWNZzQjQTc GnbgwbyXbygVG4zZGNn6oWbpO8aSYXh5Z6MgZ3RGggJRSkoBqNYEAYBYZJxAuz74Ou29znxwb zTAqC4/sYpxBBRVjK7gzNNrEp9mwgt2KcJuM88cj9AaZS1dnOpxHOxtU5HiLwOTTzEoEQmnfI uUGOIZN5ALO/ZPgkGLTBpOFbT/bVtzOvE25BJGCfIajCRHNUkUGKhaCBHtCZatMgdTVLCRRVt N5NYemOH77RZ1us5EF/M4DV9LSLM5gAcLQnIX4Tkyv7ekhU1x2wMnD0gczfjaY+wfrybxvlAM G8Vedf0HC3RtbRlMJ3m5Wds5NuCokp2HILRpH4i4iKt8mk+fHgeuyHdEzAa+Ak78VbmNfSPXY 5ajJ80razzfWcylZXkqtNktpGPuCCokVDFYKzzHlVrKVjOHVsK8kvc8wJNLkGgK7hj5EJs7I5 QSRN0lgQIQyHa8d7zyFvHoRI1Myd2SNq49G899MuXU7yRv0InKvKPCzbGfjxi4L+TeetjoqRk U0SD1dFaE0qJYa+BDEhiClfWzhrMlmcGRzO11XTAE00aM/qmet8igGsvn9E= X-Spam-Status: No, score=-99.7 required=5.0 tests=BAYES_00, GOOD_FROM_CORINNA_CYGWIN, JMQ_SPF_NEUTRAL, KAM_DMARC_NONE, KAM_DMARC_STATUS, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_NONE, SPF_NEUTRAL, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: cygwin-developers@cygwin.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Cygwin core component developers mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Sep 2021 19:00:34 -0000 On Sep 2 09:01, Ken Brown wrote: > On 9/2/2021 4:17 AM, Corinna Vinschen wrote: > > What if the readers never request more than, say, 50 or even 25% of the > > available buffer space? Our buffer is 64K and there's no guarantee that > > any read > PIPE_BUF (== 4K) is atomic anyway. This can work without > > having to check the other side of the pipe. Something like this, > > ignoring border cases: > > > > pipe::create() > > { > > [...] > > mutex = CreateMutex(); > > } > > > > pipe::raw_read(char *buf, size_t num_requested) > > { > > if (blocking) > > { > > WFSO(mutex); > > NtQueryInformationFile(FilePipeLocalInformation); > > if (!fpli.ReadDataAvailable > > && num_requested > fpli.InboundQuota / 4) > > num_requested = fpli.InboundQuota / 4; > > NtReadFile(pipe, buf, num_requested); > > ReleaseMutex(mutex); > > } > > } > > > > It's not entirely foolproof, but it should fix 99% of the cases. > > I like it! > > Do you think there's anything we can or should do to avoid a deadlock in the > rare cases where this fails? The only thing I can think of immediately is > to always impose a timeout if select is called with infinite timeout on the > write side of a pipe, after which we report that the pipe is write ready. > After all, we've lived since 2008 with a bug that caused select to *always* > report write ready. Indeed. Hmm. What timeout are you thinking of? Seconds? Minutes? > Alternatively, we could just wait and see if there's an actual use case in > which someone encounters a deadlock. Or that. Fixing up select isn't too hard in that case, I guess. Corinna