From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mout.kundenserver.de (mout.kundenserver.de [212.227.126.131]) by sourceware.org (Postfix) with ESMTPS id D9C63385840F for ; Fri, 3 Sep 2021 11:41:47 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org D9C63385840F Authentication-Results: sourceware.org; dmarc=fail (p=none dis=none) header.from=cygwin.com Authentication-Results: sourceware.org; spf=fail smtp.mailfrom=cygwin.com Received: from calimero.vinschen.de ([24.134.7.25]) by mrelayeu.kundenserver.de (mreue009 [212.227.15.167]) with ESMTPSA (Nemesis) id 1MEmAV-1mAtHt2Ajz-00GJFZ for ; Fri, 03 Sep 2021 13:41:46 +0200 Received: by calimero.vinschen.de (Postfix, from userid 500) id 0644AA80F82; Fri, 3 Sep 2021 13:41:46 +0200 (CEST) Date: Fri, 3 Sep 2021 13:41:45 +0200 From: Corinna Vinschen To: cygwin-developers@cygwin.com Subject: Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? Message-ID: Reply-To: cygwin-developers@cygwin.com Mail-Followup-To: cygwin-developers@cygwin.com References: <24138e20-aa97-cfea-bf48-198fc67755ea@cornell.edu> <9ba687eb-f4a0-18f8-b10b-76e7e51e123e@cornell.edu> <152bfc0c-2f72-c684-6fc5-aa7c36c136b8@cornell.edu> <20210903190046.663c60fb11c936e344821383@nifty.ne.jp> <20210903191340.c28ae366e79ca14799bacc1f@nifty.ne.jp> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: X-Provags-ID: V03:K1:7eHuaguBU+zxPcqxJaQJnLSZAZZOuFoW6r4msQC6W2iNrE1O/WF urOUs0mQ3sh1trIH2zW/gJCBBsNOdsgePWNIctdtvZIwdGlhWS+7MzZ0WrCzPguVdPezByd Zg72xXcfWAbp7FGF+sy0Q9ktvQPrjoDIM0sqTLwF3ahG5l39MGfADX13wJKWaBk1/xU3egr jG7HDm2jbIRgBVlgCBQWA== X-UI-Out-Filterresults: notjunk:1;V03:K0:w6Ew1/IcXKU=:sgDXpwoyVlmWo7Y2KYyYYS YYvGkrWlBlCc0g5iEMLyMUlZvDdp7BBIs8zErRL26Qf7mWLei0ag9+qqScncHcQuFjI5rc3cW V3yzxJ7js8YwNrGf1arV3HkuhOXr6vgAY0CnJSJwtDhBDJU4ziO6CBwW/eEB5gmszSqp6Zn0q 17rt3zeciKFsJGNMhYUebRO0sE0lUVqTkXK87WnmY73OGIcjt2AwgII8tPIZBXM/+onLq4qNv juwprTZJDwj7PGzKto4H8Nbmt8q6IKVgMOdHENzJ1FfMvy99n4MEh2qNSNVm1u2qsdACM3we/ yzVRau2Co9FpWCqRScBPS1AX7NZKKaFbfSSQfq/9yrQn0wvEXaE2YEgUwKQPUvT6pOEsyMDgg AhhF1eHYvnnha+x1LrtdzUZnZRf0kG2Bdcne8msbwyojEZh0g3Xjkb/O22iu35C0kM9hSkpFN yRG50iO7hppzjakn+pbXxSdBdRQih+rW4GQP65/3NG7uMubjRbhVmh674Rx7zNmVdPofeNFC8 ONQLlYfsw83RKSyUa7WSu4Yjil0Ue5zNAcr1epeauGTQKv5stWKd4IlG8n1L/XceuIMIkq8Ee qo1PhGGRMc+2nYZlMu7J4/2KnIMFXXBAoIaLSe/0zn9ys8wqJdgQERZcpBuiOH/mXOpEnteCm OAp5VccPk7Lq65YAgd+EJCbcXrhmJyj0uzZH1KI1VYUYj1K0ix/kMXh9x7Hb/vTZQv4aFM/hn oNN/AsMCEqr9RONSaPP8AovFi9n6g7vlNrehlN9hEHJse2sIx5zRKckWCDQ= X-Spam-Status: No, score=-99.7 required=5.0 tests=BAYES_00, GOOD_FROM_CORINNA_CYGWIN, JMQ_SPF_NEUTRAL, KAM_DMARC_NONE, KAM_DMARC_STATUS, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_NONE, SPF_NEUTRAL, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: cygwin-developers@cygwin.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Cygwin core component developers mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Sep 2021 11:41:49 -0000 On Sep 3 13:31, Corinna Vinschen wrote: > On Sep 3 19:13, Takashi Yano wrote: > > On Fri, 3 Sep 2021 19:00:46 +0900 > > Takashi Yano wrote: > > > On Thu, 2 Sep 2021 21:35:21 +0200 > > > Corinna Vinschen wrote: > > > > On Sep 2 21:00, Corinna Vinschen wrote: > > > > It's getting too late again. I drop off for tonight, but I attached > > > > my POC code I have so far. It also adds the snippets from my previous > > > > patch which fixes stuff Takashi found during testing. It also fixes > > > > something which looks like a bug in raw_write: > > > > > > > > - ptr = ((char *) ptr) + chunk; > > > > + ptr = ((char *) ptr) + nbytes_now; > > > > > > > > Incrementing ptr by chunk bytes while only nbytes_now have been written > > > > looks incorrect. > > > > > > > > As for the reader, it makes the # of bytes to read dependent on the > > > > number of reader handles. I don't know if that's such a bright idea, > > > > but this can be changed easily. > > > > > > > > Anyway, this runs all my testcases successfully but they are anything > > > > but thorough. > > > > > > > > Patch relativ to topic/pipe attached. Would you both mind to take a > > > > scrutinizing look? > > > > > > Thanks. > > > > > > Your code seems that read() returns only the partial data even > > > if the pipe stil has more data. Is this by design? > > > > > > This happes in both blocking and non-blocking case. > > > > The patch attached seems to fix the issue. > > I'm sorry, but I don't see what your patch is supposed to do in the > first place. What I see is that it now calls NtQueryInformationFile > even in the non-blocking case, which is not supposed to happen. > > I'm a bit puzzled what the actual bug is. > > The code changing len is only called if there's no data in the pipe. > In that case we only request a partial buffer so as not to block > the writer on select. > > If there *is* data in the pipe, it will just go straight to the > NtReadFile code without changing len. > > Where's the mistake? Oh, wait. Do you mean, if we only request less than len bytes, but after NtReadFile there's still data in the buffer, we should try to deplete the buffer up to len bytes in a subsequent NtReadFile? I thought this is unnecessary, actually, because of POSIX: The standard developers considered adding atomicity requirements to a pipe or FIFO, but recognized that due to the nature of pipes and FIFOs there could be no guarantee of atomicity of reads of {PIPE_BUF} or any other size that would be an aid to applications portability. Corinna