From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mout.kundenserver.de (mout.kundenserver.de [212.227.126.187]) by sourceware.org (Postfix) with ESMTPS id E33323858417 for ; Mon, 30 Aug 2021 14:05:38 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org E33323858417 Authentication-Results: sourceware.org; dmarc=fail (p=none dis=none) header.from=cygwin.com Authentication-Results: sourceware.org; spf=fail smtp.mailfrom=cygwin.com Received: from calimero.vinschen.de ([24.134.7.25]) by mrelayeu.kundenserver.de (mreue011 [212.227.15.167]) with ESMTPSA (Nemesis) id 1Mtf39-1n8pOo1oKt-00v3t7 for ; Mon, 30 Aug 2021 16:05:37 +0200 Received: by calimero.vinschen.de (Postfix, from userid 500) id DC8EAA80DBC; Mon, 30 Aug 2021 16:05:36 +0200 (CEST) Date: Mon, 30 Aug 2021 16:05:36 +0200 From: Corinna Vinschen To: cygwin-developers@cygwin.com Subject: Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? Message-ID: Reply-To: cygwin-developers@cygwin.com Mail-Followup-To: cygwin-developers@cygwin.com References: <4f2cb5f3-ce9c-c617-f65f-841a5eca096e@cornell.edu> <20210828022111.91ef5b4ff24f6da9fadb489e@nifty.ne.jp> <20210828184102.f2206a8a9e5fe5cf24bf5e45@nifty.ne.jp> <20210829180729.48b4e877f773cb3980c5766d@nifty.ne.jp> <20210830091314.f9a2cb71794d0f68cdb5eba7@nifty.ne.jp> <20210830092259.52f7d54fc3fa340738373af4@nifty.ne.jp> <529d7dd7-d876-ca51-cc1f-e414d3c24f71@cornell.edu> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <529d7dd7-d876-ca51-cc1f-e414d3c24f71@cornell.edu> X-Provags-ID: V03:K1:BIHc0GSVstCdXaG75Sag89DqMODqLTUSW6qVmtZc043u36mfZVc 55JDlVhpK0l+eETCa+v6GOfuxFBL5QMEIfX2sAVTBwG3tyY+R4aOqpoRJl61CxdT40uKxws 542ZBkadYDADNCDzfffdrSg9ILWiH0IbR6387IZfpiEzfUWlvEBJmDBDxf7Mg70E/K9nUoJ tmEUVmeOWYTZz4O4RW8Ew== X-UI-Out-Filterresults: notjunk:1;V03:K0:D9TqR2DMIxo=:vwkOx0ZKecAx0jUYfNuDZs o2BQm0Jy9VPMldkVysxQhuCdMQ8NZ5GhMwDfJZs3mEUlGNCg/cmmnEMuVHUJAAB72mYLhLtYj CE5IggpINhQsqiad0kTyIklKoHx4qyVxS5fZX9oauL+fFNwhdYPML+1IILlZvyM5zoknbYzbB KyXqaSsmKMhmc26svlsOtHjVTkNMHJ8JTiaBHUzuzIbvK3YCtqGnESVBxBkCfHCECpcijkvtv JuHtX5dVGKx+6iQ2fpINPb24rh6fgy9IE3h3hmm9YMFjqJpkXABMnBSIcW274fbuiSAx6fe76 hgRliwp0EClfMvtykalR9ZZBF853wx50IOgERf9RQQ6IUBYVMv1SCzYH6NhhjL7yCJWjk6qcV Kouc2+tLQMHQIJ397t9cAfT2v4rQd82H84QDg7PxwdTDaDnhYfs2lWZsW3xKA3qZ3QiyvC3NN kvVBQVN1tV9toe67ru1A844/87Bv0yLVC6O4Hcpm9rvlZiXzrIFDLdGRUWDoQRD+x8ieJnTWB t1A6T8mpWzHE+kfG0u5tXhLoVoMizNXopiN6tyZVL8ZbiyhtBOaA6j8hizJp8UdTxTmFA1NTs +d9g4SBXvEdjj2ntHy/esSSSWNCKfcZf0t0Xri7XBMFVj+lXBJKVBYsrRcnsarODkElAKrUp9 DLeJJnKFroIYnglxqbnwJ41WmSYasS0QSexZlridC8oMyHeZYv7y7EePXTgagvI44w6ixbc6I HtVP5NxF98T3PihZzPC68DtZpns74GvwPAfro7XUW6e1XNDkASaV3W4/kIE= X-Spam-Status: No, score=-104.5 required=5.0 tests=BAYES_00, BODY_8BITS, GIT_PATCH_0, GOOD_FROM_CORINNA_CYGWIN, KAM_DMARC_NONE, KAM_DMARC_STATUS, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_NONE, SPF_NEUTRAL, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: cygwin-developers@cygwin.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Cygwin core component developers mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Aug 2021 14:05:49 -0000 On Aug 30 09:36, Ken Brown wrote: > On 8/29/2021 10:15 PM, Ken Brown via Cygwin wrote: > > On 8/29/2021 8:22 PM, Takashi Yano via Cygwin wrote: > > > [...] > > > Even without this problem, select() for writing pipe has a bug > > > and does not wrok as expected. The following patch seems to be > > > needed. > > > > > > diff --git a/winsup/cygwin/select.cc b/winsup/cygwin/select.cc > > > index 83e1c00e0..ac2fd227e 100644 > > > --- a/winsup/cygwin/select.cc > > > +++ b/winsup/cygwin/select.cc > > > @@ -612,7 +612,6 @@ pipe_data_available (int fd, fhandler_base *fh, > > > HANDLE h, bool writing) > > >             that.  This means that a pipe could still block since you could > > >             be trying to write more to the pipe than is available in the > > >             buffer but that is the hazard of select().  */ > > > -      fpli.WriteQuotaAvailable = fpli.OutboundQuota - fpli.ReadDataAvailable; > > >         if (fpli.WriteQuotaAvailable > 0) > > >          { > > >            paranoid_printf ("fd %d, %s, write: size %u, avail %u", fd, > > > > > > > I agree. > > Now I'm starting to wonder. The use of fpli.OutboundQuota - > fpli.ReadDataAvailable instead of fpli.WriteQuotaAvailable was introduced in > commit a010e6abe, with no comment in the commit message or the code to > explain why. Corinna, does this make sense to you? Is it related to the > issues raised in the message > > https://cygwin.com/pipermail/cygwin-patches/2004q4/005002.html > > that you cited elsewhere in this thread? I thought so, but no. This patch was introduced by cgf in 2008, and I don't see any hint as to the why in any of my mail archives, not even in private email. The only vague hint is the release message later on: - Reworked pipe handling for better speed and better support for signal processing. I wonder if that was a typo, considering the observation from the above mail: "But there is a strange twist: When a read is pending on an empty pipe, then WriteQuotaAvailable is also decremented!" > BTW, when I was working on the pipe approach to AF_UNIX sockets > (topic/af_unix branch), I had occasion to step through > select.cc:pipe_data_available in gdb, and the use of fpli.OutboundQuota - > fpli.ReadDataAvailable definitely seemed wrong to me. So when I wrote > peek_socket_unix on that branch, I used fpli.WriteQuotaAvailable, as Takashi > is suggesting now. If that's working reliable these days (keeping fingers crossed for W7), it's ok if we use that. We may want to check if the above observation in terms on WriteQuotaAvailable on a pipe with pending read is still an issue. Corinna