From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mout.kundenserver.de (mout.kundenserver.de [217.72.192.75]) by sourceware.org (Postfix) with ESMTPS id 1EFD8385802D for ; Fri, 3 Sep 2021 09:12:40 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 1EFD8385802D Authentication-Results: sourceware.org; dmarc=fail (p=none dis=none) header.from=cygwin.com Authentication-Results: sourceware.org; spf=fail smtp.mailfrom=cygwin.com Received: from calimero.vinschen.de ([24.134.7.25]) by mrelayeu.kundenserver.de (mreue106 [212.227.15.183]) with ESMTPSA (Nemesis) id 1Mgf4k-1mu49y0AYu-00h6Ql for ; Fri, 03 Sep 2021 11:12:38 +0200 Received: by calimero.vinschen.de (Postfix, from userid 500) id 8A5CCA80F97; Fri, 3 Sep 2021 11:12:37 +0200 (CEST) Date: Fri, 3 Sep 2021 11:12:37 +0200 From: Corinna Vinschen To: cygwin-developers@cygwin.com Subject: Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? Message-ID: Reply-To: cygwin-developers@cygwin.com Mail-Followup-To: cygwin-developers@cygwin.com References: <20210901172339.1039604b7067e0492534a20f@nifty.ne.jp> <24138e20-aa97-cfea-bf48-198fc67755ea@cornell.edu> <9ba687eb-f4a0-18f8-b10b-76e7e51e123e@cornell.edu> <152bfc0c-2f72-c684-6fc5-aa7c36c136b8@cornell.edu> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: X-Provags-ID: V03:K1:n6vuULjFG8Jrvt5NnYuIneKPaLnyzDi5r4a2kab/6QLXLUu6xMl KKHMVmYkNr6k3HZwLN4WAOoNq2JBbHtzX4rJMJGM7lgt88vpxjU+wD8zejxY24qE3Vc2YET /NFnR0ZE8MF7K4RCDRGFGFTLkBauJ8xv4jX3J9Wu1DBbROZd1HubSFKT6EkMgCfZMCkPiRK ZWudjXHmGbOUE2Q0UuMKA== X-UI-Out-Filterresults: notjunk:1;V03:K0:OlZsoctvfis=:CUo3wCD2muqRrNIXmWTgfN a7/PPvSqjLuqIWXGTyKiPuVcEGe3CX/L3CyaSXHhSYu82UftT/bad/CR73y0/RXDFkHccbBAD 3fsEkxNBiwy9CCA6HgDmvZSCk7ojHuYXA+0EG/sxecxronOz3rpyXfN8Fn3uSLC9vYVxPYCmR jskR8vNo7rEChMvJBph6UoFYI5T8Wf2Oxvs2ZDJhSkwCQ+38595b2a7YV5Gd3JrBnicK7ZNa9 2WuRm/JsobdiDpEvurdCqTlGhGFPM28GnOrYTpmFOfd2JpbnHZJfj0bnnY4SQMrp7eYLGQhkC yApJbzuLet/DugF4dbDAdwT8KWkVVLH0QQ1qD310XC2qxyU2c7sP1FAeN7DLGNHquc4JrYb3y pfxU54sPNLh45Yz2rxYMO8uAaI5Al8Nygsc4K3/DtC9Z6njiFlVGv0ogX1pqJjV+rf6n2Mhug Wsd3YiKxSkcQIqX4Ho26prKH1zfY43yp/DRrqNxNYefURChDcrgSCAtr7ANweYVHtM1LlDzms udOkjYqvonqg2RNiOy9fxBCDiwrrT5yahjWG3HXeZ/A5FMcreJkVu911iH5fWehJvZo2DRKa4 obtXUmuKU7xbinTV+D2MMVvjWmlmgqBOjQALCldsMXbe/EaWYdgZC8Z6jefwHjkK3x6B74LwX Y/muWQBPMBkDCc1KudxxPduiSsoxtrzDNtwLY5YeyHfnGU9X48+fKTDPCx98E6mMCe1THiyXq 2xUXdPTECwcuJTbxh4HSywSmVowbmWv6+AOUZzUXvxn8LiAPmxhl+si+jD4= X-Spam-Status: No, score=-99.7 required=5.0 tests=BAYES_00, GOOD_FROM_CORINNA_CYGWIN, JMQ_SPF_NEUTRAL, KAM_DMARC_NONE, KAM_DMARC_STATUS, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_NONE, SPF_NEUTRAL, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: cygwin-developers@cygwin.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Cygwin core component developers mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Sep 2021 09:12:42 -0000 On Sep 2 16:19, Ken Brown wrote: > On 9/2/2021 3:35 PM, Corinna Vinschen wrote: > > On Sep 2 21:00, Corinna Vinschen wrote: > > > On Sep 2 09:01, Ken Brown wrote: > > > > Do you think there's anything we can or should do to avoid a deadlock in the > > > > rare cases where this fails? The only thing I can think of immediately is > > > > to always impose a timeout if select is called with infinite timeout on the > > > > write side of a pipe, after which we report that the pipe is write ready. > > > > After all, we've lived since 2008 with a bug that caused select to *always* > > > > report write ready. > > > > > > Indeed. Hmm. What timeout are you thinking of? Seconds? Minutes? > > > > > > > Alternatively, we could just wait and see if there's an actual use case in > > > > which someone encounters a deadlock. > > > > > > Or that. Fixing up select isn't too hard in that case, I guess. > > > > It's getting too late again. I drop off for tonight, but I attached > > my POC code I have so far. It also adds the snippets from my previous > > patch which fixes stuff Takashi found during testing. It also fixes > > something which looks like a bug in raw_write: > > > > - ptr = ((char *) ptr) + chunk; > > + ptr = ((char *) ptr) + nbytes_now; > > > > Incrementing ptr by chunk bytes while only nbytes_now have been written > > looks incorrect. > > Yes. I actually copied that bug from fhandler_base_overlapped::raw_write. > It's amazing that no one has no one has bumped into it up to now. > > As for the reader, it makes the # of bytes to read dependent on the > > number of reader handles. I don't know if that's such a bright idea, > > but this can be changed easily. > > > > Anyway, this runs all my testcases successfully but they are anything > > but thorough. > > > > Patch relativ to topic/pipe attached. Would you both mind to take a > > scrutinizing look? > > Will do. I pushed my stuff to the topic/pipe branch split into hopefully useful chunks. Kick me if anything is wrong or not working. Thanks, Corinna