From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from conssluserg-05.nifty.com (conssluserg-05.nifty.com [210.131.2.90]) by sourceware.org (Postfix) with ESMTPS id A3AFE3858D35 for ; Mon, 13 Sep 2021 08:40:46 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org A3AFE3858D35 Authentication-Results: sourceware.org; dmarc=fail (p=none dis=none) header.from=nifty.ne.jp Authentication-Results: sourceware.org; spf=fail smtp.mailfrom=nifty.ne.jp Received: from Express5800-S70 (z221123.dynamic.ppp.asahi-net.or.jp [110.4.221.123]) (authenticated) by conssluserg-05.nifty.com with ESMTP id 18D8eR0s006615 for ; Mon, 13 Sep 2021 17:40:28 +0900 DKIM-Filter: OpenDKIM Filter v2.10.3 conssluserg-05.nifty.com 18D8eR0s006615 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nifty.ne.jp; s=dec2015msa; t=1631522428; bh=XDltgA5wUsNeEiT2F+s7PmaRCjwBWBB7T6O7C33fjpY=; h=Date:From:To:Subject:In-Reply-To:References:From; b=SFiW5OrFHkFyEMv3aP3PY4MHN/UZ3c70VgN8m+xq4dpsLl8JzzapIIRCDcDq5VdOu +I+D2uUEXcCH97FzoR4LbX6tusD4csgneHRiMAlLONv8mqiTNNNML7JFNR4PHjYKYh x/TTuyLJ43V48S3dpr2zjJ9jsrX8EZCfUL4q3HdFPJ435FrZR8znc4ZMALg1IVNgLr ekan4i1HyqAWb4YDGJ21NngimRcNy8fDW2hPhaBNW5bRoBSCy9aGbtP0XB4pcET8aL BDrB3QUTxbSae/UWlK2GDZS9adTX6BPjebKeKsXDuSdjHCCeVrc6nqPKxD+ElYPnSa sJnHt8FUfyxdg== X-Nifty-SrcIP: [110.4.221.123] Date: Mon, 13 Sep 2021 17:40:27 +0900 From: Takashi Yano To: cygwin-developers@cygwin.com Subject: Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? Message-Id: <20210913174027.c0b7a03cf09e6c59752ed735@nifty.ne.jp> In-Reply-To: <20210913085431.8b13feaba276699fc3920a7b@nifty.ne.jp> References: <41A583E1-C8E7-42AB-9F24-EEC33A41EC60@house.org> <20210827202440.47706fc2fc07c5e9a1bc0047@nifty.ne.jp> <20210907122631.65452be8d021ec72259431d5@nifty.ne.jp> <20210909124115.555c6be15d675500617d284a@nifty.ne.jp> <20210909170549.506cc3c1f6029d904fece6dd@nifty.ne.jp> <20210909211940.51ef391e27d43f0421962cb8@nifty.ne.jp> <20210909214246.cd1ff1a3062fea27e51ad4ae@nifty.ne.jp> <33386baf-3b2d-d57f-2ad3-1bd328ed7935@cornell.edu> <20210911075734.aaf37697ba7db2ad14d911a3@nifty.ne.jp> <20210911113517.f74fc3ac1971bbf04c7a9bd1@nifty.ne.jp> <695ce1f4-4f7d-f3f3-6dd3-087467d67b28@cornell.edu> <20210912174849.3d38107568065a95aeb19c7c@nifty.ne.jp> <20210912200423.667e40eb1adc52461bbefa20@nifty.ne.jp> <20210913085431.8b13feaba276699fc3920a7b@nifty.ne.jp> X-Mailer: Sylpheed 3.7.0 (GTK+ 2.24.30; i686-pc-mingw32) Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-9.8 required=5.0 tests=BAYES_00, BODY_8BITS, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, NICE_REPLY_A, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: cygwin-developers@cygwin.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Cygwin core component developers mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Sep 2021 08:40:50 -0000 On Mon, 13 Sep 2021 08:54:31 +0900 Takashi Yano wrote: > On Sun, 12 Sep 2021 17:46:47 -0400 > Ken Brown wrote: > > On 9/12/2021 11:10 AM, Ken Brown wrote: > > > On 9/12/2021 7:04 AM, Takashi Yano wrote: > > >> On Sun, 12 Sep 2021 17:48:49 +0900 > > >> Takashi Yano wrote: > > >>> On Sat, 11 Sep 2021 09:12:02 -0400 > > >>> Ken Brown wrote: > > >>>> On 9/10/2021 10:35 PM, Takashi Yano wrote: > > >>>>> On Fri, 10 Sep 2021 22:17:21 -0400 > > >>>>> Ken Brown wrote: > > >>>>>> On 9/10/2021 6:57 PM, Takashi Yano wrote: > > >>>>>>> On Fri, 10 Sep 2021 11:17:58 -0400 > > >>>>>>> Ken Brown wrote: > > >>>>>>>> I've rerun your test with the latest version, and the test results are > > >>>>>>>> similar. > > >>>>>>>>      I've also run a suite of fifo tests that I've accumulated, and they > > >>>>>>>> all pass > > >>>>>>>> also, so I pushed your patch. > > >>>>>>>> > > >>>>>>>> I think we're in pretty good shape now.  The only detail remaining, > > >>>>>>>> AFAIK, is > > >>>>>>>> how to best avoid a deadlock if the pipe has been created by a non-Cygwin > > >>>>>>>> process.  I've proposed a timeout, but maybe there's a better idea. > > >>>>>>> > > >>>>>>> I am not pretty sure what is the problem, but is not the following > > >>>>>>> patch enough? > > >>>>>>> > > >>>>>>> diff --git a/winsup/cygwin/fhandler.h b/winsup/cygwin/fhandler.h > > >>>>>>> index d309be2f7..13fba9a14 100644 > > >>>>>>> --- a/winsup/cygwin/fhandler.h > > >>>>>>> +++ b/winsup/cygwin/fhandler.h > > >>>>>>> @@ -1205,6 +1205,7 @@ public: > > >>>>>>>       select_record *select_except (select_stuff *); > > >>>>>>>       char *get_proc_fd_name (char *buf); > > >>>>>>>       int open (int flags, mode_t mode = 0); > > >>>>>>> +  void open_setup (int flags); > > >>>>>>>       void fixup_after_fork (HANDLE); > > >>>>>>>       int dup (fhandler_base *child, int); > > >>>>>>>       int close (); > > >>>>>>> diff --git a/winsup/cygwin/fhandler_pipe.cc b/winsup/cygwin/fhandler_pipe.cc > > >>>>>>> index 6994a5dce..d84e6ad84 100644 > > >>>>>>> --- a/winsup/cygwin/fhandler_pipe.cc > > >>>>>>> +++ b/winsup/cygwin/fhandler_pipe.cc > > >>>>>>> @@ -191,6 +191,17 @@ out: > > >>>>>>>       return 0; > > >>>>>>>     } > > >>>>>>> > > >>>>>>> +void > > >>>>>>> +fhandler_pipe::open_setup (int flags) > > >>>>>>> +{ > > >>>>>>> +  fhandler_base::open_setup (flags); > > >>>>>>> +  if (get_dev () == FH_PIPER && !read_mtx) > > >>>>>>> +    { > > >>>>>>> +      SECURITY_ATTRIBUTES *sa = sec_none_cloexec (flags); > > >>>>>>> +      read_mtx = CreateMutexW (sa, FALSE, NULL); > > >>>>>>> +    } > > >>>>>>> +} > > >>>>>>> + > > >>>>>>>     off_t > > >>>>>>>     fhandler_pipe::lseek (off_t offset, int whence) > > >>>>>>>     { > > >>>>>>> > > >>>>>>> > > >>>>>>> AFAIK, another problem remaining is: > > >>>>>>> > > >>>>>>> On Mon, 6 Sep 2021 14:49:55 +0200 > > >>>>>>> Corinna Vinschen wrote: > > >>>>>>>> - What about calling select for writing on pipes read by non-Cygwin > > >>>>>>>>      processes?  In that case, we still can't rely on WriteQuotaAvailable, > > >>>>>>>>      just as before. > > >>>>>> > > >>>>>> This is the problem I was talking about.  In this case the non-Cygwin process > > >>>>>> might have a large pending read, so that the Cygwin process calling select on > > >>>>>> the write side will see WriteQuotaAvailable == 0.  This could lead to a > > >>>>>> deadlock > > >>>>>> with the Cygwin process waiting for write ready while the non-Cygwin > > >>>>>> process is > > >>>>>> blocked trying to read. > > >>>>> > > >>>>> Then, the above patch is for another issue. > > >>>>> The problem happes when: > > >>>>> 1) Start command prompt. > > >>>>> 2) Run 'echo AAAAAAAAAAAA | \cygwin64\bin\cat > > >>>>> This causes hang up in cat. In this case, pipe is created by cmd.exe. > > >>>>> Therefore, read_mtx is not created. > > >>>> > > >>>> Confirmed, and your patch fixes it.  Maybe you should check for error in the > > >>>> call to CreateMutexW and print a debug message in that case. > > >>>> > > >>>>>> My suggestion is that we impose a timeout in this situation, after which > > >>>>>> select > > >>>>>> reports write ready. > > >>>>> > > >>>>> Keeping read handle in write pipe (Corinna's query_hdl) causes problem > > >>>>> that write side cannot detect close on read side. > > >>>>> Is it possible to open read handle temporally when pipe_data_available() > > >>>>> is called? > > >>>> > > >>>> That would be nice, but I have no idea how you could do that. > > >>> > > >>> Hmm. Then, what about PoC code attached? This returns to Corinna's > > >>> query_hdl, and counts read/write handles to detect closing reader side. > > >>> > > >>> If the number of read handles is equal to number of write handles, > > >>> only the pairs of write handle and query_hdl are alive. So, read pipe > > >>> supposed to be closed. > > >>> > > >>> This patch depends another patch I posted a few hours ago. > > >> > > >> Revised a bit. > > > > > > I don't see how this solves the problem.  In the case we were worried about > > > where we have a non-Cygwin reader, the writer has no query_hdl, and you're just > > > always reporting write ready, aren't you?  Or am I missing something? > > > > BTW, we could just decide that always reporting write ready in this corner case > > is acceptable. But then we could just do that without going back to query_hdl. > > The various combination of cygwin and non-cygwin cases resuts in: > > W P R: current, query_hdl > c c c: OK , OK > c c n: NG , OK > c n c: OK , OK > c n n: NG , select() always report write ready Sorry, this was not correct. In fact, W P R: current, query_hdl c c c: OK , OK c c n: NG , OK c n c: OK , select() always report write ready c n n: NG , select() always report write ready :-( -- Takashi Yano