From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from conssluserg-01.nifty.com (conssluserg-01.nifty.com [210.131.2.80]) by sourceware.org (Postfix) with ESMTPS id C9C813858D29 for ; Sun, 12 Sep 2021 06:24:06 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org C9C813858D29 Authentication-Results: sourceware.org; dmarc=fail (p=none dis=none) header.from=nifty.ne.jp Authentication-Results: sourceware.org; spf=fail smtp.mailfrom=nifty.ne.jp Received: from Express5800-S70 (z221123.dynamic.ppp.asahi-net.or.jp [110.4.221.123]) (authenticated) by conssluserg-01.nifty.com with ESMTP id 18C6Nj83016485 for ; Sun, 12 Sep 2021 15:23:45 +0900 DKIM-Filter: OpenDKIM Filter v2.10.3 conssluserg-01.nifty.com 18C6Nj83016485 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nifty.ne.jp; s=dec2015msa; t=1631427825; bh=Kb+XXYsJ+tzTeWoZVeE3yKAkXxSSdBGRohRSswOcHCM=; h=Date:From:To:Subject:In-Reply-To:References:From; b=Il4Tph31EQ6zmFk5uwdLjqvECLV3qyKBZBEQ/1BV6NIM+sN9kxYYoKC5KPfQ9jgFN QbL7+rvKLYjvXgSjg/TmNqPAjSVweeqXHGiAc4YVlOMc6CHv+INJ/GEJ3ed9ARZRAn V83A4+3j9IHUo64mvBhQiJd4HUUNznIEHlqXeFU+E0+tjq6IIBrb+puzy7ue0h6vx8 pzq2b0Wt5gGEvpqLCjgqypWSTw0TrwsK/9wGSNb6t/kxyrIU3famh1pe+PojrgTHf/ XFTkO4IpA5BunM4/MqGE7t+p9b37KIC1sggWjjFEJBYuEAdhjBEqli030ChfnY+a93 T9Fp/ugmRLDqg== X-Nifty-SrcIP: [110.4.221.123] Date: Sun, 12 Sep 2021 15:23:56 +0900 From: Takashi Yano To: cygwin-developers@cygwin.com Subject: Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? Message-Id: <20210912152356.48d617068a9eda3cf860e950@nifty.ne.jp> In-Reply-To: <695ce1f4-4f7d-f3f3-6dd3-087467d67b28@cornell.edu> References: <41A583E1-C8E7-42AB-9F24-EEC33A41EC60@house.org> <20210825201845.07b6400b79dc5558a7761efe@nifty.ne.jp> <20210826062934.54f2f2216021c095bb7ba13b@nifty.ne.jp> <3b560051-ab27-f392-ca4b-d1fd9b5733b0@cornell.edu> <20210827202440.47706fc2fc07c5e9a1bc0047@nifty.ne.jp> <20210907122631.65452be8d021ec72259431d5@nifty.ne.jp> <20210909124115.555c6be15d675500617d284a@nifty.ne.jp> <20210909170549.506cc3c1f6029d904fece6dd@nifty.ne.jp> <20210909211940.51ef391e27d43f0421962cb8@nifty.ne.jp> <20210909214246.cd1ff1a3062fea27e51ad4ae@nifty.ne.jp> <33386baf-3b2d-d57f-2ad3-1bd328ed7935@cornell.edu> <20210911075734.aaf37697ba7db2ad14d911a3@nifty.ne.jp> <20210911113517.f74fc3ac1971bbf04c7a9bd1@nifty.ne.jp> <695ce1f4-4f7d-f3f3-6dd3-087467d67b28@cornell.edu> X-Mailer: Sylpheed 3.7.0 (GTK+ 2.24.30; i686-pc-mingw32) Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="Multipart=_Sun__12_Sep_2021_15_23_56_+0900_UmxbyhHlrornibOn" X-Spam-Status: No, score=-12.0 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, NICE_REPLY_A, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: cygwin-developers@cygwin.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Cygwin core component developers mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 12 Sep 2021 06:24:10 -0000 This is a multi-part message in MIME format. --Multipart=_Sun__12_Sep_2021_15_23_56_+0900_UmxbyhHlrornibOn Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit On Sat, 11 Sep 2021 09:12:02 -0400 Ken Brown wrote: > On 9/10/2021 10:35 PM, Takashi Yano wrote: > > On Fri, 10 Sep 2021 22:17:21 -0400 > > Ken Brown wrote: > >> On 9/10/2021 6:57 PM, Takashi Yano wrote: > >>> On Fri, 10 Sep 2021 11:17:58 -0400 > >>> Ken Brown wrote: > >>>> I've rerun your test with the latest version, and the test results are similar. > >>>> I've also run a suite of fifo tests that I've accumulated, and they all pass > >>>> also, so I pushed your patch. > >>>> > >>>> I think we're in pretty good shape now. The only detail remaining, AFAIK, is > >>>> how to best avoid a deadlock if the pipe has been created by a non-Cygwin > >>>> process. I've proposed a timeout, but maybe there's a better idea. > >>> > >>> I am not pretty sure what is the problem, but is not the following > >>> patch enough? > >>> > >>> diff --git a/winsup/cygwin/fhandler.h b/winsup/cygwin/fhandler.h > >>> index d309be2f7..13fba9a14 100644 > >>> --- a/winsup/cygwin/fhandler.h > >>> +++ b/winsup/cygwin/fhandler.h > >>> @@ -1205,6 +1205,7 @@ public: > >>> select_record *select_except (select_stuff *); > >>> char *get_proc_fd_name (char *buf); > >>> int open (int flags, mode_t mode = 0); > >>> + void open_setup (int flags); > >>> void fixup_after_fork (HANDLE); > >>> int dup (fhandler_base *child, int); > >>> int close (); > >>> diff --git a/winsup/cygwin/fhandler_pipe.cc b/winsup/cygwin/fhandler_pipe.cc > >>> index 6994a5dce..d84e6ad84 100644 > >>> --- a/winsup/cygwin/fhandler_pipe.cc > >>> +++ b/winsup/cygwin/fhandler_pipe.cc > >>> @@ -191,6 +191,17 @@ out: > >>> return 0; > >>> } > >>> > >>> +void > >>> +fhandler_pipe::open_setup (int flags) > >>> +{ > >>> + fhandler_base::open_setup (flags); > >>> + if (get_dev () == FH_PIPER && !read_mtx) > >>> + { > >>> + SECURITY_ATTRIBUTES *sa = sec_none_cloexec (flags); > >>> + read_mtx = CreateMutexW (sa, FALSE, NULL); > >>> + } > >>> +} > >>> + > >>> off_t > >>> fhandler_pipe::lseek (off_t offset, int whence) > >>> { > >>> > >>> > >>> AFAIK, another problem remaining is: > >>> > >>> On Mon, 6 Sep 2021 14:49:55 +0200 > >>> Corinna Vinschen wrote: > >>>> - What about calling select for writing on pipes read by non-Cygwin > >>>> processes? In that case, we still can't rely on WriteQuotaAvailable, > >>>> just as before. > >> > >> This is the problem I was talking about. In this case the non-Cygwin process > >> might have a large pending read, so that the Cygwin process calling select on > >> the write side will see WriteQuotaAvailable == 0. This could lead to a deadlock > >> with the Cygwin process waiting for write ready while the non-Cygwin process is > >> blocked trying to read. > > > > Then, the above patch is for another issue. > > The problem happes when: > > 1) Start command prompt. > > 2) Run 'echo AAAAAAAAAAAA | \cygwin64\bin\cat > > This causes hang up in cat. In this case, pipe is created by cmd.exe. > > Therefore, read_mtx is not created. > > Confirmed, and your patch fixes it. Maybe you should check for error in the > call to CreateMutexW and print a debug message in that case. I added the debug message. -- Takashi Yano --Multipart=_Sun__12_Sep_2021_15_23_56_+0900_UmxbyhHlrornibOn Content-Type: application/octet-stream; name="0001-Cygwin-pipe-Fix-deadlock-if-pipe-is-created-by-non-c.patch" Content-Disposition: attachment; filename="0001-Cygwin-pipe-Fix-deadlock-if-pipe-is-created-by-non-c.patch" Content-Transfer-Encoding: base64 RnJvbSAxYzkyOWY3ZDM2MzNkOTljMDA0MGU1MTU1MjA4MjgyOWIxN2Y1NTBkIE1vbiBTZXAgMTcg MDA6MDA6MDAgMjAwMQpGcm9tOiBUYWthc2hpIFlhbm8gPHRha2FzaGkueWFub0BuaWZ0eS5uZS5q cD4KRGF0ZTogU3VuLCAxMiBTZXAgMjAyMSAxNTowNjowNSArMDkwMApTdWJqZWN0OiBbUEFUQ0hd IEN5Z3dpbjogcGlwZTogRml4IGRlYWRsb2NrIGlmIHBpcGUgaXMgY3JlYXRlZCBieSBub24tY3ln d2luCiBhcHAuCgotLS0KIHdpbnN1cC9jeWd3aW4vZmhhbmRsZXIuaCAgICAgICB8ICAxICsKIHdp bnN1cC9jeWd3aW4vZmhhbmRsZXJfcGlwZS5jYyB8IDEzICsrKysrKysrKysrKysKIDIgZmlsZXMg Y2hhbmdlZCwgMTQgaW5zZXJ0aW9ucygrKQoKZGlmZiAtLWdpdCBhL3dpbnN1cC9jeWd3aW4vZmhh bmRsZXIuaCBiL3dpbnN1cC9jeWd3aW4vZmhhbmRsZXIuaAppbmRleCBkMzA5YmUyZjcuLjEzZmJh OWExNCAxMDA2NDQKLS0tIGEvd2luc3VwL2N5Z3dpbi9maGFuZGxlci5oCisrKyBiL3dpbnN1cC9j eWd3aW4vZmhhbmRsZXIuaApAQCAtMTIwNSw2ICsxMjA1LDcgQEAgcHVibGljOgogICBzZWxlY3Rf cmVjb3JkICpzZWxlY3RfZXhjZXB0IChzZWxlY3Rfc3R1ZmYgKik7CiAgIGNoYXIgKmdldF9wcm9j X2ZkX25hbWUgKGNoYXIgKmJ1Zik7CiAgIGludCBvcGVuIChpbnQgZmxhZ3MsIG1vZGVfdCBtb2Rl ID0gMCk7CisgIHZvaWQgb3Blbl9zZXR1cCAoaW50IGZsYWdzKTsKICAgdm9pZCBmaXh1cF9hZnRl cl9mb3JrIChIQU5ETEUpOwogICBpbnQgZHVwIChmaGFuZGxlcl9iYXNlICpjaGlsZCwgaW50KTsK ICAgaW50IGNsb3NlICgpOwpkaWZmIC0tZ2l0IGEvd2luc3VwL2N5Z3dpbi9maGFuZGxlcl9waXBl LmNjIGIvd2luc3VwL2N5Z3dpbi9maGFuZGxlcl9waXBlLmNjCmluZGV4IDY5OTRhNWRjZS4uOWI0 MjU1Y2ZkIDEwMDY0NAotLS0gYS93aW5zdXAvY3lnd2luL2ZoYW5kbGVyX3BpcGUuY2MKKysrIGIv d2luc3VwL2N5Z3dpbi9maGFuZGxlcl9waXBlLmNjCkBAIC0xOTEsNiArMTkxLDE5IEBAIG91dDoK ICAgcmV0dXJuIDA7CiB9CiAKK3ZvaWQKK2ZoYW5kbGVyX3BpcGU6Om9wZW5fc2V0dXAgKGludCBm bGFncykKK3sKKyAgZmhhbmRsZXJfYmFzZTo6b3Blbl9zZXR1cCAoZmxhZ3MpOworICBpZiAoZ2V0 X2RldiAoKSA9PSBGSF9QSVBFUiAmJiAhcmVhZF9tdHgpCisgICAgeworICAgICAgU0VDVVJJVFlf QVRUUklCVVRFUyAqc2EgPSBzZWNfbm9uZV9jbG9leGVjIChmbGFncyk7CisgICAgICByZWFkX210 eCA9IENyZWF0ZU11dGV4IChzYSwgRkFMU0UsIE5VTEwpOworICAgICAgaWYgKCFyZWFkX210eCkK KwlkZWJ1Z19wcmludGYgKCJDcmVhdGVNdXRleCBmYWlsZWQ6ICVFIik7CisgICAgfQorfQorCiBv ZmZfdAogZmhhbmRsZXJfcGlwZTo6bHNlZWsgKG9mZl90IG9mZnNldCwgaW50IHdoZW5jZSkKIHsK LS0gCjIuMzMuMAoK --Multipart=_Sun__12_Sep_2021_15_23_56_+0900_UmxbyhHlrornibOn--