From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from conssluserg-06.nifty.com (conssluserg-06.nifty.com [210.131.2.91]) by sourceware.org (Postfix) with ESMTPS id 0AE253858402 for ; Tue, 14 Sep 2021 08:07:16 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 0AE253858402 Authentication-Results: sourceware.org; dmarc=fail (p=none dis=none) header.from=nifty.ne.jp Authentication-Results: sourceware.org; spf=fail smtp.mailfrom=nifty.ne.jp Received: from Express5800-S70 (z221123.dynamic.ppp.asahi-net.or.jp [110.4.221.123]) (authenticated) by conssluserg-06.nifty.com with ESMTP id 18E873MI027784 for ; Tue, 14 Sep 2021 17:07:03 +0900 DKIM-Filter: OpenDKIM Filter v2.10.3 conssluserg-06.nifty.com 18E873MI027784 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nifty.ne.jp; s=dec2015msa; t=1631606823; bh=t1iUXJUXMQVhsxREvvKw3eEE9SnAjEw5vTIss/o9/yY=; h=Date:From:To:Subject:In-Reply-To:References:From; b=a+wJMpnaZpE0qAjepHDbluP1YQMGbbWXG5DP/GwGHUw50TYCXfGxHrECCeGdbtjm9 VOXCXOGJeKSHcTjLTZo1Vh9rOC/flt0AjEoNEeXmFLOhvWiF/Jo0H6o+IX6JV6VLVf QSKTmMGkLeILS2zvQsCiO0GVrWpkeAH9od9mJR5ZZ+7TM8KOicrbKQcdpl59tlAtPJ cvJZzAs0zQB5c8BLLkOFi64SjXeR8yGfhR5SB+9V9ySo5aS6HiD0LvpKEbgu1RqeBD uLP0gDdlfLOkeylWwajEboaSJ6PSLkvzRlVWpmPlZPsbAEk2m/qsJsrwI4nWHtbFNK L1qUQk8IsVh2A== X-Nifty-SrcIP: [110.4.221.123] Date: Tue, 14 Sep 2021 17:07:16 +0900 From: Takashi Yano To: cygwin-developers@cygwin.com Subject: Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? Message-Id: <20210914170716.39d5b48d129c9193b410b2f3@nifty.ne.jp> In-Reply-To: References: <33386baf-3b2d-d57f-2ad3-1bd328ed7935@cornell.edu> <20210911075734.aaf37697ba7db2ad14d911a3@nifty.ne.jp> <20210911113517.f74fc3ac1971bbf04c7a9bd1@nifty.ne.jp> <695ce1f4-4f7d-f3f3-6dd3-087467d67b28@cornell.edu> <20210912174849.3d38107568065a95aeb19c7c@nifty.ne.jp> <20210912200423.667e40eb1adc52461bbefa20@nifty.ne.jp> <20210914043718.f420491c6723f3dc2e2d9753@nifty.ne.jp> X-Mailer: Sylpheed 3.7.0 (GTK+ 2.24.30; i686-pc-mingw32) Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="Multipart=_Tue__14_Sep_2021_17_07_16_+0900_wIPatk9HtZ3EgkzK" X-Spam-Status: No, score=-5.4 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, NICE_REPLY_A, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: cygwin-developers@cygwin.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Cygwin core component developers mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 Sep 2021 08:07:21 -0000 This is a multi-part message in MIME format. --Multipart=_Tue__14_Sep_2021_17_07_16_+0900_wIPatk9HtZ3EgkzK Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit On Mon, 13 Sep 2021 22:15:25 +0200 Corinna Vinschen wrote: > On Sep 14 04:37, Takashi Yano wrote: > > On Mon, 13 Sep 2021 20:32:33 +0200 > > Corinna Vinschen wrote: > > > I don't understand why calling fork_fixup on query_hdl should depend > > > on the handle count. If you duplicate a writer, you always have to > > > duplicate query_hdl as well to keep the count, no? Inheritence is > > > handled by the O_CLOEXEC flag and fork_fixup will do the right thing. > > > > I thought so, however, counting query_hdl cannot work as expected > > without this check. The number of query_hdl opend seems to exceed > > the number of writer. > > If the write handle as well as the query handle are both opened, > duplicated and closed in the same manner, they should never have a > different count, unless the write side is inherited by a non-Cygwin > client. > > > There seems to be the case that handle is already inherited here > > without fork_fixup. Any idea? > > That should depend on the O_CLOEXEC setting, but identically for > all handles in the fhandler. I found the cause. set_close_on_exec() in fhandler_pipe is missing. set_no_inheritance() calls for all adjunct handles are necessary. > I pushed two more patches to topic/pipe in terms of inheritence, > maybe that gives a clue? I attached two additional patch for this issue. > > > > + if (!DuplicateHandle (GetCurrentProcess (), r, > > > > + GetCurrentProcess (), &fhs[1]->query_hdl, > > > > + GENERIC_READ, !(mode & O_CLOEXEC), 0)) > > > > > > This is a bug I introduced accidentally during testing. This > > > GENERIC_READ is actually supposed to be a FILE_READ_ATTRIBUTES. > > > Sorry about that. > > > > The PoC code uses PeekNamedPipe for query_hdl, so GENERIC_READ is > > necessary I think. > > Oh, right, that's how it's documented. Funny enough, the descriptions > of FSCTL_PIPE_PEEK does not mention any permissions at all. I tried the > permissions and it's FILE_READ_DATA which is required. Thanks. I revised the patch and attach it the subsequent mail. -- Takashi Yano --Multipart=_Tue__14_Sep_2021_17_07_16_+0900_wIPatk9HtZ3EgkzK Content-Type: application/octet-stream; name="0001-Cygwin-fhandler_base-dup-Reflect-O_CLOEXEC-to-inheri.patch" Content-Disposition: attachment; filename="0001-Cygwin-fhandler_base-dup-Reflect-O_CLOEXEC-to-inheri.patch" Content-Transfer-Encoding: base64 RnJvbSA4ZGQyYmRmZjU3OTc2MjQ5Nzg5ODA0NGU5ZTUzMDRiNGI2YWI1ZTkzIE1vbiBTZXAgMTcg MDA6MDA6MDAgMjAwMQpGcm9tOiBUYWthc2hpIFlhbm8gPHRha2FzaGkueWFub0BuaWZ0eS5uZS5q cD4KRGF0ZTogVHVlLCAxNCBTZXAgMjAyMSAxMjo0ODowMyArMDkwMApTdWJqZWN0OiBbUEFUQ0gg MS8yXSBDeWd3aW46IGZoYW5kbGVyX2Jhc2U6OmR1cCBSZWZsZWN0IE9fQ0xPRVhFQyB0bwogaW5o ZXJpdGFuY2UgZmxhZy4KCi0gQ3VycmVudGx5IGZoYW5kbGVyX2Jhc2U6OmR1cCBkdXBsaWNhdGVz IGhhbmRsZXMgd2l0aCBiSW5oZXJpdEhhbmRsZQogIFRSVUUgdW5jb25kaXRpb25hbGx5LiBUaGlz IHBhdGNoIHJlZmxlY3RzIE9fQ0xPRVhFQyBmbGFnIHRvIHRoYXQKICBwYXJhbWV0ZXIuCi0tLQog d2luc3VwL2N5Z3dpbi9maGFuZGxlci5jYyB8IDQgKystLQogMSBmaWxlIGNoYW5nZWQsIDIgaW5z ZXJ0aW9ucygrKSwgMiBkZWxldGlvbnMoLSkKCmRpZmYgLS1naXQgYS93aW5zdXAvY3lnd2luL2Zo YW5kbGVyLmNjIGIvd2luc3VwL2N5Z3dpbi9maGFuZGxlci5jYwppbmRleCAzOWZlMjY0MGEuLjlk ZmU3MGJlMyAxMDA2NDQKLS0tIGEvd2luc3VwL2N5Z3dpbi9maGFuZGxlci5jYworKysgYi93aW5z dXAvY3lnd2luL2ZoYW5kbGVyLmNjCkBAIC0xMzA4LDcgKzEzMDgsNyBAQCBmaGFuZGxlcl9iYXNl Ojppbml0IChIQU5ETEUgZiwgRFdPUkQgYSwgbW9kZV90IGJpbikKIH0KIAogaW50Ci1maGFuZGxl cl9iYXNlOjpkdXAgKGZoYW5kbGVyX2Jhc2UgKmNoaWxkLCBpbnQpCitmaGFuZGxlcl9iYXNlOjpk dXAgKGZoYW5kbGVyX2Jhc2UgKmNoaWxkLCBpbnQgZmxhZ3MpCiB7CiAgIGRlYnVnX3ByaW50ZiAo ImluIGZoYW5kbGVyX2Jhc2UgZHVwIik7CiAKQEAgLTEzMTcsNyArMTMxNyw3IEBAIGZoYW5kbGVy X2Jhc2U6OmR1cCAoZmhhbmRsZXJfYmFzZSAqY2hpbGQsIGludCkKICAgICB7CiAgICAgICBpZiAo IUR1cGxpY2F0ZUhhbmRsZSAoR2V0Q3VycmVudFByb2Nlc3MgKCksIGdldF9oYW5kbGUgKCksCiAJ CQkgICAgR2V0Q3VycmVudFByb2Nlc3MgKCksICZuaCwKLQkJCSAgICAwLCBUUlVFLCBEVVBMSUNB VEVfU0FNRV9BQ0NFU1MpKQorCQkJICAgIDAsICEoZmxhZ3MgJiBPX0NMT0VYRUMpLCBEVVBMSUNB VEVfU0FNRV9BQ0NFU1MpKQogCXsKIAkgIGRlYnVnX3ByaW50ZiAoImR1cCglcykgZmFpbGVkLCBo YW5kbGUgJXAsICVFIiwKIAkJCWdldF9uYW1lICgpLCBnZXRfaGFuZGxlICgpKTsKLS0gCjIuMzMu MAoK --Multipart=_Tue__14_Sep_2021_17_07_16_+0900_wIPatk9HtZ3EgkzK Content-Type: application/octet-stream; name="0002-Cygwin-pipe-fifo-Call-set_no_inheritance-for-adjunct.patch" Content-Disposition: attachment; filename="0002-Cygwin-pipe-fifo-Call-set_no_inheritance-for-adjunct.patch" Content-Transfer-Encoding: base64 RnJvbSBhM2MyMjNmNzJmOThiYWYxNGRmM2I1ODk0MTc2MWM3MjY5MGY1ZWVlIE1vbiBTZXAgMTcg MDA6MDA6MDAgMjAwMQpGcm9tOiBUYWthc2hpIFlhbm8gPHRha2FzaGkueWFub0BuaWZ0eS5uZS5q cD4KRGF0ZTogVHVlLCAxNCBTZXAgMjAyMSAxMjo0OTozNSArMDkwMApTdWJqZWN0OiBbUEFUQ0gg Mi8yXSBDeWd3aW46IHBpcGUsIGZpZm86IENhbGwgc2V0X25vX2luaGVyaXRhbmNlKCkgZm9yIGFk anVuY3QKIGhhbmRsZXMuCgotIEN1cnJudGx5LCBzZXRfbm9faW5oZXJpdGFuY2UoKSBpcyBub3Qg Y2FsbGVkIGZvciB0aGUgYWRqdW5jdCBoYW5kbGVzCiAgc3VjaCBhcyBzZWxlY3Rfc2VtLiBUaGlz IHBhdGNoIGZpeGVzIHRoZSBpc3N1ZS4KLS0tCiB3aW5zdXAvY3lnd2luL2ZoYW5kbGVyLmggICAg ICAgfCAgMSArCiB3aW5zdXAvY3lnd2luL2ZoYW5kbGVyX2ZpZm8uY2MgfCAgMiArKwogd2luc3Vw L2N5Z3dpbi9maGFuZGxlcl9waXBlLmNjIHwgMTAgKysrKysrKysrKwogMyBmaWxlcyBjaGFuZ2Vk LCAxMyBpbnNlcnRpb25zKCspCgpkaWZmIC0tZ2l0IGEvd2luc3VwL2N5Z3dpbi9maGFuZGxlci5o IGIvd2luc3VwL2N5Z3dpbi9maGFuZGxlci5oCmluZGV4IDEzZmJhOWExNC4uNDYzODFjMzk3IDEw MDY0NAotLS0gYS93aW5zdXAvY3lnd2luL2ZoYW5kbGVyLmgKKysrIGIvd2luc3VwL2N5Z3dpbi9m aGFuZGxlci5oCkBAIC0xMjA4LDYgKzEyMDgsNyBAQCBwdWJsaWM6CiAgIHZvaWQgb3Blbl9zZXR1 cCAoaW50IGZsYWdzKTsKICAgdm9pZCBmaXh1cF9hZnRlcl9mb3JrIChIQU5ETEUpOwogICBpbnQg ZHVwIChmaGFuZGxlcl9iYXNlICpjaGlsZCwgaW50KTsKKyAgdm9pZCBzZXRfY2xvc2Vfb25fZXhl YyAoYm9vbCB2YWwpOwogICBpbnQgY2xvc2UgKCk7CiAgIHZvaWQgX19yZWczIHJhd19yZWFkICh2 b2lkICpwdHIsIHNpemVfdCYgbGVuKTsKICAgaW50IGlvY3RsICh1bnNpZ25lZCBpbnQgY21kLCB2 b2lkICopOwpkaWZmIC0tZ2l0IGEvd2luc3VwL2N5Z3dpbi9maGFuZGxlcl9maWZvLmNjIGIvd2lu c3VwL2N5Z3dpbi9maGFuZGxlcl9maWZvLmNjCmluZGV4IGFhODlmYTdhZS4uMzc0OThmNTQ3IDEw MDY0NAotLS0gYS93aW5zdXAvY3lnd2luL2ZoYW5kbGVyX2ZpZm8uY2MKKysrIGIvd2luc3VwL2N5 Z3dpbi9maGFuZGxlcl9maWZvLmNjCkBAIC0xODE3LDQgKzE4MTcsNiBAQCBmaGFuZGxlcl9maWZv OjpzZXRfY2xvc2Vfb25fZXhlYyAoYm9vbCB2YWwpCiAJc2V0X25vX2luaGVyaXRhbmNlIChmY19o YW5kbGVyW2ldLmgsIHZhbCk7CiAgICAgICBmaWZvX2NsaWVudF91bmxvY2sgKCk7CiAgICAgfQor ICBpZiAoc2VsZWN0X3NlbSkKKyAgICBzZXRfbm9faW5oZXJpdGFuY2UgKHNlbGVjdF9zZW0sIHZh bCk7CiB9CmRpZmYgLS1naXQgYS93aW5zdXAvY3lnd2luL2ZoYW5kbGVyX3BpcGUuY2MgYi93aW5z dXAvY3lnd2luL2ZoYW5kbGVyX3BpcGUuY2MKaW5kZXggNzBjZmEzNzg0Li5kYTQ3M2ExZGMgMTAw NjQ0Ci0tLSBhL3dpbnN1cC9jeWd3aW4vZmhhbmRsZXJfcGlwZS5jYworKysgYi93aW5zdXAvY3ln d2luL2ZoYW5kbGVyX3BpcGUuY2MKQEAgLTUxNSw2ICs1MTUsMTYgQEAgZmhhbmRsZXJfcGlwZV9m aWZvOjpyYXdfd3JpdGUgKGNvbnN0IHZvaWQgKnB0ciwgc2l6ZV90IGxlbikKICAgcmV0dXJuIG5i eXRlcyA/OiAtMTsKIH0KIAordm9pZAorZmhhbmRsZXJfcGlwZTo6c2V0X2Nsb3NlX29uX2V4ZWMg KGJvb2wgdmFsKQoreworICBmaGFuZGxlcl9iYXNlOjpzZXRfY2xvc2Vfb25fZXhlYyAodmFsKTsK KyAgaWYgKHJlYWRfbXR4KQorICAgIHNldF9ub19pbmhlcml0YW5jZSAocmVhZF9tdHgsIHZhbCk7 CisgIGlmIChzZWxlY3Rfc2VtKQorICAgIHNldF9ub19pbmhlcml0YW5jZSAoc2VsZWN0X3NlbSwg dmFsKTsKK30KKwogdm9pZAogZmhhbmRsZXJfcGlwZTo6Zml4dXBfYWZ0ZXJfZm9yayAoSEFORExF IHBhcmVudCkKIHsKLS0gCjIuMzMuMAoK --Multipart=_Tue__14_Sep_2021_17_07_16_+0900_wIPatk9HtZ3EgkzK--