From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from conssluserg-03.nifty.com (conssluserg-03.nifty.com [210.131.2.82]) by sourceware.org (Postfix) with ESMTPS id 254473857016 for ; Thu, 2 Sep 2021 08:15:56 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 254473857016 Authentication-Results: sourceware.org; dmarc=fail (p=none dis=none) header.from=nifty.ne.jp Authentication-Results: sourceware.org; spf=fail smtp.mailfrom=nifty.ne.jp Received: from Express5800-S70 (z221123.dynamic.ppp.asahi-net.or.jp [110.4.221.123]) (authenticated) by conssluserg-03.nifty.com with ESMTP id 1828FafT013229 for ; Thu, 2 Sep 2021 17:15:36 +0900 DKIM-Filter: OpenDKIM Filter v2.10.3 conssluserg-03.nifty.com 1828FafT013229 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nifty.ne.jp; s=dec2015msa; t=1630570536; bh=jd5YjaMDQ/zdqwsbX0FBH6sDrwtvyk1yU7E+n4PRDp8=; h=Date:From:To:Subject:In-Reply-To:References:From; b=nQcbZpt+8LRlit3WQUaVtpuwr0rywQj3+R/OhImvvTu326UbhR6ZJTm7S5u5jm+ru NKk96L1JyRUIC9i3gq/x0Mt8pMggAwTQFYKJRN2LbEr9zXpQ0ggV1dOz9WwcivxAlm n7IgAy2P+3w9nLFdPHOuu373KG+WmCFthginwzz/9ldWzXLFCaO7jDGtzfq8w56I0N dJ4pEJZAo7GpWFeqkt+rfqPm3fT1G/IaAb3IRBXHbJkFZ2CWVP6obieR65yZy8Vz8Z Jd+Hv9mLuRaMQXUmSkKoD8q1dB66DjFpiUThTSYArWfiIKgZlp3801bIC4sMnJuNih Jm+POi2TRtxJw== X-Nifty-SrcIP: [110.4.221.123] Date: Thu, 2 Sep 2021 17:15:49 +0900 From: Takashi Yano To: cygwin-developers@cygwin.com Subject: Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? Message-Id: <20210902171549.72910cf8e01c822233bfc56a@nifty.ne.jp> In-Reply-To: References: <20210831204541.2b56702cdcd2fae5e91ba8e2@nifty.ne.jp> <583ca127-02e7-6b3c-3732-6478c0f862e3@cornell.edu> <20210901080220.ee4a5bfbea62cc1ae0a9598e@nifty.ne.jp> <20210901091652.6bf3cccbcaed4a22f6ffa6b0@nifty.ne.jp> <20210901172339.1039604b7067e0492534a20f@nifty.ne.jp> X-Mailer: Sylpheed 3.7.0 (GTK+ 2.24.30; i686-pc-mingw32) Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="Multipart=_Thu__2_Sep_2021_17_15_49_+0900_Cm5OIiTHJVI2BkND" X-Spam-Status: No, score=-5.0 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, NICE_REPLY_A, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: cygwin-developers@cygwin.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Cygwin core component developers mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Sep 2021 08:16:08 -0000 This is a multi-part message in MIME format. --Multipart=_Thu__2_Sep_2021_17_15_49_+0900_Cm5OIiTHJVI2BkND Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit On Wed, 1 Sep 2021 10:46:24 +0200 Corinna Vinschen wrote: > On Sep 1 17:23, Takashi Yano wrote: > > On Wed, 1 Sep 2021 10:07:48 +0200 > > Corinna Vinschen wrote: > > > On Sep 1 09:16, Takashi Yano wrote: > > > > On Wed, 1 Sep 2021 08:02:20 +0900 > > > > Takashi Yano wrote: > > > > > On Tue, 31 Aug 2021 17:50:14 +0200 > > > > > Corinna Vinschen wrote: > > > > > > So for the time being I suggest the below patch on top of topic/pipe. > > > > > > It contains everything we discussed so far. > > > > > > > > > > One more thing. 'git log' cannot stop normally with 'q' with your patch. > > > > > > > > The same happes with 'yes |less'. > > > > > > > > The cause is that write side cannot detect closing read side because > > > > query_hdl (read handle) is still opened. > > > > > > Oh > > > > > > my > > > > > > god. > > > > > > > > > That kills the entire idea of keeping the read handle :( > > > > One idea is: > > > > Count read handle and write handle opned using NtQueryObject(). > > If the numbers of opened handle are equal each other, only > > the write side (pair of write handle and query_hdl) is alive. > > In this case, write() returns error. > > If read side is alive, number of read handles is greater than > > number of write handles. > > Interesting idea. But where do you do the count? The event object > will not get signalled, so WFMO will not return when performing a > blocking write. I imagined something like attached patch. Unfortunately, the attached patch seems to have bug that occasionally causes the following error while building cygwin1.dll. : error: "GCC_VERSION" redefined [-Werror] : note: this is the location of the previous definition cc1plus: all warnings being treated as errors make[1]: *** [Makefile:1942: fhandler_proc.o] Error 1 -- Takashi Yano --Multipart=_Thu__2_Sep_2021_17_15_49_+0900_Cm5OIiTHJVI2BkND Content-Type: application/octet-stream; name="count-rw-handle.patch" Content-Disposition: attachment; filename="count-rw-handle.patch" Content-Transfer-Encoding: base64 ZGlmZiAtLWdpdCBhL3dpbnN1cC9jeWd3aW4vZmhhbmRsZXIuaCBiL3dpbnN1cC9jeWd3aW4vZmhh bmRsZXIuaAppbmRleCAxZjBmMjgwNzcuLjM4MzkwODQ4ZiAxMDA2NDQKLS0tIGEvd2luc3VwL2N5 Z3dpbi9maGFuZGxlci5oCisrKyBiL3dpbnN1cC9jeWd3aW4vZmhhbmRsZXIuaApAQCAtMTE3Miw2 ICsxMTcyLDcgQEAgY2xhc3MgZmhhbmRsZXJfcGlwZTogcHVibGljIGZoYW5kbGVyX2Jhc2UKIHsK IHByaXZhdGU6CiAgIEhBTkRMRSBxdWVyeV9oZGw7CisgIEhBTkRMRSByZWFkZXJfZXZ0OwogICBw aWRfdCBwb3Blbl9waWQ7CiAgIHNpemVfdCBtYXhfYXRvbWljX3dyaXRlOwogICB2b2lkIHNldF9w aXBlX25vbl9ibG9ja2luZyAoYm9vbCBub25ibG9ja2luZyk7CkBAIC0xMTgxLDYgKzExODIsNyBA QCBwdWJsaWM6CiAgIGJvb2wgaXNwaXBlKCkgY29uc3QgeyByZXR1cm4gdHJ1ZTsgfQogCiAgIEhB TkRMRSBnZXRfcXVlcnlfaGFuZGxlICgpIGNvbnN0IHsgcmV0dXJuIHF1ZXJ5X2hkbDsgfQorICBi b29sIHJlYWRlcl9jbG9zZWQgKCk7CiAKICAgdm9pZCBzZXRfcG9wZW5fcGlkIChwaWRfdCBwaWQp IHtwb3Blbl9waWQgPSBwaWQ7fQogICBwaWRfdCBnZXRfcG9wZW5fcGlkICgpIGNvbnN0IHtyZXR1 cm4gcG9wZW5fcGlkO30KZGlmZiAtLWdpdCBhL3dpbnN1cC9jeWd3aW4vZmhhbmRsZXJfcGlwZS5j YyBiL3dpbnN1cC9jeWd3aW4vZmhhbmRsZXJfcGlwZS5jYwppbmRleCAyZDllODdiYjMuLjQ3NzNk MDRkYSAxMDA2NDQKLS0tIGEvd2luc3VwL2N5Z3dpbi9maGFuZGxlcl9waXBlLmNjCisrKyBiL3dp bnN1cC9jeWd3aW4vZmhhbmRsZXJfcGlwZS5jYwpAQCAtNTEsNiArNTEsOCBAQCBmaGFuZGxlcl9w aXBlOjpzZXRfcGlwZV9ub25fYmxvY2tpbmcgKGJvb2wgbm9uYmxvY2tpbmcpCiAgIGZwaS5SZWFk TW9kZSA9IEZJTEVfUElQRV9CWVRFX1NUUkVBTV9NT0RFOwogICBmcGkuQ29tcGxldGlvbk1vZGUg PSBub25ibG9ja2luZyA/IEZJTEVfUElQRV9DT01QTEVURV9PUEVSQVRJT04KICAgICA6IEZJTEVf UElQRV9RVUVVRV9PUEVSQVRJT047CisgIGlmIChnZXRfZGV2aWNlICgpID09IEZIX1BJUEVXKQor ICAgIGZwaS5Db21wbGV0aW9uTW9kZSA9IEZJTEVfUElQRV9DT01QTEVURV9PUEVSQVRJT047CiAg IHN0YXR1cyA9IE50U2V0SW5mb3JtYXRpb25GaWxlIChnZXRfaGFuZGxlICgpLCAmaW8sICZmcGks IHNpemVvZiBmcGksCiAJCQkJIEZpbGVQaXBlSW5mb3JtYXRpb24pOwogICBpZiAoIU5UX1NVQ0NF U1MgKHN0YXR1cykpCkBAIC0zMDgsNiArMzEwLDIyIEBAIGZoYW5kbGVyX3BpcGU6OnJhd19yZWFk ICh2b2lkICpwdHIsIHNpemVfdCYgbGVuKQogICAgIH0KICAgZWxzZSBpZiAoc3RhdHVzID09IFNU QVRVU19USFJFQURfQ0FOQ0VMRUQpCiAgICAgcHRocmVhZDo6c3RhdGljX2NhbmNlbF9zZWxmICgp OworCisgIGlmICgoc3NpemVfdClsZW4gPiAwKQorICAgIFNldEV2ZW50IChyZWFkZXJfZXZ0KTsK K30KKworYm9vbAorZmhhbmRsZXJfcGlwZTo6cmVhZGVyX2Nsb3NlZCAodm9pZCkKK3sKKyAgaWYg KGdldF9kZXZpY2UgKCkgPT0gRkhfUElQRVIpCisgICAgcmV0dXJuIGZhbHNlOworICBPQkpFQ1Rf QkFTSUNfSU5GT1JNQVRJT04gb2JpOworICBOdFF1ZXJ5T2JqZWN0IChnZXRfaGFuZGxlICgpLCBP YmplY3RCYXNpY0luZm9ybWF0aW9uLCAmb2JpLCBzaXplb2Ygb2JpLCBOVUxMKTsKKyAgaW50IG5f d3JpdGVyID0gb2JpLkhhbmRsZUNvdW50OworICBOdFF1ZXJ5T2JqZWN0IChxdWVyeV9oZGwsIE9i amVjdEJhc2ljSW5mb3JtYXRpb24sICZvYmksIHNpemVvZiBvYmksIE5VTEwpOworICBpbnQgbl9y ZWFkZXIgPSBvYmkuSGFuZGxlQ291bnQ7CisgIHJldHVybiBuX3dyaXRlciA9PSBuX3JlYWRlcjsK IH0KIAogc3NpemVfdCBfX3JlZzMKQEAgLTMyMyw2ICszNDEsMTMgQEAgZmhhbmRsZXJfcGlwZTo6 cmF3X3dyaXRlIChjb25zdCB2b2lkICpwdHIsIHNpemVfdCBsZW4pCiAgIGlmICghbGVuKQogICAg IHJldHVybiAwOwogCisgIGlmIChyZWFkZXJfY2xvc2VkICgpKQorICAgIHsKKyAgICAgIHNldF9l cnJubyhFUElQRSk7CisgICAgICByYWlzZShTSUdQSVBFKTsKKyAgICAgIHJldHVybiAtMTsKKyAg ICB9CisKICAgaWYgKGxlbiA8PSBtYXhfYXRvbWljX3dyaXRlKQogICAgIGNodW5rID0gbGVuOwog ICBlbHNlIGlmIChpc19ub25ibG9ja2luZyAoKSkKQEAgLTQwMCwxMCArNDI1LDI0IEBAIGZoYW5k bGVyX3BpcGU6OnJhd193cml0ZSAoY29uc3Qgdm9pZCAqcHRyLCBzaXplX3QgbGVuKQogCSAgLyog TnRXcml0ZUZpbGUgcmV0dXJucyBzdWNjZXNzIHdpdGggIyBvZiBieXRlcyB3cml0dGVuID09IDAK IAkgICAgIGlmIHdyaXRpbmcgb24gYSBub24tYmxvY2tpbmcgcGlwZSBmYWlscyBiZWNhdXNlIHRo ZSBwaXBlCiAJICAgICBidWZmZXIgZG9lc24ndCBoYXZlIHN1ZmZpY2llbnQgc3BhY2UuICovCi0J ICBpZiAobmJ5dGVzX25vdyA9PSAwICYmIG5ieXRlcyA9PSAwKQorCSAgaWYgKG5ieXRlc19ub3cg PT0gMCAmJiBuYnl0ZXMgPT0gMCAmJiBpc19ub25ibG9ja2luZyAoKSkKIAkgICAgc2V0X2Vycm5v IChFQUdBSU4pOwotCSAgcHRyID0gKChjaGFyICopIHB0cikgKyBjaHVuazsKKwkgIHB0ciA9ICgo Y2hhciAqKSBwdHIpICsgbmJ5dGVzX25vdzsKIAkgIG5ieXRlcyArPSBuYnl0ZXNfbm93OworCSAg aWYgKHJlYWRlcl9jbG9zZWQgKCkgJiYgbmJ5dGVzID09IDApCisJICAgIHsKKwkgICAgICBzZXRf ZXJybm8oRVBJUEUpOworCSAgICAgIHJhaXNlKFNJR1BJUEUpOworCSAgICB9CisJICBpZiAoIWlz X25vbmJsb2NraW5nICgpICYmIG5ieXRlcyA8IGxlbikKKwkgICAgeworCSAgICAgIGlmIChuYnl0 ZXNfbm93ID09IDApCisJCXsKKwkJICBjeWd3YWl0IChyZWFkZXJfZXZ0KTsKKwkJICBSZXNldEV2 ZW50IChyZWFkZXJfZXZ0KTsKKwkJfQorCSAgICAgIGNvbnRpbnVlOworCSAgICB9CiAJfQogICAg ICAgZWxzZSBpZiAoU1RBVFVTX1BJUEVfSVNfQ0xPU0VEIChzdGF0dXMpKQogCXsKQEAgLTQzMCw5 ICs0NjksMTQgQEAgZmhhbmRsZXJfcGlwZTo6cmF3X3dyaXRlIChjb25zdCB2b2lkICpwdHIsIHNp emVfdCBsZW4pCiB2b2lkCiBmaGFuZGxlcl9waXBlOjpmaXh1cF9hZnRlcl9mb3JrIChIQU5ETEUg cGFyZW50KQogeworICBpZiAoY2xvc2Vfb25fZXhlYygpICYmIHF1ZXJ5X2hkbCkKKyAgICBDbG9z ZUhhbmRsZSAocXVlcnlfaGRsKTsKICAgaWYgKHF1ZXJ5X2hkbCkKICAgICBmb3JrX2ZpeHVwIChw YXJlbnQsIHF1ZXJ5X2hkbCwgInF1ZXJ5X2hkbCIpOwogICBmaGFuZGxlcl9iYXNlOjpmaXh1cF9h ZnRlcl9mb3JrIChwYXJlbnQpOworICBpZiAoIWNsb3NlX29uX2V4ZWMgKCkpCisgICAgRHVwbGlj YXRlSGFuZGxlIChwYXJlbnQsIHJlYWRlcl9ldnQsIEdldEN1cnJlbnRQcm9jZXNzICgpLCAmcmVh ZGVyX2V2dCwKKwkJICAgICAwLCAxLCBEVVBMSUNBVEVfU0FNRV9BQ0NFU1MpOwogfQogCiBpbnQK QEAgLTQ1Myw2ICs0OTcsOSBAQCBmaGFuZGxlcl9waXBlOjpkdXAgKGZoYW5kbGVyX2Jhc2UgKmNo aWxkLCBpbnQgZmxhZ3MpCiAgICAgICBmdHAtPmNsb3NlICgpOwogICAgICAgcmVzID0gLTE7CiAg ICAgfQorICBEdXBsaWNhdGVIYW5kbGUgKEdldEN1cnJlbnRQcm9jZXNzICgpLCByZWFkZXJfZXZ0 LAorCQkgICBHZXRDdXJyZW50UHJvY2VzcyAoKSwgJmZ0cC0+cmVhZGVyX2V2dCwKKwkJICAgMCwg MSwgRFVQTElDQVRFX1NBTUVfQUNDRVNTKTsKIAogICBkZWJ1Z19wcmludGYgKCJyZXMgJWQiLCBy ZXMpOwogICByZXR1cm4gcmVzOwpAQCAtNDYzLDcgKzUxMCwxMSBAQCBmaGFuZGxlcl9waXBlOjpj bG9zZSAoKQogewogICBpZiAocXVlcnlfaGRsKQogICAgIE50Q2xvc2UgKHF1ZXJ5X2hkbCk7Ci0g IHJldHVybiBmaGFuZGxlcl9iYXNlOjpjbG9zZSAoKTsKKyAgaW50IHJldCA9IGZoYW5kbGVyX2Jh c2U6OmNsb3NlICgpOworICBpZiAoZ2V0X2RldmljZSAoKSA9PSBGSF9QSVBFUikKKyAgICBTZXRF dmVudCAocmVhZGVyX2V2dCk7CisgIENsb3NlSGFuZGxlIChyZWFkZXJfZXZ0KTsKKyAgcmV0dXJu IHJldDsKIH0KIAogI2RlZmluZSBQSVBFX0lOVFJPICJcXFxcLlxccGlwZVxcY3lnd2luLSIKQEAg LTY3OCw2ICs3MjksMTAgQEAgZmhhbmRsZXJfcGlwZTo6Y3JlYXRlIChmaGFuZGxlcl9waXBlICpm aHNbMl0sIHVuc2lnbmVkIHBzaXplLCBpbnQgbW9kZSkKICAgICAgIGVsc2UKIAlyZXMgPSAwOwog ICAgIH0KKyAgZmhzWzBdLT5yZWFkZXJfZXZ0ID0gQ3JlYXRlRXZlbnQgKE5VTEwsIHRydWUsIGZh bHNlLCBOVUxMKTsKKyAgRHVwbGljYXRlSGFuZGxlIChHZXRDdXJyZW50UHJvY2VzcyAoKSwgZmhz WzBdLT5yZWFkZXJfZXZ0LAorCQkgICBHZXRDdXJyZW50UHJvY2VzcyAoKSwgJmZoc1sxXS0+cmVh ZGVyX2V2dCwKKwkJICAgMCwgMSwgRFVQTElDQVRFX1NBTUVfQUNDRVNTKTsKIAogICBkZWJ1Z19w cmludGYgKCIlUiA9IHBpcGUoJWQsICV5KSIsIHJlcywgcHNpemUsIG1vZGUpOwogICByZXR1cm4g cmVzOwpkaWZmIC0tZ2l0IGEvd2luc3VwL2N5Z3dpbi9zZWxlY3QuY2MgYi93aW5zdXAvY3lnd2lu L3NlbGVjdC5jYwppbmRleCBkYzFmNzk2MTMuLjA5MDViM2QzOCAxMDA2NDQKLS0tIGEvd2luc3Vw L2N5Z3dpbi9zZWxlY3QuY2MKKysrIGIvd2luc3VwL2N5Z3dpbi9zZWxlY3QuY2MKQEAgLTc0MSw2 ICs3NDEsMTMgQEAgb3V0OgogICAgIH0KICAgaWYgKHMtPndyaXRlX3NlbGVjdGVkICYmIGRldiAh PSBGSF9QSVBFUikKICAgICB7CisgICAgICBpZiAoZGV2ID09IEZIX1BJUEVXICYmICgoZmhhbmRs ZXJfcGlwZSAqKSBmaCktPnJlYWRlcl9jbG9zZWQgKCkpCisJeworCSAgZ290b25lICs9IHMtPndy aXRlX3JlYWR5ID0gdHJ1ZTsKKwkgIGlmIChzLT5leGNlcHRfc2VsZWN0ZWQpCisJICAgIGdvdG9u ZSArPSBzLT5leGNlcHRfcmVhZHkgPSB0cnVlOworCSAgcmV0dXJuIGdvdG9uZTsKKwl9CiAgICAg ICAvKiBGb3IgdGhlIHdyaXRlIHNpZGUgb2YgYSBwaXBlLCBmZXRjaCB0aGUgaGFuZGxlIHRvIHRo ZSByZWFkIHNpZGUuCiAJIFNlZSB0aGUgbG9uZ2lzaCBjb21tZW50IGluIHBpcGVfZGF0YV9hdmFp bGFibGUgZm9yIHRoZSByZWFzb25pbmcuICovCiAgICAgICBpZiAoZGV2ID09IEZIX1BJUEVXKQo= --Multipart=_Thu__2_Sep_2021_17_15_49_+0900_Cm5OIiTHJVI2BkND--