From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2112.outbound.protection.outlook.com [40.107.243.112]) by sourceware.org (Postfix) with ESMTPS id 8C5EF3858D34 for ; Mon, 30 Aug 2021 20:47:46 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 8C5EF3858D34 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=cornell.edu Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=cornell.edu ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=cq8bRxSUK0/vkLyg58ekrb+3urXTqLQX2RZuHGTc1qufIUM7LhaWt4J4QYTWH4DkFdcF+CWOM8SMrAXqB07ZDfGC2nlPGRkwm5uPG+SHmz7TkCACXXtxnMFA/UnyhdlUYd485okCrrUQ49bBU3xoEb70xf15sGs3BjiKPNYn/47QY6sd12wijzydIG+Jp+RcMF5oI2cQS23IKnaM8O+yI1w6ZOniCb2TFRC23wRpYpiF34BaKckyvBBun/XaxbVzGIo6bwxs94svr1/fmgRWsmCZ7vzFxQIFHESgCnepr+kny5D5EyspigUmpfGRQRx4hbd/vpN4V0EvJmxdheihXQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=kFJ5hNtu3DSDJuEoOd26J/BCcB3DDqTHRegcKciybyg=; b=OHAqwvlBDWOTNTOQecBWRMQ0OLesVdd1HZ/0k9bEpsapbU8Ap7PNmTblrXN2oWN8TBBqtJrly3GNoBqGubOlr84xOArgYAmpBWeEf5X6LpHQSKNZlt3qq1rZj2XrNxjHMqC/voxRNhm6fqFIz85Wa9z/7zmg8J2DC176h5JZ+o4qf0KVoPmXUvaz0yYmAn2UVNIpthUAony/F/3VIrznrjLtkpOKoSoKWbCm/DmpkF0Ko5KBwOTxxQxjMwoaTgcFfa2CaiBo+g48grg5GPk6h9k7mvfjO+mo7k7xANZX7t04KQKSjYRXnIt0BZbwgi5HoBPnWyzT5Sjcf8WlZM2onQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=cornell.edu; dmarc=pass action=none header.from=cornell.edu; dkim=pass header.d=cornell.edu; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cornell.edu; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=kFJ5hNtu3DSDJuEoOd26J/BCcB3DDqTHRegcKciybyg=; b=RPAUjtfy+ObsJxDY/mk4yXauhv+sYTvwUTbqY5GSFzqzvBNCU1hALP88D7TVd8A/PO88/rfai2SoEEHuj2vCEIcx5W7Js2W9U3+j6AH0zZMVioi7ficrWjm6OTWMZHtymtrSIp8+QQaKrAVIHg9zLKBSwGqb3jXSpbCsVLJjT2I= Authentication-Results: cygwin.com; dkim=none (message not signed) header.d=none;cygwin.com; dmarc=none action=none header.from=cornell.edu; Received: from BN7PR04MB4388.namprd04.prod.outlook.com (2603:10b6:406:f8::19) by BN7PR04MB4388.namprd04.prod.outlook.com (2603:10b6:406:f8::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.23; Mon, 30 Aug 2021 20:47:44 +0000 Received: from BN7PR04MB4388.namprd04.prod.outlook.com ([fe80::38bd:b608:234f:9ec6]) by BN7PR04MB4388.namprd04.prod.outlook.com ([fe80::38bd:b608:234f:9ec6%7]) with mapi id 15.20.4457.024; Mon, 30 Aug 2021 20:47:44 +0000 Subject: Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? To: cygwin-developers@cygwin.com References: <20210828184102.f2206a8a9e5fe5cf24bf5e45@nifty.ne.jp> <20210829180729.48b4e877f773cb3980c5766d@nifty.ne.jp> <20210830091314.f9a2cb71794d0f68cdb5eba7@nifty.ne.jp> <20210830092259.52f7d54fc3fa340738373af4@nifty.ne.jp> <529d7dd7-d876-ca51-cc1f-e414d3c24f71@cornell.edu> From: Ken Brown Message-ID: Date: Mon, 30 Aug 2021 16:47:42 -0400 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.13.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-ClientProxiedBy: CH0PR04CA0004.namprd04.prod.outlook.com (2603:10b6:610:76::9) To BN7PR04MB4388.namprd04.prod.outlook.com (2603:10b6:406:f8::19) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from [192.168.1.211] (74.69.128.111) by CH0PR04CA0004.namprd04.prod.outlook.com (2603:10b6:610:76::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.17 via Frontend Transport; Mon, 30 Aug 2021 20:47:43 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 91258f1b-1269-4136-6e95-08d96bf76a8a X-MS-TrafficTypeDiagnostic: BN7PR04MB4388: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:9508; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: xrR7NIURqazcR4drZe7xLpPFwLHJz/YhEjWe74wGGKF2Xse7QDma/kBrJY+4PLc5pO3DGkQ73vB69CO6JMYEKoiuPvAbZgVYuPLKW+NrbPsiCm1ZKnjj8pjuIK43OV2QmBkigZ7RoVCa1wet4HFuYR7L83XMKkVUJHoTfDi87Hz1UHtC1GJ6RiiKK4IcR2PFry71MO/9ZIVyThFvam2xXC0g1Z7Jex2uAZgNGthBBk3pfl24Fn2nQQ1cFscnOUXXshW17xD8kZWMYsrDcvoczsBl+qECJRsZeU6pPYNILsXoL8xrCPtiSpgbYDbPQnqcef25rcTkJ4jOPrVAQxAX1KNrIpvPMLrZzKX+4exKkfQ9IERhpBdn20wkedLBL5hb+0t3OWLeNS5dLjh5r8mkVnnF6UFLyaiouHlxp/8LXdfDz+mjkHsRe14ygGblo1QscvAYq0Zd/3T6gWODrWYYHRDciSZ+Mh56LDG/Qb7hlfAY2Jxk9p/F3XI/qEeSt/MnRcoI4Oskso5vQvr4mfLqTGic0afPkHwfmW6WqSSWLrzmJKbhl08eyI7y/dU3hi6GE0ebOgXFKCaS1qUTsLrVlOsIij5yNDuam9SuhGm0qpo3/smZfh09OcaA5Zg3l8TJmBqylTizAqTX3Oh4gocLH6jXXy9Q5bWhJGWaKtEJJ7PrdjAzDh1toVIUUK7QvFiCRuiqzGmm5UfxNuG5Ntfb4Vesxtlb/vUKpb+C41J95mg= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BN7PR04MB4388.namprd04.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(366004)(396003)(346002)(376002)(39860400002)(136003)(6486002)(53546011)(8676002)(6916009)(38100700002)(478600001)(8936002)(31696002)(186003)(66476007)(26005)(66946007)(2616005)(5660300002)(316002)(2906002)(31686004)(86362001)(956004)(16576012)(83380400001)(66556008)(36756003)(75432002)(786003)(43740500002)(45980500001); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?Windows-1252?Q?8kY1EkAbYO3Hu3XvAqKE0/x5JvsXpf7SnS1zPUS2SYtqe9jasF9PQYHB?= =?Windows-1252?Q?7D/kQfvTxmBJmTdjZY/pH4Te11gssuenTDuWY4NV3+UqRgD9u0dfYJCD?= =?Windows-1252?Q?PS26q5zvf4OYhS80u5kq9xJH/Dz61VP9C/Pd2DlMhMtsJEEz/uSE4rUt?= =?Windows-1252?Q?8NpnKkKDyoF6fBEqWLgyf25lQ/3Sy240RP0nK3KBoQmioUTwEHN6DXIa?= =?Windows-1252?Q?VK9PANuBx0wqzT2I9kmoD0U1dp6napqbxHJO80Op5jSGVh+ZoKRYse1b?= =?Windows-1252?Q?vjQO3Wt8LZ7bOrhCbIIxYaDufrP4QiWHV2Lak/4HQGtywMtcA83mOT2i?= =?Windows-1252?Q?kLif3sjZXpcOjpSSGJKXDbdwjZxIhB+NCo4UpB3zcCfLCMRsq9TIHUMd?= =?Windows-1252?Q?1h82lniYV4p17FPq8BPT0TPR09iOI8zRfLtLWrWdKsIsOlpxeRXSOxQO?= =?Windows-1252?Q?1lyl2ck4YV9BL8UJAa6U5KVHH8SojV0vrSq/DB0cEH0/VS71N4RFxRnL?= =?Windows-1252?Q?McPURLszW+5YXN9iRsmz0+DJ62kqTAdHyKx+4lxOMr5wNU2Z9RsvsKdT?= =?Windows-1252?Q?hWA8u/oLjQKAfZwQ5H+rkYGgLtlaKVMmkOUYY6cA2vEaGLBCWUiiMHiH?= =?Windows-1252?Q?/VAzs7cpC1b8quuIBKF3akAm+Bkv5lKI3bVvS4T8Uj1TRmjHGWSKiohB?= =?Windows-1252?Q?NlbuL2Aq14pArakFOo+xpxZatcxMgwzz5ciyif9SCfhVBS6bRTpow7gv?= =?Windows-1252?Q?GSoqkQyNq2+n74IVVICMh3Zpsz/9tR4Zry8LEMDM/ifo2uZSXkriOczN?= =?Windows-1252?Q?ZfzSRzmbmw//eFUVkRolGOJfQtSQT7trq3qzyoDUA1rwCnPzMLQL9TXo?= =?Windows-1252?Q?yxCUhBXs0lwbuqSKSJcKCZqESK3C3GxEMDRQzX9Dch4XigtqUK0x5zRJ?= =?Windows-1252?Q?g06FYSaZ6cVKGVTCdPy820UuLTQw7Fvwx0eux9JakfzfTMl8Cs5RwYft?= =?Windows-1252?Q?hWpSE2Gf/PYUqoalERXGYEL+GKlyfNhw7PMqSFmKgq5R13pj4dGOdUnm?= =?Windows-1252?Q?6T/1mpzM0w4ubxeAtKNSrKy2wANySLBPuDBagT1MVsnH+dP0F6zdMmtP?= =?Windows-1252?Q?WRbJiztXdKIOrYQUsgWXuzoblE8BnWpfYGJqyzksvg//GFv7chWotkOT?= =?Windows-1252?Q?v5hLBi67+NpYWs65fg0pVJBn9RD7ytsqsxuiC7PiLf1DmHFq7DepjA+s?= =?Windows-1252?Q?fdxLbXwdlDCYnO6aB7Tjwd3lxrxKUDu1X0Da7lAf+iEJlPyXCNgCkRC+?= =?Windows-1252?Q?qwjNjE840csG3XPffRrg5nJ5xwYZXeEt1/RsWyFNtWZ5AnkItW4xLOAU?= =?Windows-1252?Q?YgiTNpp3UrPfPULW/ujKWqQmEQd6T8u9n0/cRrvNOmH6fjEr/pPX9Xhf?= X-OriginatorOrg: cornell.edu X-MS-Exchange-CrossTenant-Network-Message-Id: 91258f1b-1269-4136-6e95-08d96bf76a8a X-MS-Exchange-CrossTenant-AuthSource: BN7PR04MB4388.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Aug 2021 20:47:43.9676 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 5d7e4366-1b9b-45cf-8e79-b14b27df46e1 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 9g4lz63ms/g2TVMsjKFnQSCrl8StG8LaGlnvwR3C5u0Ka4htkgEq2zlLueKF9bKT/PME4IrZvxkAurvBE650LA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN7PR04MB4388 X-Spam-Status: No, score=-3.8 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, MSGID_FROM_MTA_HEADER, NICE_REPLY_A, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_PASS, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: cygwin-developers@cygwin.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Cygwin core component developers mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Aug 2021 20:47:57 -0000 On 8/30/2021 4:14 PM, Corinna Vinschen wrote: > Hi Ken, Hi Takashi, > > On Aug 30 19:00, Corinna Vinschen wrote: >> Ok, let's discuss this. I added more code to my testcase and here's >> what I see. I dropped all data from the output which doesn't change. >> [...] >> - InboundQuota and OutboundQuota are always constant values and >> do not depend on the side the information has been queried on. >> That certainly makes sense. >> >> - WriteQuotaAvailable does not depend on the OutboundQuota, but on >> the InboundQuota, and very likely on the InboundQuota of the read >> side. The OutboundQuota *probably* only makes sense when using >> named pipes with remote clients, which we never do anyway. >> >> The preceeding output shows that ReadDataAvailable on the read side and >> WriteQuotaAvailable on the write side are connected. If we write 20 >> bytes, ReadDataAvailable is incremented by 20 and WriteQuotaAvailable is >> decremented by 20. >> >> So: write.WriteQuotaAvailable == InboundQuota - read.ReadDataAvailable. >> >> Except when a ReadFile is pending on the read side. It's as if the >> running ReadFile already reserved write quota. So the write side >> WriteQuotaAvailable is the number of bytes we can write without blocking, >> after all pending ReadFiles have been satisfied. >> >> Unfortunately that doesn't really make sense when looked at it from the >> user space. >> >> What that means in the first place is that WriteQuotaAvailable on the >> write side is unreliable. What we really need is InboundQuota - >> read.ReadDataAvailable. The problem with that is that the write side >> usually has no access to the read side of the pipe. >> >> Long story short, I have no idea how to fix that ATM. > > Well, what about keeping a duplicate of the read side handle on the > write side just for calling NtQueryInformationFile? > > Attached is an untested patch, can you have a look if that makes sense? I probably won't get a chance to test this until tomorrow, but at a glance it looks great!! Ken