From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2098.outbound.protection.outlook.com [40.107.92.98]) by sourceware.org (Postfix) with ESMTPS id A7C853858D34 for ; Mon, 30 Aug 2021 18:59:05 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org A7C853858D34 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=cornell.edu Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=cornell.edu ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=XAnXanFqK6m4NLlz+rv42cc1i+JluR46EuuZ6JMVkF18CwG2KaR5jrvFrkvd7rk54/WXDv1CBqCP9A2E21AHUvx8T1h2HiUXero/CIUSn5eBgF16oaTZZWZeYMvsXmFOQb5M9heZ92NxUYta8czXiD+w5B4Kh9ywdMrrtOu7pxlQQss0/OHLrNfdITMWnJ5K4n83XBFAQ3gMCVXtK12yJQzbcZR1casIH7N5TKi/qPJ0fJUwF4EbXWdWxYfhXu4Yz0h13BJ588y2u0Mne6s8wqvOFGNIN8weVEg/NV0j3TyLo73K39F8FaAXUuCYU8rqOdBojPVH+Q94iz3bljlbRA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=XQMkFnm2+pHZYj7E7kMifDGH3Yaogy8wP793tA+DDs8=; b=SxRKahEXXYye8lisphOyByshrx/9MpKwkpqoL3sHYaL3CKB2StB+UCsFbYHcLvAeNCMuO/EUODQTldfIW5RwuFmNsf0sHISsbB58roB0Wge7DRQAairbhLCJyLMAz8rVzjQ96C2/4el//KnI3AXIZlNBrYbwi50wJ0PBMgmaDpkY42hdX4vawovgAgakwtPg6I55ma34YTHBmoALUMocOkjPqY5DrLIuRpUiKiDHWgR8Kjez891FXScaHpemVw3KTbwhs+f0roSmkZPRLByq5hOkDZWT8uI46+tLSwh4cq+EXYsswWQ+Z8blBuU5pbdfJCIXHi5ekR6QpdTnzEZKfA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=cornell.edu; dmarc=pass action=none header.from=cornell.edu; dkim=pass header.d=cornell.edu; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cornell.edu; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=XQMkFnm2+pHZYj7E7kMifDGH3Yaogy8wP793tA+DDs8=; b=MuK7QRe/r2hMzEIHionEaoVD8PC5mVY++PKPeliZ6OFFNxTGe5ZYlITnTHr9o/uxjQO1rYQWeE7yxVwPicGt4l6PWtSNot/cqNpsrg2OwsIN5+DJ+yZYC6uK8w8+uyFbkzQXpZlaYj0B8I1LkdVMViGtIQcIEIgHiV752kFFzsY= Authentication-Results: cygwin.com; dkim=none (message not signed) header.d=none;cygwin.com; dmarc=none action=none header.from=cornell.edu; Received: from BN7PR04MB4388.namprd04.prod.outlook.com (2603:10b6:406:f8::19) by BN7PR04MB3953.namprd04.prod.outlook.com (2603:10b6:406:c9::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4436.19; Mon, 30 Aug 2021 18:59:03 +0000 Received: from BN7PR04MB4388.namprd04.prod.outlook.com ([fe80::38bd:b608:234f:9ec6]) by BN7PR04MB4388.namprd04.prod.outlook.com ([fe80::38bd:b608:234f:9ec6%7]) with mapi id 15.20.4457.024; Mon, 30 Aug 2021 18:59:03 +0000 Subject: Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? To: cygwin-developers@cygwin.com References: <20210828184102.f2206a8a9e5fe5cf24bf5e45@nifty.ne.jp> <20210829180729.48b4e877f773cb3980c5766d@nifty.ne.jp> <20210830091314.f9a2cb71794d0f68cdb5eba7@nifty.ne.jp> <20210830092259.52f7d54fc3fa340738373af4@nifty.ne.jp> <529d7dd7-d876-ca51-cc1f-e414d3c24f71@cornell.edu> From: Ken Brown Message-ID: <6f4d6673-0128-cc62-3a5a-5b0897fbbf3f@cornell.edu> Date: Mon, 30 Aug 2021 14:59:02 -0400 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.13.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-ClientProxiedBy: CH2PR18CA0047.namprd18.prod.outlook.com (2603:10b6:610:55::27) To BN7PR04MB4388.namprd04.prod.outlook.com (2603:10b6:406:f8::19) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from [192.168.1.211] (74.69.128.111) by CH2PR18CA0047.namprd18.prod.outlook.com (2603:10b6:610:55::27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.18 via Frontend Transport; Mon, 30 Aug 2021 18:59:03 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: d17faf37-6e21-4bc7-17bd-08d96be83c05 X-MS-TrafficTypeDiagnostic: BN7PR04MB3953: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:10000; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: iTmAkyclWDSVB3DrFdrFQlZMHJUX7z23ag4nyEeIPKftKDP27saJyrrQ6E2Hbr7/mBGzeFrKBw7nvnuXfIw7zHnt06Xb+7mO736AR6hbIaUpDUCrCnIWK5x1EkmEJIKe/EmIlUw/XNbcoKWoy76Em4A8Dj653yX4Q9jZOwF26d4bELDloeXS51yF7zqbvZSoyzbKYPs6C/HgsZBymCwqYcwQzox5HzlNssNuu+KJM6I0XvSkXvxC1+t4lZJzU2DbcFCN4lj/6XY+F8BGA+nx+HWeahRBZmC6a+vcE2PgAQ6XNkZVRGLWpM3w9UX6urmnH70OSAxVuzcDJ9FFF77Ee/V5AAKI6dW4Aqwn/V9ERBdgG8npE1+oiB3FxskP6qBf2WE7vNYlNwp8NFujhvJtBrX2fxw77WcU9U8sfxPpnHPzwVzX4FJU3CqIJn5uPOw4bkBkbH/PlJcpft1finlrY8OoChF5p3P6b4a3UimNJ0cGK64uDU9ATECTJT044+CN8C9HQp5l49KhxUhSnwKewSAhhXjdCFXLrZqkZKVv3kO4Y0rbt+cL5fSlDJ7ykone0GI7KDBKCTgf56YySCCees4Ew0s0OyOnUG6188ciYP6ScjR2vWCYjt5K5Z35fKnafai4wZebklxCBTrfjPSeuA1MdzcETQuY9OD+QfiLZsYfNIOQpXgYVfBRZPUGXHb80J4CM3/7u104W60HG1KJ0xS+P/PIiG4s8/k6YQwnQ/g= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BN7PR04MB4388.namprd04.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(136003)(366004)(376002)(346002)(396003)(39860400002)(83380400001)(38100700002)(316002)(31696002)(16576012)(75432002)(786003)(86362001)(8676002)(31686004)(53546011)(2906002)(478600001)(5660300002)(8936002)(6916009)(956004)(2616005)(186003)(6486002)(66476007)(26005)(36756003)(66556008)(66946007)(43740500002)(45980500001); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?Windows-1252?Q?4V5YJgXoKVzQF9nhAaCJy1W1B7OlcIe2/1gcm5GsakAkXxr/3/44f4q0?= =?Windows-1252?Q?jZFoxo3a4WbsizQRmSzlExrAu7g0+plm8fBo3+bMYSLRU1xGV4PGrNAH?= =?Windows-1252?Q?VUp+LZp88FAYJKttpyMPUMfDAYvJKv8j5/swig7H2nUxQMs1G99PJxUj?= =?Windows-1252?Q?iEjC773M7kLG1SJyAii5saV+ZHHnwS5n+zAth04MldXs0GAwEOH6/0Wx?= =?Windows-1252?Q?FNWY2xa7GOrzWLD/wccrdyAjQLAnOugwyN9drdijuSQGWjsvPfvqJKYr?= =?Windows-1252?Q?xtrjAVmXUc8JAr7w8imF9w7MrEK/R0v5k0dnN/IlDGHuNBNYqkIDoyYy?= =?Windows-1252?Q?X0SCfLliHvfp5FmlYnA/5LYTzPscs3RCmclm+fE+gUhk67YDuVDfoICW?= =?Windows-1252?Q?sl4uX6C1eSx88l7k6L6tA6vWPUU7W5dsvwrXPJj0FiwO9/wKoQeMHyUN?= =?Windows-1252?Q?7LQGspAyiV6PMiUGU6kgnBPvp/cBuIdNGML6LeShb52Jtj6lB6Clf+VG?= =?Windows-1252?Q?gqUVhMW2M+ixuau70qBUtpMRztBKXhzYIPQQmsisEcF2tY90nwFEOWTJ?= =?Windows-1252?Q?v9RcvYkw836f7SQrSh4bgvlZWHFE9EG2BS2tPOT5rPEDs7rkTi5MJ1eQ?= =?Windows-1252?Q?dHHprm4mopLQryyN73m34v7Ti/b5xxwZXMEqksoQw5BA50IORjLhJjHB?= =?Windows-1252?Q?0KpGBaIewdwvARHRhyqphGPsJK5iIowySuMCZQe2E+yyObHHNeOJp4ob?= =?Windows-1252?Q?I11lGV4x6Rrav/cwEWAIqeXHooRBeAzgLxoCGsPPmqx6oR/49DqEe7oT?= =?Windows-1252?Q?nsf8rbMugIG+NrWFKSzsfmkEr3Aw6RE+CExPTlHa31yl8q+elsZwbgXz?= =?Windows-1252?Q?UdM1HxnzjGra82JGZPeQPSUiZED308zW4bpTE0bBloTwXeoIYfJGA43Z?= =?Windows-1252?Q?d8Sny5+24KmStGi24GZ2xFAuIogYd8w8BeGYBJP2pNDzTxX01a4Ful6Q?= =?Windows-1252?Q?7xN9Z06wUaBDEc1GUnVNkMkgDice90PgRAFUY/7QZsWSsr1mMcsld+x4?= =?Windows-1252?Q?KcPa3JC2ornT+hAuqATUXuavDaMI9dq1bGhHiLErhOmyUELmDdtq5t22?= =?Windows-1252?Q?LkPIKVpHwBWTikd4xb1Rkf3eR2j9MyhVK8YrUMVBwjQCbtqzkUnnZ+c1?= =?Windows-1252?Q?oJxCpDiaklz/SsHWbbCJIs3TFPqki2/12x8Sj9c5icS8v9ChLfHJ7oX2?= =?Windows-1252?Q?DYSvP0bJgEQ+xpBIM4Bk/xNn5it81BQ+3ExloQZOECtNKeHKEMjG67IM?= =?Windows-1252?Q?hDpqJx7SAqhfnh18degSh1wKuLQH+37HLYXaZN5TuOV7CbAl04EP/lVd?= =?Windows-1252?Q?6G67rZd1fwAjQ9Lq1yxdUP9X6QRwEFbTjkRmxZeEKXFq4KUkmoT/2BWz?= X-OriginatorOrg: cornell.edu X-MS-Exchange-CrossTenant-Network-Message-Id: d17faf37-6e21-4bc7-17bd-08d96be83c05 X-MS-Exchange-CrossTenant-AuthSource: BN7PR04MB4388.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Aug 2021 18:59:03.4956 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 5d7e4366-1b9b-45cf-8e79-b14b27df46e1 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: jyyqxUpHcUZxePI7XWpPA4fExDx1jH66s/3zdw+HZ0pOtM9tlotnQavJUVziIz9qBloj8MFynByu7Ye9VFqE3w== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN7PR04MB3953 X-Spam-Status: No, score=-3.8 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, MSGID_FROM_MTA_HEADER, NICE_REPLY_A, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_PASS, TXREP, T_SPF_HELO_TEMPERROR autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: cygwin-developers@cygwin.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Cygwin core component developers mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Aug 2021 18:59:17 -0000 On 8/30/2021 1:00 PM, Corinna Vinschen wrote: > On Aug 30 17:53, Corinna Vinschen wrote: >> On Aug 30 16:05, Corinna Vinschen wrote: >>> On Aug 30 09:36, Ken Brown wrote: >>>> BTW, when I was working on the pipe approach to AF_UNIX sockets >>>> (topic/af_unix branch), I had occasion to step through >>>> select.cc:pipe_data_available in gdb, and the use of fpli.OutboundQuota - >>>> fpli.ReadDataAvailable definitely seemed wrong to me. So when I wrote >>>> peek_socket_unix on that branch, I used fpli.WriteQuotaAvailable, as Takashi >>>> is suggesting now. >>> >>> If that's working reliable these days (keeping fingers crossed for W7), >>> it's ok if we use that. We may want to check if the above observation >>> in terms on WriteQuotaAvailable on a pipe with pending read is still an >>> issue. >> >> Ok, I wrote a small testcase. It creates a named pipe, reads from the >> pipe, then, later, writes to the pipe. Interlaced with these calls, it >> calls NtQueryInformationFile(FilePipeLocalInformation) on the write side >> of the pipe. Kind of like this: >> >> CreatePipe >> NtQueryInformationFile >> ReadFile >> NtQueryInformationFile >> WriteFile >> NtQueryInformationFile >> >> Here's the result: >> >> Before ReadFile: >> >> InboundQuota: 65536 >> ReadDataAvailable: 0 >> OutboundQuota: 65536 >> WriteQuotaAvailable: 65536 >> >> While ReadFile is running: >> >> InboundQuota: 65536 >> ReadDataAvailable: 0 >> OutboundQuota: 65536 >> WriteQuotaAvailable: 65494 !!! >> >> After WriteFile and ReadFile succeeded: >> >> InboundQuota: 65536 >> ReadDataAvailable: 0 >> OutboundQuota: 65536 >> WriteQuotaAvailable: 65536 >> >> That means, while a reader on the reader side is waiting for data, the >> WriteQuotaAvailable on the write side is decremented by the amount of >> data requested by the reader (42 bytes in my case), just as outlined in that >> mail from 2004. And this is on W10 now. >> >> What to do with this information? TBD. > > Ok, let's discuss this. I added more code to my testcase and here's > what I see. I dropped all data from the output which doesn't change. > > What I'm trying to get a grip on are the dependencies here. > > After creating the pipe: > > read side: ReadDataAvailable: 0 > write side: WriteQuotaAvailable: 65536 > > After writing 20 bytes... > > read side: ReadDataAvailable: 20 > write side: WriteQuotaAvailable: 65516 > > After writing 40 more bytes... > > read side: ReadDataAvailable: 60 > write side: WriteQuotaAvailable: 65476 > > After reading 42 bytes... > > read side: ReadDataAvailable: 18 > write side: WriteQuotaAvailable: 65518 > > After writing 20 bytes... > > read side: ReadDataAvailable: 38 > write side: WriteQuotaAvailable: 65498 > > *While* reading 42 bytes with an empty buffer... > > read side: ReadDataAvailable: 0 > write side: WriteQuotaAvailable: 65494 > > Another important fun fact: Assuming the read and write buffer sizes > are differently specified. I called CreateNamedPipe with an outbuffer > size of 32K and an inbuffer size of 64K: > > After creating the pipe: > > read side: > InboundQuota: 65536 > ReadDataAvailable: 0 > OutboundQuota: 32768 > WriteQuotaAvailable: 32768 > write side: > InboundQuota: 65536 > ReadDataAvailable: 0 > OutboundQuota: 32768 > WriteQuotaAvailable: 65536 !!! > > This last data point shows that: > > - InboundQuota and OutboundQuota are always constant values and > do not depend on the side the information has been queried on. > That certainly makes sense. > > - WriteQuotaAvailable does not depend on the OutboundQuota, but on > the InboundQuota, and very likely on the InboundQuota of the read > side. The OutboundQuota *probably* only makes sense when using > named pipes with remote clients, which we never do anyway. > > The preceeding output shows that ReadDataAvailable on the read side and > WriteQuotaAvailable on the write side are connected. If we write 20 > bytes, ReadDataAvailable is incremented by 20 and WriteQuotaAvailable is > decremented by 20. > > So: write.WriteQuotaAvailable == InboundQuota - read.ReadDataAvailable. > > Except when a ReadFile is pending on the read side. It's as if the > running ReadFile already reserved write quota. So the write side > WriteQuotaAvailable is the number of bytes we can write without blocking, > after all pending ReadFiles have been satisfied. > > Unfortunately that doesn't really make sense when looked at it from the > user space. > > What that means in the first place is that WriteQuotaAvailable on the > write side is unreliable. What we really need is InboundQuota - > read.ReadDataAvailable. The problem with that is that the write side > usually has no access to the read side of the pipe. For the purposes of select.cc:pipe_data_available, we only need to know whether InboundQuota - read.ReadDataAvailable is positive. If WriteQuotaAvailable is positive, then we're OK, even though its precise value might be too small. But if WriteQuotaAvailable == 0, we don't know whether the buffer is actually full or there's a pending ReadFile. It's only in this case that we need access to the read side. What if we reverse the roles of the read and write sides of the pipe, so that the write side is the server and the read side is the client. We can then try to use ImpersonateNamedPipeClient to get information about the read side when WriteQuotaAvailable == 0. If we succeed, then we can determine whether or not there's space in the buffer. If we fail, we simply report, possibly incorrectly, that there is space. This is no worse than the current situation (on the master branch), in which we use OutboundQuota - ReadDataAvailable, which is always positive. Ken