From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM02-BN1-obe.outbound.protection.outlook.com (mail-bn1nam07on20724.outbound.protection.outlook.com [IPv6:2a01:111:f400:7eb2::724]) by sourceware.org (Postfix) with ESMTPS id 80CA53858D3C for ; Mon, 20 Sep 2021 19:14:16 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 80CA53858D3C Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=cornell.edu Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=cornell.edu ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=QiwdKWRcL8XMIvlzfIHHTcBleCgyscSHerQ5jSp6+jf//TyfnfhC7P8YB2vSaJFuPWD/dK2LtRi+he6Tr260iWmbdOjAdZQ+A/ykR2CUu75Z65NuePH6zeZbI3gIChqxgZi5WOuJkkFbSnBizSULTuJE6HAXkhnVCKnfmMubQd8fkVvirgT+2gCLt+TZyNjdfrvBwcccuoqbVYkjT6NDX8aigfVsbdc/fV9OhOoofxzhUcwoxcy+b7ecDXQa2c48318CEUldxGQmfHiofFPVupKr0p4K7nnBuVDL6KfkNCEuWhCbZF+DxRAk9h12MsmR7nbvAVOP90cSQk+R83Rvng== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=cZtPOcPwAm22rwJfIcfLDLFwu2eGJz+urgngqYeoRuk=; b=Vx/3fDr7MSDeS9Tq6EAsF57j86JwvC0/mZhJDuDfuHC6na3XtxQyiRz3Q7eJz2Qy3Bl4sC3DA51ZzKm1d7nWkUK4rMoqLd1rLbrXgkW+iMnO5SVbRPIY93K49ToCE9ip4HN34AxQ0O91zMZ/rd08HWOk0XAwAzi2u1v2lCtyDVqQv0QH8tohrgVCGUpztHndDeURenvPFf/Z41sxC0BNo/m233agB9AYbnuAJ2rkEihbqKVyRHzgx7hdO/C6qrl8hK7aFaaPG+Q4fu0TUgAIxJvNA+xl2fmFirpWDzduCZlM14+HW+MdCaAW1MtRA4Bh5mqOaFWqvVZyEcL6KwG2sg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=cornell.edu; dmarc=pass action=none header.from=cornell.edu; dkim=pass header.d=cornell.edu; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cornell.edu; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=cZtPOcPwAm22rwJfIcfLDLFwu2eGJz+urgngqYeoRuk=; b=QmTEz5FpVHwgpboP9Z2PLbNKYWuoaaRBrXG1rWFftH/9+OFQMm7pwujdWYSFtSxmJ9U9D7d4gEGdmtv7L9VRTqgrAT/8isZATBAriBT4IxXjhIzBsOt2C8Jfts/0Wmi3r1iB1NlQmdPxYhsd0ID5e1NWx4DjF84doqNmoq6t1r4= Authentication-Results: cygwin.com; dkim=none (message not signed) header.d=none;cygwin.com; dmarc=none action=none header.from=cornell.edu; Received: from BN7PR04MB4388.namprd04.prod.outlook.com (2603:10b6:406:f8::19) by BN6PR04MB0244.namprd04.prod.outlook.com (2603:10b6:404:1d::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.18; Mon, 20 Sep 2021 19:14:10 +0000 Received: from BN7PR04MB4388.namprd04.prod.outlook.com ([fe80::5113:e84a:b38a:7a66]) by BN7PR04MB4388.namprd04.prod.outlook.com ([fe80::5113:e84a:b38a:7a66%6]) with mapi id 15.20.4523.018; Mon, 20 Sep 2021 19:14:10 +0000 Subject: Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? To: cygwin-developers@cygwin.com References: <20210907122631.65452be8d021ec72259431d5@nifty.ne.jp> <20210909124115.555c6be15d675500617d284a@nifty.ne.jp> <20210909170549.506cc3c1f6029d904fece6dd@nifty.ne.jp> <20210909211940.51ef391e27d43f0421962cb8@nifty.ne.jp> <20210909214246.cd1ff1a3062fea27e51ad4ae@nifty.ne.jp> <33386baf-3b2d-d57f-2ad3-1bd328ed7935@cornell.edu> <20210911075734.aaf37697ba7db2ad14d911a3@nifty.ne.jp> <20210911113517.f74fc3ac1971bbf04c7a9bd1@nifty.ne.jp> <20210920215229.cea5722b2c2fcd7e03917108@nifty.ne.jp> From: Ken Brown Message-ID: <00f4eca0-2b38-f4bb-98f2-e99e7ee941f5@cornell.edu> Date: Mon, 20 Sep 2021 15:14:08 -0400 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.14.0 In-Reply-To: <20210920215229.cea5722b2c2fcd7e03917108@nifty.ne.jp> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-ClientProxiedBy: CH0PR03CA0269.namprd03.prod.outlook.com (2603:10b6:610:e5::34) To BN7PR04MB4388.namprd04.prod.outlook.com (2603:10b6:406:f8::19) MIME-Version: 1.0 Received: from [IPv6:2603:7081:7e3f:3419:d571:af58:205e:a58] (2603:7081:7e3f:3419:d571:af58:205e:a58) by CH0PR03CA0269.namprd03.prod.outlook.com (2603:10b6:610:e5::34) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14 via Frontend Transport; Mon, 20 Sep 2021 19:14:09 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 577b6bec-ffd2-4de6-8543-08d97c6ad2f9 X-MS-TrafficTypeDiagnostic: BN6PR04MB0244: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:6790; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: AjjinvZ6GLFP4WooQMEkSYBUuHly6X5E78rRl1+CZjUuOeFVyobsLt6vcDJiz/aaMRMtnLYzCuEVxqZYuF1cj9EbfiKoplHoDhqf2VWMB+LuIdg9vKimTIfHZRBimo+JsCTkjjX3MFTL5lxZnzUqAwy+tr4/AXm5RsqHqYobkzp2QYCKtntjchi7ZAU8YVp96JHTsJa9VD5W0yqBE+LvyiYINJYIQqTXBMTOolPsF6iL0d2PgcwL365JcoRp4euvaCSu8fBpssh3DSVjfReTHvI8rfP3HiEvZiHun9dnIbsnA3bns5hqSZzPE7UET1bKF2tNrNvmAp85XSZ1c0M+77f+a8pBEmwjKRgz3RBYca+cQ1SyHOgjkXGqv0lTiNNEkq4VTfDxaTpHqwgPs9ZWDJGCYsTbNdJVTJndtaO4BTvs78olJLnx5tenWEifwldbEhYlf/n5ym0p3NUqQRBCD/3k2D+WRvLh3rfojutA4EuF/k5uMWpvJxZOB/zFg0MkgXqXoyhchsap6a0GNwrk4TzKVN+9BWPckJAX+jo/GzRAxRdFd0cNNuiKvWVJVAT9U3uYUCoHXFXZmwK5SrohnMkBkj9qGX9vXOAxOM4fsuD+u8CVa+Fm8e75dHKE6pibagFU3SfbS7tgMBrOeitdDBuU/iLyWpaT1s+oe4KGLBB17xcxxwguSnEKjSzjSoDhZn/CFkMN1DgjLkY5An3d5RCFY+JA92yCaqzLjJcAlNk= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BN7PR04MB4388.namprd04.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(366004)(5660300002)(2906002)(86362001)(66476007)(83380400001)(66946007)(66556008)(6916009)(75432002)(36756003)(508600001)(8676002)(31686004)(186003)(6486002)(31696002)(38100700002)(8936002)(316002)(2616005)(53546011)(43740500002)(45980500001); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?Windows-1252?Q?2IjBxGs4MRNRwE+bK6koT6fmBfDhYAYRauETJarnMMY94o+19iY5R794?= =?Windows-1252?Q?jFcSjWmj6IigGgVluPeTUlyHBoCGLQHQJSiwbTs2QKE7e7SVTNP1wFtS?= =?Windows-1252?Q?MqmPDOVHl94rfXeXIExcojKnqhYBh1qqqxM4atnFb6m/BBJmvje4xF3w?= =?Windows-1252?Q?bJqrN2st/tBWQHG0XmQ4P+sqW8QLrMezQtViguw6oYxcVk2FnTGBMBIt?= =?Windows-1252?Q?8kTNYLePttI6jHHB0kyGQWbu6ncETdq12DvFy58epTHzJ3UJdvqIBAu/?= =?Windows-1252?Q?HCD2ntNqu28IhcKTY1c/HxHP7WVPJ58URRX1vbkzWzpwF2aArU89OOLG?= =?Windows-1252?Q?4wHbUsNpoObGiS9J4Yz+eAk1Rs4u4fMJ02Aj6lJkHsEXsPg9/0fPxWaE?= =?Windows-1252?Q?czSAtdLU1S1oTiOl3PZHlXyjSxbeX+nJNNYGoxK2CnPIPjF9FIF2c2um?= =?Windows-1252?Q?pZk78sIQEYlnXDTSmVEkLVEgC8vQZyQdTglT1NxoagEsW2uOAvmX0cLF?= =?Windows-1252?Q?pshtaY25hnQW50I0vJx9wsFoxZ0Cch9etW6IWKTZBYi9IAJMwUtP16XV?= =?Windows-1252?Q?N3N6umeoF4hexjCKhr2/CtUYbQB2+cfF4dKaCugANinhET0py7Xdyl+Q?= =?Windows-1252?Q?sEw7yuIoguQkVcjndq1NIB6UwjNHCkdyEaqg2LfGpuOi3slQ9/YqzF6y?= =?Windows-1252?Q?jb3OfSgKxK/p2mnP+1jC18Pj9nE7wUTgzic7pbZSyyee5zHSQQL3AINC?= =?Windows-1252?Q?NsgCQZHlMqFtCmd89LQ+jaR0GyFOAvsbyKAFTlUiDFo4ik6E5ARFM6bl?= =?Windows-1252?Q?Ih6cHJLHZ0RA4nwVFPkg2XZnmnhvDHggoTqO1Nbl/FmTzaIBsCbqOZn4?= =?Windows-1252?Q?gKLumrPyTb1LZqsREU9wDwjCksWUA3uwdq+FoAlvX4YE+sNyX8SLepxn?= =?Windows-1252?Q?+XoHLgT2xDVWdEwwZrD0SvsucssN3BNQPC7fX6ZrSvxZtcq16J7JhfT1?= =?Windows-1252?Q?OopaOkoo404dJRABdvVNKW8NZm2ul1fBGWxFGtHeUXesF+WKNDjSuE+F?= =?Windows-1252?Q?1ZgmbrU5OnGM+AfuPh5tHrk0rfpG2xJkRuBn2XhBwsWpXN8RA1q9kf/n?= =?Windows-1252?Q?w8PsxjKG7HQ9n2elhA3X+/92wuFMeVeR6ESO18lFqWZV3RP6wTpF9WM3?= =?Windows-1252?Q?mQ+ngahkWQRYm4wiK9WhHL2Vrkyi0BczIJAJ2MN0ltbQS5DIHnrafeZB?= =?Windows-1252?Q?CNXvMSrm5Rr3wLF4sSGDNUl12mnuGtp9fhx7tfJK89n1BA1dgc4DP2pG?= =?Windows-1252?Q?5Z7K1IERoTk2zBpUpFzJy5YuhU+W/6m3mOFKgVk0HOpHkUVgMgVYQyBp?= =?Windows-1252?Q?qmcWy5WuHY6pv6S6p+s8HLD32o8q6zqpR7wAvzW/LvLlZOxDpR8WqQrL?= =?Windows-1252?Q?hqFuVJaAJgD9Ng2TukLPKk8PehHKjFPK8HCfDUvd7QyNFed2qcqH1MUI?= =?Windows-1252?Q?0T07dN/0?= X-OriginatorOrg: cornell.edu X-MS-Exchange-CrossTenant-Network-Message-Id: 577b6bec-ffd2-4de6-8543-08d97c6ad2f9 X-MS-Exchange-CrossTenant-AuthSource: BN7PR04MB4388.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Sep 2021 19:14:09.9600 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 5d7e4366-1b9b-45cf-8e79-b14b27df46e1 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: M5nZnB+2SiqkOksPJ0qlPTm7FAQFwA3o2bQ2xhd024Chunm32dOzkTqgsozKwBvUFXBiWlaS9meg8pnQ+Q4XAQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR04MB0244 X-Spam-Status: No, score=-3.1 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, MSGID_FROM_MTA_HEADER, NICE_REPLY_A, SPF_HELO_PASS, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: cygwin-developers@cygwin.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Cygwin core component developers mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Sep 2021 19:14:17 -0000 Hi Takashi, On 9/20/2021 8:52 AM, Takashi Yano wrote: > On Mon, 13 Sep 2021 11:07:42 +0200 > Corinna Vinschen wrote: >> On Sep 11 11:35, Takashi Yano wrote: >>> Keeping read handle in write pipe (Corinna's query_hdl) causes problem >>> that write side cannot detect close on read side. >>> Is it possible to open read handle temporally when pipe_data_available() >>> is called? >> >> 1. You would have to know which process keeps the other side of the pipe. >> 2. You would have to have the permission to open the other process to >> duplicate the pipe into your own process >> 3. You would have to know the HANDLE value of the read side of your pipe >> in that other process. >> >> Point 1 is (kind of) doable using GetNamedPipeClientProcessId or >> GetNamedPipeServerProcessId. ZIt's not clear how reliable these >> functions are, given that both pipe sides are created by the same >> process and then usually inherited by two child processes communicating >> over that pipe. >> >> Point 2 is most of the time the case, especially when talking with >> native processes. >> >> Point 3 requires some sort of IPC. >> >> Having said that, I think this is too complicated. > > In the current implementation, there is a potential risk that > write side fails to detect the closure of read side if more than > one writers exist and one of them is non-cygwin process. > > Therefore, I had tried to implement the above idea (tentative > query handle). Finally, I could convince myself to some extent, > so I attach a patch for that. > > As for point 1 and 3, I used NtQuerySystemInformation() with > SystemHandleInformation. I also used NtQueryObject() with > ObjectNameInformation in order to know which handle is the > other side of the pipe. As for point 2, it is not possible to > open the process which is running as a service, so the current > query_hdl is used for such process as an exception. > > Ken, WDYT of this implementation? This looks like a great idea! I'll review it in detail later today or tomorrow. Thanks. Ken