From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2137.outbound.protection.outlook.com [40.107.220.137]) by sourceware.org (Postfix) with ESMTPS id 63C4F3858D3C for ; Mon, 20 Sep 2021 21:21:16 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 63C4F3858D3C Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=cornell.edu Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=cornell.edu ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=c7YdTRe778Iwm85R6BVhit1v5n4Fwwlit1Lq1OF8bhdK3DLJERh/gJKlLOcCL0zEBHvs5gwDIwu2v1W55GZHfy6WbqQGLm9lcItfwGKpSfRtn2BmWdKSI0o+eucMqtPrycJ3I5eG4vPSLBAUUA5eaH+qsqfUhuS11yC0Cr5zsJ3jF9zYORN8sx49u+OOIWR/q2IjYhRcbKja/+BlgFtAYo3fMtfuQqnIwR9bWFMNZWnN9hRbNk4v978gVQPEAai8meb2lkmdBAKtVnDy6Eah9gzd5D2S2UNw9xTP4ICxdmkhBTk1L+ezwm7YEM1N9breDLocF+06aViCNJfmD4cxcw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=wt0SD7G7cds5tZXEcZ826787x9eajIytUcdcU2DOjOE=; b=QsM37vnTcfWXGh1P/EaXUwPTRhQNsbEpKPnU24YHB8IcC2UaWuJL71AB5uxCTky1lH065t96KoISxHRWXFD2CT19b0KQHqiRiFjBv8q/CMD4r8jnmY0icUpUijukYbS+sFwg0wacjsTXntDqh2JfcNUVoxhguBkYcjQmwAqz2suK5K7B0fPYhCUxPY+2e97R9NhR5ioTqi80F6+7L4QP9+76hHTW9MBIT4r1nIhtmurKjoyuV29lYRDj0tXzaoo6SwDR+BAx+m80zZmS7ylZ7k+RKH43yTyH8GgXCa3rCuKKDrucmcrhfseR9iHGzDKEZuknAZQiE8r47slDqBcgEg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=cornell.edu; dmarc=pass action=none header.from=cornell.edu; dkim=pass header.d=cornell.edu; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cornell.edu; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=wt0SD7G7cds5tZXEcZ826787x9eajIytUcdcU2DOjOE=; b=NReqD/Az8S69Ya4f3n7RZMJ0ggO8ai8YbDI9UXvnVeaUZI7t4MDr5CbpB1z6FcNhMJSOhDEnZg8w3EGdJsmBgvQDEVA8yHhzhBF/xt6ISKKesGVhWd9aDZ2euW/9jXOFwlon44fQBay4G7Gax0oubvW/K2OrIgnd1KccGAC++Ls= Authentication-Results: cygwin.com; dkim=none (message not signed) header.d=none;cygwin.com; dmarc=none action=none header.from=cornell.edu; Received: from BN7PR04MB4388.namprd04.prod.outlook.com (2603:10b6:406:f8::19) by BN6PR04MB0532.namprd04.prod.outlook.com (2603:10b6:404:91::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.16; Mon, 20 Sep 2021 21:21:14 +0000 Received: from BN7PR04MB4388.namprd04.prod.outlook.com ([fe80::5113:e84a:b38a:7a66]) by BN7PR04MB4388.namprd04.prod.outlook.com ([fe80::5113:e84a:b38a:7a66%6]) with mapi id 15.20.4523.018; Mon, 20 Sep 2021 21:21:14 +0000 Subject: Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? To: cygwin-developers@cygwin.com References: <20210907122631.65452be8d021ec72259431d5@nifty.ne.jp> <20210909124115.555c6be15d675500617d284a@nifty.ne.jp> <20210909170549.506cc3c1f6029d904fece6dd@nifty.ne.jp> <20210909211940.51ef391e27d43f0421962cb8@nifty.ne.jp> <20210909214246.cd1ff1a3062fea27e51ad4ae@nifty.ne.jp> <33386baf-3b2d-d57f-2ad3-1bd328ed7935@cornell.edu> <20210911075734.aaf37697ba7db2ad14d911a3@nifty.ne.jp> <20210911113517.f74fc3ac1971bbf04c7a9bd1@nifty.ne.jp> <20210920215229.cea5722b2c2fcd7e03917108@nifty.ne.jp> <00f4eca0-2b38-f4bb-98f2-e99e7ee941f5@cornell.edu> From: Ken Brown Message-ID: Date: Mon, 20 Sep 2021 17:21:12 -0400 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.14.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-ClientProxiedBy: CH0PR03CA0331.namprd03.prod.outlook.com (2603:10b6:610:11a::22) To BN7PR04MB4388.namprd04.prod.outlook.com (2603:10b6:406:f8::19) MIME-Version: 1.0 Received: from [IPv6:2603:7081:7e3f:3419:d571:af58:205e:a58] (2603:7081:7e3f:3419:d571:af58:205e:a58) by CH0PR03CA0331.namprd03.prod.outlook.com (2603:10b6:610:11a::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14 via Frontend Transport; Mon, 20 Sep 2021 21:21:13 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 8ef08e95-1f29-4ae6-6d84-08d97c7c935d X-MS-TrafficTypeDiagnostic: BN6PR04MB0532: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:632; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: guR4gHAkL47RfOmTocAxHBaqDRxPTbfXjLEoBbpDg7tE+nLnELbk4eysuEbdcta5frV9mYMhqBtG4uiaoYH8tPLPMcMg99hYhuujON/fj/Ls+gOtnEoo1TjUSe91Cbc8HHBJ2PgI5hXNbFMw4A2Il8Q30wlW48YikB4vUG6qo0Pifr2CQAqXktRxK8CI98Y0rEXCp9QdeW7te/Z7uZe5HLVn47XOdwpROcr0t96lTlzsgtvYcmz1ml5KDYW7+gKUJrqLPhxrM/NoSlEYtkrqI73AfHXVu7fxW8HsMHoUSzluvyuOW0LI5WVY/u17F/8SfOeFHmnm1XJUiMwyxuNCvZRMDHtAQsyS2h2qEEDkaTO1Td1UyfAegSv19dXKSm1IDb6So2KLh6+gu1yfzG6ff3pWHFbavmSNago5R7VADrJH6H7KgN89CN4FC+JrqRZFqGR+Oah0sSWOcnHD2nvbdKgSTTm++Vyr8V6MDTmLZiaEpIlN0vmzc0VDvNStF9HN4ErYwyKW+o6gXorOPGI3q81ckgRsEyfiMjp+ORFCCl8EDA2GlAP5+YQ1u4+byiubeLagiIqcqYtlWqGt0MBwmDf1GuFIV1OZie9W7VLliSUrn8hrEuNuIn8d+Ze/k/XdEu6OAg1VOzMroarwCcXyxIxaoNlPh8lBBLJl0uDwY3KnBZAhjuOH8eGfOIVkVcwEChmOhHCzyIrlpJWJfnr4Vmvh4NJZsB8CJcOfUTAYSls= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BN7PR04MB4388.namprd04.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(366004)(38100700002)(6916009)(53546011)(5660300002)(86362001)(2616005)(508600001)(36756003)(31696002)(186003)(6486002)(2906002)(66946007)(66556008)(66476007)(83380400001)(8936002)(316002)(8676002)(31686004)(75432002)(43740500002)(45980500001); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?Windows-1252?Q?jc5lP6fBMfLnIEhW2QEa49tudl7bcnILaQV22UQZ1Z5yfqDnH4yemSYZ?= =?Windows-1252?Q?I68/1sCThPO8hpZyYbsgGlYawp1C6NsCnijFKmuQazrm9UFHoRdJKqXN?= =?Windows-1252?Q?UC4dGmwaX1xDh1nJFy2dSNFKutTP8qZScAt9aSm8idoFyA1Turzd1eCU?= =?Windows-1252?Q?JNRHWF+KosSs34SPP6qh2u1c7FfQpoIF5fir/mcOrPPI9SugbuCtFuUr?= =?Windows-1252?Q?DGBTBouW2mj9AvrJXUGil+Sc+gWmuvejLqyJn02GN87jsFp8Jv15ym9F?= =?Windows-1252?Q?BNm4V4RCqjvacG1EaTBKlOC/0b0BjqdpJPhg3OYEFbU7TD6LBOA48NZw?= =?Windows-1252?Q?LuddR4JTHJu3YYZw6wCfQkWcj6iVVmKQa3lbxYe01E19trIVj9f8CPIm?= =?Windows-1252?Q?RxzLiBOI+TlrCiKKbnP+P+E9opn6XIQDh/ceb6umiDWYJ+xXrdGfgjA5?= =?Windows-1252?Q?Ew71F4sQO3bTcajl+Gk+0LGTRWcUQuG/wqE/kFPLk+gIyHsYCYzi5T8O?= =?Windows-1252?Q?wkdSUuMy0VtWosU9+ltwb2klX1bkMfDsHzYjMOyCMD4PLo2J/idkpFq7?= =?Windows-1252?Q?612WvekFmzJs1ggHEFZ1+iydgDJ8Lmio3+qdQk8SD29KciH4ZKaMTtgS?= =?Windows-1252?Q?MIJRQdvWdi7nSp1p1pTuhtWxKqnm5ExlcpGkMO7kI8Km9DXL0zKDPdBw?= =?Windows-1252?Q?pnh1lCBPqlJ7SBJZmfIJrDIoTy5kORWw2gnF5tgWkmXAQSbW0Fwb2KG7?= =?Windows-1252?Q?yweHHnX+hnGfz7jlkB0vC8j+7Q97CfGTN5goYyyBCyyff6vXD9q1F1fs?= =?Windows-1252?Q?UrVrYkPEMoL3L2NvQBV+Xr77szMcMBh4IgWDpV5+aYGA8o5Xp8Uz7Fv4?= =?Windows-1252?Q?UEZ/ZgeksGbFETnQR9KMmRPE2Uts3CQ3arVOpeGEdi3w4NGo9AMNvPN9?= =?Windows-1252?Q?xKOD6OA8DXVT4OoHsuMQLr1Y8TCWmj8TG/lOkftbVb0uy/Wp2j5az7eY?= =?Windows-1252?Q?YJO3PhbV+s3lgUoDqETkjInApvar1tAo2Oo84e7poGquy74jErPQjgAO?= =?Windows-1252?Q?UenbG+Xt65vBrWQQtw/2pHt8JlHpedSuVs2KQMK2p9TpNVUt7bug0I7F?= =?Windows-1252?Q?WxojG3FhsoOMbeGLnTv7gK4njd/hKg3bSjCmU/YGK3T4S8KJacY7U627?= =?Windows-1252?Q?hoPIiEOYB720ebfAnP/58dXORZgeARt0/GQTptio8U5GtEf+tqEuzlqU?= =?Windows-1252?Q?v4gdBfzzTkPtvBe0hgo23w51HORzSfCdu5DKV6eig3yZbSf1o+PVKPvV?= =?Windows-1252?Q?QysS6rkNZl8TsA73C2n1+tFcbTfroSjufTYLMyefmhqaXWp8XqHMEVrX?= =?Windows-1252?Q?ZRlKVM+w7M677SPJRno+NB+TgW0VmO/vpUM0iS3HKxLKVir9ofn2GmSP?= =?Windows-1252?Q?0+J+ebSPQcdDFEt61IJpnCayOprG7XXeGtkagZ3kBK6JQnSrfRkp/Rpt?= =?Windows-1252?Q?Fy/036dG?= X-OriginatorOrg: cornell.edu X-MS-Exchange-CrossTenant-Network-Message-Id: 8ef08e95-1f29-4ae6-6d84-08d97c7c935d X-MS-Exchange-CrossTenant-AuthSource: BN7PR04MB4388.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Sep 2021 21:21:14.1657 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 5d7e4366-1b9b-45cf-8e79-b14b27df46e1 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 8AgsLMkClTOOmDXFiivfqNuwAg6/FsLU6a4zRQQKXZjc6Wu7s4MebYiPU9pjFKnFzSHQhtVOQQQo+L0WtXXGmg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR04MB0532 X-Spam-Status: No, score=-0.2 required=5.0 tests=BAYES_00, DKIM_INVALID, DKIM_SIGNED, KAM_DMARC_STATUS, MSGID_FROM_MTA_HEADER, NICE_REPLY_A, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_PASS, SPF_PASS, TXREP autolearn=no autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: cygwin-developers@cygwin.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Cygwin core component developers mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Sep 2021 21:21:18 -0000 On 9/20/2021 5:09 PM, Ken Brown wrote: > On 9/20/2021 3:14 PM, Ken Brown wrote: >> Hi Takashi, >> >> On 9/20/2021 8:52 AM, Takashi Yano wrote: >>> On Mon, 13 Sep 2021 11:07:42 +0200 >>> Corinna Vinschen wrote: >>>> On Sep 11 11:35, Takashi Yano wrote: >>>>> Keeping read handle in write pipe (Corinna's query_hdl) causes problem >>>>> that write side cannot detect close on read side. >>>>> Is it possible to open read handle temporally when pipe_data_available() >>>>> is called? >>>> >>>> 1. You would have to know which process keeps the other side of the pipe. >>>> 2. You would have to have the permission to open the other process to >>>>     duplicate the pipe into your own process >>>> 3. You would have to know the HANDLE value of the read side of your pipe >>>>     in that other process. >>>> >>>> Point 1 is (kind of) doable using GetNamedPipeClientProcessId or >>>> GetNamedPipeServerProcessId.  ZIt's not clear how reliable these >>>> functions are, given that both pipe sides are created by the same >>>> process and then usually inherited by two child processes communicating >>>> over that pipe. >>>> >>>> Point 2 is most of the time the case, especially when talking with >>>> native processes. >>>> >>>> Point 3 requires some sort of IPC. >>>> >>>> Having said that, I think this is too complicated. >>> >>> In the current implementation, there is a potential risk that >>> write side fails to detect the closure of read side if more than >>> one writers exist and one of them is non-cygwin process. >>> >>> Therefore, I had tried to implement the above idea (tentative >>> query handle). Finally, I could convince myself to some extent, >>> so I attach a patch for that. >>> >>> As for point 1 and 3, I used NtQuerySystemInformation() with >>> SystemHandleInformation. I also used NtQueryObject() with >>> ObjectNameInformation in order to know which handle is the >>> other side of the pipe. As for point 2, it is not possible to >>> open the process which is running as a service, so the current >>> query_hdl is used for such process as an exception. >>> >>> Ken, WDYT of this implementation? >> >> This looks like a great idea!  I'll review it in detail later today or tomorrow. > > I only saw one thing that doesn't look right to me: > >> +      /* Check object name */ >> +      status = NtQueryObject (h, ObjectNameInformation, ntfn, 65536, &len); >> +      if (!NT_SUCCESS (status)) >> +    goto close_handle; >> +      ntfn->Name.Buffer[ntfn->Name.Length / sizeof (WCHAR)] = L'\0'; >> +      if (wcscmp (name, ntfn->Name.Buffer) == 0) >> +    { >> +      query_hdl_proc = proc; >> +      query_hdl_value = (HANDLE)(intptr_t) shi->Handles[i].HandleValue; >> +      qh = h; >> +      break; >> +    } >> +close_handle: >> +      CloseHandle (h); >> +close_proc: >> +      CloseHandle (proc); > > Doesn't this mean that query_hdl_proc is not a valid handle any more?  So the > attempt to reduce overhead at the beginning of tentative_query_hdl() will never > work. Never mind. I see my stupid mistake. > Other than that, I still think it looks great.  But it's complicated enough that > I think Corinna should review it too when she returns.  (It also uses Windows > functions that I have no experience with.)  Nevertheless, I'm tempted to push it > so that it can get testing, even if Corinna changes it or reverts it later. > > Ken