From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM04-BN8-obe.outbound.protection.outlook.com (mail-bn8nam08on2096.outbound.protection.outlook.com [40.107.100.96]) by sourceware.org (Postfix) with ESMTPS id 4E5A03858420 for ; Tue, 31 Aug 2021 16:19:42 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 4E5A03858420 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=cornell.edu Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=cornell.edu ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=KlrxMKh/PUDeSB4HyRGF2zNo3rx7vgh33iP7C4OF5IyVEtNvMdSy+ASp8NLliipKb8AJUS6NsayLfbQNlDoQ2cvxPUJuBYgw2dr4c+bFWC11HVnDFm7pBQjvMzzPyS6hev69CQSbTdHJXYIkhawqGeIipSAHcmmgmxTcuM8fVVhaIvOauDGz28NbfJfh2/62uNaezRpezqUmeedYHBavbdApq53gtuB8ltSf0JV+mgFvD/dm7Ogzy57qUj7fJ1UXpaZFnYPN+jnW2RZkkXI1+YQP2kYcIBUiJACFWVRyHWzApH4BkmeQ6ZVatskhWnJMaEZBuPi6t0aQIXi+mHBU8w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=mQ7Bi40Lc5ZGckwrb82zYvzONtIPhGabFl9VcYthnz0=; b=JX3kJHcC+i2bVVwkSKoaz+eJEJ0l1y+0mSbMzGX9tZ21bC26JjY4NUr9wy7PqfaGSr5wvkKaIDnqfChMRkWgMomKL1DgzosWJS5gFCFarS4IyNpsi7NgSnRYrnO6Dj9BJqVIJrFEU69m8Uo/kfbpJ21uIFVjLhRMYTm9mIgDNoA49OnDUMM26m9N6F5WGKA6KKfuaYWXdzQZSqPhmlfOjkOozao15FSxRo7rs+FS7scm5oxP41ZWWqUWCKQB7cN53XLXZ8bGtoKqD89eIywjKUu8foMuSl32r96tmUlm2yDcce9s33tFOqheY6LGa8ROar5TxDlsD0dL/5GvDlkGkw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=cornell.edu; dmarc=pass action=none header.from=cornell.edu; dkim=pass header.d=cornell.edu; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cornell.edu; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=mQ7Bi40Lc5ZGckwrb82zYvzONtIPhGabFl9VcYthnz0=; b=ZrZbVFts8lTQQgVNYDCpN/DWNLlzqp0xXGeon8MF/wVXOmGMDYemeFVqjsAGXUFmWadlrF5oCoztF1AmBlXTXGQSaQoDT9HpgDh2QOqO5IkUvqnxbEDcquOt+jK0RDah6yHkzR8Nd56qEwBCsa7MDWMJYlzcFuJ8nT9MyeL9a1s= Authentication-Results: cygwin.com; dkim=none (message not signed) header.d=none;cygwin.com; dmarc=none action=none header.from=cornell.edu; Received: from BN7PR04MB4388.namprd04.prod.outlook.com (2603:10b6:406:f8::19) by BN8PR04MB5713.namprd04.prod.outlook.com (2603:10b6:408:76::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.24; Tue, 31 Aug 2021 16:19:41 +0000 Received: from BN7PR04MB4388.namprd04.prod.outlook.com ([fe80::38bd:b608:234f:9ec6]) by BN7PR04MB4388.namprd04.prod.outlook.com ([fe80::38bd:b608:234f:9ec6%7]) with mapi id 15.20.4457.024; Tue, 31 Aug 2021 16:19:40 +0000 Subject: Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? To: cygwin-developers@cygwin.com References: <20210831175534.21edae2356d74a0750c686de@nifty.ne.jp> <20210831182500.4c1c67dae13f51ca68964f63@nifty.ne.jp> <20210831204541.2b56702cdcd2fae5e91ba8e2@nifty.ne.jp> <583ca127-02e7-6b3c-3732-6478c0f862e3@cornell.edu> From: Ken Brown Message-ID: <6dc7d21d-05c2-5cb2-f04c-935153d73214@cornell.edu> Date: Tue, 31 Aug 2021 12:19:39 -0400 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.13.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-ClientProxiedBy: BL1PR13CA0063.namprd13.prod.outlook.com (2603:10b6:208:2b8::8) To BN7PR04MB4388.namprd04.prod.outlook.com (2603:10b6:406:f8::19) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from [IPv6:2603:7081:7e3f:3419:14b8:c8c6:fea8:df06] (2603:7081:7e3f:3419:14b8:c8c6:fea8:df06) by BL1PR13CA0063.namprd13.prod.outlook.com (2603:10b6:208:2b8::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.6 via Frontend Transport; Tue, 31 Aug 2021 16:19:40 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: d2735696-895e-4915-8b53-08d96c9b227c X-MS-TrafficTypeDiagnostic: BN8PR04MB5713: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:10000; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: k2zObPqUPUwXonVrB20fdadRrS9Fp3BZkuwQH+3rcZir+6X5rtO+8GvLzVkjh29mA6f5vEH9HrX/XTxcLPGrVqSYSFmBujUEoU9grY+AEPaAGdJEpoH6DBbr3/0DCVVISNJ+z4dpOp7flSk/HX/Q91krz39jj7OtTAgxHFEyhL0CTvUPlcjUuycE29mFCfWMbj8cEJj0RG7MyqgkkOzh3jOJnNC8SwMR/c3lMtQylf5x4vmxGCIU9/iBmRBBjQOKnq36cjJNRLmRRygazdggBHkdwOgPtsJUM7e4UdMFabO0ObfRDg7J4JTplOXw64gkTAU8b2T6K8ZLjinIjHqZL8C5VIgl0AAxfhUy+nnjUchhIDqAFKBJJLaFvhalzQ/0v8XPx6RtusKDsvWs/cgMrKosP0dD+NJmIOELszs4K0dZiynb/5bGHXURayhHo1XsZzUWhn69bzm64Y6PjILags76d5yqBTyJb4mpnXKyQNtO4+AgMCSNdggFhu42ZpOW1YWW/pS+Nq80nW29UEomS0nBhjhN9sJFkjHhc3ixaE0mC2aoyK8kEyKFWe2Dbwss/9YmWmcLTRwp1V9yzh3GxcjfJZuzDfW2iDAL7RK2Ym9jmAm2xRcoYlCp6NIK1qzSQBRyEjUASzLk6RRWo3hAQ0KS/KLWzDTQn+j0hhySWnozWIArOT81+VyCoZSS4wUuNvW/GBPNFuTZDohpBZaqTvqrnLB6jlsfj1UbgiKVQo0= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BN7PR04MB4388.namprd04.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(396003)(136003)(366004)(39860400002)(346002)(376002)(38100700002)(316002)(5660300002)(31696002)(66556008)(53546011)(86362001)(8676002)(478600001)(75432002)(6486002)(786003)(66946007)(8936002)(6916009)(2616005)(2906002)(83380400001)(36756003)(31686004)(66476007)(186003)(43740500002)(45980500001); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?Windows-1252?Q?jINg5OyK/OxX2nJlpWM2ZUYJTTgiKWRFgnkZM1a6eo+EezQSh2cvzSw8?= =?Windows-1252?Q?eMSJGwW6gDWfyIS9iwkw4zGZjhF9eOM9BIY6R588Fu2KcOAJiacP2mxC?= =?Windows-1252?Q?UnFk2tSa8JbhBthUd7M8AzhtQKXicv1WFsSsBSjO8QmWSxulYklJmoJO?= =?Windows-1252?Q?pM8jlCQePlno0ULW/ow+DDB7W1y/FayLGQloxbtYoxpAjIBukNvSVciF?= =?Windows-1252?Q?nNK6nIdIwJY7jMD4O+LW/OmzRBYJI7mJpcZFod5WK+SbyIg1moRnilA0?= =?Windows-1252?Q?iu8FTZQUAdJ63P/MizkfyBk42paM/UNm+VFyqjLbrRDqqv+0/6G7j5et?= =?Windows-1252?Q?boOfxgjlDx2Zx0LFJJNnMS2xUSLG+8ivGen8kVcLhyRuMQ/Bp1b2GdU6?= =?Windows-1252?Q?sLmaRaQLLbD5ra6HrErqTzdOv/dSipp4p4qTqkXgySa0JenqBKcIOWom?= =?Windows-1252?Q?6nKYRJgBJgMANxDw1Q971p9PBNcSpAuo6E6++8pLeJcl1GF6ljznoO8b?= =?Windows-1252?Q?vTcxOOq/smaF4rufgicd0uTnnADB/GAeBoK00F3MXDtABLrtgAiKL6cy?= =?Windows-1252?Q?DkGhA9n7lMzpp5mL1yBOuRfVn36bA9nl0p4V6SRemdBUYhXXAi0NvsOc?= =?Windows-1252?Q?A4//ThDG5u18adP1mvzsTpVRObqcMDB0jjVCdnoVTl22FGGd5odDZWKr?= =?Windows-1252?Q?ta74eIWtMMEb3dNWqCH4YMh2lnlGPRchTItA4SGlBgIOhWgan8EitLfs?= =?Windows-1252?Q?Bmiw2SsT0YNizdwQHkfbJnIkhcs2OaUJa9S++dyvA01LfsoCzER9muYi?= =?Windows-1252?Q?7CTdBqjN7mFifNB0975PAfmRtf/4NFoMm3eByy2jrL2R7JzuAyWDoqc6?= =?Windows-1252?Q?8tKxjK9B+VHf52wffrJG77aReLfBlj4Y03r2pJ8WRshQ5OMLCc0Kqv3J?= =?Windows-1252?Q?0/mpbKunPk9mkycLQuaY5ELoVHwiAYqV7VxMH36D2AoSsmO1u/LmdEH4?= =?Windows-1252?Q?p3V6ny0M9/84KM5evYPFbbgQn5x84y1TYIQRnLXa6WzPOOTvURhHWb5h?= =?Windows-1252?Q?uZ/NhaugY25x0FRJXcFXWy/FuDbzXAitAR35ZQZurkhUx8SC7t2vRUAX?= =?Windows-1252?Q?/ucS7EK6umaYM0w+VHdbq7eKLDjYUFucWJCjco/9eN064jvjuJ/roSW0?= =?Windows-1252?Q?6zfnsdTt1zlXbXtF1QvQvKAhzA6oUxAsWIV0vL6pZ7AWFfMQXSg0uqW6?= =?Windows-1252?Q?7qBEIogF+5oWX0pqt1ELddwgrSh6lfEs6jfkr+CwBom6gFrBl8MiEv+n?= =?Windows-1252?Q?zyOmBgpE9D77zJ9HsPaKK20OA6htpGoZaPrGLIA88fFz2AdBF1M9oZEP?= =?Windows-1252?Q?XjnxiJPrftjw1uKza5q4oy9oMrWCnoC1lxYNKqghYAWEL9uZNGNkpxHN?= =?Windows-1252?Q?dUVHO4f89/2qkpAktFNk76LwbvnlZskEp7lnhgf0hezn4YIigSSVJRXF?= =?Windows-1252?Q?S4era6v6?= X-OriginatorOrg: cornell.edu X-MS-Exchange-CrossTenant-Network-Message-Id: d2735696-895e-4915-8b53-08d96c9b227c X-MS-Exchange-CrossTenant-AuthSource: BN7PR04MB4388.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 31 Aug 2021 16:19:40.6082 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 5d7e4366-1b9b-45cf-8e79-b14b27df46e1 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: /DunQwq4MfHG+tUj6Emfon6ZKNIb+pNL6BG+EoIS/9wDkx9ms/Vzn590Q/9hnG//XagRkvyZza/IB2BGA7QmTQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN8PR04MB5713 X-Spam-Status: No, score=-3.6 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, MSGID_FROM_MTA_HEADER, NICE_REPLY_A, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_PASS, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: cygwin-developers@cygwin.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Cygwin core component developers mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Aug 2021 16:19:52 -0000 On 8/31/2021 11:50 AM, Corinna Vinschen wrote: > On Aug 31 17:27, Corinna Vinschen wrote: >> On Aug 31 17:18, Corinna Vinschen wrote: >>> On Aug 31 08:33, Ken Brown wrote: >>>> On 8/31/2021 7:45 AM, Takashi Yano wrote: >>>>> On Tue, 31 Aug 2021 12:18:57 +0200 >>>>> Corinna Vinschen wrote: >>>>>> Please try the attached patch on top of topic/pipe. >>>>> >>>>> Thanks for the new patch. I have confirmed that above issue >>>>> is fixed and select() for write pipe seems to work as expected. >>>>> >>>>> >>>>> BTW, I found one minor difference between Linux and this pipe >>>>> implementation. >>>>> [...] >>>>> Is this difficult to be fixed? >>>> Two other remarks: >>>> >>>> 1. I think query_hdl needs to be initialized in the fhandler_pipe constructor. >>> >>> No, that's not necessary. The fhandlers are always ccalloc'ed so they >>> are all 0 anyway. >>> >>>> 2. When the read side of the pipe is non-blocking, there can be no pending >>>> reads, so shouldn't we be able to use WriteQuotaAvailable reliably on the >>>> write side? (I can't test this at the moment.) >>> >>> In theory, yes, but is it a safe bet that non-blocking reads won't change >>> WriteQuotaAvailable on the write side, at least for a very short time? >>> The question is, of course, if that really makes much of a difference. >> >> Oh, btw... why do you want to use WriteQuotaAvailable for normal >> pipes, even though the read side information is available anyway? >> >> We can do that for fifos, no problem, but it doesn't make much sense >> to differ between blocking and non-blocking pipes, the code flow is the >> same. Agreed. It was mainly the fifo case that I was concerned about, and your way of handling that is much better than what I suggested. [I also wondered about the pty case, but I see you've dealt with that in your latest patch.] > So for the time being I suggest the below patch on top of topic/pipe. > It contains everything we discussed so far. > > One question left is, do we want to switch to FILE_PIPE_BYTE_STREAM_TYPE > entirely for pipes? I don't see that it's still necessary to use > FILE_PIPE_MESSAGE_TYPE for pipes. Everything seems to work normally > with byte-type pipes. Since no one remembers why we're defaulting to FILE_PIPE_MESSAGE_TYPE, I agree. If a problem shows up, we can always rethink it. I suggest that we still retain the CYGWIN option, at least for a while, in case someone encounters a problem and wants to switch back to message type for testing. I'm afraid I still haven't had a chance to do any testing of your patch, but I expect to be able to do that later today. Ken