From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2133.outbound.protection.outlook.com [40.107.220.133]) by sourceware.org (Postfix) with ESMTPS id 66AAC3858411 for ; Wed, 1 Sep 2021 12:56:09 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 66AAC3858411 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=cornell.edu Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=cornell.edu ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=RMsGgS7tVmXcW3XZ/1gfX4Q4u4rWJp12JMQZUKsOnmubDRN+WyJOWRaBGtLbqVkzjtzXVejt0ClkBY0ae/japIfViK599P2jnAi/e0U68YrkCfqr2TfMmyXrFEb69oCU0tikDJjQiTglnvkCgovGPHauBVXA12+WHz9yZia6zfsDgKK9PiIYCDGPldzJxo1ZNnRbXKjyEjbOXn3fg/MO7kTsqPJSdGJUQ1ebyoRN74+tObefz27Hx/i6QGAwq6DDSf5vb06mU28wubp6M0sS1t7toYeZRTNtXtzdfCioHqcz/hoAvdMJgDhTfPVhyMrksxOCnH99GTxhwNNE0iLByg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=sfcSz/Gium4c0ttMp9QewK/RekBZZqDV2taAxtsj6lk=; b=UXzPdPNAfoOE4MnqqTgf8+3vReIV02z5hW4f7PAlJEHpdcad7khrsCdsYX7RwMSbQspy552AzamURUKdlOhK2eCaYn2khvu50+ie9XoCqYiMHv58lgF3eigqgumqsNUzIz1C0w7qjxGcvXDd+p1KMEoqvZkgI+2Cg5loPSJLrlcAvKGZPWpK68aSaVP4RqLGj0R0h3DGsxzyiPbnPLQDPqrc3d1Fkln+0dVC6Wv0YJSZWaBHT1+fmcEInIHqUHbbAGwzTWrKqaB2pRv5sn0uRyukshVZRR2pjl3Me7Nqn1gGYsOr580+fY0I5EultqKateY5RIUV/tIhMlzgQI4V3g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=cornell.edu; dmarc=pass action=none header.from=cornell.edu; dkim=pass header.d=cornell.edu; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cornell.edu; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=sfcSz/Gium4c0ttMp9QewK/RekBZZqDV2taAxtsj6lk=; b=V91wGXBQVUsP8h5heNbuOeTkaaOK7LJEDDRmRqwRw7WGL5jdKl5StLwGlM4kf7HaWK2m1Md3E1Iw/J4GhQXhElsddlzQt6tfKxyy9QUNCPQxkwi22e+YdaI3LhxglG+Z4J6YgzOKwA6c5A9gKjUbWl+U2GG2CvNQjpEC30L9eHY= Authentication-Results: cygwin.com; dkim=none (message not signed) header.d=none;cygwin.com; dmarc=none action=none header.from=cornell.edu; Received: from BN7PR04MB4388.namprd04.prod.outlook.com (2603:10b6:406:f8::19) by BN8PR04MB5825.namprd04.prod.outlook.com (2603:10b6:408:a5::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.23; Wed, 1 Sep 2021 12:56:07 +0000 Received: from BN7PR04MB4388.namprd04.prod.outlook.com ([fe80::b510:889b:1fd0:d80e]) by BN7PR04MB4388.namprd04.prod.outlook.com ([fe80::b510:889b:1fd0:d80e%7]) with mapi id 15.20.4478.020; Wed, 1 Sep 2021 12:56:06 +0000 Subject: Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? To: cygwin-developers@cygwin.com References: <20210831204541.2b56702cdcd2fae5e91ba8e2@nifty.ne.jp> <583ca127-02e7-6b3c-3732-6478c0f862e3@cornell.edu> <20210901080220.ee4a5bfbea62cc1ae0a9598e@nifty.ne.jp> <20210901091652.6bf3cccbcaed4a22f6ffa6b0@nifty.ne.jp> <20210901172339.1039604b7067e0492534a20f@nifty.ne.jp> From: Ken Brown Message-ID: <24138e20-aa97-cfea-bf48-198fc67755ea@cornell.edu> Date: Wed, 1 Sep 2021 08:56:05 -0400 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.13.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-ClientProxiedBy: CH0PR03CA0187.namprd03.prod.outlook.com (2603:10b6:610:e4::12) To BN7PR04MB4388.namprd04.prod.outlook.com (2603:10b6:406:f8::19) MIME-Version: 1.0 Received: from [IPv6:2603:7081:7e3f:3419:b8a4:79e4:4201:c4c9] (2603:7081:7e3f:3419:b8a4:79e4:4201:c4c9) by CH0PR03CA0187.namprd03.prod.outlook.com (2603:10b6:610:e4::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.17 via Frontend Transport; Wed, 1 Sep 2021 12:56:06 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: c45efda0-a77b-4f53-eadc-08d96d47dced X-MS-TrafficTypeDiagnostic: BN8PR04MB5825: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:7691; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: aEkyD9Lkq0Akufu3Iz59YM9HEtNUuMKR6ql+xzIzC3OGaAnCn1Kmm8aoS5rX6VhWNPhm2EFHqTjLU6OEFGnZ6Z/4ly3rDk5DA+MmzTrQINVZMszlltO0yAqcIkFwgAACsO4piL/ycvheNCc4ELmIh2Y02yNTyLsXLxecyJNFKSWaTQKSgv27F6BRjGW+7fmCXguSFVKty/ZD5pPd6ok+c0PCKl/voRqPtK1SCvLorOJW05Y9RmOGFzvqQHzU8G12wimJfPUeahryDDBtSmBOGq33WLudVI36AHUPQrpazADbhSosid+IzrpmklNBcbjYQNuXpluogO/v0VXemgXrY+4SQDCm8/6Q/JLRs76Gf2WqxtBT7zF27V1kNtVJQbymMrY+8ERr7R14fXsFXSGpuAvwgi7kq/ro39Xl3rAzk5QosdGAV2UTcQwDTlL4ws2shdWI7/HG+haGBwZT6r7R7nMrt57F99vp+BD5UrbOfNJxAT5gsMxi/EfAcPpFvPW79f6Z8onvpQuC2R7CBG7ItqF5/GreWUQCpeSIxGk7QHDGEuD+nPTpA2JlBaIjpcTOsf9q2qmh7Xuhx7Eo9/z7rmAJiWIGTVJqPVQdxeMmI4V9r8ohA/LvXASdAaBQMn1dbw4WIaY9rQtjlw8QWz4tcyQvcj7ZBmsR2wya09pYttvx3jTVdTAN48zQwVXhSgYX18qLAwYdVRvgX55p1XgfDulBQKi8T/0+zDVZohUP1jY= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BN7PR04MB4388.namprd04.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(136003)(376002)(396003)(366004)(39860400002)(346002)(31686004)(8936002)(36756003)(6916009)(316002)(66476007)(186003)(83380400001)(8676002)(2906002)(75432002)(31696002)(38100700002)(478600001)(53546011)(5660300002)(6486002)(2616005)(86362001)(66946007)(66556008)(43740500002)(45980500001); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?Windows-1252?Q?1qUU+Pjanyc10l7K4eKpVDQihH10h/9kKNLAWSbNC7pC2mL23m22RyK6?= =?Windows-1252?Q?nEotAKfj59mPd5M0f8EEsteQFjcWKWZdffOTSawntNKw8tvK67sPS5Rk?= =?Windows-1252?Q?Z+SyC2G2y7E2nZqWMxE24dgac6GpJsn05i7wLDWwLXoI2Nr87SoTdgM5?= =?Windows-1252?Q?edlQ35rjEU4VXzt1URb2lNO1Wx4UFBIYkLICC8X+eJND+oIT9p9sU20N?= =?Windows-1252?Q?MGOSUtSQojGg+RE2rs/Oj+gZPPcVm2HqpG25x1CzXO+4Et2o+iMNG8dA?= =?Windows-1252?Q?ag5UQsGnMHitt4tPTY/8w4PLgUgTXGzVujiNxQPhM9Nbw/OQDumV8zs7?= =?Windows-1252?Q?tWlMl8t0MazRxrQTnYf+pz4B/8tfV+uIiDl+jfiuycaRzEtw6NG6FB+1?= =?Windows-1252?Q?DswMfxvWFGVMIo0lFBZx8bILFgBM1XTQMccwg55fKNW1I3ePkBw7yggN?= =?Windows-1252?Q?lrUnGbi5bNt61rELI5gwZSQ1P/siqLTpSX4073WsvBl+Y2zZUaSX+8FF?= =?Windows-1252?Q?zX4wQWcg70LJ0G4oHdM7m2hs1cEC2DoPMBew4hCGCCLXoKr+fg6m8tV7?= =?Windows-1252?Q?YAJoMCnuxMSTkJZtD/MkhtcrG60FJgfA7/H1obkKyJG4S/dHdQv/kANA?= =?Windows-1252?Q?BIr+HYLPZ0Dtr3HgwwYNIXleAwHwSazNzwrrbESBmkBiZyCixTgxhuKD?= =?Windows-1252?Q?DEfZ7g9wQXwgjSUN+Qizdzy4v1D5HGv3FCaZ6geo9RjFa7Oj7y/wD/cs?= =?Windows-1252?Q?9otufh3rD7lW4snwAIOcZraOIhrQ0+RIbsNkhqq32JM7snfYZHvaP9l9?= =?Windows-1252?Q?yciRYVnp14TxObfu0c7ANLNogAcKPuncIyE8jFKLsflMO1qonwbPOIlg?= =?Windows-1252?Q?jsOrJRR1dZ9WqfVMBDDqKl+ZtGJxzKaeZ2BiLC3EKEU1OkJ1pmeMGV4x?= =?Windows-1252?Q?4EaXUovawlOcB/jL5jq0W22TSsqICmv2uk/ReAT5vkEmdNhlOiO0w7Zh?= =?Windows-1252?Q?WmgQAVFYv7uF8w3Ki5d6+kS+QBDPPX5ZyIyQ+CdvVBXh+SrQJXZbZUBj?= =?Windows-1252?Q?nGrKf2RoFlkKHI2Q6WRczOhGBt+GWFmctzPhaqFUHYx72qByVBRcDgaG?= =?Windows-1252?Q?hajgxQNfbt38K6le6zb2WGSwcD+edMMrOM7GaefUa/kTl5R4brDeNpCe?= =?Windows-1252?Q?C3u4D2AipZmeE0BZtQd/nbch+8DV7CxfLyKwe9ps6QOJNbpK4px9LzPt?= =?Windows-1252?Q?n+m66SbPeXhixFuVphFcNoMg50BI53IO6X2YudM/LhOAbYjyIo4fuCIn?= =?Windows-1252?Q?DQjDQ0XQ4R3oNKN0sweUg3EtlHWiJqsEdJMtWHvkazO/43YhhfITUH1o?= =?Windows-1252?Q?k1hQzY9kEDuXY2Z9pRT9fWabPffGBqj8cDND4+/meMyLgDdpP/bR6JNy?= =?Windows-1252?Q?K8mzggink3SpGEtjxZlV/RWkJoIjYuxicyv4iJ9yh1NlQ1vUwdK2+FMy?= =?Windows-1252?Q?mmGRVH3M?= X-OriginatorOrg: cornell.edu X-MS-Exchange-CrossTenant-Network-Message-Id: c45efda0-a77b-4f53-eadc-08d96d47dced X-MS-Exchange-CrossTenant-AuthSource: BN7PR04MB4388.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Sep 2021 12:56:06.8160 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 5d7e4366-1b9b-45cf-8e79-b14b27df46e1 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: EVNdTNXVvE4VAK+GdTHUTFlowXBuhdLrK5GEVoUg470wVCT611f/8BS+xDSkGvQ0qvoDwGMeMIzj6ndvX+iy9Q== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN8PR04MB5825 X-Spam-Status: No, score=-3.7 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, MSGID_FROM_MTA_HEADER, NICE_REPLY_A, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_PASS, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: cygwin-developers@cygwin.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Cygwin core component developers mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Sep 2021 12:56:20 -0000 On 9/1/2021 4:46 AM, Corinna Vinschen wrote: > On Sep 1 17:23, Takashi Yano wrote: >> On Wed, 1 Sep 2021 10:07:48 +0200 >> Corinna Vinschen wrote: >>> On Sep 1 09:16, Takashi Yano wrote: >>>> On Wed, 1 Sep 2021 08:02:20 +0900 >>>> Takashi Yano wrote: >>>>> On Tue, 31 Aug 2021 17:50:14 +0200 >>>>> Corinna Vinschen wrote: >>>>>> So for the time being I suggest the below patch on top of topic/pipe. >>>>>> It contains everything we discussed so far. >>>>> >>>>> One more thing. 'git log' cannot stop normally with 'q' with your patch. >>>> >>>> The same happes with 'yes |less'. >>>> >>>> The cause is that write side cannot detect closing read side because >>>> query_hdl (read handle) is still opened. >>> >>> Oh >>> >>> my >>> >>> god. >>> >>> >>> That kills the entire idea of keeping the read handle :( >> >> One idea is: >> >> Count read handle and write handle opned using NtQueryObject(). >> If the numbers of opened handle are equal each other, only >> the write side (pair of write handle and query_hdl) is alive. >> In this case, write() returns error. >> If read side is alive, number of read handles is greater than >> number of write handles. > > Interesting idea. But where do you do the count? The event object > will not get signalled, so WFMO will not return when performing a > blocking write. What if we create an event that we signal every time a reader closes, and we add that to the events that WFMO is waiting for? If this doesn't work for some reason, a different (but more complicated) idea is to keep a count of the number of open readers in shared memory. When this is 0, write returns an error. I'm thinking of shared memory as in topic/af_unix (which I copied in the fifo implementation), but maybe something simpler would work since we only have a single variable to keep track of. Ken