From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2134.outbound.protection.outlook.com [40.107.236.134]) by sourceware.org (Postfix) with ESMTPS id 48F0E385840C for ; Tue, 31 Aug 2021 16:38:13 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 48F0E385840C Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=cornell.edu Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=cornell.edu ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=fOeHFM7+hHoYi8OkHfdMciloKiseCZSyJ0APt0O4w9HVB0t12hZjCUoZqekrtPatYmsqFJWXu3fAyNAdMEZE1rRVnYed4tyk7In9lMCCtkHsTcMLVOKbGezMt9VdzcqM3wW1uqXJKnOWoFE7JT25HH0Lm7U52B52M4cwHWflK4dOIfeLj0MgT+q0Hq0pNldb4gj/HYaT/eTqUbERNRddl46jqCCcDKBhvRqcEu3B+jqRQnOdav4MO5PnJz/stOBgcOH9oBE/4b92VJgMFC/XS7UsWOV1wQpHpXEIUm2kZGYXQelrWIaME89syvs80XuMQR86KluIwJtkYqfpv/TzrQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=NwqTn5x+1Ji0YZoJuB+aSHr8H8KBLeSPrqZ3aaG/IPQ=; b=knrId4H/M+Sk8nsG3yKF0RhdMKz7O0WJqxl3oQAYaq+ZZcoI6aykW9IwH0Nysp3Qf339nT6fOj9/eZFgciz9K9HtJy6Ii9mKA/nT9PbpS9KEsX2FWqNYV3SdOxh6q+TntCRnzUKmSS1+BcrAWsBQiYYNwhWBcqtPDKVwBnVCfWHjBBL/4yC9KIn86lIHMnWC3X0LLS1n8Pahat1EhauB2rDZo+Em5ax8ARvxityIolbLkIdY9idmJzMI7agBqL7JOdS7p/sxY6sPFxl0avn3qCJquDaZ8qoN7qiP8CoElH2aTCP2d08L880Rj3OUxdkm2cnUxlgYVWasFlwe6kE+3A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=cornell.edu; dmarc=pass action=none header.from=cornell.edu; dkim=pass header.d=cornell.edu; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cornell.edu; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=NwqTn5x+1Ji0YZoJuB+aSHr8H8KBLeSPrqZ3aaG/IPQ=; b=FdXx56mIYshEbyS1JPmsSlJ9exOOwzlGUU6IP8l+/LgHd1v1v/8+vevRZ2uC2qXkWfzqkYxdqe/LXZCuY7+aEfgosvRlAw6UkX5Yrzb4MRmJV6KUNeno63MK6xGZ5GZDAw1Hd84WNSInZc8x+BQ7J+977D0o3VNDg2Ys3mIbm78= Authentication-Results: cygwin.com; dkim=none (message not signed) header.d=none;cygwin.com; dmarc=none action=none header.from=cornell.edu; Received: from BN7PR04MB4388.namprd04.prod.outlook.com (2603:10b6:406:f8::19) by BN6PR04MB0866.namprd04.prod.outlook.com (2603:10b6:405:44::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.24; Tue, 31 Aug 2021 16:38:12 +0000 Received: from BN7PR04MB4388.namprd04.prod.outlook.com ([fe80::38bd:b608:234f:9ec6]) by BN7PR04MB4388.namprd04.prod.outlook.com ([fe80::38bd:b608:234f:9ec6%7]) with mapi id 15.20.4457.024; Tue, 31 Aug 2021 16:38:12 +0000 Subject: Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? To: cygwin-developers@cygwin.com References: <20210831175534.21edae2356d74a0750c686de@nifty.ne.jp> <20210831182500.4c1c67dae13f51ca68964f63@nifty.ne.jp> <20210831204541.2b56702cdcd2fae5e91ba8e2@nifty.ne.jp> <583ca127-02e7-6b3c-3732-6478c0f862e3@cornell.edu> <6dc7d21d-05c2-5cb2-f04c-935153d73214@cornell.edu> From: Ken Brown Message-ID: Date: Tue, 31 Aug 2021 12:38:09 -0400 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.13.0 In-Reply-To: <6dc7d21d-05c2-5cb2-f04c-935153d73214@cornell.edu> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-ClientProxiedBy: CH0PR08CA0009.namprd08.prod.outlook.com (2603:10b6:610:33::14) To BN7PR04MB4388.namprd04.prod.outlook.com (2603:10b6:406:f8::19) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from [IPv6:2603:7081:7e3f:3419:14b8:c8c6:fea8:df06] (2603:7081:7e3f:3419:14b8:c8c6:fea8:df06) by CH0PR08CA0009.namprd08.prod.outlook.com (2603:10b6:610:33::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.17 via Frontend Transport; Tue, 31 Aug 2021 16:38:11 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 708f4aae-3f02-4316-e20d-08d96c9db8f9 X-MS-TrafficTypeDiagnostic: BN6PR04MB0866: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:10000; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: wXjOTw49qGRGCOi1HCiRFuJ9mqdA0btuwhpPHgvZsCJIfzUCuGhb9oVwboTp/BFwaaQ554z4xAaTb+CC/PFgFCbADdVRpdG5YmppHRLY6B3FVmOBaNMRIdOs2YSq/GdrllITats1CtK0mIYKHGSSrk5QIYyBpa4UX3pDKi4eyOfoKn4Ux+WdhBe4qugCDY69O4LYB0Dwav69mrB711hfdU4diMRE/I5XLvOpKxsF1Fax4SVsf1Gbbn0S8gbiSx0BTbnyzqHt+T5Sj96MfDW74NNiO4Znnj6+4WxsnvMFq4qJf2VcOyvCujjS8qOynm95pQ3YY70CNml30MuysJGHQVMvKpfYRzcDQrpjl57YDkj8kCnffULQTexSPeV0LjgeraM1LuV6tpxe48A/nlxrUkiC417JNsUsXxAJHbpNKt7kPQG1g7N+DzWMM4VTL6RT4WbgEDhDZJoySecdClJNAKK64kjuk6CQqTOBV9Aw2ykd0dxA8Iin4NLJH+Vv1YWH4rpG4gMKJmSgcGRxatBVKXfHWQEr6yfpjh9nlsc13cvsN74CXNLKXCHPFzDnIYFKITfL5m4gFrkdkeuIw/IPpjucmgpH8pqjF98x+03LYiYi1XW1YrxL6a1GuhjZn7IyDI222wqKe9eHOqxNoE3oTF47x0YqTkTB8Clt3PGPXtcjOL3opitacsnwgdihxbIfv9dObyoungXAt4wh/3xZqsxvw2hl/tCptjsg2ylTTBY= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BN7PR04MB4388.namprd04.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(136003)(366004)(396003)(39860400002)(376002)(346002)(2616005)(31696002)(53546011)(66946007)(75432002)(66476007)(8936002)(83380400001)(66556008)(6486002)(38100700002)(6916009)(2906002)(316002)(5660300002)(86362001)(478600001)(186003)(8676002)(36756003)(786003)(31686004)(45980500001)(43740500002); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?Windows-1252?Q?ZVfTz0QqlzMbOpCJ92TEeaVfiopTpUQ+ebhsi0iD2m7zmXabQVyXISnS?= =?Windows-1252?Q?/eL1UzZKVc/YG3VGSaQvVcxDRVvkb2ZVZ4djzp98DpYPi164CgfPbxm0?= =?Windows-1252?Q?uv5i+1u1/4F3wIhfuxQVUIX4tZMGMqZnf+L2GYpcsaw398LTtJv3AMOE?= =?Windows-1252?Q?hC0J1ESxQAKfs6h+PRpGagYdqGDs9CTNO4EFWHtuVk3AHe4i0uHxkewQ?= =?Windows-1252?Q?cLsbzFBIFer34u9B5MbK+3PHTpHZTQ712Bg5uYTphu5XQTM+FzpT6gDo?= =?Windows-1252?Q?YCV+V5y7DIpZbiBYnTp7A22OScqkNOrA/uzuIIlkeiYoQVmcLRJGClum?= =?Windows-1252?Q?yZnztjB7l7oAZ5JIG7obgdAZtH3vbMBfrCp+ajnceNM6SFsGX4nNToMc?= =?Windows-1252?Q?rVChuT9AwAU7l1bFoCcpLg21HAMN9pvnegaPa5wdFWRaHRVxM5YZ26OJ?= =?Windows-1252?Q?SKDqnrsRC+Dp6KwUTIl0FO/986F/suW0AnZc0ERFlhb/csZoGHn4Jftv?= =?Windows-1252?Q?2qZ/GXnGAciOG3sF16LoGdHwOmZEKNmFZWoJOYlQQxsr0EqckB9FCVUc?= =?Windows-1252?Q?3E7ShBa2mkvD9rOwIn0BEGB96BC1/riOh84BCfOnzy39H5stdAeqkVlH?= =?Windows-1252?Q?Dqw/OEMR+AxMUhB/5kL+4HwvnFWOr+P3xLHJO8DuNiux5QoVQG8+4KCH?= =?Windows-1252?Q?CnLNgmCdisJywdBo64n3SD5RMl0vW3SF00MvnkWCHSAtM98ti1C/JxVE?= =?Windows-1252?Q?+NENkl+oQYtgpebPmJKCGpM1XK479oDAJrOt/CmmurDC1eG2xmOdbEr/?= =?Windows-1252?Q?ns7T2Ccz4sZcIK75gn0kSIMhTV3mwD0PtXoP+9IwBA4PEA0/SPE9XdsL?= =?Windows-1252?Q?fAgZvE2/a/NWy//KHfF86/fnefeoDpCAAlQY0dhzf84K81PeqxkacO/b?= =?Windows-1252?Q?Gf8xwXoMxcNxOOfHcxmVaEx/GrPJDeRrKBMJQ5ZEKhJXZ8INihFFWiQB?= =?Windows-1252?Q?V3zkFWOhbUw9h4lNRrPbtcTB4+bD6fbZhgLrubY7Zi0mTDegWMF1MeIh?= =?Windows-1252?Q?EqDBxKvJq72jaJ3OkW6SBqqISqj2I34Qios9bUw/Zhu0YvltPbzCnFSM?= =?Windows-1252?Q?yBl2oeQjG04dDglPWTaoDnFeQd1zHe9ofQn8lQDdORkB4+bQgjTZEe8V?= =?Windows-1252?Q?asZq08O0hGEyLbCCRxd/uHz0N5ST87otrYQyvMLE9mhxU9M29ML4pNn0?= =?Windows-1252?Q?Vs7VrQr+C/UMF+/yPpiGEFYHwMQeG9UwYloNC5eB5ltVxwjAaZ3piyor?= =?Windows-1252?Q?g4t7o7FGBlEOEPFFQ3+KQgs9u9qowAuILMsjnJRQysxkhdGrrk4RxfzL?= =?Windows-1252?Q?o/rUk8oEXvMg6Pctp5Rp/HC52PcCyPGCs2+owSvj+MQHeEWB+7fh+dWQ?= =?Windows-1252?Q?ezo/RVHHugx21PjeuVDV1UlPM74X3kfYLyKeucjBZfpP4ldse8/o5VSQ?= =?Windows-1252?Q?Si8OwVHq?= X-OriginatorOrg: cornell.edu X-MS-Exchange-CrossTenant-Network-Message-Id: 708f4aae-3f02-4316-e20d-08d96c9db8f9 X-MS-Exchange-CrossTenant-AuthSource: BN7PR04MB4388.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 31 Aug 2021 16:38:12.0244 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 5d7e4366-1b9b-45cf-8e79-b14b27df46e1 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Ez89Yyw8oMNWjnh0Z8sA2a9e1+2jOpP1DDCkp4LmehyhSV2s/PrCMeo9JauxxYkt6zLkWh9UovQmESUkaF+tMw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR04MB0866 X-Spam-Status: No, score=-0.8 required=5.0 tests=BAYES_00, DKIM_INVALID, DKIM_SIGNED, KAM_DMARC_STATUS, MSGID_FROM_MTA_HEADER, NICE_REPLY_A, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_PASS, SPF_PASS, TXREP autolearn=no autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: cygwin-developers@cygwin.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Cygwin core component developers mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Aug 2021 16:38:23 -0000 On 8/31/2021 12:19 PM, Ken Brown wrote: > On 8/31/2021 11:50 AM, Corinna Vinschen wrote: >> On Aug 31 17:27, Corinna Vinschen wrote: >>> On Aug 31 17:18, Corinna Vinschen wrote: >>>> On Aug 31 08:33, Ken Brown wrote: >>>>> On 8/31/2021 7:45 AM, Takashi Yano wrote: >>>>>> On Tue, 31 Aug 2021 12:18:57 +0200 >>>>>> Corinna Vinschen wrote: >>>>>>> Please try the attached patch on top of topic/pipe. >>>>>> >>>>>> Thanks for the new patch. I have confirmed that above issue >>>>>> is fixed and select() for write pipe seems to work as expected. >>>>>> >>>>>> >>>>>> BTW, I found one minor difference between Linux and this pipe >>>>>> implementation. >>>>>> [...] >>>>>> Is this difficult to be fixed? >>>>> Two other remarks: >>>>> >>>>> 1. I think query_hdl needs to be initialized in the fhandler_pipe constructor. >>>> >>>> No, that's not necessary.  The fhandlers are always ccalloc'ed so they >>>> are all 0 anyway. >>>> >>>>> 2. When the read side of the pipe is non-blocking, there can be no pending >>>>> reads, so shouldn't we be able to use WriteQuotaAvailable reliably on the >>>>> write side?  (I can't test this at the moment.) >>>> >>>> In theory, yes, but is it a safe bet that non-blocking reads won't change >>>> WriteQuotaAvailable on the write side, at least for a very short time? >>>> The question is, of course, if that really makes much of a difference. >>> >>> Oh, btw... why do you want to use WriteQuotaAvailable for normal >>> pipes, even though the read side information is available anyway? >>> >>> We can do that for fifos, no problem, but it doesn't make much sense >>> to differ between blocking and non-blocking pipes, the code flow is the >>> same. > > Agreed.  It was mainly the fifo case that I was concerned about, and your way of > handling that is much better than what I suggested.  [I also wondered about the > pty case, but I see you've dealt with that in your latest patch.] > >> So for the time being I suggest the below patch on top of topic/pipe. >> It contains everything we discussed so far. >> >> One question left is, do we want to switch to FILE_PIPE_BYTE_STREAM_TYPE >> entirely for pipes?  I don't see that it's still necessary to use >> FILE_PIPE_MESSAGE_TYPE for pipes.  Everything seems to work normally >> with byte-type pipes. > > Since no one remembers why we're defaulting to FILE_PIPE_MESSAGE_TYPE, I agree. >  If a problem shows up, we can always rethink it.  I suggest that we still > retain the CYGWIN option, at least for a while, in case someone encounters a > problem and wants to switch back to message type for testing. > > I'm afraid I still haven't had a chance to do any testing of your patch, but I > expect to be able to do that later today. And here's a really trivial comment about your patch to raw_write: Where you have len1 = fpli.InboundQuota - fpli.ReadDataAvailable; I think the code would be slightly clearer if you wrote len1 = fpli.WriteQuotaAvailable; Ken