From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on2106.outbound.protection.outlook.com [40.107.223.106]) by sourceware.org (Postfix) with ESMTPS id 7868A3858C27 for ; Tue, 31 Aug 2021 18:54:36 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 7868A3858C27 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=cornell.edu Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=cornell.edu ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=JUoiOtMk5/1TVNh0ISOSJ6GA1WK0SWZ+OiwS9/1tn7QZ03XW7Gg1+KVhi8+8TxAg2qLkhLq/6Ha8IwA4WQGPZP1zv23Fuk6CfP2sNYH3mq3LOyXBVFozJV7xVOHSoyGELq2bA0viRroNZU0UYMzVYS9JIM2J4Pm2r8Wwtnte2+cgkkB3y5feTmQm9PgYpGp/fb9AGxEoehFumATEyE1u6+DcrxHlHq5msHk4Xt4sNIxujnBfwrO+Oi7cLfPEuewIdLjutwuoYettMWonMJIMH1og8hVTZq2blHT+LFhto689QNdMuBfRptrKqaCnVf+YUTztP5a6OGao+vtW2gSUSA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=yKISt/b4u2verWw3q3n8sNwFKZCknyRD4MwzEF7+XyM=; b=nQ7YXYCG77KWb9fNQ7naxEVRh4sq4QbtCDrwAETr3/pOxe1gG2blCN6EWOlJUjZuwKmaIBbhgfFo8C2z65YkAtQsmzSZFO9Y8MzUZfeMks2E/VADN5yJ2flwG0Y/1UX/zN46Hmb1CC6/90GB5Ph3a6HaEILfEvaHDjC74VVxrmdfEhJmvjEI8+pZqQ93JwCtNRvqrW39o/TdBoKc6+stdBWMpM6Ib0cwPDZE0nrtIAaSuSFouDXqSm/T7GiuAY1wTUTfy9USAcRB1T0XDxtgrYtDOfRCLNFLnUfyN2CpbxfsBbtyf5Z6x7Agz15GpcAGkBTrFsIIsyYyxAu6RaMmrQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=cornell.edu; dmarc=pass action=none header.from=cornell.edu; dkim=pass header.d=cornell.edu; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cornell.edu; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=yKISt/b4u2verWw3q3n8sNwFKZCknyRD4MwzEF7+XyM=; b=NMzA15XKEDihv8MLEtUdhmD1wBHYr6iUWDepO7kA/oIbZg5OUks2KbQLsj68JQ3WP7gim/yOSLiOqOhAC4gcltrZtmjHJ0kSnUJ82aywHlfTvQ3dh1b72ZXzWH7ia5NaiWBwWl9GnUNj3S8V/ozTDWQOsi2N/SHe7o3v7pqonL8= Authentication-Results: cygwin.com; dkim=none (message not signed) header.d=none;cygwin.com; dmarc=none action=none header.from=cornell.edu; Received: from BN7PR04MB4388.namprd04.prod.outlook.com (2603:10b6:406:f8::19) by BN8PR04MB5713.namprd04.prod.outlook.com (2603:10b6:408:76::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.24; Tue, 31 Aug 2021 18:54:35 +0000 Received: from BN7PR04MB4388.namprd04.prod.outlook.com ([fe80::38bd:b608:234f:9ec6]) by BN7PR04MB4388.namprd04.prod.outlook.com ([fe80::38bd:b608:234f:9ec6%7]) with mapi id 15.20.4457.024; Tue, 31 Aug 2021 18:54:34 +0000 Subject: Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? To: cygwin-developers@cygwin.com References: <20210831182500.4c1c67dae13f51ca68964f63@nifty.ne.jp> <20210831204541.2b56702cdcd2fae5e91ba8e2@nifty.ne.jp> <583ca127-02e7-6b3c-3732-6478c0f862e3@cornell.edu> <6dc7d21d-05c2-5cb2-f04c-935153d73214@cornell.edu> From: Ken Brown Message-ID: <0b2f7e89-5013-d99c-5425-925f98fc214f@cornell.edu> Date: Tue, 31 Aug 2021 14:54:33 -0400 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.13.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-ClientProxiedBy: CH2PR05CA0005.namprd05.prod.outlook.com (2603:10b6:610::18) To BN7PR04MB4388.namprd04.prod.outlook.com (2603:10b6:406:f8::19) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from [IPv6:2603:7081:7e3f:3419:4c8e:6050:e136:61a5] (2603:7081:7e3f:3419:4c8e:6050:e136:61a5) by CH2PR05CA0005.namprd05.prod.outlook.com (2603:10b6:610::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.14 via Frontend Transport; Tue, 31 Aug 2021 18:54:34 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: d1def710-37bb-4ca4-a73e-08d96cb0c646 X-MS-TrafficTypeDiagnostic: BN8PR04MB5713: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:9508; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: sJeqd0ujGCVcdXfbooZeRyNBxVJ3ZniWfv7nwtaZ8nTOZ8bsAfjUe9xYY0QT4o+XP2QyO6ZHSbX5JyRzfQkFuGv66zbUXUOTgjAPs1qtSHGvUdICK83LepyAVKnO+jbJo+JzITa8AbjwwmYAA/j4j0YvsUTyPskd1Ob35e7DiDlcC/4Dq4A4LfI5bcRGn4baa13inaetxDU8tuDUgJvNY79vBFHPYw4saS6Tk0ox7/flJmn+cRLjQK3fmXVMJ6g+nTNBufVVph0MqRrX3TZYnybE8LtBzhPzTuQHpiN+7dXK2Jlm2yIkfF6pyuFMDDz5cymykjueDuy6JqxwaDAUaowFKwl4W5peDIABRXaevY494N3BlE+VOLS2eoeiW6idgp0ZuiWEHHTveMb9QAIfd8A6NDrE1xR/9a81I4PKcphaVDdK1Pm6WBLC9uLSofEThnvXG7QAQ/smuV0S94PW901scnmNT3+aJxDpxQcmONOGbamyB0XQ1sQ2YhGcn8Frs/e52V1w2zuzpQQQqZeswoeKzHGDi+Q13T6qJ3BCAove4XiDzAimsL2hxqFDoN1mN6ykTzc52LgK+icJJ6PAcxQdCAWQzytz+HaJ8ZFE1t2tKrlFn1vAh/mSskZM34E6epYJFFBBkAv4pIgDRiDR+ShO44he+AvCIZcWIdWPvwW6LpDhn+UsLteqazsk2X3k6hE21cFKsjElQBoXHMDcnCgK2MzdmId3kLxnJO+2J21BvgwNOx95LDiH55akqIY9tVz+cl05j+U5XKUpkR6fzj0nPi+0Lou/XH8mX4xTWKKbysFee3sgqEMYltu8RJfk X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BN7PR04MB4388.namprd04.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(396003)(376002)(366004)(136003)(346002)(39860400002)(38100700002)(316002)(31696002)(5660300002)(966005)(66556008)(86362001)(53546011)(478600001)(8676002)(75432002)(6486002)(786003)(66946007)(6916009)(8936002)(2616005)(2906002)(83380400001)(66476007)(36756003)(31686004)(186003)(43740500002)(45980500001); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?Windows-1252?Q?bFqLR9MCMVDPkhM/IWgyCLEEvoKmi/cvCSR3Kkh7ATIq3cXIfKc3tM4v?= =?Windows-1252?Q?Vy/nRdHiPxu9sTyZdjAzN6EUxPDvA+ieURLPB2S7C1Il8AOpMvGD0p9R?= =?Windows-1252?Q?XslsEg9wXysNKg+LLGl7yM0rZ58wj6ovFiyLTBsfpcKtEVAulJoghsFB?= =?Windows-1252?Q?YYuLULsOtmdxTlxPAqh8qhCbvPAGjLchN5UmWeSltGARcwh2mQs8uYa7?= =?Windows-1252?Q?4D6HDQWuUyZgvMKO5yIt5HDtdT8QjUGwpRU2yhsVu/fqdsj58G2cP1ST?= =?Windows-1252?Q?KoweaC1FjQ8Uhz5pQazFR5b01wezyj4+rUjnK8ueE4suI7qHX9vDY4yW?= =?Windows-1252?Q?V8s6jBLEARRMSBCKnuVeak3DOJ767tlcqIVsgmWdUnJ2vauTTAzEO64A?= =?Windows-1252?Q?97WvzHTc980wn3y3jVuf5sCKd/x0DlZZNumLMTximxrOHwV85WCGjZPm?= =?Windows-1252?Q?QObtaMU0CWK0Amf9VLHiY5Lfr0MuTLHUtBUNHFJbva4JD9Thpivw47mI?= =?Windows-1252?Q?xBxfMuMrWc26qOdlJk6cTmswDKjtmk5PuRzd7uJEVFcmeWSfuLbOvEpP?= =?Windows-1252?Q?wa/m7TeNv5R6B13Jm75g+31hVCG+GajreO/Em879vvk9sckNE6xnCQjr?= =?Windows-1252?Q?CqjHgvccXDySViKPdkiaBNHHbKBHOlLK74piIPwZEb08q5jAMoCbF6If?= =?Windows-1252?Q?AdaXALIqzza5HKPWj2UgWEVuHEpSlzZuCrz2yQVVRA+XVaFHWvNc/dyy?= =?Windows-1252?Q?lk2DJ+q1Uqgq1BSE+sIqYqrBUMhgORSsrUY+Zjm++7WlmtA24RMyhn49?= =?Windows-1252?Q?ocsCUUKqQMYUI0KW2caa8q2U4TjopbwyLVUNkX/QfdBjVtk/b10+oyur?= =?Windows-1252?Q?ZnWlmYgPflkc7JUaGnIAGC5T+yqp/4YLZYivf4bp2bQtaV/UI2eepSWa?= =?Windows-1252?Q?173lLlgaFU+HZPEiTIlPvPSTCxiyejob0g2Ry3Oi/FsAZ0z3kG7snwu3?= =?Windows-1252?Q?wLT5Fbr6Rcf8cnpL1LooXvBrRiJv2i43zQ6eG3hBP3/fLdWrthv7CT2C?= =?Windows-1252?Q?brWdCqIKx6n/G9yxdph6oSYhkSV6r86HBU5CGZaxGrhmLgD5sIDLTQLu?= =?Windows-1252?Q?nrBalqlZE31LO3p5xiR28yATbJMVUgQ/YXsXM6DsBSMbuYJSXnIPv7uv?= =?Windows-1252?Q?O3Uu18PWbh+6cRocLrrp3gKOHfRt2ifUmeuR5LKpKmTGJp9ugMOY/Qff?= =?Windows-1252?Q?VdLTODJUIegWLXNzhijT8VBvwEQwfUo3ar27usPIeKI3eskufYlFcJ7A?= =?Windows-1252?Q?7SozgHXn13kR8vcgD5B7OzdaSaZgxA5cTivlQovZi5b8iWxc7O6iG0De?= =?Windows-1252?Q?2AOVMzQSPP1+yix1DyC9IRe/ha/rB9/FgI/tIjrAO7QAy0nqaG3ig/n+?= =?Windows-1252?Q?TNznw3vzx2hO4QXyxwVJmFuXgvvQ4rokTGPtNvCfFttJJdaJmmSBhR/3?= =?Windows-1252?Q?QEaQF6pZ?= X-OriginatorOrg: cornell.edu X-MS-Exchange-CrossTenant-Network-Message-Id: d1def710-37bb-4ca4-a73e-08d96cb0c646 X-MS-Exchange-CrossTenant-AuthSource: BN7PR04MB4388.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 31 Aug 2021 18:54:34.8557 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 5d7e4366-1b9b-45cf-8e79-b14b27df46e1 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: r+UYcQ7jYawTIl63L87oXCyKKNldAUTCtJesDGvdHIljnl9+vV+uYTk8/Qfhg/jZfNYp31M9uQ60X3wQzDaS0Q== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN8PR04MB5713 X-Spam-Status: No, score=-3.3 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, JMQ_SPF_NEUTRAL, MSGID_FROM_MTA_HEADER, NICE_REPLY_A, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_PASS, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: cygwin-developers@cygwin.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Cygwin core component developers mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Aug 2021 18:54:47 -0000 On 8/31/2021 1:30 PM, Corinna Vinschen wrote: > On Aug 31 12:38, Ken Brown wrote: >> And here's a really trivial comment about your patch to raw_write: Where you have >> >> len1 = fpli.InboundQuota - fpli.ReadDataAvailable; >> >> I think the code would be slightly clearer if you wrote >> >> len1 = fpli.WriteQuotaAvailable; > > D'oh! That was the idea. Aparently I forgot it in mid-air... One more thing. For a non-blocking write, according to POSIX, "A write request for {PIPE_BUF} or fewer bytes shall have the following effect: if there is sufficient space available in the pipe, write() shall transfer all the data and return the number of bytes requested. Otherwise, write() shall transfer no data and return -1 with errno set to [EAGAIN]." So I think the condition for breaking from the retry loop has to be changed from evt || !NT_SUCCESS (status) || io.Information > 0 to evt || !NT_SUCCESS (status) || io.Information > 0 || len <= PIPE_BUF And I wonder if we've now uncovered a reason for using message mode: If the pipe was created in byte mode, might we get a partial write when len <= PIPE_BUF? I see the following under "Pipes" at https://docs.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-writefile "When writing to a non-blocking, byte-mode pipe handle with insufficient buffer space, WriteFile returns TRUE with *lpNumberOfBytesWritten < nNumberOfBytesToWrite." Ken