From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2129.outbound.protection.outlook.com [40.107.236.129]) by sourceware.org (Postfix) with ESMTPS id C45AA3857815 for ; Sat, 28 Aug 2021 20:55:56 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org C45AA3857815 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=gSqEG2yyvQxi+BvpWj+JnBu6vVNgvqw13z81ygv0FOE0wMNXBAsld50a2wgQre5XyoNmlOg+H0i0Xs8c6tCw4kuqMBL8h1JmPZN7LBaC2H2g23zevnbdkJhxJeue1kodGYey0vex56JS3FtG4sY3fST9duAhwDXKo7xQeSbUrDQ6XDhqHjsvaa4YUXL4vKm1cV5hOBMe0CX6varIvakkbYVScgvqth4SssoYz32FUylV+cIh00oh/lNFsqJpYP7HzTP/tBLAgBl2g0SfFbiFYq23Zf8ks1ePMFJISXyneqlfNDz09t8sInKQeeCRhwaWSRMr+8LYNc4aCSBH7bL3GQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=HSabgIcac7CGru0qBSDURodQlxaM200ukqYot1kBIW8=; b=mhXDpvHmYsHr0I9g0qZ8acYYx5s+hR6vaPeu5zoVk548Va/Gjfew3voXUgSGrTMuHIX04y5Lr9m94hDGJU1SbBV6MRRhw7EJerrZLLBMf7oB97AqcnETkbfQI1EyfZDFPf6o2yriQRsI6YtD9eZLNEjvTSEcuqnLxUxebMT6jAIlpwi4/cJu6mrjdufdcMbs7/JApmUCNH/HPm/eCIxV9F58sJZcOa1bkvJCf5tnX2TiMIAjqSg2K5cs93TvJog9OkYPCsG9Z0Kt5UEzfxErvSdID/7qLhEG+lpufPe9UeTAccDEoaSYKcw9SiSGNtu3UWYd6H1qxeRwYSHf3x6SuA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=cornell.edu; dmarc=pass action=none header.from=cornell.edu; dkim=pass header.d=cornell.edu; arc=none Received: from BN7PR04MB4388.namprd04.prod.outlook.com (2603:10b6:406:f8::19) by BN3PR04MB2180.namprd04.prod.outlook.com (2a01:111:e400:7bb8::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.23; Sat, 28 Aug 2021 20:55:53 +0000 Received: from BN7PR04MB4388.namprd04.prod.outlook.com ([fe80::38bd:b608:234f:9ec6]) by BN7PR04MB4388.namprd04.prod.outlook.com ([fe80::38bd:b608:234f:9ec6%7]) with mapi id 15.20.4457.023; Sat, 28 Aug 2021 20:55:53 +0000 Subject: Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? To: cygwin@cygwin.com References: <20210825201845.07b6400b79dc5558a7761efe@nifty.ne.jp> <20210826062934.54f2f2216021c095bb7ba13b@nifty.ne.jp> <3b560051-ab27-f392-ca4b-d1fd9b5733b0@cornell.edu> <20210827202440.47706fc2fc07c5e9a1bc0047@nifty.ne.jp> <4f2cb5f3-ce9c-c617-f65f-841a5eca096e@cornell.edu> <20210828022111.91ef5b4ff24f6da9fadb489e@nifty.ne.jp> <20210828184102.f2206a8a9e5fe5cf24bf5e45@nifty.ne.jp> <20210829004346.c2f80469abc3a07fd4b2918d@nifty.ne.jp> From: Ken Brown Message-ID: Date: Sat, 28 Aug 2021 16:55:52 -0400 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.13.0 In-Reply-To: <20210829004346.c2f80469abc3a07fd4b2918d@nifty.ne.jp> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-ClientProxiedBy: CH2PR10CA0021.namprd10.prod.outlook.com (2603:10b6:610:4c::31) To BN7PR04MB4388.namprd04.prod.outlook.com (2603:10b6:406:f8::19) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from [192.168.1.211] (74.69.128.111) by CH2PR10CA0021.namprd10.prod.outlook.com (2603:10b6:610:4c::31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.17 via Frontend Transport; Sat, 28 Aug 2021 20:55:52 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: ab91d725-e04b-4f6b-9c51-08d96a663936 X-MS-TrafficTypeDiagnostic: BN3PR04MB2180: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:7219; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 8c4uYBD9K1BmKs0N/d5jbr70eMxp2G38ZSYLkve41aWDmFQK5Afs+hVKkzRRXKYms5fryAfl9yRD0ZU8LBzKdRfWr3401GrB8GPMZ/8ai/7PH/ZWOE1G7bfHYHOzunSDTJjlxb2wX2gkzvplc3wKG7/eWqY7IhUDd+i/lF5zXRbS3vSA4kBbIbAP7R+WERcYeS2A95eBmgEJpX8CzwfUydPXm9Ty2VvBc3zgaLe0vjnzlRjJPD4jBW5/vlFhsO5/ijWiC5TTtxm5EJCnpu0jlIq+d7WjiLvu+LtcjfnOWPbKDfmr7gDSKSHLZqE4IGzVtCP2nsLxhAuqqNebF4ONsXMMgllwcm0NYEwpxQhNCMqsSScLlLE19FxmAwZyvKU+8tF9fzeSRAH5o2WbVIVXHfwErYqup9Ud60VBplerLaR+NaKgY+aWGD8H3oP4CrYs/FnFTyYCIO9mTvqUqESA+DoCELcxoNc4qnmeZE5M4VRG9Olk1WeBy5GsM3bZEWGz0dUFsaQkgS4NUOOPR2v+8EmxBAfO+5HeKKGGhrdNbIutXnyLuCJzPJHo9XtioN3DRSBW9EEaQh4cdotQ8tv+S8KnGpwkdGLm+q+h3ftBumo+tDEssCqnVeVNTrZXO2+arcia4JcsoeLnIuYAhwsRfrOalfmOGXqI2xZHXyBNhIoophEFxpaEf4BlqnE9kTnSdXlAQ91Mz8txCqpBlEYpRP6zsEEhe2fJWTsgkjpS7eUCfo9K7AexltfsS1jHkOTl6i3Zbpfhf8eqawZfb1n1ZwGy4SGOIS3PWs8RyiX787ZSWQk2pjK73DRvGdrMgIm0 X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BN7PR04MB4388.namprd04.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(346002)(39860400002)(366004)(136003)(396003)(376002)(26005)(186003)(38100700002)(31686004)(6486002)(53546011)(8676002)(5660300002)(8936002)(966005)(956004)(2616005)(6916009)(36756003)(75432002)(478600001)(786003)(316002)(16576012)(83380400001)(66556008)(66476007)(66946007)(86362001)(2906002)(31696002)(43740500002)(45980500001); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?Windows-1252?Q?iK7Ocxy0r4DQ1yN9BTyN/eL8Y4btmtYQ71oG0aiKZiA41htyss/T9B44?= =?Windows-1252?Q?sFx7SlSRySAC9OPkdMCAriasNym4e+aGhy0YXvW+Cwg6fVO6nKB01t5V?= =?Windows-1252?Q?S6qe6v8G60XFIIUJcum3bvPqDXZ53q80kT8tGrPlPWCm4VaigrIZTorg?= =?Windows-1252?Q?2wyfJnqIFgh9TUnSCCWXACA7RjvStr2zrGI7zjyv3WI5VRAc17oxGEEF?= =?Windows-1252?Q?3HM1OiRnZynVrHD9UDqxpRl7pKTPTqZwWb4i18S18DcKp8YSBejaajJW?= =?Windows-1252?Q?tPpDXhllMf/6I8TTJjkATyMElFlBRMniUX1xFv1c6NTw01lkZELLe7AL?= =?Windows-1252?Q?uFvhN38RTPcznxoE6XUP4QniIH5OtzsOeTMXkOX6+4ziMbvMV0XWwsR2?= =?Windows-1252?Q?wbcwRNDHH9SboSQ2/wOMrvB1iBe9fvYQ352QFV3ssZg1OhKG+Kw1XBFv?= =?Windows-1252?Q?T7hsqcZyA2pkDC9sHRxwh4G5HiLqWsyTuzOvHCSIQQHkKorRFY/qnrrq?= =?Windows-1252?Q?JoKgyQ0J7anrFwc5jObY3LzDub3E6KEIMXJghEY0Z4taS3uglbkc/pwp?= =?Windows-1252?Q?3V6ffIBN4xYXMuXPLYaASX3NGrTgSx6tQtCLSDt1Z2hR2GzeVeK9PsUC?= =?Windows-1252?Q?zYokQWS4MPJktop+JnrPy9CDuz9YIZi0PsquuNrXM20cLZMmxOnZBTnC?= =?Windows-1252?Q?GsCZ7xzL4RIW11zzcUqGzfL6+XXeu3jyJCtjDJFz6vnELX5r9V0AXPDQ?= =?Windows-1252?Q?3ron+vVa/YdslGxL4I8ZoCdujuOrG2euuxIVM1Sl9MjCm/Sh/pnh9Ram?= =?Windows-1252?Q?3ECD1U56CmeisJNj1M48BWzfqeArm0zUCHYfXgjJYCBEkrimEUTNZMd+?= =?Windows-1252?Q?6JyE6nZ3hVoXa2M6r3HiIX2C+/oxodh+Q5ADmRPFEhfK/EtOP3ik3Cni?= =?Windows-1252?Q?AQ8qTrSy7PlAxI6Q6f8ByW7OETt03Nfj3zbr2eMHMjxOb0YTtMLnbk2c?= =?Windows-1252?Q?aTAdabi1pp4kFF39HMEacldmr3Ya5C8QZRljKt7398F0d/GG0V58ggIy?= =?Windows-1252?Q?tGQ/uI4oER5wnoq3AO/Rv+/poj4LYQCVl3v/ttMLOk20Y9RyaQ8fwC8M?= =?Windows-1252?Q?1B1PD8bYgnCacFdBukec8LpFlGfmKUj70c13Y36MrQVyayvqb5LuFL5l?= =?Windows-1252?Q?mEPYddyRu0vne3ebZax5o70rV95A3B1wxc544uZy/ddjTH4C12zdFw4F?= =?Windows-1252?Q?N6za7LHzo38DMZGKLuF/1vRLJbynjYyssGmQUMUWyyxDU6HficSldFKC?= =?Windows-1252?Q?PF1L8fBaBruu7+dXTKkh+Qqrsm/BiwWWhhKxsKFIudnzN87QOd9oRBMR?= =?Windows-1252?Q?1o91/6GMB2sf9nPhhG3v/SEfrbs8SAz+TBnuFzo8t4Y8Cyxe2OlEE5Zw?= X-OriginatorOrg: cornell.edu X-MS-Exchange-CrossTenant-Network-Message-Id: ab91d725-e04b-4f6b-9c51-08d96a663936 X-MS-Exchange-CrossTenant-AuthSource: BN7PR04MB4388.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Aug 2021 20:55:52.9903 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 5d7e4366-1b9b-45cf-8e79-b14b27df46e1 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Y6KWauKehyOBbvpIcF434WtkWDCfrsgdM7u398Powoi0W47emndsIKLTSazfN6btcS/8m1kv0OnN0wDNkiZLzQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN3PR04MB2180 X-Spam-Status: No, score=-3.3 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, JMQ_SPF_NEUTRAL, MSGID_FROM_MTA_HEADER, NICE_REPLY_A, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_PASS, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: cygwin@cygwin.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: General Cygwin discussions and problem reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Aug 2021 20:56:07 -0000 On 8/28/2021 11:43 AM, Takashi Yano via Cygwin wrote: > On Sat, 28 Aug 2021 13:58:08 +0200 > Corinna Vinschen wrote: >> On Aug 28 18:41, Takashi Yano via Cygwin wrote: >>> On Sat, 28 Aug 2021 10:43:27 +0200 >>> Corinna Vinschen wrote: >>>> On Aug 28 02:21, Takashi Yano via Cygwin wrote: >>>>> On Fri, 27 Aug 2021 12:00:50 -0400 >>>>> Ken Brown wrote: >>>>>> Two years ago I thought I needed nt_create to avoid problems when calling >>>>>> set_pipe_non_blocking. Are you saying that's not an issue? Is >>>>>> set_pipe_non_blocking unnecessary? Is that the point of your modification to >>>>>> raw_read? >>>>> >>>>> Yes. Instead of making windows read function itself non-blocking, >>>>> it is possible to check if the pipe can be read before read using >>>>> PeekNamedPipe(). If the pipe cannot be read right now, EAGAIN is >>>>> returned. >>>> >>>> The problem is this: >>>> >>>> if (PeekNamedPipe()) >>>> ReadFile(blocking); >>>> >>>> is not atomic. I. e., if PeekNamedPipe succeeds, nothing keeps another >>>> thread from draining the pipe between the PeekNamedPipe and the ReadFile >>>> call. And as soon as ReadFile runs, it hangs indefinitely and we can't >>>> stop it via a signal. >>> >>> Hmm, you are right. Mutex guard seems to be necessary like pty code >>> if we go this way. >>> >>>> Is a blocking ReadFile actually faster than a non-blocking read? Or >>>> does it mainly depend on BYTE vs. MESSAGE mode? >>> >>> Actually, I don't think so. Perhaps it is not essential problem of >>> overlapped I/O but something is wrong with current pipe code. >>> >>>> What if the pipe is created non-blocking and stays non-blocking all the >>>> time and uses BYTE mode all the time? Just as sockets, it would always >>>> only emulate blocking mode. Wouldn't that drop code size a lot and fix >>>> most problems? >>> >>> If 'non-blocking' means overlapped I/O, only the problem will be: >>> https://cygwin.com/pipermail/cygwin/2021-March/247987.html >> >> Sorry if that wasn't clear, but I was not talking about overlapped I/O, >> which we should get rid off, but of real non-blocking mode, which >> Windows pipes are fortunately capable of. > > Do you mean, PIPE_NOWAIT flag? If this flags is specified in > the read pipe, non-cygwin apps cannot read the pipe correctly. While waiting for Corinna's response to this, I have one more question. Do you understand why nt_create() failed and you had to revert to create()? Was it an access problem because nt_create requested FILE_WRITE_ATTRIBUTES? Or did I make some careless mistake in writing nt_create? Ken