From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2090.outbound.protection.outlook.com [40.107.243.90]) by sourceware.org (Postfix) with ESMTPS id D96AC3858417 for ; Mon, 30 Aug 2021 15:00:26 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org D96AC3858417 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=cornell.edu Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=cornell.edu ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=fNT/cE753SXQujBDNWbvQBcFU8DSm5hnf1aUISurphwFEXHAZL/iC/4XtGfiWwXdAJooZSxnkKSurKkykRTNKN61x1x4vfv6NB47OUbJ88ysTU4igRR7wLvsEbqUQuA3sLtlLZ40EYUxAlW0CPx1mLHSDjukDCLLP3LAoP2+SyGpM9SkzJUVbEHZlSle+GZikCxMn5+avoSImD88OX20SkysuOCVIy35vQraKqd6mgoOcNHBcihAUeuF1i/oXLCoHTqJzgn+opMYGHMiy3nsdTUTAXfJ5rJSYUDa4O26UiXVMr02qSL7m1gCT0q7TwpGV9I2skLhC7prcpDir3MKlw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=EAKynuVKhwlzKHJLiPVScrGTMcyqiT2BOU7W4U5g8XU=; b=W6iARgrxTHL7qziU+XyfXAu8unkMZbAFbjUI2XlNUTlQ4+bTO4Vgj2wzUQyGxfLi3pKRk4AMJk7/GyhGDYUx3Wb3zLcUmJqAvdSLL0KDzkM2DV/086Bdw7xc5n4uwmYz/8rEP2DHVGdOZ6pWEaBmenkVGkCLsY/YSBB04HoYlqGONsF/+NcA/gfpga1aBfCU8GCHCQiayT1dVfOm640MfIBeN8KxaIRfmMb4+XP0QYwE6ohTYGSNmGamxcBuinrBE9iS8RFQjaJ2/Eb/cmgMS/+4eSyKAXvNTs2ifG0YgiN5/IOhd4gcybEkJNtGeCBpqp/TsC5sVz28/puUVmfSFA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=cornell.edu; dmarc=pass action=none header.from=cornell.edu; dkim=pass header.d=cornell.edu; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cornell.edu; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=EAKynuVKhwlzKHJLiPVScrGTMcyqiT2BOU7W4U5g8XU=; b=CWPttNedAug7N45FEq2sQhi+AuTpOFk2vz6DlOlbQlxG7z3RcaM2f4hhSpKMkADLx0FQPKrhl7HQhkGU3mJjk5HViPo7zCDDNM9I1Z6kJ/u7OEgUum7XKsnURgEAP1o/ucjPwWQeokM6ZAG1k6v/fBxZ5bQlnO7y1P2Zh5eYVVQ= Authentication-Results: cygwin.com; dkim=none (message not signed) header.d=none;cygwin.com; dmarc=none action=none header.from=cornell.edu; Received: from BN7PR04MB4388.namprd04.prod.outlook.com (2603:10b6:406:f8::19) by BN3PR04MB2323.namprd04.prod.outlook.com (2a01:111:e400:7bb6::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.23; Mon, 30 Aug 2021 15:00:23 +0000 Received: from BN7PR04MB4388.namprd04.prod.outlook.com ([fe80::38bd:b608:234f:9ec6]) by BN7PR04MB4388.namprd04.prod.outlook.com ([fe80::38bd:b608:234f:9ec6%7]) with mapi id 15.20.4457.024; Mon, 30 Aug 2021 15:00:23 +0000 Subject: Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? To: cygwin-developers@cygwin.com References: <20210828184102.f2206a8a9e5fe5cf24bf5e45@nifty.ne.jp> <20210829180729.48b4e877f773cb3980c5766d@nifty.ne.jp> <20210830091314.f9a2cb71794d0f68cdb5eba7@nifty.ne.jp> <20210830092259.52f7d54fc3fa340738373af4@nifty.ne.jp> <20210830170204.fa91eaf110f310f13b67abc3@nifty.ne.jp> <20210830210423.00df7f37473b0ac1251e880f@nifty.ne.jp> <932300c9-2e09-5ee5-bbb1-3c060d33e3e1@cornell.edu> From: Ken Brown Message-ID: <474e1343-9cba-6b3c-b952-c92004968d8f@cornell.edu> Date: Mon, 30 Aug 2021 11:00:22 -0400 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.13.0 In-Reply-To: <932300c9-2e09-5ee5-bbb1-3c060d33e3e1@cornell.edu> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-ClientProxiedBy: CH0PR04CA0076.namprd04.prod.outlook.com (2603:10b6:610:74::21) To BN7PR04MB4388.namprd04.prod.outlook.com (2603:10b6:406:f8::19) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from [192.168.1.211] (74.69.128.111) by CH0PR04CA0076.namprd04.prod.outlook.com (2603:10b6:610:74::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.17 via Frontend Transport; Mon, 30 Aug 2021 15:00:22 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 19b3c59f-e8f5-4733-4f7a-08d96bc6e449 X-MS-TrafficTypeDiagnostic: BN3PR04MB2323: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:9508; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: KcL8xDM2kz5jyepWI7armF3B0GmjKtgFUG7Fyjp26OfY1V3GU4yYY/0C0wxfzdWGs77efdE8RwTMZxu53AUIKPm7QxITLKVL0DyzqLmiHkmmfPxiQGMgm0Gh1SxZ3dZnyjNdc93YGmJoM3b8HJ9JmBhGxtMRFklPFK+wqSjr08t147VVoJWS59QyLF2VvtcK2phxc8JBXDJEiFAnz5YbF6EciaiJui4vQrD6GX+c2U4B9GjFfqMwVZ5qLpoOTXs+7LE/J7J396QTXBBO536wl6EnezQMaztldeB6K3iubFvh/OTyEsrDr98BBlleKUpxZGqgetTGN9Du3OkLBjr5UJJTH7d8BSxLk6pJHIKldfm7SXDMU7xReJlgVzOgWCsQg9jsH+Gfv8eavnoZ7Zcf3mzRumaGhgO4yBullacq474F0RMz4nt2KWxctbgvOCWzREGDz886iY0lTiGScGRIPBJ9E9FxteRV1Kf+HjczedwNV6S6RqS+zXnbrLIQUd5uO8z2HyPtEs26C0ObpDOKznhgw5Tl4ldbakQJ+RMRjMwXhSgTedcbEr7UAElkqQkd7UYU6Ai/tw2nM+7nY8rsIh0dkl/dM4z7e9TlVS+nh2P/YFOmUeHr4hkG/tE5hzhM4lcv+WQ7cRted5TH0bk7UOMtEN5a5JczcmW0QmAj1qXfm+8eXIMquGskpCJOiC3535cvSZ/l2oWmoFAWhE8ds5Qqn7m4ewOSZj5zswMo447QksUfd+YTWpLMPn3C4rwRTviQWKONTr1xrw5SMaSKCIlHIwwCqXabP/fDasBGGXYrEMElD+D6yncvb9jhAM5g X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BN7PR04MB4388.namprd04.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(346002)(39860400002)(136003)(396003)(366004)(376002)(16576012)(36756003)(86362001)(956004)(5660300002)(2616005)(966005)(8936002)(66946007)(66476007)(31696002)(53546011)(66556008)(26005)(83380400001)(2906002)(8676002)(31686004)(6486002)(75432002)(6916009)(786003)(478600001)(186003)(38100700002)(316002)(43740500002)(45980500001); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?Windows-1252?Q?HG7Fwqdl8B/HSyeZF3z0ydPb/ytfQ0mIhAUCH/pm1ZpRuWECuImd2xMs?= =?Windows-1252?Q?oijeDliyRAWCRrm1JKS7e5PMyVJwbrAzXJUkKAkXyjsevnoOTCcJT3PL?= =?Windows-1252?Q?ttV7RnfVeV4VdbIuIuhsX7c/WlPK8ZCbtBOF56pcCqVYyH3lU10KPlG8?= =?Windows-1252?Q?8E1QRk7t3Aj3s84x1v1Sd136IosHcjlYyNxspTjFDZQfiK/ljUt3hIdV?= =?Windows-1252?Q?UaWdFL6X05aJFdpDZqI57N7GBVpb/8+texlhyRPkgTE8AnoxW2ZwTWrI?= =?Windows-1252?Q?U/3zgtpHiRZ0H1iF+sA1+xqn6mgX1Z+AXlKd9oszxdZ/0+jD80lMkAJS?= =?Windows-1252?Q?VHyOfObmnbrNU6L8/vA17m9+fi6At3zRmXNbVfo5rfkEIiwu1N4ib8FB?= =?Windows-1252?Q?w5fvFjnoLzq50OjLnRTKOYrl1yb2Z8jLXw6uUYyrdSml6hi86+GHq8W9?= =?Windows-1252?Q?Bk8FRcN+a84S0aW6tkZ+ZOaW0b3jSOabMmQPDxwGtQo/2Uiei6QqCuRk?= =?Windows-1252?Q?gnZYhOBNXeS1FSwp73ERpj5coxIKRGyjHma7dxppRSBlWwuLELCi4TFl?= =?Windows-1252?Q?4z/I5I0Ysav+CzajRz5FeMonRwFxFdr/h57JVoAoCOXkGDufwSqjM/hB?= =?Windows-1252?Q?tiM9i4cqkFEwLE1rER6yRde66dSNnaxQvBfuBalKY6x/Ke2Zx6J45TNV?= =?Windows-1252?Q?+sHn6JNl9LFTdB/l8G0hpt3EktqgdnHzNdyXzgowkWAfA+gd6BbQ0lC+?= =?Windows-1252?Q?S3dePW75GttCENg1efB37uyhhIlXAei2TKMn4HD6mrgIq9Dw4+UWb2RR?= =?Windows-1252?Q?QRAFBKs80InvoA7D9Ii6EcscQjp4tB3Xs2k+YM9TL76qow1vdDcqPuyg?= =?Windows-1252?Q?lCJr14hS+WwPKgG+KZffY9IwvBLN3L/g+Pg9YwUI2OnjHs7i652zH/h5?= =?Windows-1252?Q?Caq4B9h7oOFIpLpZlgqNAHdA5uV5FA4TboR1gwDxXXlDSC9shMalzbKI?= =?Windows-1252?Q?yS1D4E8mwUtn5dYHq4WExAPq9pXuthmiFvoPLBdF5U5+zGjS+rXi/xSL?= =?Windows-1252?Q?SfP9sp1bLzghqa2lUc9iZS4FU4VYqyzKBxbv4LDtWJvfrbJRs1cR8+3t?= =?Windows-1252?Q?JT70OHlkI94NwoxMOaKUzG4/6rMBnOEU57W3svOCbrzr0xR0sdxevw86?= =?Windows-1252?Q?6zk8XByiFyWEzHIzd0ls4TvyqYzS3oKhHoV1rm25aDeRvds+E+l8ziQO?= =?Windows-1252?Q?7tjbHdJ43UoHyY7r2L2cmxRk4Wfjx192xZcRteBU9zqwZ48+DEDE4/iA?= =?Windows-1252?Q?7q6ctH5EgbUBZrhEnTOeaebAYWSnt+bpC94+43cnBJXUD//duQ0L6F6r?= =?Windows-1252?Q?btrmdcBGcrWegw/MJF6xwI5r5VjYehuJfwTEmDrxSxhewEfNZO+yUGuE?= X-OriginatorOrg: cornell.edu X-MS-Exchange-CrossTenant-Network-Message-Id: 19b3c59f-e8f5-4733-4f7a-08d96bc6e449 X-MS-Exchange-CrossTenant-AuthSource: BN7PR04MB4388.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Aug 2021 15:00:22.8821 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 5d7e4366-1b9b-45cf-8e79-b14b27df46e1 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: o44c7mgBlTGzQTurGn7Ul4YwBLUWMImL/RJOs/CyfPdC4MlHshE4GnorN28D4mp7LWb5NVJOhTdx69kW0zlbXw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN3PR04MB2323 X-Spam-Status: No, score=0.3 required=5.0 tests=BAYES_00, BODY_8BITS, DKIM_INVALID, DKIM_SIGNED, KAM_DMARC_STATUS, MSGID_FROM_MTA_HEADER, NICE_REPLY_A, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_PASS, SPF_PASS, TXREP autolearn=no autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: cygwin-developers@cygwin.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Cygwin core component developers mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Aug 2021 15:00:37 -0000 On 8/30/2021 9:51 AM, Ken Brown wrote: > On 8/30/2021 8:55 AM, Corinna Vinschen wrote: >> On Aug 30 21:04, Takashi Yano wrote: >>> On Mon, 30 Aug 2021 12:20:30 +0200 >>> Corinna Vinschen wrote: >>>> [Move discussion to cygwin-developers] >>>> >>>> On Aug 30 17:02, Takashi Yano via Cygwin wrote: >>>>> [...] >>>>> Is naming the pipe really necessary? >>>> >>>> It's not, but CreatePipe is doing this anyway. >>>> >>>> "Anonymous pipes are implemented using a named pipe with a unique name." >>>> https://docs.microsoft.com/en-us/windows/win32/api/namedpipeapi/nf-namedpipeapi-createpipe >>>> >>>> >>>> The reason CreateNamedPipe was used in the first place was that >>>> FILE_READ_ATTRIBUTES isn't set by CreatePipe for the write side >>>> of the pipe, however, it creates full duplex pipe: >>>> >>>> https://cygwin.com/pipermail/cygwin-patches/2004q3/004912.html >>>> >>>> Given the fact that CreatePipe is implemented in terms of >>>> NtCreateNamedPipeFile anyway, why should the pipe created with >>>> NtCreateNamedPipeFile fail where the pipe created with CreatePipe works? >>>> >>>> The only reason can be some missing flag, I think.  Checking >>>> fhandler_pipe.cc::nt_create and comparing that with the default flags >>>> for files and other devices, it occurs to me that the SYNCHRONIZE stuff >>>> is missing.  So, Takashi, what if you call NtCreateNamedPipeFile like >>>> this in nt_create: >>>> >>>>    status = NtCreateNamedPipeFile (r, access | SYNCHRONIZE, &attr, &io, >>>>                   FILE_SHARE_READ | FILE_SHARE_WRITE, >>>>                   FILE_CREATE, FILE_SYNCHRONOUS_IO_NONALERT, >>>>                   pipe_type, FILE_PIPE_BYTE_STREAM_MODE, >>>>                   0, 1, psize, psize, &timeout); >>>> >>>> Does that fix the above problems, too? >>> >>> Yes it does! Now, if CYGWIN=pipe_byte is also set, the piping issue >>> of C# program is gone! >>> >>> In fact, I've already tested adding the SYNCHRONIZE access flag, >>> but it didn't solve the problem. It seems that the cause was >>> that FILE_SYNCHRONOUS_IO_NONALERT was missing. >>> >>> Thank you for figuring out the solution! >> >> No worries.  The same should apply to the NtCreateFile side of the >> pipe, btw. > > I'll add my thanks.  I should have checked the default flags that are typically > used for other devices when I wrote nt_create.  I'm glad you caught this. > > So I'll reinstate the use of nt_create and then let Takashi recheck everything. I've done this now. I'm still not sure I've got all the flags right. For unknown reasons, I've used FILE_SHARE_READ | FILE_SHARE_WRITE in the call to NtCreateNamedPipeFile, and no sharing in the call to NtOpenFile. Should I also use FILE_SHARE_READ | FILE_SHARE in NtOpenFile? Is sharing even relevant in this context? Ken