From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2101.outbound.protection.outlook.com [40.107.220.101]) by sourceware.org (Postfix) with ESMTPS id 4D22E3858419 for ; Wed, 15 Sep 2021 11:41:43 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 4D22E3858419 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=cornell.edu Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=cornell.edu ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=OU8g8DEvRvAiOA1bEoqnky6iJLflnzPHLVTXiJelovVp46hyLyDyWWc2kIzEkfV2nzcBQvT66hB+6dcypwiJUbsog2iNR6V9MVbU2mjT7w5E0rxRhG465WKHznBhR65KCQ/ShTpDTH7JMIGdeVM9dXxIWWIfo0SS9oqK3COc5yKbpsSYoKLXTF8lxW1l0pRLT+q7VECS9W/qMAE59+9JPQGRg2udPzDNRIrZrJKU1FSVk0k79LmSfnDX+ZGhsm0uw2ZOHUWUxs9PeHpJ7Q28ZCy6t/nJwnMgPJ0WncAxId8mNEn/Wge1IlQn8P3LTbaDsAOZxpqiRyNKEBDYM7NxHA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=wA+iVeyP79PkMjDYZPJUveprs2/JdWgI3q4dvsSTxLg=; b=Pb9w5UYelgT40bGKROEj44rDusX6T+NQZjLb2PE1hauRC8yqPckBF8CXgCL+wSPLhNWlrMZ//+7OwIBTzAWRa+s57MGXe2my+s1K8bVuieHTcQcfxwdR13xN1psTVjvVWMsnjjXFYJFusVVpmONaPDw7dXyDMS7La3sxfpPxeNDu/iuzj9ISsuTdlKDPSTa0I8Msw4d6Zf3PMAJUKT3Hthg2SqQWcARVKCG+HXZl/bGXnxnfnWJFHmihyAJdsXt2JQ0ElYn4FY8rXkDlxLpJWCEMQ+bzZxcGCQm2DHO6B2csTmbIe1ScZI8DSRuVWWhAi/S+VkyGQ/vVeFqHfagU3w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=cornell.edu; dmarc=pass action=none header.from=cornell.edu; dkim=pass header.d=cornell.edu; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cornell.edu; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=wA+iVeyP79PkMjDYZPJUveprs2/JdWgI3q4dvsSTxLg=; b=He0kldJoEySF0+7rg8wUkJPSJRKftL0KNZRTCgaK04gdfzy/0Dn/xnjaZTxRduH2sSA18Mc5DddMQ7lgeVvyUBv63J4rXoSQzX2hgtim9XrmVV2F6gQYOiZ5MH7OJQClrpC67UOzJIMaz0fXS8WLD2/UHxtbXhMT97Y+OPfKei8= Authentication-Results: cygwin.com; dkim=none (message not signed) header.d=none;cygwin.com; dmarc=none action=none header.from=cornell.edu; Received: from BN7PR04MB4388.namprd04.prod.outlook.com (2603:10b6:406:f8::19) by BN8PR04MB6354.namprd04.prod.outlook.com (2603:10b6:408:df::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.15; Wed, 15 Sep 2021 11:41:41 +0000 Received: from BN7PR04MB4388.namprd04.prod.outlook.com ([fe80::5113:e84a:b38a:7a66]) by BN7PR04MB4388.namprd04.prod.outlook.com ([fe80::5113:e84a:b38a:7a66%6]) with mapi id 15.20.4500.019; Wed, 15 Sep 2021 11:41:41 +0000 Subject: Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled? To: cygwin-developers@cygwin.com References: <20210914185623.0488635bf6d1e925baee5fe8@nifty.ne.jp> <20210914191924.5092f63762d62c88431b5313@nifty.ne.jp> <20210914210507.be6c950c948033704542bc42@nifty.ne.jp> <20210915092100.796985ae33039f3184483450@nifty.ne.jp> <20210915094454.42ef1d085d9bf92a1af8d39a@nifty.ne.jp> <20210915095911.415d174ea38fb2d2ea8e5432@nifty.ne.jp> <20210915194858.845bcd54c0c63246b40703a8@nifty.ne.jp> From: Ken Brown Message-ID: <5be76f40-cf62-6fdd-46ab-a66e88879089@cornell.edu> Date: Wed, 15 Sep 2021 07:41:40 -0400 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.14.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-ClientProxiedBy: MN2PR14CA0005.namprd14.prod.outlook.com (2603:10b6:208:23e::10) To BN7PR04MB4388.namprd04.prod.outlook.com (2603:10b6:406:f8::19) MIME-Version: 1.0 Received: from [IPv6:2603:7081:7e3f:3419:1d84:cb99:9c15:8cd0] (2603:7081:7e3f:3419:1d84:cb99:9c15:8cd0) by MN2PR14CA0005.namprd14.prod.outlook.com (2603:10b6:208:23e::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14 via Frontend Transport; Wed, 15 Sep 2021 11:41:40 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 2aac52e1-7ddc-4e20-ed82-08d9783dc90d X-MS-TrafficTypeDiagnostic: BN8PR04MB6354: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:1051; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: fJevkyEZFcDSCLejdUxQppHwmDKRcfmVPrWf3mrNN95TF3fs+40oF/itBCFHjpn/lZEEBYr9aOGCxW+mrCXXegKl9mLli4Py2N7yrGCRGML2TPwKmS0lypp/mjlJF5lKpV1RLhb98U/Cc8h3m27Y31X6pbBM3xTxWfhJ/IMPQRSXJJbasriEBJ/+LXJEiBDh5H2trSM8bTCsMQ31lKN1TCiHOy2IrcSOlRJ+EqafdNEQuFxuATTe9/OsmhZnlVEirm/gcKs3mZ9GYr4qIjE/JsH9WvkvBJH/Yhlg00mklUeOpPTwEYALN4qVAm1C6k+/5WXlD6sADOwHNJLJ9Ajkl34UJw2DWvdr2jBGU4OCVuhiTi5EC7HSvSlmzR5yxohrkayeoSN0rNW1EcmOy5b6hpNgRHnNM/g8N3vib1WmPQ/k4mQ+ZN+O+WZ4s8PDNm8H4MieXN6ohiGeovuTlRnYB8pm7iFIuOn8g9dStgMB63UAys/UrlmPHe/xHJYQwZkBgyDXlFQxR9QaM2+YIEDzJaPror0o3k4phCP3LVWf8tF0/+od9TW8Rc0XTk0eoQvoI3D83m7kmeXew/oBlwH603wVdMfu0/KpT1R0fSwngH7jlzWnmt/DYANvCGEDtn5j8+7bPjlehAnj/onb6dIccPsPdCXcNrtg+8LA3jWlO+lgWlRlje6ynKQddGUmfoepUZSzydUg6me+TC73PYWrFDgjoneGtoQYnicMWuwj3tM= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BN7PR04MB4388.namprd04.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(396003)(366004)(136003)(39860400002)(376002)(346002)(75432002)(53546011)(478600001)(66556008)(86362001)(66476007)(36756003)(186003)(2616005)(31696002)(8676002)(83380400001)(66946007)(31686004)(5660300002)(38100700002)(8936002)(316002)(6486002)(2906002)(6916009)(45980500001)(43740500002); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?Windows-1252?Q?su3MURGd3WR2fIzF7uEg3rxoeRa9AuOjk8P/fp3VH1p+uVaoE9dBgn1p?= =?Windows-1252?Q?jHYtL9Ct/PQrz2MOWQX3ODwFd8GdNk9nGtMrsNQkjAbodHhvsYMX3T1O?= =?Windows-1252?Q?chDTCDE7GKkD0OgBansv9gCoTI6Zsis15OnbF3hUmQx2S9g8mYg1khbf?= =?Windows-1252?Q?Of9qOAavIhNv0SEex3TSL9UMO7ABdW+II2Mrcrhdi8YqmSgquqJNSoug?= =?Windows-1252?Q?mASdyqMNX8p2GQwY9PZ47Cf+TV6khaZh4CI5lRz/sPWn9QufAijKRELA?= =?Windows-1252?Q?Y+V8tfSwClbXHUsQE4jYDEyInQVdKOiFxuAv3okoEDYsvJNid5KYAF9+?= =?Windows-1252?Q?bTHjw8rhfJTXOZHsvbTm5UZFztRFdVwJZlMUB4YWPlpyJ27iN7MgPUJV?= =?Windows-1252?Q?FCyv8mGVGl37rWYPdVvhLCpNx8q0knsqC2fbSKDE2d7zpmA/vX0PADsB?= =?Windows-1252?Q?MEWfVp/xB7MwDRifO6b27r1Gdu05a/5p71i68FO0bnmL7ePcvKEuJQk0?= =?Windows-1252?Q?zGLyi7yQoqzstMtiFcMIvJ0rX/FBk3wCDV0g0c/NFESwDtPZPj2xeAKI?= =?Windows-1252?Q?Pieka6GF1fbgAUM5wgKbPxcmmRzTlbnYk7hMk/222F8W/WKB2eFkahX8?= =?Windows-1252?Q?K4zkZLZXP8aeesKNTpGIlLC9ZRS3GHpfCnvXMQlKOpUG2jhBrj1liHGa?= =?Windows-1252?Q?dtaxjzy+710tUTXKXo+psZDIAYCdG2jd1lBVFvjPDEaKs5MhlMEtMSGa?= =?Windows-1252?Q?N+z4fH+TLZsP1QID7STDAF4BRzIGxlpf+/oKU8W44J1d7r/N95bXtLD6?= =?Windows-1252?Q?mkceJOA2s/+DLtISRoZnO1LirOqv3I7BcxquDVRjD4qXZmp9Jocggf6u?= =?Windows-1252?Q?awHIi5Ex0/uUEqg5kNK7HUneg1bT1Id3vN2wQGpR3NEIARi1XcSG5vZq?= =?Windows-1252?Q?cKRD36kCBsgW2ex/rlaThjh6JFTxvqfGkNtUnkl+tpEYOtGPj0rdacnu?= =?Windows-1252?Q?M0gNcSTqnkLvxPe1Vt8Pqf7I/+d13ykrG5E62FPy6wSZd3e1AfHg7YIM?= =?Windows-1252?Q?5S1GioTQpWVkt5tDAucV3CT9X3il4yTa9k9fCBevPLGZLiwnYj5GZ9jc?= =?Windows-1252?Q?KykwuK9tR7cln5Tfa/oX2djWktt71tqv1O7vgbPEJKBkeXlC1PiyQcVW?= =?Windows-1252?Q?L2YMGya3dpgps2LL9MiGR75wYuhGqKg3vqXSyirutfEseGxrP5E9YldT?= =?Windows-1252?Q?Vpw6q2gSrkEsEy+gviW9DByNWiwSdsHP5rbjteJjni+EOIEqIqeXjCPU?= =?Windows-1252?Q?U18BksSOc+/CYD9BFs7Kt0wAOTQOwT51hYmN7GhiJ6R+OhNtq8Jn+9h/?= =?Windows-1252?Q?osHHQpzdnoJSnN/TKQ6xO+kZkqDnyrSouWGWKHd6XwMbD+D1bJ4BUTr6?= =?Windows-1252?Q?yx9xjp9TiLoyQ5Xq/oLnyhzGzr7mIZKbtRUaw07x3wTTZITS+VB+l/qw?= =?Windows-1252?Q?IVdXPl+0?= X-OriginatorOrg: cornell.edu X-MS-Exchange-CrossTenant-Network-Message-Id: 2aac52e1-7ddc-4e20-ed82-08d9783dc90d X-MS-Exchange-CrossTenant-AuthSource: BN7PR04MB4388.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Sep 2021 11:41:41.2706 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 5d7e4366-1b9b-45cf-8e79-b14b27df46e1 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: QPS+aFFasR15JY8Nbuy1i75i3AMW8bYVuhtTH0N6Vm1U3Tp6vxd4NIMEV/375sl/l02huf9HATk7/zBpQDvGFg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN8PR04MB6354 X-Spam-Status: No, score=-11.9 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, MSGID_FROM_MTA_HEADER, NICE_REPLY_A, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_PASS, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: cygwin-developers@cygwin.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Cygwin core component developers mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 15 Sep 2021 11:41:44 -0000 On 9/15/2021 7:13 AM, Corinna Vinschen wrote: > On Sep 15 19:48, Takashi Yano wrote: >> On Wed, 15 Sep 2021 11:57:44 +0200 >> Corinna Vinschen wrote: >>> ...but we're unconditionally using NtQueryInformationFile at the start >>> of pipe_data_available. If this is the read side of the pipe, isn't >>> there a chance that another (non-Cygwin) process is performing a blocking >>> read and this call will hang? >> >> I will check that. > > Something like that comes to mind: > > diff --git a/winsup/cygwin/select.cc b/winsup/cygwin/select.cc > index fd23122988c3..cf9ad87538f2 100644 > --- a/winsup/cygwin/select.cc > +++ b/winsup/cygwin/select.cc > @@ -594,18 +594,6 @@ pipe_data_available (int fd, fhandler_base *fh, HANDLE h, bool writing) > if (fh->has_ongoing_io ()) > return 0; > > - status = NtQueryInformationFile (h, &iosb, &fpli, sizeof (fpli), > - FilePipeLocalInformation); > - if (!NT_SUCCESS (status)) > - { > - /* If NtQueryInformationFile fails, optimistically assume the > - pipe is writable. This could happen if we somehow > - inherit a pipe that doesn't permit FILE_READ_ATTRIBUTES > - access on the write end. */ > - select_printf ("fd %d, %s, NtQueryInformationFile failed, status %y", > - fd, fh->get_name (), status); > - return writing ? 1 : -1; > - } > if (writing) > { > /* If there is anything available in the pipe buffer then signal > @@ -633,6 +621,18 @@ pipe_data_available (int fd, fhandler_base *fh, HANDLE h, bool writing) > /* Note: Do not use NtQueryInformationFile() for query_hdl because > NtQueryInformationFile() seems to interfere with reading pipes > in non-cygwin apps. Instead, use PeekNamedPipe() here. */ > + status = NtQueryInformationFile (h, &iosb, &fpli, sizeof (fpli), > + FilePipeLocalInformation); > + if (!NT_SUCCESS (status)) > + { > + /* If NtQueryInformationFile fails, optimistically assume the > + pipe is writable. This could happen if we somehow > + inherit a pipe that doesn't permit FILE_READ_ATTRIBUTES > + access on the write end. */ > + select_printf ("fd %d, %s, NtQueryInformationFile failed, status %y", > + fd, fh->get_name (), status); > + return 1; > + } > if (fh->get_device () == FH_PIPEW && fpli.WriteQuotaAvailable == 0) > { > HANDLE query_hdl = ((fhandler_pipe *) fh)->get_query_handle (); > @@ -652,11 +652,15 @@ pipe_data_available (int fd, fhandler_base *fh, HANDLE h, bool writing) > } > /* TODO: Buffer really full or non-Cygwin reader? */ > } > - else if (fpli.ReadDataAvailable) > + else > { > - paranoid_printf ("fd %d, %s, read avail %u", fd, fh->get_name (), > - fpli.ReadDataAvailable); > - return 1; > + DWORD nbytes_in_pipe; > + if (PeekNamedPipe (h, NULL, 0, NULL, &nbytes_in_pipe, NULL)) > + { > + paranoid_printf ("fd %d, %s, read avail %u", fd, fh->get_name (), > + nbytes_in_pipe); > + return nbytes_in_pipe > 0; > + } > } > if (fpli.NamedPipeState & FILE_PIPE_CLOSING_STATE) > return -1; Maybe we want to call NtQueryInformationFile in all cases except FH_PIPER, keeping in mind that there are callers of pipe_data_available other than pipes. That way we can still test for FILE_PIPE_CLOSING_STATE at the end in many cases. Ken