From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from omta002.cacentral1.a.cloudfilter.net (omta002.cacentral1.a.cloudfilter.net [3.97.99.33]) by sourceware.org (Postfix) with ESMTPS id 9D2D23858C56 for ; Thu, 21 Dec 2023 20:32:54 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 9D2D23858C56 Authentication-Results: sourceware.org; dmarc=fail (p=none dis=none) header.from=kylheku.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=shaw.ca ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 9D2D23858C56 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=3.97.99.33 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1703190776; cv=none; b=LEi2jojP5qTF236NhhnZpb985rHqfPXjkyhFliRY7Gw28rU6/pAyzTANPRM8eSSjWu/5dI1Do7gKmKd1ic1sYgZaIat3Zl/VfgZIecRQ2dfusVxpHtBUfiei3NQXm0OLd6AS7VEb2mCU4YpqG6XTdgqUzUc6E9gNzqZRKSN8lLg= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1703190776; c=relaxed/simple; bh=yQTG3SABJp8esGLrtB0PlIY3kFtODAdJAXbjsT2f4BY=; h=DKIM-Signature:MIME-Version:Date:From:To:Subject:Message-ID; b=lEZ+EfkF/OMA+AvTmopRPSAyCHYJlDrGw0lT78aRKXDILzVstwDJsyqslq1aHp9lTNVaPDY6rMhP+e1UNvLpoXqjYZAusuq4S3vpiJRbrED5nh1CzrpJfGLHbNI6vdLzgdqGOnQ647QPV1Ncy/x2DYyVZKr6nxfvgYdOft4h160= ARC-Authentication-Results: i=1; server2.sourceware.org Received: from shw-obgw-4004a.ext.cloudfilter.net ([10.228.9.227]) by cmsmtp with ESMTPS id GLMWrhVsXB0n0GPiwrOQSC; Thu, 21 Dec 2023 20:32:54 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=shaw.ca; s=s20180605; t=1703190774; bh=yQTG3SABJp8esGLrtB0PlIY3kFtODAdJAXbjsT2f4BY=; h=Date:From:To:Cc:Subject:In-Reply-To:References; b=tvE0hu+XTZcXkRoGMNgYWxG/rPIP02btm9/U3wcra/9LIkXxSCc9VaFjZm9ymNX7S 1Iqz81An0gvUi8DwM6ulsmNSlr+6GcZDXmSGe3qz/k5950Bm71o5KQ3E0NVR4yvdiG PZRXLXAIjwZttzkTBjPJ+js5zFQRPEXOMFFpwmNXLovGlY+LXdUVbnzyoOvlLdYe9b NFGhPfi0A3ZQqeVq7OkN7FBY8/d4F9l+dAVYz56X+r+0E50EhWo+IE5UKcb1Xd2Lcy Bp+YVm/YozieC9aP3NzT3kNb8Eg7/ZbxfTGm8OC/xZOqphfEeXrLh5ZFkb5fcD07Zg T1zUl5+g31y9w== Received: from kylheku.com ([70.79.182.7]) by cmsmtp with ESMTPSA id GPiur3uTsDqGYGPivrp62O; Thu, 21 Dec 2023 20:32:54 +0000 Authentication-Results: ; auth=pass (PLAIN) smtp.auth=kkylheku@shaw.ca X-Authority-Analysis: v=2.4 cv=Cousz10D c=1 sm=1 tr=0 ts=6584a0f6 a=pMSlDXUwMa7SJ1EIez8PdQ==:117 a=pMSlDXUwMa7SJ1EIez8PdQ==:17 a=xqWC_Br6kY4A:10 a=IkcTkHD0fZMA:10 a=e2cXIFwxEfEA:10 a=w_pzkKWiAAAA:8 a=eZNnFSucP04RNPoSW7UA:9 a=QEXdDO2ut3YA:10 a=sRI3_1zDfAgwuvI8zelB:22 Received: from localhost ([::1] helo=mail.kylheku.com) by kylheku.com with esmtp (Exim 4.94.2) (envelope-from ) id 1rGPit-00Afgk-Pz; Thu, 21 Dec 2023 12:32:51 -0800 MIME-Version: 1.0 Date: Thu, 21 Dec 2023 12:32:51 -0800 From: Kaz Kylheku To: Martin Wege Cc: cygwin@cygwin.com Subject: Re: rfe: CYGWIN fslinktypes option? Re: Catastrophic Cygwin find . -ls, grep performance on samba share compared to WSL&Linux In-Reply-To: References: <07c7379e983c9f436ebf86e3818ca843@kylheku.com> User-Agent: Roundcube Webmail/1.4.15 Message-ID: <4723aab7e2b331cb81946eff0fb4e862@kylheku.com> X-Sender: kaz@kylheku.com Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-CMAE-Envelope: MS4xfDUSa6PrHD1PN+vfJHKvbEGzRUUMpbncIvaqzClTgMNMlts7IREelzTDr8m6gTkvZmfHHDHKf72k3640Vs23HDBbxPdT80z2XkRuhOjerhzBHGkMynJI QucdsefXcir3tXb2BifEVBv8V/beBuVYXLFv7bfgU6GmCMXogccuhymOZLJoaFeyOO6HLXau/F7CZhRefZP9XQYQRL5rWeqAl1k= X-Spam-Status: No, score=-3.0 required=5.0 tests=BAYES_00,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_LOW,RCVD_IN_MSPIKE_H4,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_PASS,TXREP,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: On 2023-12-21 04:16, Martin Wege via Cygwin wrote: > On Wed, Dec 20, 2023 at 6:21 PM Kaz Kylheku via Cygwin > wrote: >> >> On 2023-12-17 22:22, Dan Shelton via Cygwin wrote: >> > It would be nice if someone from the Cygwin authors could assist me in >> > figuring out why this happens. >> >> Cygwin is famously slow; this is nothing new. We are grateful >> for Cygwin because it makes stuff work at all; if it were blazing >> fast that would be a bonus. >> >> E.g. git operations (clone, rebase, ...); ./configure scripts; ...: all >> run like molasses. >> >> The following is just my fast and loose opinion, shot from the hip, >> and possibly off or wrong, but it likely has to do with the layering. >> Cygwin's core API is based on a C library called Newlib. Cygwin bolts >> Newlib to Windows by means of an additional shim below Newlib that >> is based on C++ objects, where there is path munging going on and such, >> and that's where the Win32 calls get made. It's an additional abstraction. > > I disagree with that. Ok, part of that is that the layering causes > more memory allocations and copies, but this is not the root cause. I seem to recall that most operations that take a path argument have to convert the path from Cygwin to Win32, and I think that also involves going from 8 bit to UTF-16 also. That's gotta hurt a bit. > The root cause is IMO the extra Win32 syscalls (>= 3 per file lookup, > compared to 1 on Linux) to lookup the *.lnk and *.exe.lnk files on > filesystems which have native link support (NTFS, ReFS, SMBFS, NFS). > On SMBFS and NFS it hurts the most, because access latency is the > highest for networked filesystems. Could some intelligent caching be added there? (Discussion of associated invalidation problem in 3... 2.... 1... ) Can you discuss more details, so people don't have to dive into code to understand it? If we are accessing some file "foo", the application or user may actually be referring to a "foo.lnk" link. But in the happy case that "foo" exists, why would we bother looking for "foo.lnk"? If "foo" does not exist, but "foo.lnk" does, that could probably be cached, so that next time "foo" is accessed, we go straight for "foo.lnk", and keep using that while it exists. If someone has both "foo" and "foo.lnk" in the same directory, that's a bit of a degenerate case; how important is it to be "correct", anyway. > So my proposal would be to add an option ('fslinktypes') to the CYGWIN > environment variable to define which types of links are supported: > default 'all'. which is an shortcut for 'native,lnk,lnkexe'. > So in case people do not want 'lnk' link support they just add > CYGWIN+=' fslinktypes:native' to env, to turn off support for > lnk/lnk.exe style links, and be happy. So this complements the winsymlinks option? winsymlinks has to do with how the Cygwin DLL creates symbolic links, whereas this has to do with what objects are recognized as links. The implementation would probably want to compare fslinktypes and winsymlinks to make sure they are harmonized together; if winsymlinks tells Cygwin to make .lnk files, but then fslinktypes banishes them, that's something diagnosable somehow.