From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 47412 invoked by alias); 19 Jun 2016 11:20:34 -0000 Mailing-List: contact cygwin-help@cygwin.com; run by ezmlm Precedence: bulk List-Id: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: cygwin-owner@cygwin.com Mail-Followup-To: cygwin@cygwin.com Received: (qmail 47401 invoked by uid 89); 19 Jun 2016 11:20:33 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-1.7 required=5.0 tests=AWL,BAYES_00,FREEMAIL_FROM,RCVD_IN_DNSWL_LOW,SPF_PASS autolearn=ham version=3.3.2 spammy=narrow, hallo, guest, lnk X-HELO: mout.gmx.net Received: from mout.gmx.net (HELO mout.gmx.net) (212.227.15.18) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Sun, 19 Jun 2016 11:20:22 +0000 Received: from [127.0.0.1] ([46.163.75.188]) by mail.gmx.com (mrgmx002) with ESMTPSA (Nemesis) id 0Lx83d-1bUGy00bUX-016fuH; Sun, 19 Jun 2016 13:20:17 +0200 Subject: Re: FUSE for Cygwin To: cygwin@cygwin.com References: From: Herbert Stocker Message-ID: <57667FEF.5070801@gmx.de> Date: Sun, 19 Jun 2016 18:53:00 -0000 User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:38.0) Gecko/20100101 Thunderbird/38.1.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-UI-Out-Filterresults: notjunk:1;V01:K0:z9JNrvNPQp0=:I2w7DNB66WcXQ5lPVH6w15 oRAswwTOwDNYgfWoWk7vOkmfuq/CoF8gWgFY8XoDXsZTr2RYDXvrQXKEJg0pjreBtFkgdqq7M dyYuzpNzSXn2Qr35MUpOeq/SWnX/YkhlAg1KVuuiCZ0T4wU9Ah2NcxXTx0259dalv50KRv5H1 +g2b7SmpE82GDSV3VJVtJRSVpCzmi5tWBgKnsovMdUfcP7MWM9lmfLHMs7m/dqY/esBxE74wL dRtydwgXkYRygjUOSKBiNAvIx6dgNBKKWcWOKi2e+Ml1yKZsaqbj05ce9LDk0KZXHis1Yw2mx QpZTjg6HCVrm7NN9JvO/sIYq0msvxkHNdD9PXossKhIdVQXKe2Fk70Zbw4RAHuKPEtikE/k/3 cAFE9rne95xDfhYt3gfyv1JvyGSEG2GTYPiSJkpGVUJdqLvGWwezGL1HEJo86jAc/BFXoeQ3H 3LuNVGU7OM9lgQOdpRUntHQfN9o5dWVjlRSK9S0M35D1aV72cmN5HKnV6CX8dXmowYqoJEpmZ XD4KyiyO46+Gd60R84hZvVZMc/8XiJvl/J/Z6jlZqVA5nuOEBCbUV57ZnaDUsFB/m0ebxVCoQ WPwZQBxTWO59OgSojsOYcPMBYriMRsGkjjc5/X5zt0rbc7AwwDaSqSK2dKuaQnKQ8fXgpd/Fv w0U5GkRMeiJFJsFR+ZJ6vzyHK5v2mZ7F8UrX/sXkY7+8UlxdRDfEiYs+2FGGDviRnAE+2MjsJ 995YardxLleveugZoqlOQEZFT4RnTlIXfoS7xo8exJgJIAyIus30MKjI+EjfczwBmnNrPHTm9 dZ3akTo X-IsSubscribed: yes X-SW-Source: 2016-06/txt/msg00255.txt.bz2 Hi Bill, this is now my proposal of an alternative mode for WinFsp to support Cygwin based FUSE file systems. I'll call it mode (4). It's actually my initial idea that i have in mind for some time but did not propose to implement because i have some constraints that prevent me from completing it. Maybe i should first note that i'm not part of Cygwin, i'm just a user of Cygwin who is happy to have Cygwin. However i know how to code. To repeat your 3 modes of operation (in my words): (1) WinFsp's own API. The file system is a Windows process that can use all file system features that Windows (and WinFsp) provides. (2) A FUE API for Win32. The file system is a native Windows process but receives requests as defined in fuse.h . It eases porting for the file system and does not require Cygwin be installed. (3) The file system is a Cygwin process and thus can use all POSIX features of Cygwin. It needs no porting or only little. What i don't like on (3) is that when a Cygwin process accesses the FUSE file system there are two Cygwin processes whose communication is translated from Posix to Win32 and then back (which is done again for the response). You sketched this as follows: > Let's examine the lifetime of a call to creat(). Suppose a Cygwin process > does creat("/cygdrive/z/foo*bar"). In the following OP is the "originating > process", CW is the "Cygwin runtime", NT is NTOS, WD is the "WinFsp FSD", > WL is the "WinFsp DLL", FL is the "FUSE layer", and FS is the "user mode > FUSE file system". > > OP: creat("/cygdrive/z/foo*bar") > CW: NtCreateFile(L"\\foo\xf02abar") <--- Cygwin translation > NT: IRP_MJ_CREATE L"\\foo\xf02abar" > WD: FspFsctlTransactCreateKind L"\\foo\xf02abar" > WL: FSP_FILE_SYSTEM_INTERFACE::Create L"\\foo\xf02abar" > FL: fuse_operations::create "/foo*bar" <--- WinFsp/FUSE > translation > FS: somehow satisfies fuse_operations::create > [snip return path] So my proposal is basically this: (4) The file system is a Cygwin process and Cygwin is extended to pass file system requests to that process if they fall into the respective path (where the file system is mounted.) This way the file system is exported to Cygwin only. But another tool exports this to the Win32 world. Initially i thought for the second part we could use that part of VirtualBox Guest Additions that makes the host file system available in a windows guest. But now we have WinFsp for this... For the first part of (4), Cygwin does have a place that translates Unix paths to Win32 paths. This part follows the settings in /etc/fstab so that arbitrary Win32 paths can be "mounted" to arbitrary Posix paths with settings applied. This could be extended so that processes that link to a Cygwin version of libfuse can register for certain paths. Cygwin does not have a "kernel", but a Cygwin1.dll which is loaded into every Cygwin process. On places where it does path lookups or interferes with files (read, write, extended attributes, etc) it would require some means to communicate with the file system process and to wait for the response. This would make Cygwin processes accessing the file system 1st class citicens while Windows processes wouldn't lose compared to mode (3). In mode (3) Cygwin processes would be 2nd class, as i'll explain below. But the drawback of my approach is that it requires to reimplement FUSE for Cygwin. If i could, i would volunteer to work on it. i like Cygwin and i like FUSE, so i sure would like to work on that. Mode (4) would avoid the following issues coming from the double translation of mode (3): A) Besides the double double translations of every request, there is also the need for four Kernel/Usermode transitions, which take their time. B) We could have hardlinks for the Cygwin world only, and you have no need to implement them in FSD or the DLL. Though you could add that feature later not not at all. (Or would that be an issue e.g. with oplocs?) C) i guess this way we could support fuse with all other languages besides C/C++. D) Symlinks won't go through reparse points. You'd have no need to implement that or to follow Cygwin should Cygwin improve its way of translation. BTW, are code pages involved there? E) Same for uid/gid to SID mapping. No need to implement or follow Cygwin. And how about the case where a uid/gid has no correspoinding SID? Can this happen? F) Pipes: > [Quick experiment: > > $ mkfifo foo; cmd /c dir 'foo*' | grep foo; rm foo > 06/16/2016 11:02 PM 130 foo.lnk > ] > > Ok, so they are shortcuts. Naturally they are supported. I think they are not. The mkfifo system call will have Cygwin create a .lnk file and WinFsp will forward it as such to the file system process. The sytem calls readdir or open will then have the file system process tell WinFsp that there is a .lnk file and Cygwin will translate this back to a fifo, so in this sense it does work. But the file system will see a file (with name *.lnk) where it should see a pipe (mknod call with 'mode' set to S_IFIFO). IMHO one could say this is a break of the FUSE API. Practically it will break: - file systems that special-treat pipe files (or .lnk files). - If one uses sshfs to connect to a Linux based server and issues the command mkfifo foo from Cygwin, the server will end up with a .lnk file instead of a pipe special file. - Imagine something like mysqlfs, which stores the stuff in a database. When you run SQL statements to analyze the data in the file system, you won't see the pipes as such. Or if you open the file system from Linux you'll see the .lnk files. To fix this with mode (3) you'd have to recognize these .lnk files and forward them to the file system as pipes, and you'd have to generate .lnk files on the fly when the file system says there is a pipe file (e.g. on readdir). G) Case sensitivity. Windows is usually case insensitive but case preserving on file names. But there is a registry setting that allows one to activate case sensitivity. Then programs that use the Win32 API in a certain way can have case sensitivity. Cygwin has an option to do so, too. It is an option in the /etc/fstab file. (And i use it for all drives.) With this you can say touch a A in the Cygwin command line and you'll see two files in Windows Explorer. Double clicking one of them will always open the first file though. i guess FUSE file systems ported to mode (3) or (2) will expect file names to be case sensitive. https://cygwin.com/cygwin-ug-net/using-specialnames.html#pathnames-casesensitive What happens if you use sshfs to connect to a Linux box and then do the following in cmd.exe: # echo Hallo >Foo.txt # type foo.Txt H) Currently you have either "disk style" file system that you mount as a drive letter or a "network style" file sytem that you mount via UNC path, e.g. \\sshfs\name@host . Drive letters are a name space of only 26 entries which is very narrow, but you can also mount them to empty directories on other drives (which must support reparse points i think). I don't know if that's possible with UNC paths too. That means that Cygwin processes will see them as //sshfs/name@host and have to use symlinks or /etc/fstab settings to "mount" them elsewhere. It looks like mode (4) is much more work because it requires FUSE to be reimplemented/ported to Cygwin. But it's lesser work on WinFsp and would have no issues due to double translations. It would make Cygwin processes 1st class on FUSE file systems. So my question is to Corinna, would it be possible and feasible to implement this in Cygwin? best regards, Herbert -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple