FUSE, symbolic links and special files

FUSE, symbolic links and special files

[Bill Zissimopoulos]

While on vacation I have been (slowly) working to add reparse point and
symbolic link support for WinFsp and FUSE for Cygwin. This work is mostly
complete and is currently being tested.

I am writing to the Cygwin list because I want to resolve a problem that
Herbert Stocker originally brought up:

>F) Pipes:
>> [Quick experiment:
>>
>> $ mkfifo foo; cmd /c dir 'foo*' | grep foo; rm foo
>> 06/16/2016  11:02 PM               130 foo.lnk
>> ]
>>
>> Ok, so they are shortcuts. Naturally they are supported.
>
>    I think they are not.
>    The mkfifo system call will have Cygwin create a .lnk file and
>    WinFsp will forward it as such to the file system process. The
>    sytem calls readdir or open will then have the file system
>    process tell WinFsp that there is a .lnk file and Cygwin will
>    translate this back to a fifo, so in this sense it does work.
>
>    But the file system will see a file (with name *.lnk) where it
>    should see a pipe (mknod call with 'mode' set to S_IFIFO).
>    IMHO one could say this is a break of the FUSE API.
>
>    Practically it will break:
>     - file systems that special-treat pipe files (or .lnk files).
>
>     - If one uses sshfs to connect to a Linux based server and
>       issues the command mkfifo foo from Cygwin, the server will
>       end up with a .lnk file instead of a pipe special file.
>
>     - Imagine something like mysqlfs, which stores the stuff in a
>       database. When you run SQL statements to analyze the data
>       in the file system, you won't see the pipes as such. Or if
>       you open the file system from Linux you'll see the .lnk
>       files.

Herbert is of course right. A .lnk file is not a FIFO to any system other
than Cygwin. I have been thinking about how to solve this problem and I
believe I have an answer in the form of reparse points.

Reparse points can be viewed as a form of special metadata that can be
attached to a file or directory. The interesting thing about reparse
points is that they can have special meaning to a file system driver
(NTFS/WinFsp), a filter driver (e.g. a hierarchical storage system) or
even an application (Cygwin).

NTFS uses reparse points to implement symbolic links and places severe
limitations on them. For one, it differentiates between file and directory
symlinks. It also requires the SE_CREATE_SYMBOLIC_LINK_PRIVILEGE privilege
to be held to create a symbolic link account.

Interestingly it does not perform any extraneous access checks on other
reparse points. IMO this makes reparse points very interesting because it
offers a path for WinFsp and FUSE for Cygwin to provide special file
support.

Turns out that Microsoft already has a solution for special files on NFS:

https://msdn.microsoft.com/en-us/library/dn617178.aspx

I see no reason that the same solution cannot be used for FUSE for Cygwin
as well. In the following OP is the originating process, CW is the Cygwin
layer, WL is the WinFsp layer and FL is the FUSE layer.

OP: mkfifo("myfifo")
CW: NtCreateFile, NtDeviceIoControlFile(FSCTL_SET_REPARSE_POINT)
[NFS_SPECFILE_FIFO]
WL: IRP_MJ_FILE_SYSTEM_CONTROL/FSCTL_SET_REPARSE_POINT [NFS_SPECFILE_FIFO]
FL: fuse_operations::mknod("myfifo", S_IFIFO)

Regarding symbolic link support specifically I am still undecided on
whether the right thing to do is to use NTFS symbolic links
(IO_REPARSE_TAG_SYMLINK) or use NFS_SPECFILE_LNK for the FUSE layer. My
inclination is to support both and let the FUSE file system developer
decide on their preference.

Any comments welcome.

Bill

Re: FUSE, symbolic links and special files

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 2552 bytes --]

On Aug 25 11:46, Bill Zissimopoulos wrote:
> Turns out that Microsoft already has a solution for special files on NFS:
> 
> https://msdn.microsoft.com/en-us/library/dn617178.aspx
> 
> I see no reason that the same solution cannot be used for FUSE for Cygwin
> as well.

On NFS Cygwin utilizes the information available via extended attributes
so far.  There's an extended attribute you can give to NtCreateFile
to indicate that you want to act on the link, not on the file the link
points to.  The EA method is used for stata() as well to fetch the NFS
FATTR3 structure.  See winsup/cygwin/nfs.{cc,h} for details.

What bugs me is that I wasn't aware that there's also a reparse point
implementation for NFS.  When I wrote the original code accessing
NFS, the MSFT guys working on NFS gave me a hint or two how to access
this information, but there was no talk about reparse points at all :(

Since when is this RP method available?  Unfortunately the above MSDN
page doesn't tell...  Was it already available with Vista?  Does anybody
know?

Hmm.

This might allow to speed up symlink_info::check since the entire
code to read the EA could go away and the functionality of check_nfs_symlink
could be folded into check_reparse_point.  But...

In the following OP is the originating process, CW is the Cygwin
> layer, WL is the WinFsp layer and FL is the FUSE layer.
> 
> OP: mkfifo("myfifo")
> CW: NtCreateFile, NtDeviceIoControlFile(FSCTL_SET_REPARSE_POINT)
> [NFS_SPECFILE_FIFO]
> WL: IRP_MJ_FILE_SYSTEM_CONTROL/FSCTL_SET_REPARSE_POINT [NFS_SPECFILE_FIFO]
> FL: fuse_operations::mknod("myfifo", S_IFIFO)
> 
> Regarding symbolic link support specifically I am still undecided on
> whether the right thing to do is to use NTFS symbolic links
> (IO_REPARSE_TAG_SYMLINK) or use NFS_SPECFILE_LNK for the FUSE layer. My
> inclination is to support both and let the FUSE file system developer
> decide on their preference.
> 
> Any comments welcome.

...it needs thorough testing(*).  There's a good chance that the NFS RP
buffer is not exposed to user space, but instead only handled by the NFS
driver.  *If* the RP method works fine in user space, I'm inclined to do
as outlined above and get rid of the EA stuff in symlink_info::check
since it could be transparently shared between NFS and WinFSP.


Corinna

(*) https://cygwin.com/acronyms/#SHTDI

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: FUSE, symbolic links and special files

[Jeffrey Altman]

[-- Attachment #1: Type: text/plain, Size: 294 bytes --]

On 8/25/2016 8:45 AM, Corinna Vinschen wrote:
> Since when is this RP method available?  Unfortunately the above MSDN
> page doesn't tell...  Was it already available with Vista?  Does anybody
> know?

#define IO_REPARSE_TAG_NFS

was added in the Windows 8.0 DDK.

Jeffrey Altman



[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4349 bytes --]

Re: FUSE, symbolic links and special files

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 586 bytes --]

On Aug 25 09:04, Jeffrey Altman wrote:
> On 8/25/2016 8:45 AM, Corinna Vinschen wrote:
> > Since when is this RP method available?  Unfortunately the above MSDN
> > page doesn't tell...  Was it already available with Vista?  Does anybody
> > know?
> 
> #define IO_REPARSE_TAG_NFS
> 
> was added in the Windows 8.0 DDK.

Thank you.  Too bad, so we have to stick to the EA method to accommodate
Vista and W7.


Corinna



-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: FUSE, symbolic links and special files

[Jeffrey Altman]

[-- Attachment #1: Type: text/plain, Size: 947 bytes --]

On 8/25/2016 9:16 AM, Corinna Vinschen wrote:
> On Aug 25 09:04, Jeffrey Altman wrote:
>> On 8/25/2016 8:45 AM, Corinna Vinschen wrote:
>>> Since when is this RP method available?  Unfortunately the above MSDN
>>> page doesn't tell...  Was it already available with Vista?  Does anybody
>>> know?
>>
>> #define IO_REPARSE_TAG_NFS
>>
>> was added in the Windows 8.0 DDK.
> 
> Thank you.  Too bad, so we have to stick to the EA method to accommodate
> Vista and W7.

I don't believe there is any restriction from using this tag on Vista or
Win7.  It will be stored on NTFS just as any other Reparse Tag would be.
 NTFS and ReFS will ignore it just as they would on Win8 or Win10.

The only file system for which this tag is known to be interpreted is
the Microsoft NFS provider that will report its

  FILE_REMOTE_PROTOCOL_INFORMATION.Protocol

value as

  #define     WNNC_NET_MS_NFS      0x00420000

Jeffrey Altman



[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4349 bytes --]

Re: FUSE, symbolic links and special files

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1463 bytes --]

On Aug 25 10:46, Jeffrey Altman wrote:
> On 8/25/2016 9:16 AM, Corinna Vinschen wrote:
> > On Aug 25 09:04, Jeffrey Altman wrote:
> >> On 8/25/2016 8:45 AM, Corinna Vinschen wrote:
> >>> Since when is this RP method available?  Unfortunately the above MSDN
> >>> page doesn't tell...  Was it already available with Vista?  Does anybody
> >>> know?
> >>
> >> #define IO_REPARSE_TAG_NFS
> >>
> >> was added in the Windows 8.0 DDK.
> > 
> > Thank you.  Too bad, so we have to stick to the EA method to accommodate
> > Vista and W7.
> 
> I don't believe there is any restriction from using this tag on Vista or
> Win7.  It will be stored on NTFS just as any other Reparse Tag would be.
>  NTFS and ReFS will ignore it just as they would on Win8 or Win10.

Sorry, but I don't grok that.  Why would I create a reparse point on
NTFS with this value?  This RP type would only be interesting if I can
access the information for files on a non-Windows remote filesystem to
indicate special file types.

Granted, it *could* be used by Cygwin on NTFS to indicate Cygwin's own
implementations of AF_LOCAL sockets or fifos.  Or even for symlinks.
But that would only introduce YA symlink type which would be unusable
from non-Cygwin applications.

Or am I missing something?


Thanks,
Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: FUSE, symbolic links and special files

[Jeffrey Altman]

[-- Attachment #1: Type: text/plain, Size: 2977 bytes --]

On 8/25/2016 11:21 AM, Corinna Vinschen wrote:
> On Aug 25 10:46, Jeffrey Altman wrote:
>> On 8/25/2016 9:16 AM, Corinna Vinschen wrote:
>>> On Aug 25 09:04, Jeffrey Altman wrote:
>>>> On 8/25/2016 8:45 AM, Corinna Vinschen wrote:
>>>>> Since when is this RP method available?  Unfortunately the above MSDN
>>>>> page doesn't tell...  Was it already available with Vista?  Does anybody
>>>>> know?
>>>>
>>>> #define IO_REPARSE_TAG_NFS
>>>>
>>>> was added in the Windows 8.0 DDK.
>>>
>>> Thank you.  Too bad, so we have to stick to the EA method to accommodate
>>> Vista and W7.
>>
>> I don't believe there is any restriction from using this tag on Vista or
>> Win7.  It will be stored on NTFS just as any other Reparse Tag would be.
>>  NTFS and ReFS will ignore it just as they would on Win8 or Win10.
> 
> Sorry, but I don't grok that.  Why would I create a reparse point on
> NTFS with this value?  This RP type would only be interesting if I can
> access the information for files on a non-Windows remote filesystem to
> indicate special file types.

If you want the underlying remote file system to parse the value then
you can't set it anywhere except on the file system that returns
protocol type

  WNNC_NET_MS_NFS  0x00420000

because that is the only file system for which it is known supports the
Microsoft reparse tag

  IO_REPARSE_TAG_NFS                      (0x80000014L)

Since it is a Microsoft TAG it means that it can be interpreted by any
file system.  For example, I could add support for it to OpenAFS as an
alternative method of creating and reporting symlinks instead of

  IO_REPARSE_TAG_SYMLINK                  (0xA000000CL)

or

  IO_REPARSE_TAG_OPENAFS_DFS              (0x00000037L)

Only one of the reparse tag types can be reported at a time so the
choice of which reparse tag to use could be left up to configuration or
be set via an FSCTL on a per process basis.

Ideally Cygwin would obtain its own IO_REPARSE_TAG_xxxx value to use and
then any file system that wants to support Cygwin can simply add support
for it.  Whether that be FUSE or OpenAFS or an NFS implementation, etc.

FUSE should also be sure to obtain its own WNNC_NET_xxxx value to use.

> Granted, it *could* be used by Cygwin on NTFS to indicate Cygwin's own
> implementations of AF_LOCAL sockets or fifos.  Or even for symlinks.
> But that would only introduce YA symlink type which would be unusable
> from non-Cygwin applications.

Correct.

With its own reparse tag Cygwin could store exactly the same metadata it
stores today in the data stream of the .lnk file as reparse tag data.
The benefit of applying a reparse tag is that the .lnk will no longer be
confused for a regular file.   On file systems that do not support
reparse points it can continue to store the data in the data stream.

> Or am I missing something?

I doubt it.

> 
> Thanks,
> Corinna

Anytime.

Jeffrey Altman



[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4349 bytes --]

Re: FUSE, symbolic links and special files

[Bill Zissimopoulos]

On 8/25/16, 3:45 PM, Corinna Vinschen wrote:

>On Aug 25 11:46, Bill Zissimopoulos wrote:
>>In the following OP is the originating process, CW is the Cygwin
>> layer, WL is the WinFsp layer and FL is the FUSE layer.
>> 
>> OP: mkfifo("myfifo")
>> CW: NtCreateFile, NtDeviceIoControlFile(FSCTL_SET_REPARSE_POINT)
>> [NFS_SPECFILE_FIFO]
>> WL: IRP_MJ_FILE_SYSTEM_CONTROL/FSCTL_SET_REPARSE_POINT
>>[NFS_SPECFILE_FIFO]
>> FL: fuse_operations::mknod("myfifo", S_IFIFO)
>> 
>> Regarding symbolic link support specifically I am still undecided on
>> whether the right thing to do is to use NTFS symbolic links
>> (IO_REPARSE_TAG_SYMLINK) or use NFS_SPECFILE_LNK for the FUSE layer. My
>> inclination is to support both and let the FUSE file system developer
>> decide on their preference.
>> 
>> Any comments welcome.
>
>...it needs thorough testing(*).  There's a good chance that the NFS RP
>buffer is not exposed to user space, but instead only handled by the NFS
>driver.  *If* the RP method works fine in user space, I'm inclined to do
>as outlined above and get rid of the EA stuff in symlink_info::check
>since it could be transparently shared between NFS and WinFSP.

I agree. FYI I have not tested the use of NFS reparse points yet, although
I intend to.

My expectation is that there should not be any issue accessing such
reparse points from user mode. My understanding of the reparse point
mechanism is that it comes into play in a couple of cases:

- The first case is during the processing of NtCreateFile (without the
FILE_OPEN_REPARSE_POINT flag set). In this case the file system driver
will return STATUS_REPARSE and either:
	- Return a path that NTOS will resend to the file system driver for
further processing (useful for symlink-style processing), or...
	- Return a reparse data buffer that some higher-level component (e.g. a
filter driver) may interpret specially. For example, a hierarchical
storage management system may download a file pointed to by a reparse
point locally. If there is no higher-level component that knows about the
reparse point, I believe the NTOS IO manager will return
STATUS_IO_REPARSE_TAG_NOT_HANDLED.

- The second case is through direct manipulation of the reparse point
using FSCTL_GET_REPARSE_POINT, FSCTL_SET_REPARSE_POINT and
FSCTL_DELETE_REPARSE_POINT.

Let us consider the expected behavior of an NFS_SPECFILE_LNK reparse point
(this is speculation) during NtCreateFile:

- On NTFS prior to Win8:
	- STATUS_IO_REPARSE_TAG_NOT_HANDLED

- On NFS prior to Win8:
	- STATUS_IO_REPARSE_TAG_NOT_HANDLED

- On FUSE for Cygwin prior to Win8:
	- STATUS_SUCCESS with file pointed by symlink opened (because the WinFsp
FSD would know how to handle NFS reparse points).

- On NTFS after Win8:
	- Likely STATUS_IO_REPARSE_TAG_NOT_HANDLED, although there is a
possibility that Microsoft has built knowledge of the NFS_SPECFILE_LNK
reparse point into the NTFS FSD or another filter driver, in which case
the file would be opened and the return would be STATUS_SUCCESS.

- On NFS after Win8:
	- STATUS_SUCCESS with file pointed by symlink opened.

- On FUSE for Cygwin prior to Win8:
	- STATUS_SUCCESS with file pointed by symlink opened.


This is not ideal, because sometimes an NtCreateFile of an
NFS_SPECFILE_LNK reparse point will result in STATUS_SUCCESS and sometimes
it will result in STATUS_IO_REPARSE_TAG_NOT_HANDLED. I can imagine of
course that the NtCreateFile call could be retried with
FILE_OPEN_REPARSE_POINT followed by FSCTL_GET_REPARSE_POINT and symlink
resolution could happen in user mode.

Bill

Re: FUSE, symbolic links and special files

[Bill Zissimopoulos]

On 8/25/16, 7:14 PM, Jeffrey Altman wrote:

>On 8/25/2016 11:21 AM, Corinna Vinschen wrote:
>>Granted, it *could* be used by Cygwin on NTFS to indicate Cygwin's own
>> implementations of AF_LOCAL sockets or fifos.  Or even for symlinks.
>> But that would only introduce YA symlink type which would be unusable
>> from non-Cygwin applications.
>
>Correct.
>
>With its own reparse tag Cygwin could store exactly the same metadata it
>stores today in the data stream of the .lnk file as reparse tag data.
>The benefit of applying a reparse tag is that the .lnk will no longer be
>confused for a regular file.   On file systems that do not support
>reparse points it can continue to store the data in the data stream.

I agree with Jeffrey, and I did think of this as a potential solution for
FUSE for Cygwin (and perhaps Cygwin itself).

But since I learned about the NFS reparse points I find them a very good
solution, especially because they indicate that someone in Microsoft has
already thought about this problem.

Bill

Re: FUSE, symbolic links and special files

[Bill Zissimopoulos]

On 8/25/16, 5:46 PM, Jeffrey Altman wrote:

>The only file system for which this tag is known to be interpreted is
>the Microsoft NFS provider that will report its
>
>  FILE_REMOTE_PROTOCOL_INFORMATION.Protocol
>
>value as
>
>  #define     WNNC_NET_MS_NFS      0x00420000

I missed this. Jeffrey do you have a pointer for this information?

Bill

Re: FUSE, symbolic links and special files

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1636 bytes --]

On Aug 25 19:04, Bill Zissimopoulos wrote:
> On 8/25/16, 3:45 PM, Corinna Vinschen wrote:
> >...it needs thorough testing(*).  There's a good chance that the NFS RP
> >buffer is not exposed to user space, but instead only handled by the NFS
> >driver.  *If* the RP method works fine in user space, I'm inclined to do
> >as outlined above and get rid of the EA stuff in symlink_info::check
> >since it could be transparently shared between NFS and WinFSP.
> 
> I agree. FYI I have not tested the use of NFS reparse points yet, although
> I intend to.
> 
> My expectation is that there should not be any issue accessing such
> reparse points from user mode. My understanding of the reparse point
> mechanism is that it comes into play in a couple of cases:

No, me neither, but the MSDN documentation is, shall we say, limited...

> - The first case is during the processing of NtCreateFile (without the
> FILE_OPEN_REPARSE_POINT flag set).

This case doesn't matter to us.  Cygwin always opens the file with
FILE_OPEN_REPARSE_POINT set...

> - The second case is through direct manipulation of the reparse point
> using FSCTL_GET_REPARSE_POINT, FSCTL_SET_REPARSE_POINT and
> FSCTL_DELETE_REPARSE_POINT.
> 
> Let us consider the expected behavior of an NFS_SPECFILE_LNK reparse point
> (this is speculation) during NtCreateFile:
> 
> - On NTFS prior to Win8:
> 	- STATUS_IO_REPARSE_TAG_NOT_HANDLED

...so this shouldn't happen to us, right?


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: FUSE, symbolic links and special files

[Bill Zissimopoulos]

On 8/26/16, 11:05 AM, Corinna Vinschen wrote:

>On Aug 25 19:04, Bill Zissimopoulos wrote:
>>- The first case is during the processing of NtCreateFile (without the
>> FILE_OPEN_REPARSE_POINT flag set).
>
>This case doesn't matter to us.  Cygwin always opens the file with
>FILE_OPEN_REPARSE_POINT set...
>
>> - The second case is through direct manipulation of the reparse point
>> using FSCTL_GET_REPARSE_POINT, FSCTL_SET_REPARSE_POINT and
>> FSCTL_DELETE_REPARSE_POINT.
>> 
>> Let us consider the expected behavior of an NFS_SPECFILE_LNK reparse
>>point
>> (this is speculation) during NtCreateFile:
>> 
>> - On NTFS prior to Win8:
>> 	- STATUS_IO_REPARSE_TAG_NOT_HANDLED
>
>...so this shouldn't happen to us, right?

I think so.

I will continue with the implementation/testing of reparse points and
report back when I have more.

Bill

Re: FUSE, symbolic links and special files

[Bill Zissimopoulos]

On 8/26/16, 4:59 PM, cygwin-owner@cygwin.com Bill Zissimopoulos wrote:


>On 8/26/16, 11:05 AM, Corinna Vinschen wrote:
>
>>On Aug 25 19:04, Bill Zissimopoulos wrote:
>>>- The first case is during the processing of NtCreateFile (without the
>>> FILE_OPEN_REPARSE_POINT flag set).
>>
>>This case doesn't matter to us.  Cygwin always opens the file with
>>FILE_OPEN_REPARSE_POINT set...
>>
>>> - The second case is through direct manipulation of the reparse point
>>> using FSCTL_GET_REPARSE_POINT, FSCTL_SET_REPARSE_POINT and
>>> FSCTL_DELETE_REPARSE_POINT.
>>> 
>>> Let us consider the expected behavior of an NFS_SPECFILE_LNK reparse
>>>point
>>> (this is speculation) during NtCreateFile:
>>> 
>>> - On NTFS prior to Win8:
>>> 	- STATUS_IO_REPARSE_TAG_NOT_HANDLED
>>
>>...so this shouldn't happen to us, right?
>
>I think so.
>
>I will continue with the implementation/testing of reparse points and
>report back when I have more.

I have finally completed the implementation of reparse points for WinFsp.
I have also implemented NFS reparse point support for WinFsp-FUSE.

I tested this implementation with Cygwin although my testing was general
command line use and not rigorous. For this purpose I cobbled together a
patch for Cygwin; the patch is low quality and I hesitate to post it here,
but I can if so desired. Effectively I changed the implementation of
Cygwin’s mknod_worker, symlink_info::check_reparse_point and
readdir_check_reparse_point to understand and use NFS reparse points.

I am unclear on how to proceed from here. Although I do not understand the
Cygwin internals well enough and have rather limited time at the moment I
could try cleaning up the patch and officially submitting.

I am also thinking that using reparse points for POSIX special files on
other file systems that support it (i.e. NTFS) may be something that the
list should consider.

Bill