mkfifo: cannot set permissions of 'x.fifo': Not a directory

mkfifo: cannot set permissions of 'x.fifo': Not a directory

[Martin Wege]

Hello,

We get a weird mkfifo failure in Cygwin on NTFS:

/usr/bin/mkfifo -m 600 x.fifo
mkfifo: cannot set permissions of 'x.fifo': Not a directory

Is there a known workaround? So far named fifos cannot be created somehow.

Thanks,
Martin

Re: mkfifo: cannot set permissions of 'x.fifo': Not a directory

[Brian Inglis]

[-- Attachment #1: Type: text/plain, Size: 1336 bytes --]

On 2023-08-07 08:42, Martin Wege via Cygwin wrote:
> We get a weird mkfifo failure in Cygwin on NTFS:
> 
> /usr/bin/mkfifo -m 600 x.fifo
> mkfifo: cannot set permissions of 'x.fifo': Not a directory

This error is also reproducible with default fifo permissions 666 or anything 
else, and appears to be returned by gen_full_path_at() in syscalls.cc when 
called with a fifo: see attached strace extract - underlying Windows file is 
Windows shortcut x.fifo.lnk.

> Is there a known workaround? So far named fifos cannot be created somehow.

For now, create with default permissions then chmod:

$ mkfifo x.fifo && chmod 600 x.fifo
mode of 'x.fifo' changed from 0666 (rw-rw-rw-) to 0600 (rw-------)
$ ls -glo x.fifo{,.lnk}
prw------- 1 0 Aug  7 12:07 x.fifo
prw------- 1 0 Aug  7 12:07 x.fifo.lnk
$ readshortcut -af x.fifo
Target: /usr/src/
Working Directory: /usr/src/
Arguments:
Show Command: Normal
Icon Library: /usr/src/
Icon Library Offset: 0
Description: :\0:c4:1180

-- 
Take care. Thanks, Brian Inglis              Calgary, Alberta, Canada

La perfection est atteinte                   Perfection is achieved
non pas lorsqu'il n'y a plus rien à ajouter  not when there is no more to add
mais lorsqu'il n'y a plus rien à retirer     but when there is no more to cut
                                 -- Antoine de Saint-Exupéry

[-- Attachment #2: mkfifo-m666.strace.log --]
[-- Type: text/plain, Size: 1890 bytes --]

open: open(x.fifo, 0x2140000)
normalize_posix_path: src x.fifo
cwdstuff::get: posix /usr/src
cwdstuff::get: (/usr/src) = cwdstuff::get (0x1250720, 32768, 1, 0), errno 0
normalize_posix_path: /usr/src/x.fifo = normalize_posix_path (x.fifo)
mount_info::conv_to_win32_path: conv_to_win32_path (/usr/src/x.fifo)
mount_info::conv_to_win32_path: src_path /usr/src/x.fifo, dst C:/.../cygwin64/usr/src/x.fifo, flags 0x30008, rc 0
symlink_info::check: 0xC0000034 = NtCreateFile (/??/C:/.../cygwin64/usr/src/x.fifo)
symlink_info::check: 0xC0000034 = NtQueryInformationFile (/??/C:/.../cygwin64/usr/src/x.fifo)
symlink_info::check: 0xC0000034 = NtCreateFile (/??/C:/.../cygwin64/usr/src/x.fifo.exe)
symlink_info::check: 0xC0000034 = NtQueryInformationFile (/??/C:/.../cygwin64/usr/src/x.fifo.exe)
symlink_info::check: 0x0 = NtCreateFile (/??/C:/.../cygwin64/usr/src/x.fifo.lnk)
symlink_info::check: is a device
symlink_info::check: 0 = symlink.check(C:/.../cygwin64/usr/src/x.fifo.lnk, 0x7FFFFB760) (mount_flags 0x30008, path_flags 0x14)
path_conv::check: this->path(C:/.../cygwin64/usr/src/x.fifo.lnk), has_acls(1)
build_fh_pc: fh 0x800008DF0, dev 000000C4
fhandler_base::open: (/??/C:/.../cygwin64/usr/src/x.fifo.lnk, 0x2148000)
fhandler_base::set_flags: flags 0x2148000, supplied_bin 0x10000
fhandler_base::set_flags: filemode set to binary
fhandler_base::open: 0x0 = NtCreateFile (0x1F8, 0x20080, /??/C:/.../cygwin64/usr/src/x.fifo.lnk, io, NULL, 0x0, 0x7, 0x1, 0x4000, NULL, 0)
fhandler_base::open: 1 = fhandler_base::open(/??/C:/.../cygwin64/usr/src/x.fifo.lnk, 0x2148000)
fhandler_base::open_fs: 1 = fhandler_disk_file::open(/??/C:/.../cygwin64/usr/src/x.fifo.lnk, 0x2148000)
open: 3 = open(x.fifo, 0x2148000)
__set_errno: int gen_full_path_at(char*, int, const char*, bool):4445 setting errno 20
close: close(3)
fhandler_base::close: closing '/usr/src/x.fifo' handle 0x1F8
close: 0 = close(3)

Re: mkfifo: cannot set permissions of 'x.fifo': Not a directory

[Corinna Vinschen]

On Aug  7 12:26, Brian Inglis via Cygwin wrote:
> On 2023-08-07 08:42, Martin Wege via Cygwin wrote:
> > We get a weird mkfifo failure in Cygwin on NTFS:
> > 
> > /usr/bin/mkfifo -m 600 x.fifo
> > mkfifo: cannot set permissions of 'x.fifo': Not a directory
> 
> This error is also reproducible with default fifo permissions 666 or
> anything else, and appears to be returned by gen_full_path_at() in
> syscalls.cc when called with a fifo:

This will be fixed in 3.4.8, see
https://cygwin.com/git/?p=newlib-cygwin.git;a=commit;h=b56b4d7fd85b


Corinna

Re: mkfifo: cannot set permissions of 'x.fifo': Not a directory

[Martin Wege]

On Mon, Aug 7, 2023 at 9:14 PM Corinna Vinschen
<corinna-cygwin@cygwin.com> wrote:
>
> On Aug  7 12:26, Brian Inglis via Cygwin wrote:
> > On 2023-08-07 08:42, Martin Wege via Cygwin wrote:
> > > We get a weird mkfifo failure in Cygwin on NTFS:
> > >
> > > /usr/bin/mkfifo -m 600 x.fifo
> > > mkfifo: cannot set permissions of 'x.fifo': Not a directory
> >
> > This error is also reproducible with default fifo permissions 666 or
> > anything else, and appears to be returned by gen_full_path_at() in
> > syscalls.cc when called with a fifo:
>
> This will be fixed in 3.4.8,

Thank you

> see
> https://cygwin.com/git/?p=newlib-cygwin.git;a=commit;h=b56b4d7fd85b

First, Can I download a beta of  3.4.8, or any other version with that
fix? It's pretty urgent

Secondly, how are fifos, other device nodes and mknod implemented? You
encode them as softlinks on ntfs? What about Windows native NFSv3
driver, do you use native mknod then to create a fifo? Where can I
find the code in cygwin for that?

Thanks,
Martin

Re: mkfifo: cannot set permissions of 'x.fifo': Not a directory

[Corinna Vinschen]

On Aug  7 23:14, Martin Wege via Cygwin wrote:
> On Mon, Aug 7, 2023 at 9:14 PM Corinna Vinschen
> <corinna-cygwin@cygwin.com> wrote:
> >
> > On Aug  7 12:26, Brian Inglis via Cygwin wrote:
> > > On 2023-08-07 08:42, Martin Wege via Cygwin wrote:
> > > > We get a weird mkfifo failure in Cygwin on NTFS:
> > > >
> > > > /usr/bin/mkfifo -m 600 x.fifo
> > > > mkfifo: cannot set permissions of 'x.fifo': Not a directory
> > >
> > > This error is also reproducible with default fifo permissions 666 or
> > > anything else, and appears to be returned by gen_full_path_at() in
> > > syscalls.cc when called with a fifo:
> >
> > This will be fixed in 3.4.8,
> 
> Thank you
> 
> > see
> > https://cygwin.com/git/?p=newlib-cygwin.git;a=commit;h=b56b4d7fd85b
> 
> First, Can I download a beta of  3.4.8, or any other version with that
> fix? It's pretty urgent

You can install a test release of Cygwin, simply by using the setup tool
These test releases are always created from the git main branch, though,
so they are always test releases of the next major version, currently
3.5.0.

> Secondly, how are fifos, other device nodes and mknod implemented? You
> encode them as softlinks on ntfs?

Not only on NTFS.  FIFOs on the disk are always created as symlinks
with a special target string, recognized by Cygwin.  No differece on
NFS.  Microsoft NFSv3 has no way to specify creating a "real" FIFO
on the remote filesystem.  It wouldn't make sense anyway, because
there's no way to share the FIFO across systems.


Corinna

Re: mkfifo: cannot set permissions of 'x.fifo': Not a directory

[Martin Wege]

On Tue, Aug 8, 2023 at 10:20 AM Corinna Vinschen
<corinna-cygwin@cygwin.com> wrote:
>
> On Aug  7 23:14, Martin Wege via Cygwin wrote:
> > On Mon, Aug 7, 2023 at 9:14 PM Corinna Vinschen
> > <corinna-cygwin@cygwin.com> wrote:
> > >
> > > On Aug  7 12:26, Brian Inglis via Cygwin wrote:
> > > > On 2023-08-07 08:42, Martin Wege via Cygwin wrote:
> > > > > We get a weird mkfifo failure in Cygwin on NTFS:
> > > > >
> > > > > /usr/bin/mkfifo -m 600 x.fifo
> > > > > mkfifo: cannot set permissions of 'x.fifo': Not a directory
> > > >
> > > > This error is also reproducible with default fifo permissions 666 or
> > > > anything else, and appears to be returned by gen_full_path_at() in
> > > > syscalls.cc when called with a fifo:
> > >
> > > This will be fixed in 3.4.8,
> >
> > Thank you
> >
> > > see
> > > https://cygwin.com/git/?p=newlib-cygwin.git;a=commit;h=b56b4d7fd85b
> >
> > First, Can I download a beta of  3.4.8, or any other version with that
> > fix? It's pretty urgent
>
> You can install a test release of Cygwin, simply by using the setup tool
> These test releases are always created from the git main branch, though,
> so they are always test releases of the next major version, currently
> 3.5.0.

Where can I download the Setup.exe tool for those test releases?

>
> > Secondly, how are fifos, other device nodes and mknod implemented? You
> > encode them as softlinks on ntfs?
>
> Not only on NTFS.  FIFOs on the disk are always created as symlinks
> with a special target string, recognized by Cygwin.  No differece on
> NFS.  Microsoft NFSv3 has no way to specify creating a "real" FIFO
> on the remote filesystem.  It wouldn't make sense anyway, because
> there's no way to share the FIFO across systems.

So no Windows ioctl to do a mknod?

What does a Linux/FreeBSD FIFO file look like, if you export that
filesystem via NFSv3, and mount it via the native Windows NFSv3
driver, and do a ls -l in Cygwin?

Thanks,
Martin

Re: mkfifo: cannot set permissions of 'x.fifo': Not a directory

[Corinna Vinschen]

On Aug  8 21:38, Martin Wege via Cygwin wrote:
> On Tue, Aug 8, 2023 at 10:20 AM Corinna Vinschen
> <corinna-cygwin@cygwin.com> wrote:
> >
> > On Aug  7 23:14, Martin Wege via Cygwin wrote:
> > > On Mon, Aug 7, 2023 at 9:14 PM Corinna Vinschen
> > > <corinna-cygwin@cygwin.com> wrote:
> > > >
> > > > On Aug  7 12:26, Brian Inglis via Cygwin wrote:
> > > > > On 2023-08-07 08:42, Martin Wege via Cygwin wrote:
> > > > > > We get a weird mkfifo failure in Cygwin on NTFS:
> > > > > >
> > > > > > /usr/bin/mkfifo -m 600 x.fifo
> > > > > > mkfifo: cannot set permissions of 'x.fifo': Not a directory
> > > > >
> > > > > This error is also reproducible with default fifo permissions 666 or
> > > > > anything else, and appears to be returned by gen_full_path_at() in
> > > > > syscalls.cc when called with a fifo:
> > > >
> > > > This will be fixed in 3.4.8,
> > >
> > > Thank you
> > >
> > > > see
> > > > https://cygwin.com/git/?p=newlib-cygwin.git;a=commit;h=b56b4d7fd85b
> > >
> > > First, Can I download a beta of  3.4.8, or any other version with that
> > > fix? It's pretty urgent
> >
> > You can install a test release of Cygwin, simply by using the setup tool
> > These test releases are always created from the git main branch, though,
> > so they are always test releases of the next major version, currently
> > 3.5.0.
> 
> Where can I download the Setup.exe tool for those test releases?

You already have it: https://cygwin.com/install.html

> > > Secondly, how are fifos, other device nodes and mknod implemented? You
> > > encode them as softlinks on ntfs?
> >
> > Not only on NTFS.  FIFOs on the disk are always created as symlinks
> > with a special target string, recognized by Cygwin.  No differece on
> > NFS.  Microsoft NFSv3 has no way to specify creating a "real" FIFO
> > on the remote filesystem.  It wouldn't make sense anyway, because
> > there's no way to share the FIFO across systems.
> 
> So no Windows ioctl to do a mknod?

Windows does not have provisions to store FIFOs on disk.  The closest
you get in Windows are Windows named pipes, and those are stored
non-permanently in a special, temporary filesystem inside the native NT
namespace.  Try this in Cygwin:

  $ cd /proc/sys/Device/NamedPipe/    <-- trailing slash!
  $ ls -l

FIFOs in the normal Windows filesystem are a Cygwin fake.

> What does a Linux/FreeBSD FIFO file look like, if you export that
> filesystem via NFSv3, and mount it via the native Windows NFSv3
> driver, and do a ls -l in Cygwin?

It looks like a FIFO because MSFT NFSv3 exports the stat(2) info
verbatim:

linux$ mkfifo blubb

cygwin$ ls -l blubb
prw-r--r-- 1 corinna vinschen 0 Aug  8 10:22 blubb

In Windows Explorer and cmd.exe its a file of size 0 with the
"System" attribute set.

In Samba it looks like an ordinary file of size 0.


Corinna

Re: mkfifo: cannot set permissions of 'x.fifo': Not a directory

[Martin Wege]

On Wed, Aug 9, 2023 at 10:01 AM Corinna Vinschen
<corinna-cygwin@cygwin.com> wrote:
>
> On Aug  8 21:38, Martin Wege via Cygwin wrote:
> > On Tue, Aug 8, 2023 at 10:20 AM Corinna Vinschen
> > <corinna-cygwin@cygwin.com> wrote:
> > >
> > > On Aug  7 23:14, Martin Wege via Cygwin wrote:
> > > > On Mon, Aug 7, 2023 at 9:14 PM Corinna Vinschen
> > > > <corinna-cygwin@cygwin.com> wrote:
> > > > >
> > > > > On Aug  7 12:26, Brian Inglis via Cygwin wrote:
> > > > > > On 2023-08-07 08:42, Martin Wege via Cygwin wrote:
> > > > > > > We get a weird mkfifo failure in Cygwin on NTFS:
> > > > > > >
> > > > > > > /usr/bin/mkfifo -m 600 x.fifo
> > > > > > > mkfifo: cannot set permissions of 'x.fifo': Not a directory
> > > > > >
> > > > > > This error is also reproducible with default fifo permissions 666 or
> > > > > > anything else, and appears to be returned by gen_full_path_at() in
> > > > > > syscalls.cc when called with a fifo:
> > > > >
> > > > > This will be fixed in 3.4.8,
> > > >
> > > > Thank you
> > > >
> > > > > see
> > > > > https://cygwin.com/git/?p=newlib-cygwin.git;a=commit;h=b56b4d7fd85b
> > > >
> > > > First, Can I download a beta of  3.4.8, or any other version with that
> > > > fix? It's pretty urgent
> > >
> > > You can install a test release of Cygwin, simply by using the setup tool
> > > These test releases are always created from the git main branch, though,
> > > so they are always test releases of the next major version, currently
> > > 3.5.0.
> >
> > Where can I download the Setup.exe tool for those test releases?
>
> You already have it: https://cygwin.com/install.html

Thanks

>
> > > > Secondly, how are fifos, other device nodes and mknod implemented? You
> > > > encode them as softlinks on ntfs?
> > >
> > > Not only on NTFS.  FIFOs on the disk are always created as symlinks
> > > with a special target string, recognized by Cygwin.  No differece on
> > > NFS.  Microsoft NFSv3 has no way to specify creating a "real" FIFO
> > > on the remote filesystem.  It wouldn't make sense anyway, because
> > > there's no way to share the FIFO across systems.
> >
> > So no Windows ioctl to do a mknod?
>
> Windows does not have provisions to store FIFOs on disk.  The closest
> you get in Windows are Windows named pipes, and those are stored
> non-permanently in a special, temporary filesystem inside the native NT
> namespace.  Try this in Cygwin:
>
>   $ cd /proc/sys/Device/NamedPipe/    <-- trailing slash!
>   $ ls -l
>
> FIFOs in the normal Windows filesystem are a Cygwin fake.

Maybe make pipes in Cygwin symlinks to /proc/sys/Device/NamedPipe/?

> > What does a Linux/FreeBSD FIFO file look like, if you export that
> > filesystem via NFSv3, and mount it via the native Windows NFSv3
> > driver, and do a ls -l in Cygwin?
>
> It looks like a FIFO because MSFT NFSv3 exports the stat(2) info
> verbatim:
>
> linux$ mkfifo blubb
>
> cygwin$ ls -l blubb
> prw-r--r-- 1 corinna vinschen 0 Aug  8 10:22 blubb

Will cygwin treat this as a FIFO?

Can cygwin set the 'p' flag via a magic ioctl?

Thanks,
Martin

Re: mkfifo: cannot set permissions of 'x.fifo': Not a directory

[Corinna Vinschen]

On Aug  9 11:12, Martin Wege via Cygwin wrote:
> On Wed, Aug 9, 2023 at 10:01 AM Corinna Vinschen
> <corinna-cygwin@cygwin.com> wrote:
> >
> > On Aug  8 21:38, Martin Wege via Cygwin wrote:
> > > On Tue, Aug 8, 2023 at 10:20 AM Corinna Vinschen
> > > <corinna-cygwin@cygwin.com> wrote:
> > > >
> > > > On Aug  7 23:14, Martin Wege via Cygwin wrote:
> > > > > Secondly, how are fifos, other device nodes and mknod implemented? You
> > > > > encode them as softlinks on ntfs?
> > > >
> > > > Not only on NTFS.  FIFOs on the disk are always created as symlinks
> > > > with a special target string, recognized by Cygwin.  No differece on
> > > > NFS.  Microsoft NFSv3 has no way to specify creating a "real" FIFO
> > > > on the remote filesystem.  It wouldn't make sense anyway, because
> > > > there's no way to share the FIFO across systems.
> > >
> > > So no Windows ioctl to do a mknod?
> >
> > Windows does not have provisions to store FIFOs on disk.  The closest
> > you get in Windows are Windows named pipes, and those are stored
> > non-permanently in a special, temporary filesystem inside the native NT
> > namespace.  Try this in Cygwin:
> >
> >   $ cd /proc/sys/Device/NamedPipe/    <-- trailing slash!
> >   $ ls -l
> >
> > FIFOs in the normal Windows filesystem are a Cygwin fake.
> 
> Maybe make pipes in Cygwin symlinks to /proc/sys/Device/NamedPipe/?

Why?  Windows Named Pipes don't act like UNIX FIFOs.

> > > What does a Linux/FreeBSD FIFO file look like, if you export that
> > > filesystem via NFSv3, and mount it via the native Windows NFSv3
> > > driver, and do a ls -l in Cygwin?
> >
> > It looks like a FIFO because MSFT NFSv3 exports the stat(2) info
> > verbatim:
> >
> > linux$ mkfifo blubb
> >
> > cygwin$ ls -l blubb
> > prw-r--r-- 1 corinna vinschen 0 Aug  8 10:22 blubb
> 
> Will cygwin treat this as a FIFO?

No.  Cygwin FIFOs != Unix FIFOs.

> Can cygwin set the 'p' flag via a magic ioctl?

No.  I don't understand your insistence to use FIFOs on NFS.  There is
no value in that, because you can't share them between systems anyway.

https://www.unix.com/unix-for-advanced-and-expert-users/10861-fifo-over-nfs.html
https://stackoverflow.com/questions/1038788/cant-write-to-fifo-file-mouted-via-nfs


Corinna

Re: mkfifo: cannot set permissions of 'x.fifo': Not a directory

[Martin Wege]

On Wed, Aug 9, 2023 at 11:56 AM Corinna Vinschen
<corinna-cygwin@cygwin.com> wrote:
>
> On Aug  9 11:12, Martin Wege via Cygwin wrote:
> > On Wed, Aug 9, 2023 at 10:01 AM Corinna Vinschen
> > <corinna-cygwin@cygwin.com> wrote:
> > >
> > > On Aug  8 21:38, Martin Wege via Cygwin wrote:
> > > > On Tue, Aug 8, 2023 at 10:20 AM Corinna Vinschen
> > > > <corinna-cygwin@cygwin.com> wrote:
> > > > >
> > > > > On Aug  7 23:14, Martin Wege via Cygwin wrote:
> > > > > > Secondly, how are fifos, other device nodes and mknod implemented? You
> > > > > > encode them as softlinks on ntfs?
> > > > >
> > > > > Not only on NTFS.  FIFOs on the disk are always created as symlinks
> > > > > with a special target string, recognized by Cygwin.  No differece on
> > > > > NFS.  Microsoft NFSv3 has no way to specify creating a "real" FIFO
> > > > > on the remote filesystem.  It wouldn't make sense anyway, because
> > > > > there's no way to share the FIFO across systems.
> > > >
> > > > So no Windows ioctl to do a mknod?
> > >
> > > Windows does not have provisions to store FIFOs on disk.  The closest
> > > you get in Windows are Windows named pipes, and those are stored
> > > non-permanently in a special, temporary filesystem inside the native NT
> > > namespace.  Try this in Cygwin:
> > >
> > >   $ cd /proc/sys/Device/NamedPipe/    <-- trailing slash!
> > >   $ ls -l
> > >
> > > FIFOs in the normal Windows filesystem are a Cygwin fake.
> >
> > Maybe make pipes in Cygwin symlinks to /proc/sys/Device/NamedPipe/?
>
> Why?  Windows Named Pipes don't act like UNIX FIFOs.

OK, dumb-user-question: Where is the difference?

>
> > > > What does a Linux/FreeBSD FIFO file look like, if you export that
> > > > filesystem via NFSv3, and mount it via the native Windows NFSv3
> > > > driver, and do a ls -l in Cygwin?
> > >
> > > It looks like a FIFO because MSFT NFSv3 exports the stat(2) info
> > > verbatim:
> > >
> > > linux$ mkfifo blubb
> > >
> > > cygwin$ ls -l blubb
> > > prw-r--r-- 1 corinna vinschen 0 Aug  8 10:22 blubb
> >
> > Will cygwin treat this as a FIFO?
>
> No.  Cygwin FIFOs != Unix FIFOs.
>
> > Can cygwin set the 'p' flag via a magic ioctl?
>
> No.  I don't understand your insistence to use FIFOs on NFS.  There is
> no value in that, because you can't share them between systems anyway.
>
> https://www.unix.com/unix-for-advanced-and-expert-users/10861-fifo-over-nfs.html
> https://stackoverflow.com/questions/1038788/cant-write-to-fifo-file-mouted-via-nfs

I'm aware of that.

But think about clusters of machines, or similar setups where many
machines from different architectures share one NFS filesystem, and
cooperatively work on that.
So for example a master machine sets up everything for the build
client machines, including generating FIFOs, which shall be used
locally on the client. But then... oopsie, client cannot use the FIFO
prepared by the cluster master, because it doesn't create the FIFO
mknod nodes the same way as other platforms.

Most of the concerns are about portability.

Thanks,
Martin

Re: mkfifo: cannot set permissions of 'x.fifo': Not a directory

[Martin Wege]

On Mon, Aug 7, 2023 at 4:42 PM Martin Wege <martin.l.wege@gmail.com> wrote:
>
> Hello,
>
> We get a weird mkfifo failure in Cygwin on NTFS:
>
> /usr/bin/mkfifo -m 600 x.fifo
> mkfifo: cannot set permissions of 'x.fifo': Not a directory
>
> Is there a known workaround? So far named fifos cannot be created somehow.

This is fixed for NTFS, but still broken for Windows builtin NFS v3 client:
$ mkfifo -m 0600 hellofifo.fifo
mkfifo: cannot set permissions of 'hellofifo.fifo': No such file or directory
$ ls -l hellofifo.fifo
lrwxrwxrwx 1 Unix_User+3640 Unix_Group+3640 11 Aug 18 05:58
hellofifo.fifo -> ':\0:c4:1180'
$ uname -a
CYGWIN_NT-10.0-19045 wincluster01_40_88 3.4.8-1.x86_64 2023-08-17
17:02 UTC x86_64 Cygwin

Thanks,
Martin

Re: mkfifo: cannot set permissions of 'x.fifo': Not a directory

[Corinna Vinschen]

On Aug 18 06:02, Martin Wege via Cygwin wrote:
> On Mon, Aug 7, 2023 at 4:42 PM Martin Wege <martin.l.wege@gmail.com> wrote:
> >
> > Hello,
> >
> > We get a weird mkfifo failure in Cygwin on NTFS:
> >
> > /usr/bin/mkfifo -m 600 x.fifo
> > mkfifo: cannot set permissions of 'x.fifo': Not a directory
> >
> > Is there a known workaround? So far named fifos cannot be created somehow.
> 
> This is fixed for NTFS, but still broken for Windows builtin NFS v3 client:

It was actually never supposed to work on NFS.  Cygwin FIFOs are
created as symlinks of the type Windows shortcut with the R/O attribute
set.  Those are only generated on NTFS and FAT filesystems.  NFS symlinks
are generated using special magic.  NFS doesn't support DOS attributes.

Corinna

Re: mkfifo: cannot set permissions of 'x.fifo': Not a directory

[Martin Wege]

On Fri, Aug 18, 2023 at 10:34 AM Corinna Vinschen
<corinna-cygwin@cygwin.com> wrote:
>
> On Aug 18 06:02, Martin Wege via Cygwin wrote:
> > On Mon, Aug 7, 2023 at 4:42 PM Martin Wege <martin.l.wege@gmail.com> wrote:
> > >
> > > Hello,
> > >
> > > We get a weird mkfifo failure in Cygwin on NTFS:
> > >
> > > /usr/bin/mkfifo -m 600 x.fifo
> > > mkfifo: cannot set permissions of 'x.fifo': Not a directory
> > >
> > > Is there a known workaround? So far named fifos cannot be created somehow.
> >
> > This is fixed for NTFS, but still broken for Windows builtin NFS v3 client:
>
> It was actually never supposed to work on NFS.  Cygwin FIFOs are
> created as symlinks of the type Windows shortcut with the R/O attribute
> set.  Those are only generated on NTFS and FAT filesystems.  NFS symlinks
> are generated using special magic.  NFS doesn't support DOS attributes.

So what would be a way to fix this for users with NFS mounted
projects, data or home dirs?

Thanks,
Martin

Re: mkfifo: cannot set permissions of 'x.fifo': Not a directory

[Cedric Blancher]

On Fri, 18 Aug 2023 at 10:35, Corinna Vinschen via Cygwin
<cygwin@cygwin.com> wrote:
>
> On Aug 18 06:02, Martin Wege via Cygwin wrote:
> > On Mon, Aug 7, 2023 at 4:42 PM Martin Wege <martin.l.wege@gmail.com> wrote:
> > >
> > > Hello,
> > >
> > > We get a weird mkfifo failure in Cygwin on NTFS:
> > >
> > > /usr/bin/mkfifo -m 600 x.fifo
> > > mkfifo: cannot set permissions of 'x.fifo': Not a directory
> > >
> > > Is there a known workaround? So far named fifos cannot be created somehow.
> >
> > This is fixed for NTFS, but still broken for Windows builtin NFS v3 client:
>
> It was actually never supposed to work on NFS.

Are you sure? AT&T AST and ARB (rDNA data cruncher) certainly have
worked with Cygwin on NFS shares (Windows builtin, I think you call it
MSFT NFSv3) in the past in our institute.

Tested with Cygwin 3.4.8: AST is not even building without such
support, and ARB throws nasty errors in the cluster run setup phase
II, because it cannot create the command fifo used for each cluster
node.

>
> Cygwin FIFOs are
> created as symlinks of the type Windows shortcut with the R/O attribute
> set.  Those are only generated on NTFS and FAT filesystems.  NFS symlinks
> are generated using special magic.  NFS doesn't support DOS attributes.

So no Linux mkfifo support on NFS with Cygwin?

Ced
-- 
Cedric Blancher <cedric.blancher@gmail.com>
[https://plus.google.com/u/0/+CedricBlancher/]
Institute Pasteur

Re: mkfifo: cannot set permissions of 'x.fifo': Not a directory

[Corinna Vinschen]

On Aug 21 06:49, Cedric Blancher via Cygwin wrote:
> On Fri, 18 Aug 2023 at 10:35, Corinna Vinschen via Cygwin
> <cygwin@cygwin.com> wrote:
> >
> > On Aug 18 06:02, Martin Wege via Cygwin wrote:
> > > On Mon, Aug 7, 2023 at 4:42 PM Martin Wege <martin.l.wege@gmail.com> wrote:
> > > >
> > > > Hello,
> > > >
> > > > We get a weird mkfifo failure in Cygwin on NTFS:
> > > >
> > > > /usr/bin/mkfifo -m 600 x.fifo
> > > > mkfifo: cannot set permissions of 'x.fifo': Not a directory
> > > >
> > > > Is there a known workaround? So far named fifos cannot be created somehow.
> > >
> > > This is fixed for NTFS, but still broken for Windows builtin NFS v3 client:
> >
> > It was actually never supposed to work on NFS.
> 
> Are you sure? AT&T AST and ARB (rDNA data cruncher) certainly have
> worked with Cygwin on NFS shares (Windows builtin, I think you call it
> MSFT NFSv3) in the past in our institute.
> 
> Tested with Cygwin 3.4.8: AST is not even building without such
> support, and ARB throws nasty errors in the cluster run setup phase
> II, because it cannot create the command fifo used for each cluster
> node.
> 
> >
> > Cygwin FIFOs are
> > created as symlinks of the type Windows shortcut with the R/O attribute
> > set.  Those are only generated on NTFS and FAT filesystems.  NFS symlinks
> > are generated using special magic.  NFS doesn't support DOS attributes.
> 
> So no Linux mkfifo support on NFS with Cygwin?

You're talking about Linux FIFOs, which don't make *any* sense on
Cygwin, and I'm talking about Cygwin FIFOs which don't make *any* sense
on Linux or, FWIW, a remote NFS fileystem.

So if you want Cygwin to be able to create FIFOs on a remote NFS, which
Cygwin won't be able to use, great.  I don't see any sense in that, but,
as usual, https://cygwin.com/acronyms/#PTC


Corinna

Re: mkfifo: cannot set permissions of 'x.fifo': Not a directory

[Martin Wege]

On Fri, Aug 18, 2023 at 10:34 AM Corinna Vinschen
<corinna-cygwin@cygwin.com> wrote:
>
> On Aug 18 06:02, Martin Wege via Cygwin wrote:
> > On Mon, Aug 7, 2023 at 4:42 PM Martin Wege <martin.l.wege@gmail.com> wrote:
> > >
> > > Hello,
> > >
> > > We get a weird mkfifo failure in Cygwin on NTFS:
> > >
> > > /usr/bin/mkfifo -m 600 x.fifo
> > > mkfifo: cannot set permissions of 'x.fifo': Not a directory
> > >
> > > Is there a known workaround? So far named fifos cannot be created somehow.
> >
> > This is fixed for NTFS, but still broken for Windows builtin NFS v3 client:
>
> It was actually never supposed to work on NFS.  Cygwin FIFOs are
> created as symlinks of the type Windows shortcut with the R/O attribute
> set.  Those are only generated on NTFS and FAT filesystems.  NFS symlinks
> are generated using special magic.  NFS doesn't support DOS attributes.

NFS supports symlinks!
Easy solution would be to say that symlinks on NFS pointing to
:\0:c4:$inodenum are pipes on the local Cygwin machine.
/usr/bin/mkfifo on Cygwin creates a symlink :\0:c4:$inodenum. Cygwin
stat() functions turn symlinks with a value of :\0:c4:... to pipes.
Everyone is happy!

It does not have to be portable across different platforms. Just
portable enough that mkfifo in with bash scripts works on NFS.

Thanks,
Martin
>
> Corinna

RE: [EXTERNAL] Re: mkfifo: cannot set permissions of 'x.fifo': Not a directory

[Lavrentiev, Anton (NIH/NLM/NCBI) [C]]

> FIFOs which don't make *any* sense
> ... FWIW, a remote NFS fileystem.

I got an impression that the OP is trying to deploy something (maybe the entire Cygwin) onto an NFS share.  So the named FIFO "file" is also created in there.

It's pointless to assume that the FIFO can be used as a communication device between the hosts that can mount the share, but it can be quite feasible to envision a scenario, in which the same host opens the FIFO located on the share from two processes and establish the communication using that special "file" (which is basically a special data-less i-node).

Anton Lavrentiev
Contractor NIH/NLM/NCBI

How to fix |mkfifo()| failure if |pathname| is on NFS ? / was: Re: [EXTERNAL] Re: mkfifo: cannot set permissions of 'x.fifo': Not a directory

[Roland Mainz]

On Tue, Aug 22, 2023 at 4:52 PM Lavrentiev, Anton (NIH/NLM/NCBI) [C]
via Cygwin <cygwin@cygwin.com> wrote:
> > FIFOs which don't make *any* sense
> > ... FWIW, a remote NFS fileystem.
>
> I got an impression that the OP is trying to deploy something (maybe the entire Cygwin) onto an
> NFS share.  So the named FIFO "file" is also created in there.

I agree with that impression. This is basically what large sites
(universities etc) do with UNIX and Linux: The machines mount an
user's ${HOMR} directory via automounter over NFS, and users are
discouraged (e.g. grumpy admin visiting you in person, blocking all
escape routes... =:-) ) from using the machine's local filesystems (in
Cygwin's case that includes "C:"!).

In that case people want to use |mkfifo()|/|mkfifoat()| and/or
/usr/bin/mkfifo in their home directory, and don't expect that it does
not work.

But that is what happens on Cygwin 3.4.8 right now, if someone tries
to do a |mkfifo()| on a NFS home directory (tested with MS NFSv3 and
CITI NFSv4 clients):
|mkfifo()| succeeds, but the resulting inode is *NOT* a FIFO as requested

Example (/cygdrive/h/ is my home directory shared from my Linux machine):
---- snip ----
roland_mainz@winkrakra1 /cygdrive/h/work/cygwin_mkfifo_on_nfs
$ uname -a
CYGWIN_NT-10.0-19045 winkrakra1 3.4.8-1.x86_64 2023-08-17 17:02 UTC
x86_64 Cygwin

roland_mainz@winkrakra1 /cygdrive/h/work/cygwin_mkfifo_on_nfs
$ mount
C:/cygwin64/bin on /usr/bin type ntfs (binary,auto)
C:/cygwin64/lib on /usr/lib type ntfs (binary,auto)
C:/cygwin64 on / type ntfs (binary,auto)
C: on /cygdrive/c type ntfs (binary,posix=0,user,noumount,auto)
H: on /cygdrive/h type nfs (binary,posix=0,user,noumount,auto)

roland_mainz@winkrakra1 /cygdrive/h/work/cygwin_mkfifo_on_nfs
$ ls -l
total 1
-rw-rw-rw- 1 Unix_User+0 Unix_Group+0 330 Aug 22 23:58 cygwin_mkfifo_on_nfs.c

roland_mainz@winkrakra1 /cygdrive/h/work/cygwin_mkfifo_on_nfs
$ cat -n cygwin_mkfifo_on_nfs.c
     1  #include <stdlib.h>
     2  #include <stdio.h>
     3  #include <stdio.h>
     4  #include <errno.h>
     5  #include <sys/types.h>
     6  #include <sys/stat.h>
     7
     8  int main(int ac, char *av[])
     9  {
    10          (void)puts("# start");
    11
    12          if
(mkfifo("/cygdrive/h/work/cygwin_mkfifo_on_nfs/myfifo.fifo", 0) != 0)
    13                  perror("mkfifo failed");
    14          (void)puts("# done.");
    15          return EXIT_SUCCESS;
    16  }
    17

roland_mainz@winkrakra1 /cygdrive/h/work/cygwin_mkfifo_on_nfs
$ gcc -g cygwin_mkfifo_on_nfs.c

roland_mainz@winkrakra1 /cygdrive/h/work/cygwin_mkfifo_on_nfs
$ ./a
# start
# done.

roland_mainz@winkrakra1 /cygdrive/h/work/cygwin_mkfifo_on_nfs
$ ls -l
total 68
-rwxr-xr-x 1 Unix_User+0 Unix_Group+0 66951 Aug 23 00:12 a.exe
-rw-rw-rw- 1 Unix_User+0 Unix_Group+0   330 Aug 22 23:58 cygwin_mkfifo_on_nfs.c
lrwxrwxrwx 1 Unix_User+0 Unix_Group+0    11 Aug 23 00:12 myfifo.fifo
-> ':\0:c4:1000'

roland_mainz@winkrakra1 /cygdrive/h/work/cygwin_mkfifo_on_nfs
$ cat <myfifo.fifo
-bash: myfifo.fifo: No such file or directory
---- snip ----

Note that Cygwin does not interpret the file |myfifo.fifo| as FIFO,
instead it comes back as a symlink "myfifo.fifo -> ':\0:c4:1000'".

AFAIK there are (at least) these two options to fix the problems:
1. Check whether the filesystem for the fifos path is NFS
(cgywin.dll's |fs.fs_is_nfs()|), and if it is a symlink check if it
starts with ':\0:c4:' (assuming "c4" is the prefix for inodes created
with |mkfifo()|). If this condition is |true|, then cygwin |stat()|,
|open()| etc. should treat this inode as FIFO.
2. Check whether the filesystem for the fifos path is NFS
(cgywin.dll's |fs.fs_is_nfs()|), and then just refuse |mkfifo()| with
|ENOSYS| (not implemented)

Better algorithm for [1] might be to check whether the inode is a
symlink, and then do a |fs.fs_is_nfs()| on the symlinks |pathname|,
assuming this is more performant.

> It's pointless to assume that the FIFO can be used as a communication
> device between the hosts that can mount the share, but it can be quite
> feasible to envision a scenario, in which the same host opens the FIFO
> located on the share from two processes and establish the
> communication using that special "file" (which is basically a special
> data-less i-node).

Well, this is what RFS (see
https://en.wikipedia.org/wiki/Remote_File_Sharing) was doing - but it
was removed in Solaris 2.4, because its complexity was too great
(well, the original implementation was simple and clean, and then it
grew all over the kernel just to handle all corner cases of POSIX&co.)
- and it would be nice not to repeat the mistakes of the past.

----

Bye,
Roland
-- 
  __ .  . __
 (o.\ \/ /.o) roland.mainz@nrubsig.org
  \__\/\/__/  MPEG specialist, C&&JAVA&&Sun&&Unix programmer
  /O /==\ O\  TEL +49 641 3992797
 (;O/ \/ \O;)

RE: [EXTERNAL] Re: mkfifo: cannot set permissions of 'x.fifo': Not a directory

[Lavrentiev, Anton (NIH/NLM/NCBI) [C]]

> What happens when the user executes two copies of an 
> application /*such as PyCharm*/ on two separate machines sharing the same 
> home directory? Does the directory entry and inode get reused on startup 
> and/or deleted on exit? How does that impact the process instance on the 
> other machine? The conclusion I came to long ago is that if pipes are to 
> be implemented in a network file system namespace then the pipes must be 
> fully functional network pipes.

A p-type i-node (FIFO on Unix) is a local named pipe regardless of its location.  Two pairs of applications running on two different hosts that happen to mount the same network share, do not have to "know" of each other, the i-node is only used to identify the FIFO within the namespace of the same node.  So both pairs can actually be communicating locally within their respective nodes, without affecting one another, between the processes within that same node.  Deleting / creating a FIFO has no significance, as it's just an i-node to "name" the pipe (via the i-node number).

Unlike UNIX sockets (which too can be placed on network shares, BTW), FIFO files do not need to be re-created afresh every time they are used.  The only requirement is that the same i-node is opened for both reading and writing, and that's all it is necessary to create a communication channel:  "man 7 fifo".

Someone wanting a truly networking device should be using sockets.

Anton Lavrentiev
Contractor NIH/NLM/NCBI

Re: [EXTERNAL] Re: mkfifo: cannot set permissions of 'x.fifo': Not a directory

[Cedric Blancher]

On Wed, 23 Aug 2023 at 17:44, Jeffrey Altman via Cygwin
<cygwin@cygwin.com> wrote:
>
> On 8/22/2023 10:52 AM, Lavrentiev, Anton (NIH/NLM/NCBI) [C] via Cygwin
> wrote:
> >> FIFOs which don't make *any* sense
> >> ... FWIW, a remote NFS fileystem.
> > I got an impression that the OP is trying to deploy something (maybe the entire Cygwin) onto an NFS share.  So the named FIFO "file" is also created in there.
> >
> > It's pointless to assume that the FIFO can be used as a communication device between the hosts that can mount the share, but it can be quite feasible to envision a scenario, in which the same host opens the FIFO located on the share from two processes and establish the communication using that special "file" (which is basically a special data-less i-node).
>
> I've been following this thread with quite a bit of curiosity. For those
> who do not know me, I'm the lead developer of the AFS filesystem on
> Windows. There have been requests for more than two decades for AFS
> clients to add support for locally created pipe files because AFS
> volumes are often used as home directories (even on Windows) and so many
> applications allocate a pipe in the home directory as a method of
> inter-process communication or a lock. The problem is that even if the
> intended usage of the file is entirely local, the directory object, the
> directory entry and the allocated inode (or equivalent) is network
> visible. What happens when the user executes two copies of an
> application such as PyCharm on two separate machines sharing the same
> home directory? Does the directory entry and inode get reused on startup
> and/or deleted on exit? How does that impact the process instance on the
> other machine? The conclusion I came to long ago is that if pipes are to
> be implemented in a network file system namespace then the pipes must be
> fully functional network pipes. In just about all cases applications can
> be configured to use a non-default paths. In my opinion they should not
> be placed in a shared file system.

This is NOT what POSIX says. We're talking about mkfifo(), which
creates pipes on the local system. The filesystem is just used to use
filesystem paths for identification.

If you want truly networked fifos, I suggest socket(), preferably with
SCTP. But this is OFFTOPIC here. We just want mkfifo() to work on NFS.

>
> I've also been following this thread because both the Microsoft NFSv3
> and the Citi NFSv4 clients are rather incomplete filesystem
> implementations from the perspective of the Installable File System
> interface and the Multiple UNC Provider interface.

Good. This is actually worth writing a bug report against the CITI
NFSv4 client, because the Microsoft NFSv3 client is only extended or
fixed if someone is willing to pay a five-digit euro sum to M$.
Literally.  We asked, and got a longer document describing the
expenses.
So better fix the NFSv4 client.

> In my opinion,
> Microsoft NFSv3 is the bare minimum that must be implemented for
> Microsoft to advertise that NFSv3 is available on Windows. The NFSv3
> symlink interface leveraging extended attributes as a method of setting
> and reading the target predates the introduction of
> IO_REPARSE_TAG_SYMLINK for NTFS. Prior to the allocation of
> IO_REPARSE_TAG_SYMLINK I obtained a proprietary tag allocation from
> Microsoft for AFS symlinks but switched to IO_REPARSE_TAG_SYMLINK once
> it was available because then AFS symlinks work the same as NTFS.
>
> The Citi NFSv4 implementation is not only incomplete from a Windows
> interface perspective but Citi never obtained from Microsoft the
> required filesystem registrations.

Please elaborate

> For example, it doesn't have a
> Microsoft assigned FLT_FILESYSTEM_TYPE, WNNC network type, name or
> NodeTypeCode values. FLT_FSTYPE_NFS, WNNC_NET_MS_NFS, and "nfs" are
> specific to the Microsoft NFSv3 (aka \FileSystem\NfsRdr) implementation.

This is not correct. The CITI NFSv4 client was originally written
using $$$ from a grant from Microsoft, and is intended to be a drop-in
replacement for the NFSv3 client. Hence it shares the same "nfs" name,
uses the same extension apis (fsattr3), and should behave almost the
same.

> The NodeTypeCode range used by the Citi NFSv4 filesystem (0xFC00) is
> outside the block allocated to third party filesystems and appears to be
> from a published sample which could lead to corruption if two
> filesystems allocating FileControlBlocks with the same NodeTypeCodes are
> present on the system. The returned WNNC value is also from a sample.

Please file a bug report. This should be fixable.

> Perhaps the original poster knows where development of the Citi NFSv4
> client is currently taking place. The source code repos I've been able
> to find do not have any commits since 2012 (tag v1.0.0) and have been
> flagged as archival on GitHub.

Seriously just my personal opinion, not speaking for my employer:
2013 on the github, and the DFG (Deutsche Forschungs Gemeinschaft) has
an ongoing fork since 2019, more or less the same year M$ came up with
the insane cost estimate for "fixing" their own horrible NFSv3 client.
Basically their retaliation to M$ wanting money for broken things. The
hard part is now to get them to throw the patches over the fence, get
them reviewed and tested.

Also, I don't like that you hit on the CITI NFSv4 client work. It is
good and stable enough for routine cluster usage, and much faster than
Microsoft's own NFSv3 client. There are issues, but none which happen
in said environment.

The only REAL issue is the lack of public binaries with signed
binaries, as any self-compiled ones cannot be loaded into a Windows
kernel with SecureBoot enabled.

>
> In my opinion, for Cygwin to reliably work with either of these
> implementations is going to require on-going developer effort and
> continuous integration testing. Not only of Cygwin but in the case of
> Citi NFSv4 from the team maintaining the product. Unless something has
> significantly changed since 2013 I do not expect Microsoft to be willing
> to invest much effort into enhancing the NFSv3 implementation. The
> likely recommendation would be to re-export the NFS file shares via
> Samba and access them via CIFS.

I don't want to bash, or rant about CIFS, but getting CIFS into
embedded medical devices is a security nightmare. I know, we tried,
and seriously ended up with both the Linux NFSv4 kernel client, and
the NFS4j for JAVA standalone applications.

Ced
-- 
Cedric Blancher <cedric.blancher@gmail.com>
[https://plus.google.com/u/0/+CedricBlancher/]
Institute Pasteur

Re: [EXTERNAL] Re: mkfifo: cannot set permissions of 'x.fifo': Not a directory

[Martin Wege]

On Wed, Aug 23, 2023 at 5:44 PM Jeffrey Altman via Cygwin
<cygwin@cygwin.com> wrote:
>
> On 8/22/2023 10:52 AM, Lavrentiev, Anton (NIH/NLM/NCBI) [C] via Cygwin
> wrote:
> >> FIFOs which don't make *any* sense
> >> ... FWIW, a remote NFS fileystem.
> > I got an impression that the OP is trying to deploy something (maybe the entire Cygwin) onto an NFS share.  So the named FIFO "file" is also created in there.
> >
> > It's pointless to assume that the FIFO can be used as a communication device between the hosts that can mount the share, but it can be quite feasible to envision a scenario, in which the same host opens the FIFO located on the share from two processes and establish the communication using that special "file" (which is basically a special data-less i-node).
>
> I've been following this thread with quite a bit of curiosity. For those
> who do not know me, I'm the lead developer of the AFS filesystem on
> Windows. There have been requests for more than two decades for AFS
> clients to add support for locally created pipe files because AFS
> volumes are often used as home directories (even on Windows) and so many
> applications allocate a pipe in the home directory as a method of
> inter-process communication or a lock. The problem is that even if the
> intended usage of the file is entirely local, the directory object, the
> directory entry and the allocated inode (or equivalent) is network
> visible.

Who cares? It is up to the user, script or application author to do
'the right thing(tm)'.

>
> What happens when the user executes two copies of an
> application such as PyCharm on two separate machines sharing the same
> home directory? Does the directory entry and inode get reused on startup
> and/or deleted on exit? How does that impact the process instance on the
> other machine? The conclusion I came to long ago is that if pipes are to
> be implemented in a network file system namespace then the pipes must be
> fully functional network pipes. In just about all cases applications can
> be configured to use a non-default paths. In my opinion they should not
> be placed in a shared file system.

Oh my god. Please have this debate at the Austin Group. They do the
POSIX standard. We're here about implementing&using the POSIX
standard.

We only want that mkfifo() works with Cygwin on a NFS filesystem, as
specified in https://pubs.opengroup.org/onlinepubs/009696799/functions/mkfifo.html

My intention was not to invent something new. Just code using mkfifo,
and scripts using /usr/bin/mkfifo, should work on NFS. On UNIX&Linux
this works.

Have a look at https://cygwin.com/pipermail/cygwin/2023-August/254266.html,
sounds like there is a feasible way to implement this.

Thanks,
Martin

Re: How to fix |mkfifo()| failure if |pathname| is on NFS ? / was: Re: [EXTERNAL] Re: mkfifo: cannot set permissions of 'x.fifo': Not a directory

[Martin Wege]

On Wed, Aug 23, 2023 at 1:06 AM Roland Mainz via Cygwin
<cygwin@cygwin.com> wrote:
>
> On Tue, Aug 22, 2023 at 4:52 PM Lavrentiev, Anton (NIH/NLM/NCBI) [C]
> via Cygwin <cygwin@cygwin.com> wrote:
> > > FIFOs which don't make *any* sense
> > > ... FWIW, a remote NFS fileystem.
> >
> > I got an impression that the OP is trying to deploy something (maybe the entire Cygwin) onto an
> > NFS share.  So the named FIFO "file" is also created in there.
>
> I agree with that impression. This is basically what large sites
> (universities etc) do with UNIX and Linux: The machines mount an
> user's ${HOMR} directory via automounter over NFS, and users are
> discouraged (e.g. grumpy admin visiting you in person, blocking all
> escape routes... =:-) ) from using the machine's local filesystems (in
> Cygwin's case that includes "C:"!).
>
> In that case people want to use |mkfifo()|/|mkfifoat()| and/or
> /usr/bin/mkfifo in their home directory, and don't expect that it does
> not work.
>
> But that is what happens on Cygwin 3.4.8 right now, if someone tries
> to do a |mkfifo()| on a NFS home directory (tested with MS NFSv3 and
> CITI NFSv4 clients):
> |mkfifo()| succeeds, but the resulting inode is *NOT* a FIFO as requested
>
> Example (/cygdrive/h/ is my home directory shared from my Linux machine):
> ---- snip ----
> roland_mainz@winkrakra1 /cygdrive/h/work/cygwin_mkfifo_on_nfs
> $ uname -a
> CYGWIN_NT-10.0-19045 winkrakra1 3.4.8-1.x86_64 2023-08-17 17:02 UTC
> x86_64 Cygwin
>
> roland_mainz@winkrakra1 /cygdrive/h/work/cygwin_mkfifo_on_nfs
> $ mount
> C:/cygwin64/bin on /usr/bin type ntfs (binary,auto)
> C:/cygwin64/lib on /usr/lib type ntfs (binary,auto)
> C:/cygwin64 on / type ntfs (binary,auto)
> C: on /cygdrive/c type ntfs (binary,posix=0,user,noumount,auto)
> H: on /cygdrive/h type nfs (binary,posix=0,user,noumount,auto)
>
> roland_mainz@winkrakra1 /cygdrive/h/work/cygwin_mkfifo_on_nfs
> $ ls -l
> total 1
> -rw-rw-rw- 1 Unix_User+0 Unix_Group+0 330 Aug 22 23:58 cygwin_mkfifo_on_nfs.c
>
> roland_mainz@winkrakra1 /cygdrive/h/work/cygwin_mkfifo_on_nfs
> $ cat -n cygwin_mkfifo_on_nfs.c
>      1  #include <stdlib.h>
>      2  #include <stdio.h>
>      3  #include <stdio.h>
>      4  #include <errno.h>
>      5  #include <sys/types.h>
>      6  #include <sys/stat.h>
>      7
>      8  int main(int ac, char *av[])
>      9  {
>     10          (void)puts("# start");
>     11
>     12          if
> (mkfifo("/cygdrive/h/work/cygwin_mkfifo_on_nfs/myfifo.fifo", 0) != 0)
>     13                  perror("mkfifo failed");
>     14          (void)puts("# done.");
>     15          return EXIT_SUCCESS;
>     16  }
>     17
>
> roland_mainz@winkrakra1 /cygdrive/h/work/cygwin_mkfifo_on_nfs
> $ gcc -g cygwin_mkfifo_on_nfs.c
>
> roland_mainz@winkrakra1 /cygdrive/h/work/cygwin_mkfifo_on_nfs
> $ ./a
> # start
> # done.
>
> roland_mainz@winkrakra1 /cygdrive/h/work/cygwin_mkfifo_on_nfs
> $ ls -l
> total 68
> -rwxr-xr-x 1 Unix_User+0 Unix_Group+0 66951 Aug 23 00:12 a.exe
> -rw-rw-rw- 1 Unix_User+0 Unix_Group+0   330 Aug 22 23:58 cygwin_mkfifo_on_nfs.c
> lrwxrwxrwx 1 Unix_User+0 Unix_Group+0    11 Aug 23 00:12 myfifo.fifo
> -> ':\0:c4:1000'
>
> roland_mainz@winkrakra1 /cygdrive/h/work/cygwin_mkfifo_on_nfs
> $ cat <myfifo.fifo
> -bash: myfifo.fifo: No such file or directory
> ---- snip ----
>
> Note that Cygwin does not interpret the file |myfifo.fifo| as FIFO,
> instead it comes back as a symlink "myfifo.fifo -> ':\0:c4:1000'".
>
> AFAIK there are (at least) these two options to fix the problems:
> 1. Check whether the filesystem for the fifos path is NFS
> (cgywin.dll's |fs.fs_is_nfs()|), and if it is a symlink check if it
> starts with ':\0:c4:' (assuming "c4" is the prefix for inodes created
> with |mkfifo()|). If this condition is |true|, then cygwin |stat()|,
> |open()| etc. should treat this inode as FIFO.
> 2. Check whether the filesystem for the fifos path is NFS
> (cgywin.dll's |fs.fs_is_nfs()|), and then just refuse |mkfifo()| with
> |ENOSYS| (not implemented)
>
> Better algorithm for [1] might be to check whether the inode is a
> symlink, and then do a |fs.fs_is_nfs()| on the symlinks |pathname|,
> assuming this is more performant.

We would like to see option 1 implemented, maybe as an option in the
CYGWIN environment variable
(CYGWIN=nfs_emulate_dev_special_files_as_symlink ?)

Thanks,
Martin

Re: How to fix |mkfifo()| failure if |pathname| is on NFS ? / was: Re: [EXTERNAL] Re: mkfifo: cannot set permissions of 'x.fifo': Not a directory

[Cedric Blancher]

On Thu, 24 Aug 2023 at 18:45, Martin Wege via Cygwin <cygwin@cygwin.com> wrote:
>
> On Wed, Aug 23, 2023 at 1:06 AM Roland Mainz via Cygwin
> <cygwin@cygwin.com> wrote:
> >
> > On Tue, Aug 22, 2023 at 4:52 PM Lavrentiev, Anton (NIH/NLM/NCBI) [C]
> > via Cygwin <cygwin@cygwin.com> wrote:
> > > > FIFOs which don't make *any* sense
> > > > ... FWIW, a remote NFS fileystem.
> > >
> > > I got an impression that the OP is trying to deploy something (maybe the entire Cygwin) onto an
> > > NFS share.  So the named FIFO "file" is also created in there.
> >
> > I agree with that impression. This is basically what large sites
> > (universities etc) do with UNIX and Linux: The machines mount an
> > user's ${HOMR} directory via automounter over NFS, and users are
> > discouraged (e.g. grumpy admin visiting you in person, blocking all
> > escape routes... =:-) ) from using the machine's local filesystems (in
> > Cygwin's case that includes "C:"!).
> >
> > In that case people want to use |mkfifo()|/|mkfifoat()| and/or
> > /usr/bin/mkfifo in their home directory, and don't expect that it does
> > not work.
> >
> > But that is what happens on Cygwin 3.4.8 right now, if someone tries
> > to do a |mkfifo()| on a NFS home directory (tested with MS NFSv3 and
> > CITI NFSv4 clients):
> > |mkfifo()| succeeds, but the resulting inode is *NOT* a FIFO as requested
> >
> > Example (/cygdrive/h/ is my home directory shared from my Linux machine):
> > ---- snip ----
> > roland_mainz@winkrakra1 /cygdrive/h/work/cygwin_mkfifo_on_nfs
> > $ uname -a
> > CYGWIN_NT-10.0-19045 winkrakra1 3.4.8-1.x86_64 2023-08-17 17:02 UTC
> > x86_64 Cygwin
> >
> > roland_mainz@winkrakra1 /cygdrive/h/work/cygwin_mkfifo_on_nfs
> > $ mount
> > C:/cygwin64/bin on /usr/bin type ntfs (binary,auto)
> > C:/cygwin64/lib on /usr/lib type ntfs (binary,auto)
> > C:/cygwin64 on / type ntfs (binary,auto)
> > C: on /cygdrive/c type ntfs (binary,posix=0,user,noumount,auto)
> > H: on /cygdrive/h type nfs (binary,posix=0,user,noumount,auto)
> >
> > roland_mainz@winkrakra1 /cygdrive/h/work/cygwin_mkfifo_on_nfs
> > $ ls -l
> > total 1
> > -rw-rw-rw- 1 Unix_User+0 Unix_Group+0 330 Aug 22 23:58 cygwin_mkfifo_on_nfs.c
> >
> > roland_mainz@winkrakra1 /cygdrive/h/work/cygwin_mkfifo_on_nfs
> > $ cat -n cygwin_mkfifo_on_nfs.c
> >      1  #include <stdlib.h>
> >      2  #include <stdio.h>
> >      3  #include <stdio.h>
> >      4  #include <errno.h>
> >      5  #include <sys/types.h>
> >      6  #include <sys/stat.h>
> >      7
> >      8  int main(int ac, char *av[])
> >      9  {
> >     10          (void)puts("# start");
> >     11
> >     12          if
> > (mkfifo("/cygdrive/h/work/cygwin_mkfifo_on_nfs/myfifo.fifo", 0) != 0)
> >     13                  perror("mkfifo failed");
> >     14          (void)puts("# done.");
> >     15          return EXIT_SUCCESS;
> >     16  }
> >     17
> >
> > roland_mainz@winkrakra1 /cygdrive/h/work/cygwin_mkfifo_on_nfs
> > $ gcc -g cygwin_mkfifo_on_nfs.c
> >
> > roland_mainz@winkrakra1 /cygdrive/h/work/cygwin_mkfifo_on_nfs
> > $ ./a
> > # start
> > # done.
> >
> > roland_mainz@winkrakra1 /cygdrive/h/work/cygwin_mkfifo_on_nfs
> > $ ls -l
> > total 68
> > -rwxr-xr-x 1 Unix_User+0 Unix_Group+0 66951 Aug 23 00:12 a.exe
> > -rw-rw-rw- 1 Unix_User+0 Unix_Group+0   330 Aug 22 23:58 cygwin_mkfifo_on_nfs.c
> > lrwxrwxrwx 1 Unix_User+0 Unix_Group+0    11 Aug 23 00:12 myfifo.fifo
> > -> ':\0:c4:1000'
> >
> > roland_mainz@winkrakra1 /cygdrive/h/work/cygwin_mkfifo_on_nfs
> > $ cat <myfifo.fifo
> > -bash: myfifo.fifo: No such file or directory
> > ---- snip ----
> >
> > Note that Cygwin does not interpret the file |myfifo.fifo| as FIFO,
> > instead it comes back as a symlink "myfifo.fifo -> ':\0:c4:1000'".
> >
> > AFAIK there are (at least) these two options to fix the problems:
> > 1. Check whether the filesystem for the fifos path is NFS
> > (cgywin.dll's |fs.fs_is_nfs()|), and if it is a symlink check if it
> > starts with ':\0:c4:' (assuming "c4" is the prefix for inodes created
> > with |mkfifo()|). If this condition is |true|, then cygwin |stat()|,
> > |open()| etc. should treat this inode as FIFO.
> > 2. Check whether the filesystem for the fifos path is NFS
> > (cgywin.dll's |fs.fs_is_nfs()|), and then just refuse |mkfifo()| with
> > |ENOSYS| (not implemented)
> >
> > Better algorithm for [1] might be to check whether the inode is a
> > symlink, and then do a |fs.fs_is_nfs()| on the symlinks |pathname|,
> > assuming this is more performant.
>
> We would like to see option 1 implemented,

+1

> maybe as an option in the
> CYGWIN environment variable
> (CYGWIN=nfs_emulate_dev_special_files_as_symlink ?)

+1

Ced
-- 
Cedric Blancher <cedric.blancher@gmail.com>
[https://plus.google.com/u/0/+CedricBlancher/]
Institute Pasteur

Re: How to fix |mkfifo()| failure if |pathname| is on NFS ? / was: Re: [EXTERNAL] Re: mkfifo: cannot set permissions of 'x.fifo': Not a directory

[Corinna Vinschen]

On Aug 23 01:05, Roland Mainz via Cygwin wrote:
> Note that Cygwin does not interpret the file |myfifo.fifo| as FIFO,
> instead it comes back as a symlink "myfifo.fifo -> ':\0:c4:1000'".
> 
> AFAIK there are (at least) these two options to fix the problems:
> 1. Check whether the filesystem for the fifos path is NFS
> (cgywin.dll's |fs.fs_is_nfs()|), and if it is a symlink check if it
> starts with ':\0:c4:' (assuming "c4" is the prefix for inodes created
> with |mkfifo()|). If this condition is |true|, then cygwin |stat()|,
> |open()| etc. should treat this inode as FIFO.

The downside is that it is not possible to diffentiate between Cygwin
FIFOs and real FIFOs created from the remote side in `ls -l'
output.  Note that Cygwin returns the NFS stat info verbatim, so
a real FIFO is returned as a real FIFO:

  linux$ mkfifo bar
  cygwin$ ls -l bar
  prw-r--r-- 1 corinna vinschen  0 Aug 25 13:58 bar

The idea was always to use NFS as exchange medium, but not as
installation medium for the entire distro or to keep Cygwin home
dirs on NFS.  There were times where NFS was pretty unstable.
I used NFS for quite some time to build Cygwin packages, but at one
point I got trouble (performance problems with multiple concurrent
processes accessing an NFS share, build errors out of the blue),
so I switched to Samba shares, albeit grudgingly.  I'm not yet
sure if the problems are fixed.  At least a recent OpenSSH build
ran through without problems...

Anyway.  How would you like to make sure that a Cygwin application
can differ between real FIFOs and Cygwin FIFOs?

> 2. Check whether the filesystem for the fifos path is NFS
> (cgywin.dll's |fs.fs_is_nfs()|), and then just refuse |mkfifo()| with
> |ENOSYS| (not implemented)

I like the idea.

> Better algorithm for [1] might be to check whether the inode is a
> symlink, and then do a |fs.fs_is_nfs()| on the symlinks |pathname|,
> assuming this is more performant.

Even better would be if we could just create and use real FIFOs
on NFS(*).  But while NtQueryEaFile can be used to fetch real
NFS file info, and while NtCreateFile can be used to create real
synmlinks via NFS, I don't see an interface resembling mknod/mkfifo.

(*) Not using them in the sense that Cygwin can actually communicate
with the real remote FIFO, but in the sense that the fake is good enough
so that our fhandler_fifo class can actually utilize the file just like
FIFOs on NTFS.


Corinna

Re: How to fix |mkfifo()| failure if |pathname| is on NFS ? / was: Re: [EXTERNAL] Re: mkfifo: cannot set permissions of 'x.fifo': Not a directory

[Corinna Vinschen]

On Aug 25 14:18, Corinna Vinschen via Cygwin wrote:
> On Aug 23 01:05, Roland Mainz via Cygwin wrote:
> > 2. Check whether the filesystem for the fifos path is NFS
> > (cgywin.dll's |fs.fs_is_nfs()|), and then just refuse |mkfifo()| with
> > |ENOSYS| (not implemented)
> 
> I like the idea.

EPERM, btw.  See https://man7.org/linux/man-pages/man2/mknod.2.html


Corinna

Re: How to fix |mkfifo()| failure if |pathname| is on NFS ? / was: Re: [EXTERNAL] Re: mkfifo: cannot set permissions of 'x.fifo': Not a directory

[Roland Mainz]

On Fri, Aug 25, 2023 at 2:18 PM Corinna Vinschen via Cygwin
<cygwin@cygwin.com> wrote:
>
> On Aug 23 01:05, Roland Mainz via Cygwin wrote:
> > Note that Cygwin does not interpret the file |myfifo.fifo| as FIFO,
> > instead it comes back as a symlink "myfifo.fifo -> ':\0:c4:1000'".
> >
> > AFAIK there are (at least) these two options to fix the problems:
> > 1. Check whether the filesystem for the fifos path is NFS
> > (cgywin.dll's |fs.fs_is_nfs()|), and if it is a symlink check if it
> > starts with ':\0:c4:' (assuming "c4" is the prefix for inodes created
> > with |mkfifo()|). If this condition is |true|, then cygwin |stat()|,
> > |open()| etc. should treat this inode as FIFO.
>
> The downside is that it is not possible to diffentiate between Cygwin
> FIFOs and real FIFOs created from the remote side in `ls -l'
> output.  Note that Cygwin returns the NFS stat info verbatim, so
> a real FIFO is returned as a real FIFO:
>
>   linux$ mkfifo bar
>   cygwin$ ls -l bar
>   prw-r--r-- 1 corinna vinschen  0 Aug 25 13:58 bar

I know.

> The idea was always to use NFS as exchange medium, but not as
> installation medium for the entire distro or to keep Cygwin home
> dirs on NFS.  There were times where NFS was pretty unstable.
> I used NFS for quite some time to build Cygwin packages, but at one
> point I got trouble (performance problems with multiple concurrent
> processes accessing an NFS share, build errors out of the blue),
> so I switched to Samba shares, albeit grudgingly.  I'm not yet
> sure if the problems are fixed.  At least a recent OpenSSH build
> ran through without problems...

I think most issues have been fixed for the Microsoft NFSv3 client,
and for the CITI NFSv4.1 client (technically it's called
"ms-nfsv41-client") the situation is even better since it's
opensource, and we can fix problems even faster there.
From what I see the ms-nfsv41-client is stable enough for daily
routine usage, and I know that other institutions like DFG and CERN
use it for daily work too. The only nasty part is the damn lack of
documentation, and that there are no signed binaries, so any kernel
with UEFI/SecureBook cannot use them.

> Anyway.  How would you like to make sure that a Cygwin application
> can differ between real FIFOs and Cygwin FIFOs?

For now I can provide a migration script, and in the medium term we
should get Microsoft to provide some kind of |mknod()| API. see below.

> > 2. Check whether the filesystem for the fifos path is NFS
> > (cgywin.dll's |fs.fs_is_nfs()|), and then just refuse |mkfifo()| with
> > |ENOSYS| (not implemented)
>
> I like the idea.

See below.

> > Better algorithm for [1] might be to check whether the inode is a
> > symlink, and then do a |fs.fs_is_nfs()| on the symlinks |pathname|,
> > assuming this is more performant.
>
> Even better would be if we could just create and use real FIFOs
> on NFS(*).  But while NtQueryEaFile can be used to fetch real
> NFS file info, and while NtCreateFile can be used to create real
> synmlinks via NFS, I don't see an interface resembling mknod/mkfifo.

Looking at the ms-nfs41-client source code, there is no API for that *YET*.

So my plan would be like this:
1. Cygwin: implements the proposed devnodes-as-symlink emulation code,
if the env var CYGWIN has "nfs_emulate_dev_special_files_as_symlink"
set
2. Cygwin: By default |mkfifo()| will fail with |ENOSYS| (not |EPERM|,
as we intend to fix the issue!) on NFS filesystems, unless
CYGWIN=nfs_emulate_dev_special_files_as_symlink
3. (CITI) ms-nfsv41-client: adds a private API to do a |mknod()|
4. Cygwin: Implements the new API added in [3] as *prototype*, so
|mknod()| will properly work in a portable manner
5. We ask Microsoft to review the private API created in [3], and ask
them to implement it for the Microsoft NFSv3 client too (because we
have a precedent with [4]!)

Optional:
6. Cygwin: Like [1] add  "afs_emulate_dev_special_files_as_symlink" to
do the same for AFS
7. Cygwin: Like [2] |mkfifo()| should fail on AFS, unless [6] ...

Issues:
- [6] and [7] should be decided by the owner of the Microsoft AFS
filesystem driver.
- Should the new Windows NFS mknod-API be only available for accounts
with administrator rights/permissions ?

----

Bye,
Roland
-- 
  __ .  . __
 (o.\ \/ /.o) roland.mainz@nrubsig.org
  \__\/\/__/  MPEG specialist, C&&JAVA&&Sun&&Unix programmer
  /O /==\ O\  TEL +49 641 3992797
 (;O/ \/ \O;)

RE: How to fix |mkfifo()| failure if |pathname| is on NFS ? / was: Re: [EXTERNAL] Re: mkfifo: cannot set permissions of 'x.fifo': Not a directory

[Lavrentiev, Anton (NIH/NLM/NCBI) [C]]

> it is not possible to diffentiate between Cygwin
> FIFOs and real FIFOs created from the remote side in `ls -l'
> output.

Why would that be necessary?  If it's a FIFO, it can be used as a FIFO, regardless where and how it was created..

Anton Lavrentiev
Contractor NIH/NLM/NCBI

Re: How to fix |mkfifo()| failure if |pathname| is on NFS ? / was: Re: [EXTERNAL] Re: mkfifo: cannot set permissions of 'x.fifo': Not a directory

[Cedric Blancher]

On Fri, 25 Aug 2023 at 14:26, Corinna Vinschen via Cygwin
<cygwin@cygwin.com> wrote:
>
> On Aug 25 14:18, Corinna Vinschen via Cygwin wrote:
> > On Aug 23 01:05, Roland Mainz via Cygwin wrote:
> > > 2. Check whether the filesystem for the fifos path is NFS
> > > (cgywin.dll's |fs.fs_is_nfs()|), and then just refuse |mkfifo()| with
> > > |ENOSYS| (not implemented)
> >
> > I like the idea.
>
> EPERM, btw.  See https://man7.org/linux/man-pages/man2/mknod.2.html
>

I am vehemently opposed to that idea. That will IMHO not only be the
wrong solution, it will break actual things like autoconf&friends on
NFS - they create fifos for testing, but do not use them. If you just
make this return EPERM, then you break stuff we have to rely on.

mkfifo should just work with Cygwin on NFS.

Ced
-- 
Cedric Blancher <cedric.blancher@gmail.com>
[https://plus.google.com/u/0/+CedricBlancher/]
Institute Pasteur

Re: How to fix |mkfifo()| failure if |pathname| is on NFS ? / was: Re: [EXTERNAL] Re: mkfifo: cannot set permissions of 'x.fifo': Not a directory

[Cedric Blancher]

On Fri, 25 Aug 2023 at 17:15, Roland Mainz via Cygwin <cygwin@cygwin.com> wrote:
>
> On Fri, Aug 25, 2023 at 2:18 PM Corinna Vinschen via Cygwin
> <cygwin@cygwin.com> wrote:
> >
> > On Aug 23 01:05, Roland Mainz via Cygwin wrote:
> > > Note that Cygwin does not interpret the file |myfifo.fifo| as FIFO,
> > > instead it comes back as a symlink "myfifo.fifo -> ':\0:c4:1000'".
> > >
> > > AFAIK there are (at least) these two options to fix the problems:
> > > 1. Check whether the filesystem for the fifos path is NFS
> > > (cgywin.dll's |fs.fs_is_nfs()|), and if it is a symlink check if it
> > > starts with ':\0:c4:' (assuming "c4" is the prefix for inodes created
> > > with |mkfifo()|). If this condition is |true|, then cygwin |stat()|,
> > > |open()| etc. should treat this inode as FIFO.
> >
> > The downside is that it is not possible to diffentiate between Cygwin
> > FIFOs and real FIFOs created from the remote side in `ls -l'
> > output.  Note that Cygwin returns the NFS stat info verbatim, so
> > a real FIFO is returned as a real FIFO:
> >
> >   linux$ mkfifo bar
> >   cygwin$ ls -l bar
> >   prw-r--r-- 1 corinna vinschen  0 Aug 25 13:58 bar
>
> I know.
>
> > The idea was always to use NFS as exchange medium, but not as
> > installation medium for the entire distro or to keep Cygwin home
> > dirs on NFS.  There were times where NFS was pretty unstable.
> > I used NFS for quite some time to build Cygwin packages, but at one
> > point I got trouble (performance problems with multiple concurrent
> > processes accessing an NFS share, build errors out of the blue),
> > so I switched to Samba shares, albeit grudgingly.  I'm not yet
> > sure if the problems are fixed.  At least a recent OpenSSH build
> > ran through without problems...
>
> I think most issues have been fixed for the Microsoft NFSv3 client,
> and for the CITI NFSv4.1 client (technically it's called
> "ms-nfsv41-client") the situation is even better since it's
> opensource, and we can fix problems even faster there.
> From what I see the ms-nfsv41-client is stable enough for daily
> routine usage, and I know that other institutions like DFG and CERN
> use it for daily work too. The only nasty part is the damn lack of
> documentation, and that there are no signed binaries, so any kernel
> with UEFI/SecureBook cannot use them.
>
> > Anyway.  How would you like to make sure that a Cygwin application
> > can differ between real FIFOs and Cygwin FIFOs?
>
> For now I can provide a migration script, and in the medium term we
> should get Microsoft to provide some kind of |mknod()| API. see below.
>
> > > 2. Check whether the filesystem for the fifos path is NFS
> > > (cgywin.dll's |fs.fs_is_nfs()|), and then just refuse |mkfifo()| with
> > > |ENOSYS| (not implemented)
> >
> > I like the idea.
>
> See below.
>
> > > Better algorithm for [1] might be to check whether the inode is a
> > > symlink, and then do a |fs.fs_is_nfs()| on the symlinks |pathname|,
> > > assuming this is more performant.
> >
> > Even better would be if we could just create and use real FIFOs
> > on NFS(*).  But while NtQueryEaFile can be used to fetch real
> > NFS file info, and while NtCreateFile can be used to create real
> > synmlinks via NFS, I don't see an interface resembling mknod/mkfifo.
>
> Looking at the ms-nfs41-client source code, there is no API for that *YET*.
>
> So my plan would be like this:
> 1. Cygwin: implements the proposed devnodes-as-symlink emulation code,
> if the env var CYGWIN has "nfs_emulate_dev_special_files_as_symlink"
> set
> 2. Cygwin: By default |mkfifo()| will fail with |ENOSYS| (not |EPERM|,
> as we intend to fix the issue!) on NFS filesystems, unless
> CYGWIN=nfs_emulate_dev_special_files_as_symlink

I think the emulation should be on by default, or you break things.

Ced
-- 
Cedric Blancher <cedric.blancher@gmail.com>
[https://plus.google.com/u/0/+CedricBlancher/]
Institute Pasteur

Re: How to fix |mkfifo()| failure if |pathname| is on NFS ? / was: Re: [EXTERNAL] Re: mkfifo: cannot set permissions of 'x.fifo': Not a directory

[Corinna Vinschen]

On Aug 26 13:26, Cedric Blancher via Cygwin wrote:
> On Fri, 25 Aug 2023 at 14:26, Corinna Vinschen via Cygwin
> <cygwin@cygwin.com> wrote:
> >
> > On Aug 25 14:18, Corinna Vinschen via Cygwin wrote:
> > > On Aug 23 01:05, Roland Mainz via Cygwin wrote:
> > > > 2. Check whether the filesystem for the fifos path is NFS
> > > > (cgywin.dll's |fs.fs_is_nfs()|), and then just refuse |mkfifo()| with
> > > > |ENOSYS| (not implemented)
> > >
> > > I like the idea.
> >
> > EPERM, btw.  See https://man7.org/linux/man-pages/man2/mknod.2.html
> >
> 
> I am vehemently opposed to that idea. That will IMHO not only be the
> wrong solution, it will break actual things like autoconf&friends on
> NFS - they create fifos for testing, but do not use them. If you just
> make this return EPERM, then you break stuff we have to rely on.

https://cygwin.com/acronyms/#STC, please.


Corinna

Re: How to fix |mkfifo()| failure if |pathname| is on NFS ? / was: Re: [EXTERNAL] Re: mkfifo: cannot set permissions of 'x.fifo': Not a directory

[Corinna Vinschen]

On Aug 25 23:21, Lavrentiev, Anton (NIH/NLM/NCBI) [C] via Cygwin wrote:
> > it is not possible to diffentiate between Cygwin
> > FIFOs and real FIFOs created from the remote side in `ls -l'
> > output.
> 
> Why would that be necessary?  If it's a FIFO, it can be used as a
> FIFO, regardless where and how it was created..

You don't seem to understand the problem.  A Cygwin FIFO is not a real
FIFO.  We're running on Windows, right?  Show me how to create a FIFO on
NTFS or FAT by OS calls only, please.

Cygwin's way of creating FIFOs is to create a Cygwin syumlink
with special content which in turn is faked to be a FIFO when running
Cygwin's stat(2) call.  Usually that's a Windows shortcut (one of the
.lnk thingies), but on NFS we create a real symlink with weird content
*accidentally*, because that's how we create symlinks on NFS, and nobody
ever cared to that anyway.  Again, there is no API to create native
FIFOs on a remote NFS filesystem.  

But, ultimately, you're still right.  If I can recognize a file as FIFO,
I can use it as FIFO, regardless if it's a native FIFO or a Cygwin FIFO.
This just isn't done yet.


Corinna

RE: How to fix |mkfifo()| failure if |pathname| is on NFS ? / was: Re: [EXTERNAL] Re: mkfifo: cannot set permissions of 'x.fifo': Not a directory

[Lavrentiev, Anton (NIH/NLM/NCBI) [C]]

> You don't seem to understand the problem.  

I think I do, and that aligns with your explanation how Cygwin machinery works to fake the FIFOs.

> If I can recognize a file as FIFO, I can use it as FIFO, regardless if it's a native FIFO or a Cygwin FIFO.

That's exactly what I meant!

> Show me how to create a FIFO on NTFS or FAT by OS calls only, please.
> Again, there is no API to create native FIFOs on a remote NFS filesystem.

And because of the above, Cygwin can keep creating FIFOs as before, on any filesystem.

Anton Lavrentiev
Contractor NIH/NLM/NCBI

Re: How to fix |mkfifo()| failure if |pathname| is on NFS ? / was: Re: [EXTERNAL] Re: mkfifo: cannot set permissions of 'x.fifo': Not a directory

[Martin Wege]

On Sat, Aug 26, 2023 at 4:16 PM Corinna Vinschen via Cygwin
<cygwin@cygwin.com> wrote:
>
> On Aug 26 13:26, Cedric Blancher via Cygwin wrote:
> > On Fri, 25 Aug 2023 at 14:26, Corinna Vinschen via Cygwin
> > <cygwin@cygwin.com> wrote:
> > >
> > > On Aug 25 14:18, Corinna Vinschen via Cygwin wrote:
> > > > On Aug 23 01:05, Roland Mainz via Cygwin wrote:
> > > > > 2. Check whether the filesystem for the fifos path is NFS
> > > > > (cgywin.dll's |fs.fs_is_nfs()|), and then just refuse |mkfifo()| with
> > > > > |ENOSYS| (not implemented)
> > > >
> > > > I like the idea.
> > >
> > > EPERM, btw.  See https://man7.org/linux/man-pages/man2/mknod.2.html
> > >
> >
> > I am vehemently opposed to that idea. That will IMHO not only be the
> > wrong solution, it will break actual things like autoconf&friends on
> > NFS - they create fifos for testing, but do not use them. If you just
> > make this return EPERM, then you break stuff we have to rely on.
>
> https://cygwin.com/acronyms/#STC, please.#

Real world failure for Cygwin on NFS:

wget 'http://deb.debian.org/debian/pool/main/k/ksh93u+m/ksh93u+m_1.0.4.orig.tar.gz
cd ksh-1.0.4
$PWD/bin/package make

This fails with Cygwin 3.4.8-1:

cat buildlog.log
package: update
/cygdrive/z/arbeitdfn/windows_ksh/build_nfsv3/ksh-1.0.4/bin/execrate
package: initialize the
/cygdrive/z/arbeitdfn/windows_ksh/build_nfsv3/ksh-1.0.4/arch/cygwin.i386-64
view
package: update
/cygdrive/z/arbeitdfn/windows_ksh/build_nfsv3/ksh-1.0.4/arch/cygwin.i386-64/lib/probe/C/make/probe
package: update
/cygdrive/z/arbeitdfn/windows_ksh/build_nfsv3/ksh-1.0.4/arch/cygwin.i386-64/bin/mamake
package: make with mamake
package: make output captured in
/cygdrive/z/arbeitdfn/windows_ksh/build_nfsv3/ksh-1.0.4/arch/cygwin.i386-64/lib/package/gen/make.out
mkfifo: cannot set permissions of
'/cygdrive/z/arbeitdfn/windows_ksh/build_nfsv3/ksh-1.0.4/arch/cygwin.i386-64/lib/package/gen/make.out.fifo':
No such file or directory
package: make done at Fri Aug 26 19:24:26 CEST 2023 in
/cygdrive/z/arbeitdfn/windows_ksh/build_nfsv3/ksh-1.0.4/arch/cygwin.i386-64

If you need more examples just let me now, we have MUCH more of that...

Thanks,
Martin

Documenting Cygwin on NFS, no longer only exchange-medium! Re: How to fix |mkfifo()| failure if |pathname| is on NFS ? / was: Re: [EXTERNAL] Re: mkfifo: cannot set permissions of 'x.fifo': Not a directory

[Martin Wege]

On Fri, Aug 25, 2023 at 2:19 PM Corinna Vinschen via Cygwin
<cygwin@cygwin.com> wrote:
>
> On Aug 23 01:05, Roland Mainz via Cygwin wrote:
> > Note that Cygwin does not interpret the file |myfifo.fifo| as FIFO,
> > instead it comes back as a symlink "myfifo.fifo -> ':\0:c4:1000'".
> >
> > AFAIK there are (at least) these two options to fix the problems:
> > 1. Check whether the filesystem for the fifos path is NFS
> > (cgywin.dll's |fs.fs_is_nfs()|), and if it is a symlink check if it
> > starts with ':\0:c4:' (assuming "c4" is the prefix for inodes created
> > with |mkfifo()|). If this condition is |true|, then cygwin |stat()|,
> > |open()| etc. should treat this inode as FIFO.
>
> The downside is that it is not possible to diffentiate between Cygwin
> FIFOs and real FIFOs created from the remote side in `ls -l'
> output.  Note that Cygwin returns the NFS stat info verbatim, so
> a real FIFO is returned as a real FIFO:
>
>   linux$ mkfifo bar
>   cygwin$ ls -l bar
>   prw-r--r-- 1 corinna vinschen  0 Aug 25 13:58 bar
>
> The idea was always to use NFS as exchange medium, but not as
> installation medium for the entire distro or to keep Cygwin home
> dirs on NFS.  There were times where NFS was pretty unstable.
> I used NFS for quite some time to build Cygwin packages, but at one
> point I got trouble (performance problems with multiple concurrent
> processes accessing an NFS share, build errors out of the blue),
> so I switched to Samba shares, albeit grudgingly.  I'm not yet
> sure if the problems are fixed.  At least a recent OpenSSH build
> ran through without problems...

Corinna, could you please update the Cygwin documentation then? We use
NFS, both Windows builtin and CITI NFSv4.1, with Cygwin for years
without major problems, so that comment about exchange-medium-only is
clearly wrong!

Thanks,
Martin

Re: How to fix |mkfifo()| failure if |pathname| is on NFS ? / was: Re: [EXTERNAL] Re: mkfifo: cannot set permissions of 'x.fifo': Not a directory

[Martin Wege]

On Thu, Aug 24, 2023 at 6:45 PM Martin Wege <martin.l.wege@gmail.com> wrote:
>
> On Wed, Aug 23, 2023 at 1:06 AM Roland Mainz via Cygwin
> <cygwin@cygwin.com> wrote:
> >
> > On Tue, Aug 22, 2023 at 4:52 PM Lavrentiev, Anton (NIH/NLM/NCBI) [C]
> > via Cygwin <cygwin@cygwin.com> wrote:
> > > > FIFOs which don't make *any* sense
> > > > ... FWIW, a remote NFS fileystem.
> > >
> > > I got an impression that the OP is trying to deploy something (maybe the entire Cygwin) onto an
> > > NFS share.  So the named FIFO "file" is also created in there.
> >
> > I agree with that impression. This is basically what large sites
> > (universities etc) do with UNIX and Linux: The machines mount an
> > user's ${HOMR} directory via automounter over NFS, and users are
> > discouraged (e.g. grumpy admin visiting you in person, blocking all
> > escape routes... =:-) ) from using the machine's local filesystems (in
> > Cygwin's case that includes "C:"!).
> >
> > In that case people want to use |mkfifo()|/|mkfifoat()| and/or
> > /usr/bin/mkfifo in their home directory, and don't expect that it does
> > not work.
> >
> > But that is what happens on Cygwin 3.4.8 right now, if someone tries
> > to do a |mkfifo()| on a NFS home directory (tested with MS NFSv3 and
> > CITI NFSv4 clients):
> > |mkfifo()| succeeds, but the resulting inode is *NOT* a FIFO as requested
> >
> > Example (/cygdrive/h/ is my home directory shared from my Linux machine):
> > ---- snip ----
> > roland_mainz@winkrakra1 /cygdrive/h/work/cygwin_mkfifo_on_nfs
> > $ uname -a
> > CYGWIN_NT-10.0-19045 winkrakra1 3.4.8-1.x86_64 2023-08-17 17:02 UTC
> > x86_64 Cygwin
> >
> > roland_mainz@winkrakra1 /cygdrive/h/work/cygwin_mkfifo_on_nfs
> > $ mount
> > C:/cygwin64/bin on /usr/bin type ntfs (binary,auto)
> > C:/cygwin64/lib on /usr/lib type ntfs (binary,auto)
> > C:/cygwin64 on / type ntfs (binary,auto)
> > C: on /cygdrive/c type ntfs (binary,posix=0,user,noumount,auto)
> > H: on /cygdrive/h type nfs (binary,posix=0,user,noumount,auto)
> >
> > roland_mainz@winkrakra1 /cygdrive/h/work/cygwin_mkfifo_on_nfs
> > $ ls -l
> > total 1
> > -rw-rw-rw- 1 Unix_User+0 Unix_Group+0 330 Aug 22 23:58 cygwin_mkfifo_on_nfs.c
> >
> > roland_mainz@winkrakra1 /cygdrive/h/work/cygwin_mkfifo_on_nfs
> > $ cat -n cygwin_mkfifo_on_nfs.c
> >      1  #include <stdlib.h>
> >      2  #include <stdio.h>
> >      3  #include <stdio.h>
> >      4  #include <errno.h>
> >      5  #include <sys/types.h>
> >      6  #include <sys/stat.h>
> >      7
> >      8  int main(int ac, char *av[])
> >      9  {
> >     10          (void)puts("# start");
> >     11
> >     12          if
> > (mkfifo("/cygdrive/h/work/cygwin_mkfifo_on_nfs/myfifo.fifo", 0) != 0)
> >     13                  perror("mkfifo failed");
> >     14          (void)puts("# done.");
> >     15          return EXIT_SUCCESS;
> >     16  }
> >     17
> >
> > roland_mainz@winkrakra1 /cygdrive/h/work/cygwin_mkfifo_on_nfs
> > $ gcc -g cygwin_mkfifo_on_nfs.c
> >
> > roland_mainz@winkrakra1 /cygdrive/h/work/cygwin_mkfifo_on_nfs
> > $ ./a
> > # start
> > # done.
> >
> > roland_mainz@winkrakra1 /cygdrive/h/work/cygwin_mkfifo_on_nfs
> > $ ls -l
> > total 68
> > -rwxr-xr-x 1 Unix_User+0 Unix_Group+0 66951 Aug 23 00:12 a.exe
> > -rw-rw-rw- 1 Unix_User+0 Unix_Group+0   330 Aug 22 23:58 cygwin_mkfifo_on_nfs.c
> > lrwxrwxrwx 1 Unix_User+0 Unix_Group+0    11 Aug 23 00:12 myfifo.fifo
> > -> ':\0:c4:1000'
> >
> > roland_mainz@winkrakra1 /cygdrive/h/work/cygwin_mkfifo_on_nfs
> > $ cat <myfifo.fifo
> > -bash: myfifo.fifo: No such file or directory
> > ---- snip ----
> >
> > Note that Cygwin does not interpret the file |myfifo.fifo| as FIFO,
> > instead it comes back as a symlink "myfifo.fifo -> ':\0:c4:1000'".
> >
> > AFAIK there are (at least) these two options to fix the problems:
> > 1. Check whether the filesystem for the fifos path is NFS
> > (cgywin.dll's |fs.fs_is_nfs()|), and if it is a symlink check if it
> > starts with ':\0:c4:' (assuming "c4" is the prefix for inodes created
> > with |mkfifo()|). If this condition is |true|, then cygwin |stat()|,
> > |open()| etc. should treat this inode as FIFO.
> > 2. Check whether the filesystem for the fifos path is NFS
> > (cgywin.dll's |fs.fs_is_nfs()|), and then just refuse |mkfifo()| with
> > |ENOSYS| (not implemented)
> >
> > Better algorithm for [1] might be to check whether the inode is a
> > symlink, and then do a |fs.fs_is_nfs()| on the symlinks |pathname|,
> > assuming this is more performant.
>
> We would like to see option 1 implemented, maybe as an option in the
> CYGWIN environment variable
> (CYGWIN=nfs_emulate_dev_special_files_as_symlink ?)

One problem I see is how we can set machine-wide defaults, as the
CYGWIN variable is only inherited from process to process, and
/usr/bin/env - myprog will start myprog with an empty environment.

So if there is no way to set machine-wide defaults, I would prefer to
have the nfs_emulate_dev_special_files_as_symlink enabled by default.

Thanks,
Martin

Re: How to fix |mkfifo()| failure if |pathname| is on NFS ? / was: Re: [EXTERNAL] Re: mkfifo: cannot set permissions of 'x.fifo': Not a directory

[Brian Inglis]

On 2023-08-26 11:40, Martin Wege via Cygwin wrote:
> On Sat, Aug 26, 2023 at 4:16 PM Corinna Vinschen via Cygwin
> <cygwin@cygwin.com> wrote:
>>
>> On Aug 26 13:26, Cedric Blancher via Cygwin wrote:
>>> On Fri, 25 Aug 2023 at 14:26, Corinna Vinschen via Cygwin
>>> <cygwin@cygwin.com> wrote:
>>>>
>>>> On Aug 25 14:18, Corinna Vinschen via Cygwin wrote:
>>>>> On Aug 23 01:05, Roland Mainz via Cygwin wrote:
>>>>>> 2. Check whether the filesystem for the fifos path is NFS
>>>>>> (cgywin.dll's |fs.fs_is_nfs()|), and then just refuse |mkfifo()| with
>>>>>> |ENOSYS| (not implemented)
>>>>>
>>>>> I like the idea.
>>>>
>>>> EPERM, btw.  See https://man7.org/linux/man-pages/man2/mknod.2.html
>>>>
>>>
>>> I am vehemently opposed to that idea. That will IMHO not only be the
>>> wrong solution, it will break actual things like autoconf&friends on
>>> NFS - they create fifos for testing, but do not use them. If you just
>>> make this return EPERM, then you break stuff we have to rely on.
>>
>> https://cygwin.com/acronyms/#STC, please.#
> 
> Real world failure for Cygwin on NFS:
> 
> wget 'http://deb.debian.org/debian/pool/main/k/ksh93u+m/ksh93u+m_1.0.4.orig.tar.gz
> cd ksh-1.0.4
> $PWD/bin/package make
> 
> This fails with Cygwin 3.4.8-1:
> 
> cat buildlog.log
> package: update
> /cygdrive/z/arbeitdfn/windows_ksh/build_nfsv3/ksh-1.0.4/bin/execrate
> package: initialize the
> /cygdrive/z/arbeitdfn/windows_ksh/build_nfsv3/ksh-1.0.4/arch/cygwin.i386-64
> view
> package: update
> /cygdrive/z/arbeitdfn/windows_ksh/build_nfsv3/ksh-1.0.4/arch/cygwin.i386-64/lib/probe/C/make/probe
> package: update
> /cygdrive/z/arbeitdfn/windows_ksh/build_nfsv3/ksh-1.0.4/arch/cygwin.i386-64/bin/mamake
> package: make with mamake
> package: make output captured in
> /cygdrive/z/arbeitdfn/windows_ksh/build_nfsv3/ksh-1.0.4/arch/cygwin.i386-64/lib/package/gen/make.out
> mkfifo: cannot set permissions of
> '/cygdrive/z/arbeitdfn/windows_ksh/build_nfsv3/ksh-1.0.4/arch/cygwin.i386-64/lib/package/gen/make.out.fifo':
> No such file or directory
> package: make done at Fri Aug 26 19:24:26 CEST 2023 in
> /cygdrive/z/arbeitdfn/windows_ksh/build_nfsv3/ksh-1.0.4/arch/cygwin.i386-64
> 
> If you need more examples just let me now, we have MUCH more of that...

No - the DLL developers need fewer examples, just *ONE* Simple Test Case, boiled 
down to the essentials, with notes on preconditions, if you can not recreate the 
required conditions in a program or script.

Alternatively, Cygwin strace output with notes from a failing program, or a gdb 
script and log of the failing code with enough calling context and notes.

Please read the problem reporting guidelines linked at the bottom of all posts 
and follow them.

-- 
Take care. Thanks, Brian Inglis              Calgary, Alberta, Canada

La perfection est atteinte                   Perfection is achieved
non pas lorsqu'il n'y a plus rien à ajouter  not when there is no more to add
mais lorsqu'il n'y a plus rien à retirer     but when there is no more to cut
                                 -- Antoine de Saint-Exupéry

Re: Documenting Cygwin on NFS, no longer only exchange-medium! Re: How to fix |mkfifo()| failure if |pathname| is on NFS ? / was: Re: [EXTERNAL] Re: mkfifo: cannot set permissions of 'x.fifo': Not a directory

[Brian Inglis]

On 2023-08-26 11:44, Martin Wege via Cygwin wrote:
> On Fri, Aug 25, 2023 at 2:19 PM Corinna Vinschen via Cygwin
> <cygwin@cygwin.com> wrote:
>>
>> On Aug 23 01:05, Roland Mainz via Cygwin wrote:
>>> Note that Cygwin does not interpret the file |myfifo.fifo| as FIFO,
>>> instead it comes back as a symlink "myfifo.fifo -> ':\0:c4:1000'".
>>>
>>> AFAIK there are (at least) these two options to fix the problems:
>>> 1. Check whether the filesystem for the fifos path is NFS
>>> (cgywin.dll's |fs.fs_is_nfs()|), and if it is a symlink check if it
>>> starts with ':\0:c4:' (assuming "c4" is the prefix for inodes created
>>> with |mkfifo()|). If this condition is |true|, then cygwin |stat()|,
>>> |open()| etc. should treat this inode as FIFO.
>>
>> The downside is that it is not possible to diffentiate between Cygwin
>> FIFOs and real FIFOs created from the remote side in `ls -l'
>> output.  Note that Cygwin returns the NFS stat info verbatim, so
>> a real FIFO is returned as a real FIFO:
>>
>>    linux$ mkfifo bar
>>    cygwin$ ls -l bar
>>    prw-r--r-- 1 corinna vinschen  0 Aug 25 13:58 bar
>>
>> The idea was always to use NFS as exchange medium, but not as
>> installation medium for the entire distro or to keep Cygwin home
>> dirs on NFS.  There were times where NFS was pretty unstable.
>> I used NFS for quite some time to build Cygwin packages, but at one
>> point I got trouble (performance problems with multiple concurrent
>> processes accessing an NFS share, build errors out of the blue),
>> so I switched to Samba shares, albeit grudgingly.  I'm not yet
>> sure if the problems are fixed.  At least a recent OpenSSH build
>> ran through without problems...
> 
> Corinna, could you please update the Cygwin documentation then? We use
> NFS, both Windows builtin and CITI NFSv4.1, with Cygwin for years
> without major problems, so that comment about exchange-medium-only is
> clearly wrong!

Feel free to submit doc patches if you feel there is a lack.

We are all volunteers here - and that includes you!

-- 
Take care. Thanks, Brian Inglis              Calgary, Alberta, Canada

La perfection est atteinte                   Perfection is achieved
non pas lorsqu'il n'y a plus rien à ajouter  not when there is no more to add
mais lorsqu'il n'y a plus rien à retirer     but when there is no more to cut
                                 -- Antoine de Saint-Exupéry

Re: Documenting Cygwin on NFS, no longer only exchange-medium! Re: How to fix |mkfifo()| failure if |pathname| is on NFS ? / was: Re: [EXTERNAL] Re: mkfifo: cannot set permissions of 'x.fifo': Not a directory

[Corinna Vinschen]

On Aug 26 19:44, Martin Wege via Cygwin wrote:
> On Fri, Aug 25, 2023 at 2:19 PM Corinna Vinschen via Cygwin
> <cygwin@cygwin.com> wrote:
> >
> > On Aug 23 01:05, Roland Mainz via Cygwin wrote:
> > > Note that Cygwin does not interpret the file |myfifo.fifo| as FIFO,
> > > instead it comes back as a symlink "myfifo.fifo -> ':\0:c4:1000'".
> > >
> > > AFAIK there are (at least) these two options to fix the problems:
> > > 1. Check whether the filesystem for the fifos path is NFS
> > > (cgywin.dll's |fs.fs_is_nfs()|), and if it is a symlink check if it
> > > starts with ':\0:c4:' (assuming "c4" is the prefix for inodes created
> > > with |mkfifo()|). If this condition is |true|, then cygwin |stat()|,
> > > |open()| etc. should treat this inode as FIFO.
> >
> > The downside is that it is not possible to diffentiate between Cygwin
> > FIFOs and real FIFOs created from the remote side in `ls -l'
> > output.  Note that Cygwin returns the NFS stat info verbatim, so
> > a real FIFO is returned as a real FIFO:
> >
> >   linux$ mkfifo bar
> >   cygwin$ ls -l bar
> >   prw-r--r-- 1 corinna vinschen  0 Aug 25 13:58 bar
> >
> > The idea was always to use NFS as exchange medium, but not as
> > installation medium for the entire distro or to keep Cygwin home
> > dirs on NFS.  There were times where NFS was pretty unstable.
> > I used NFS for quite some time to build Cygwin packages, but at one
> > point I got trouble (performance problems with multiple concurrent
> > processes accessing an NFS share, build errors out of the blue),
> > so I switched to Samba shares, albeit grudgingly.  I'm not yet
> > sure if the problems are fixed.  At least a recent OpenSSH build
> > ran through without problems...
> 
> Corinna, could you please update the Cygwin documentation then? We use
> NFS, both Windows builtin and CITI NFSv4.1, with Cygwin for years
> without major problems, so that comment about exchange-medium-only is
> clearly wrong!

I'm talking about the idea how users would use NFS, and the idea was
that nobody in their right mind would use a crippled filesystem (from
the Windows POV) for the entire installation.  Before you came up with
your complaints, nobody here even knew that somebody actually did that.

NFS is a niche filesystem on Windows and was never in the focus of
anybody here, except me, because I was toying around with it and was
happy to have it working.  As far as I knew until a few days ago,
I was the only user of Cygwin's NFS support.

As for FIFOs on NFS, as I wrote, they never worked as desired on NFS.
For kicks I tried back until Cygwin 3.3.  Given this isn't a regression,
it won't be fixed for Cygwin 3.4.

As Brian wrote in his reply to your mail, we are all volunteers here.
Everybody working on Cygwin is working in their spare time and/or on the
goodwill of their employer.

Cygwin is open source.  You can fix or implement stuff yourself and send
the patches to the cygwin-patches ML for review.  Otherwise you will
have to let us, the people actually working on the project, decide when
and how something gets implemented.

Having said that, I have a local patch enabling FIFOs on NFS.  I might
push it at one point to the main branch, but the pressure you guys are
currently building up on this mailing list puts me right off.


Corinna

Re: Documenting Cygwin on NFS, no longer only exchange-medium! Re: How to fix |mkfifo()| failure if |pathname| is on NFS ? / was: Re: [EXTERNAL] Re: mkfifo: cannot set permissions of 'x.fifo': Not a directory

[Cedric Blancher]

On Sun, 27 Aug 2023 at 14:35, Corinna Vinschen via Cygwin
<cygwin@cygwin.com> wrote:
>
> On Aug 26 19:44, Martin Wege via Cygwin wrote:
> > On Fri, Aug 25, 2023 at 2:19 PM Corinna Vinschen via Cygwin
> > <cygwin@cygwin.com> wrote:
> > >
> > > On Aug 23 01:05, Roland Mainz via Cygwin wrote:
> > > > Note that Cygwin does not interpret the file |myfifo.fifo| as FIFO,
> > > > instead it comes back as a symlink "myfifo.fifo -> ':\0:c4:1000'".
> > > >
> > > > AFAIK there are (at least) these two options to fix the problems:
> > > > 1. Check whether the filesystem for the fifos path is NFS
> > > > (cgywin.dll's |fs.fs_is_nfs()|), and if it is a symlink check if it
> > > > starts with ':\0:c4:' (assuming "c4" is the prefix for inodes created
> > > > with |mkfifo()|). If this condition is |true|, then cygwin |stat()|,
> > > > |open()| etc. should treat this inode as FIFO.
> > >
> > > The downside is that it is not possible to diffentiate between Cygwin
> > > FIFOs and real FIFOs created from the remote side in `ls -l'
> > > output.  Note that Cygwin returns the NFS stat info verbatim, so
> > > a real FIFO is returned as a real FIFO:
> > >
> > >   linux$ mkfifo bar
> > >   cygwin$ ls -l bar
> > >   prw-r--r-- 1 corinna vinschen  0 Aug 25 13:58 bar
> > >
> > > The idea was always to use NFS as exchange medium, but not as
> > > installation medium for the entire distro or to keep Cygwin home
> > > dirs on NFS.  There were times where NFS was pretty unstable.
> > > I used NFS for quite some time to build Cygwin packages, but at one
> > > point I got trouble (performance problems with multiple concurrent
> > > processes accessing an NFS share, build errors out of the blue),
> > > so I switched to Samba shares, albeit grudgingly.  I'm not yet
> > > sure if the problems are fixed.  At least a recent OpenSSH build
> > > ran through without problems...
> >
> > Corinna, could you please update the Cygwin documentation then? We use
> > NFS, both Windows builtin and CITI NFSv4.1, with Cygwin for years
> > without major problems, so that comment about exchange-medium-only is
> > clearly wrong!
>
> I'm talking about the idea how users would use NFS, and the idea was
> that nobody in their right mind would use a crippled filesystem (from
> the Windows POV) for the entire installation.  Before you came up with
> your complaints, nobody here even knew that somebody actually did that.

What is missing in the Windows NFS filesystem to uncripple (if there
is such a word in English) it from a Windows perspective?

>
> NFS is a niche filesystem on Windows and was never in the focus of
> anybody here, except me, because I was toying around with it and was
> happy to have it working.  As far as I knew until a few days ago,
> I was the only user of Cygwin's NFS support.

Well, surprise, you were wrong. The Windows NFS filesystem has lots of
users, but there are no centralised statistics.

Just as an indicator, NFS on Windows has at least enough users to feed
a couple of engineers at OpenText to work on their OpenText (once
Hummingbird) NFS client product. Plus the two free implementations,
Windows builtin NFSv3, and the CITI ms-nfsv41-client.

>
> As for FIFOs on NFS, as I wrote, they never worked as desired on NFS.
> For kicks I tried back until Cygwin 3.3.  Given this isn't a regression,
> it won't be fixed for Cygwin 3.4.

I am pretty sure this is a regression, as blast, staden and other
software were built and operated on Cygwin.

Just my PAIN job is how to rebuild our compute cluster (after total
loss due water damage), new Windows, new blast&friends, new Cygwin.
And promptly we got hit by this issue.

>
> As Brian wrote in his reply to your mail, we are all volunteers here.
> Everybody working on Cygwin is working in their spare time and/or on the
> goodwill of their employer.
>
> Cygwin is open source.  You can fix or implement stuff yourself and send
> the patches to the cygwin-patches ML for review.  Otherwise you will
> have to let us, the people actually working on the project, decide when
> and how something gets implemented.

I agree.

>
> Having said that, I have a local patch enabling FIFOs on NFS.  I might
> push it at one point to the main branch, but the pressure you guys are
> currently building up on this mailing list puts me right off.

This was not intended, at least not from my side. But in any case, I
humbly apologise.

Ced
-- 
Cedric Blancher <cedric.blancher@gmail.com>
[https://plus.google.com/u/0/+CedricBlancher/]
Institute Pasteur

Re: Documenting Cygwin on NFS, no longer only exchange-medium! Re: How to fix |mkfifo()| failure if |pathname| is on NFS ? / was: Re: [EXTERNAL] Re: mkfifo: cannot set permissions of 'x.fifo': Not a directory

[Corinna Vinschen]

On Aug 28 07:35, Cedric Blancher via Cygwin wrote:
> On Sun, 27 Aug 2023 at 14:35, Corinna Vinschen via Cygwin
> <cygwin@cygwin.com> wrote:
> > As for FIFOs on NFS, as I wrote, they never worked as desired on NFS.
> > For kicks I tried back until Cygwin 3.3.  Given this isn't a regression,
> > it won't be fixed for Cygwin 3.4.
> 
> I am pretty sure this is a regression, as blast, staden and other
> software were built and operated on Cygwin.

The fact that some software works doesn't mean they use a certain
feature.  What, for instance, if they just created their FIFOs on a
local tmp dir?

I actually *tested* it on

- Cygwin 3.3.1-1 from 2021-10-28
- Cygwin 3.2.0-1 from 2021-03-29
- Cygwin 3.1.1-1 from 2019-12-18
- Cygwin 3.0.7-1 from 2019-04-30
- Cygwin 2.9.0-3 from 2017-09-12

FIFOs on NFS didn't work in all cases.

We can go back in time prior to 2017 if you like, but Cygwin FIFOs were
always just created as symlinks, and NFS symlinks were never tested for
being a device, so they never have been treated as FIFOs.  Also, NFS
FIFOs have never been recognized by the file handling code as Cygwin
FIFOs, they were just shown as such in stat(2).


Corinna

Re: Documenting Cygwin on NFS, no longer only exchange-medium! Re: How to fix |mkfifo()| failure if |pathname| is on NFS ? / was: Re: [EXTERNAL] Re: mkfifo: cannot set permissions of 'x.fifo': Not a directory

[Joshuah Hurst]

On Sun, Aug 27, 2023 at 2:35 PM Corinna Vinschen via Cygwin
<cygwin@cygwin.com> wrote:
>
> On Aug 26 19:44, Martin Wege via Cygwin wrote:
> > On Fri, Aug 25, 2023 at 2:19 PM Corinna Vinschen via Cygwin
> > <cygwin@cygwin.com> wrote:
> > >
> > > On Aug 23 01:05, Roland Mainz via Cygwin wrote:
> > > > Note that Cygwin does not interpret the file |myfifo.fifo| as FIFO,
> > > > instead it comes back as a symlink "myfifo.fifo -> ':\0:c4:1000'".
> > > >
> > > > AFAIK there are (at least) these two options to fix the problems:
> > > > 1. Check whether the filesystem for the fifos path is NFS
> > > > (cgywin.dll's |fs.fs_is_nfs()|), and if it is a symlink check if it
> > > > starts with ':\0:c4:' (assuming "c4" is the prefix for inodes created
> > > > with |mkfifo()|). If this condition is |true|, then cygwin |stat()|,
> > > > |open()| etc. should treat this inode as FIFO.
> > >
> > > The downside is that it is not possible to diffentiate between Cygwin
> > > FIFOs and real FIFOs created from the remote side in `ls -l'
> > > output.  Note that Cygwin returns the NFS stat info verbatim, so
> > > a real FIFO is returned as a real FIFO:
> > >
> > >   linux$ mkfifo bar
> > >   cygwin$ ls -l bar
> > >   prw-r--r-- 1 corinna vinschen  0 Aug 25 13:58 bar
> > >
> > > The idea was always to use NFS as exchange medium, but not as
> > > installation medium for the entire distro or to keep Cygwin home
> > > dirs on NFS.  There were times where NFS was pretty unstable.
> > > I used NFS for quite some time to build Cygwin packages, but at one
> > > point I got trouble (performance problems with multiple concurrent
> > > processes accessing an NFS share, build errors out of the blue),
> > > so I switched to Samba shares, albeit grudgingly.  I'm not yet
> > > sure if the problems are fixed.  At least a recent OpenSSH build
> > > ran through without problems...
> >
> > Corinna, could you please update the Cygwin documentation then? We use
> > NFS, both Windows builtin and CITI NFSv4.1, with Cygwin for years
> > without major problems, so that comment about exchange-medium-only is
> > clearly wrong!
>
> I'm talking about the idea how users would use NFS, and the idea was
> that nobody in their right mind would use a crippled filesystem (from
> the Windows POV) for the entire installation.  Before you came up with
> your complaints, nobody here even knew that somebody actually did that.
>
> NFS is a niche filesystem on Windows and was never in the focus of
> anybody here, except me, because I was toying around with it and was
> happy to have it working.  As far as I knew until a few days ago,
> I was the only user of Cygwin's NFS support.
>
> As for FIFOs on NFS, as I wrote, they never worked as desired on NFS.
> For kicks I tried back until Cygwin 3.3.  Given this isn't a regression,
> it won't be fixed for Cygwin 3.4.
>
> As Brian wrote in his reply to your mail, we are all volunteers here.
> Everybody working on Cygwin is working in their spare time and/or on the
> goodwill of their employer.
>
> Cygwin is open source.  You can fix or implement stuff yourself and send
> the patches to the cygwin-patches ML for review.  Otherwise you will
> have to let us, the people actually working on the project, decide when
> and how something gets implemented.
>
> Having said that, I have a local patch enabling FIFOs on NFS.  I might
> push it at one point to the main branch, but the pressure you guys are
> currently building up on this mailing list puts me right off.

Corinna, please do not let people put you off. I don't even think it
was intended as pressure.

Could you please share the patch for testing?
-- 
Josh