[ANNOUNCEMENT] cygwin 3.4.9-1

[ANNOUNCEMENT] cygwin 3.4.9-1

[Corinna Vinschen via Cygwin-announce]

The following packages have been uploaded to the Cygwin distribution:

* cygwin-3.4.9-1
* cygwin-devel-3.4.9-1
* cygwin-doc-3.4.9-1

Bug Fixes
---------

- Fix a bug introduced in cygwin 3.4.0 that switch_to_nat_pipe flag is
  not cleared properly when non-cygwin app is terminated in the case
  where pseudo console is not activated.

- For the time being, disable creating special files using mknod/mkfifo
  on NFS.
  Addresses: https://cygwin.com/pipermail/cygwin/2023-August/254266.html

- Fix segfault when too many command line args are specified.
  Addresses: https://cygwin.com/pipermail/cygwin/2023-August/254333.html

- Fix build problems in terms of sys/cpuset.h.
  Addresses: https://cygwin.com/pipermail/cygwin/2023-August/254283.html

NFS mkfifo support in cygwin 3.5.0 Re: [ANNOUNCEMENT] cygwin 3.4.9-1

[Martin Wege]

On Wed, Sep 6, 2023 at 5:27 PM Corinna Vinschen via Cygwin-announce
via Cygwin <cygwin@cygwin.com> wrote:
>
> The following packages have been uploaded to the Cygwin distribution:
>
> * cygwin-3.4.9-1
> * cygwin-devel-3.4.9-1
> * cygwin-doc-3.4.9-1
>
> Bug Fixes
> ---------
>
> - Fix a bug introduced in cygwin 3.4.0 that switch_to_nat_pipe flag is
>   not cleared properly when non-cygwin app is terminated in the case
>   where pseudo console is not activated.
>
> - For the time being, disable creating special files using mknod/mkfifo
>   on NFS.
>   Addresses: https://cygwin.com/pipermail/cygwin/2023-August/254266.html

While I am unhappy that you disabled this for Cygwin 3.4.9-1, we
noticed that you added mkfifo support for Cygwin 3.5.0 in
https://cygwin.com/git/?p=newlib-cygwin.git;a=commit;h=622fb0776ea333dd708ff312f08ec98311138fbe
🙏😁 :-)

Thanks

We are testing 3.5.0-0.423.g7ced682549ae.x86_64 - which includes
622fb0776ea333dd708ff312f08ec98311138fbe - right now, feedback will
come on Monday

Thanks,
Martin

Re: NFS mkfifo support in cygwin 3.5.0 Re: [ANNOUNCEMENT] cygwin 3.4.9-1

[Cedric Blancher]

On Thu, 7 Sept 2023 at 23:20, Martin Wege via Cygwin <cygwin@cygwin.com> wrote:
>
> On Wed, Sep 6, 2023 at 5:27 PM Corinna Vinschen via Cygwin-announce
> via Cygwin <cygwin@cygwin.com> wrote:
> >
> > The following packages have been uploaded to the Cygwin distribution:
> >
> > * cygwin-3.4.9-1
> > * cygwin-devel-3.4.9-1
> > * cygwin-doc-3.4.9-1
> >
> > Bug Fixes
> > ---------
> >
> > - Fix a bug introduced in cygwin 3.4.0 that switch_to_nat_pipe flag is
> >   not cleared properly when non-cygwin app is terminated in the case
> >   where pseudo console is not activated.
> >
> > - For the time being, disable creating special files using mknod/mkfifo
> >   on NFS.
> >   Addresses: https://cygwin.com/pipermail/cygwin/2023-August/254266.html
>
> While I am unhappy that you disabled this for Cygwin 3.4.9-1, we
> noticed that you added mkfifo support for Cygwin 3.5.0 in
> https://cygwin.com/git/?p=newlib-cygwin.git;a=commit;h=622fb0776ea333dd708ff312f08ec98311138fbe
> 🙏😁 :-)
>
> Thanks
>
> We are testing 3.5.0-0.423.g7ced682549ae.x86_64 - which includes
> 622fb0776ea333dd708ff312f08ec98311138fbe - right now, feedback will
> come on Monday

That's amazing news.

But an old bug crept in:
H is NFS file system
$ uname -a
CYGWIN_NT-10.0-19045 bastille0x44ef 3.5.0-0.423.g7ced682549ae.x86_64
2023-09-07 17:19 UTC x86_64 Cygwin
$ mount
C:/cygwin64/bin on /usr/bin type ntfs (binary,auto)
C:/cygwin64/lib on /usr/lib type ntfs (binary,auto)
C:/cygwin64 on / type ntfs (binary,auto)
C: on /cygdrive/c type ntfs (binary,posix=0,user,noumount,auto)
H: on /cygdrive/h type nfs (binary,posix=0,user,noumount,auto)
M: on /cygdrive/m type nfs (binary,posix=0,user,noumount,auto)
N: on /cygdrive/n type nfs (binary,posix=0,user,noumount,auto)
O: on /cygdrive/o type nfs (binary,posix=0,user,noumount,auto)
P: on /cygdrive/p type nfs (binary,posix=0,user,noumount,auto)
$ cd /cygdrive/h/tmp/
$ mkfifo f1.fifo ; echo $?
0
$ mkfifo -m 0600 f2.fifo ; echo $?
mkfifo: cannot set permissions of 'f2.fifo': No such file or directory
1

So chmod() for a FIFO inode on NFS fails. Tested with MSFT NFSv3 and
new builds of the NFSv4.1 ms-nfs41-client filesystems.

Ced
-- 
Cedric Blancher <cedric.blancher@gmail.com>
[https://plus.google.com/u/0/+CedricBlancher/]
Institute Pasteur

Re: NFS mkfifo support in cygwin 3.5.0 Re: [ANNOUNCEMENT] cygwin 3.4.9-1

[Corinna Vinschen]

On Sep  8 06:48, Cedric Blancher via Cygwin wrote:
> On Thu, 7 Sept 2023 at 23:20, Martin Wege via Cygwin <cygwin@cygwin.com> wrote:
> >
> > On Wed, Sep 6, 2023 at 5:27 PM Corinna Vinschen via Cygwin-announce
> > via Cygwin <cygwin@cygwin.com> wrote:
> > >
> > > The following packages have been uploaded to the Cygwin distribution:
> > >
> > > * cygwin-3.4.9-1
> > > * cygwin-devel-3.4.9-1
> > > * cygwin-doc-3.4.9-1
> > >
> > > Bug Fixes
> > > ---------
> > >
> > > - Fix a bug introduced in cygwin 3.4.0 that switch_to_nat_pipe flag is
> > >   not cleared properly when non-cygwin app is terminated in the case
> > >   where pseudo console is not activated.
> > >
> > > - For the time being, disable creating special files using mknod/mkfifo
> > >   on NFS.
> > >   Addresses: https://cygwin.com/pipermail/cygwin/2023-August/254266.html
> >
> > While I am unhappy that you disabled this for Cygwin 3.4.9-1, we
> > noticed that you added mkfifo support for Cygwin 3.5.0 in
> > https://cygwin.com/git/?p=newlib-cygwin.git;a=commit;h=622fb0776ea333dd708ff312f08ec98311138fbe
> > 🙏😁 :-)
> >
> > Thanks
> >
> > We are testing 3.5.0-0.423.g7ced682549ae.x86_64 - which includes
> > 622fb0776ea333dd708ff312f08ec98311138fbe - right now, feedback will
> > come on Monday
> 
> That's amazing news.
> 
> But an old bug crept in:
> H is NFS file system
> $ uname -a
> CYGWIN_NT-10.0-19045 bastille0x44ef 3.5.0-0.423.g7ced682549ae.x86_64
> 2023-09-07 17:19 UTC x86_64 Cygwin
> $ mount
> C:/cygwin64/bin on /usr/bin type ntfs (binary,auto)
> C:/cygwin64/lib on /usr/lib type ntfs (binary,auto)
> C:/cygwin64 on / type ntfs (binary,auto)
> C: on /cygdrive/c type ntfs (binary,posix=0,user,noumount,auto)
> H: on /cygdrive/h type nfs (binary,posix=0,user,noumount,auto)
> M: on /cygdrive/m type nfs (binary,posix=0,user,noumount,auto)
> N: on /cygdrive/n type nfs (binary,posix=0,user,noumount,auto)
> O: on /cygdrive/o type nfs (binary,posix=0,user,noumount,auto)
> P: on /cygdrive/p type nfs (binary,posix=0,user,noumount,auto)
> $ cd /cygdrive/h/tmp/
> $ mkfifo f1.fifo ; echo $?
> 0
> $ mkfifo -m 0600 f2.fifo ; echo $?
> mkfifo: cannot set permissions of 'f2.fifo': No such file or directory
> 1
> 
> So chmod() for a FIFO inode on NFS fails. Tested with MSFT NFSv3 and
> new builds of the NFSv4.1 ms-nfs41-client filesystems.

Did you actually test this with 3.4.8?  It never worked on NFS.

Just to be clear, the above creates a Cygwin FIFO.  The situation
is different with native FIFOs, created on the host.  With those,
chmod worked before because native FIFOs were handled like normal files,
except in stat(2).  Now that they are handled as FIFOs, the mechanism
to change the file mode doesn't work anymore, because it depends on
FIFOs being WIndows shortcuts.

I will look into that at one point, but it's not a regression.


Corinna

Re: NFS mkfifo support in cygwin 3.5.0 Re: [ANNOUNCEMENT] cygwin 3.4.9-1

[Corinna Vinschen]

On Sep  8 12:59, Corinna Vinschen via Cygwin wrote:
> On Sep  8 06:48, Cedric Blancher via Cygwin wrote:
> > So chmod() for a FIFO inode on NFS fails. Tested with MSFT NFSv3 and
> > new builds of the NFSv4.1 ms-nfs41-client filesystems.
> 
> Did you actually test this with 3.4.8?  It never worked on NFS.
> 
> Just to be clear, the above creates a Cygwin FIFO.  The situation
> is different with native FIFOs, created on the host.  With those,
> chmod worked before because native FIFOs were handled like normal files,
> except in stat(2).  Now that they are handled as FIFOs, the mechanism
> to change the file mode doesn't work anymore, because it depends on
> FIFOs being WIndows shortcuts.
> 
> I will look into that at one point, but it's not a regression.

For the records:

Fixing this for native FIFOs is relatively easy. Fixing this for Cygwin
emulated FIFOs is rightout impossible ATM:

Cygwin FIFOs are symlinks with special content, basically containing
device major, device minor and mode bits.

Overwriting the mode via chmod(2) requires to change the symlink target,
an operation which isn't supported for real symlinks on NFS-mounted
filesystems.  Consequentially, the FILE_OVERWRITE_IF creation mode
doesn't work when trying to create a symlink.  It always fails with
STATUS_OBJECT_PATH_SYNTAX_BAD.

The next idea coming to mind is to unlink and recreate the symlink
(albeit not an atomic operation).  This works if you call, e. g.,
chmod(1).

Unfortunately this does *not* work if there's an open handle to the
file.  This occurs, for instance, in mkfifo(1) when using the -m option.
It calls mkfifo(3) with the correct mode bits, and then, if that worked,
and the -m option has been specified, it calls chmod(2) on the FIFO to
set the same mode again.  It does so while it opened the symlink with
the O_PATH flag.  I don't know why it does this, but albeit it looks
like double work, the operation itself is allowed and should work.

What happens in this case is this:

As soon as chmod(2) got called, the first NtCreateFile trying to create
the symlink fails with STATUS_OBJECT_NAME_COLLISION.  So just check for
this status code and then call unlink on the FIFO symlink.

Given the symlink is in use, the unlink operation renames the FIFO to
some temporary filename (that's builtin functionality of MSFT NFSv3).

Theoretically the original FIFO name is now free to be reused for the
FIFO symlink, but the next NtCreateFile call trying to create the symlink
still fails with STATUS_OBJECT_NAME_COLLISION.  Rinse and repeat.

So it's still a bad idea to use FIFOs on NFS.

I'll be offline for some time now for personal reasons, but if somebody
has an idea how to fix this *and* understands Cygwin's inner workings,
*and* can come up with a patch, feel free to send it to the
cygwin-patches mailing list.

Other than that, no idle musings, please.  It just distracts from
finding a working solution.


Corinna

Re: NFS mkfifo support in cygwin 3.5.0

[Corinna Vinschen]

On Sep  8 22:31, Corinna Vinschen via Cygwin wrote:
> On Sep  8 12:59, Corinna Vinschen via Cygwin wrote:
> > On Sep  8 06:48, Cedric Blancher via Cygwin wrote:
> > > So chmod() for a FIFO inode on NFS fails. Tested with MSFT NFSv3 and
> > > new builds of the NFSv4.1 ms-nfs41-client filesystems.
> > 
> > Did you actually test this with 3.4.8?  It never worked on NFS.
> > 
> > Just to be clear, the above creates a Cygwin FIFO.  The situation
> > is different with native FIFOs, created on the host.  With those,
> > chmod worked before because native FIFOs were handled like normal files,
> > except in stat(2).  Now that they are handled as FIFOs, the mechanism
> > to change the file mode doesn't work anymore, because it depends on
> > FIFOs being WIndows shortcuts.
> > 
> > I will look into that at one point, but it's not a regression.
> 
> For the records:
> 
> Fixing this for native FIFOs is relatively easy. Fixing this for Cygwin
> emulated FIFOs is rightout impossible ATM:
> [...]

I found a solution by switching Cygwin FIFOs on NFS from symlinks to
shortcut files.  I disabled symlink FIFOs entirely, so you have to
remove the Cygwin FIFOs you created on NFS in the last days and create
them anew.

The patches are in the test release cygwin 3.5.0-0.432.g5bbe0e56c617
and later.

> I'll be offline for some time now for personal reasons, 

That's definitely true starting right now.


Corinna