/dev/fd/N not synonymous with file descriptor N; it is on Linux

/dev/fd/N not synonymous with file descriptor N; it is on Linux

[Houder]

[-- Attachment #1: Type: text/plain, Size: 956 bytes --]

L.S.,

/dev/fd/N not synonymous with file descriptor N; it is on Linux

64-@@ cat /dev/fd/0 <<\EOF
> Hi
> EOF
cat: /dev/fd/0: No such file or directory

fails on Cygwin; not on Linux.

Also see:

     https://cygwin.com/ml/cygwin/2018-12/msg00028.html
     ( Bash heredoc on FD 3 )

Based on the output of strace on Linux, I composed an STC, that 
duplicates
the steps taken by bash (and cat).

This STC succeeds on Linux, but fails on Cygwin.

What does the STC do:

  - it creates a (temporary) file in the same way that bash does
  - the file is written to, the file descriptor is closed and the file 
unlinked
  - however, before the file is unlinked, it is opened a second time, 
like bash
    would do

Next
  - the file /dev/fd/N is opened, where N is the file descriptor that has 
been
    left open; this is what the "cat command" would do.

The "cat command" on Linux succeeds; it fails on Cygwin.

Regards,
Henri

STC attached (hopefully)

=====

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: stc.c --]
[-- Type: text/x-c; name=stc.c, Size: 3256 bytes --]

// gcc -Wall -o stc stc.c

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h> // strerror()
#include <errno.h>
#include <fcntl.h>

/*
 LPI, 5.11 The /dev/fd Directory (Linux Programming Interface, Michael Kerrisk)
 "For each process, the kernel provides the special virtual directory /dev/fd.
  This directory contains filenames of the form /dev/fd/n, where n is a number
  corresponding to one of the open file descriptors for the process."
 "... Opening one of the files in the /dev/fd directory is equivalent to duplicating
  the corresponding file descriptor. ..."
 "... The files in the /dev/fd directory are rarely used within programs. Their most
  common use is in the shell. Many user-level commands take filename arguments, and
  sometimes we would like to put them in a pipeline and have one of the arguments be
  standard input or output instead. ..."
 */

void
errExit(const char *str)
{
    printf("id = %s, errno = %d, errstr = %s\n", str, errno, strerror(errno));
    fflush(stdout);
    exit(EXIT_FAILURE);
}

int
main()
{
    int fd1, fd2, fd3;

    errno = 0;
    // create a tmpfile in the same way that bash would do ...
    fd1 = open("/tmp/stc.txt", O_WRONLY|O_CREAT|O_EXCL|O_TRUNC, 0600);
    if (fd1 == -1)
        errExit("openfd1");

    errno = 0;
    if (write(fd1, "Hello, world!\n", 14) == -1) errExit("writefd1");

    errno = 0;
    // also open this tmpfile for reading like bash would do ...
    fd2 = open("/tmp/stc.txt", O_RDONLY);
    if (fd2 == -1)
        errExit("openfd2");

    errno = 0;
    // close fd 1 like bash would do ...
    if (close(fd1) == -1) errExit("closefd1");

    errno = 0;
    // delete the tmpfile like bash would do ...
    if (unlink("/tmp/stc.txt") == -1) errExit("unlink");

    // kludge: compose a string (using fd2) representing "the device file"
    // in /dev/fd (a symlnk to /proc/self/fd) for the file descriptor fd 2
    // that is still open ...
    char devfile[12] = "/dev/fd/"; devfile[8] = fd2 + 0x30; devfile[9] = '\0';

    printf("devfile = %s\n", devfile);

    errno = 0;
    // open this device file; it succeeds on Linux, but fails on Cygwin ...
    fd3 = open(devfile, O_RDONLY);

    const char *id = "openfd3";
    if (fd3 == -1) {
#if 0
        errExit(id);
#else
	printf("%s: Cannot open!, id = %s, errno = %d, errmsg = %s\n", devfile,
            id, errno, strerror(errno));
        char buf[16] = { 0 };
        // however the file to which the symlnk refers, is still present ...
        // Q: does Cygwin attempt to read the /tmp directory? (an attempt that
        //    will fail, because the file has been unlinked)
        // it appears that a readlink of a file in /dev/fd must be diverted to
        // the open file descriptor of the process ...
        errno = 0;
        if (read(fd2, buf, sizeof(buf) ) == -1) errExit("readfd2");
        printf("buf = %s", buf);
        if (close(fd2) == -1) printf("closefd2 failed\n");
        exit(EXIT_FAILURE);
#endif
    }

    char buf[16] = { 0 };
    errno = 0;
    if (read(fd3, buf, sizeof(buf) ) == -1) errExit("readfd3");
    printf("buf = %s", buf);

    if (close(fd3) == -1) printf("closefd3 failed\n");
    if (close(fd2) == -1) printf("closefd2 failed\n");
}

//=====

[-- Attachment #3: Type: text/plain, Size: 219 bytes --]


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: /dev/fd/N not synonymous with file descriptor N; it is on Linux

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1770 bytes --]

On Dec 16 17:31, Houder wrote:
> L.S.,
> 
> /dev/fd/N not synonymous with file descriptor N; it is on Linux

Yes, it is.  Most of the time.  Try this:

$ echo foo | cat /dev/fd/0

The problem is that some of the concepts don't work as desired:

> 64-@@ cat /dev/fd/0 <<\EOF

If you observe what happens in tcsh in this situation you see that it
doesn't even execute cat as long as you didn't type EOF.  What you type
is written to a tmpfile:

$ ls -l /proc/5980/fd
total 0
lrwxrwxrwx 1 corinna vinschen 0 Dec 16 21:15 0 -> /tmp/sh.lVQq04
lrwxrwxrwx 1 corinna vinschen 0 Dec 16 21:15 15 -> /dev/pty0
lrwxrwxrwx 1 corinna vinschen 0 Dec 16 21:15 16 -> /dev/pty0
lrwxrwxrwx 1 corinna vinschen 0 Dec 16 21:15 17 -> /dev/pty0
lrwxrwxrwx 1 corinna vinschen 0 Dec 16 21:15 18 -> /dev/pty0
lrwxrwxrwx 1 corinna vinschen 0 Dec 16 21:15 19 -> /dev/pty0

However, this tmpfile has been unlinked already, so it has been moved to the
recycle bin:

$ ls -l /tmp/sh.lVQq04
ls: /tmp/sh.lVQq04: No such file or directory

So the path in the fd subdir doesn't reflect the actual file path.

But after starting cat, cat tries to open /proc/self/fd/0 which
is in fact the non-existing path /tmp/sh.lVQq04.  Bad luck.

In contrast to Linux the symlinks are not just faked symlinks with the
underlying OS having direct access to the file descriptors.  The way
it's implemented in Cygwin uses the actual file path resolution and then
either works or fails as above.  I'm not sure how to fix that easily.
I guess the fd/0 symlink would have to show the actual file path pointing
to the recycle bin.  But that's often not what you want either since it
hows a patch outside the POSIX namespace.


Corinna

-- 
Corinna Vinschen
Cygwin Maintainer

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: /dev/fd/N not synonymous with file descriptor N; it is on Linux

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1922 bytes --]

On Dec 16 21:28, Corinna Vinschen wrote:
> On Dec 16 17:31, Houder wrote:
> > L.S.,
> > 
> > /dev/fd/N not synonymous with file descriptor N; it is on Linux
> 
> Yes, it is.  Most of the time.  Try this:
> 
> $ echo foo | cat /dev/fd/0
> 
> The problem is that some of the concepts don't work as desired:
> 
> > 64-@@ cat /dev/fd/0 <<\EOF
> 
> If you observe what happens in tcsh in this situation you see that it
> doesn't even execute cat as long as you didn't type EOF.  What you type
> is written to a tmpfile:
> 
> $ ls -l /proc/5980/fd
> total 0
> lrwxrwxrwx 1 corinna vinschen 0 Dec 16 21:15 0 -> /tmp/sh.lVQq04
> lrwxrwxrwx 1 corinna vinschen 0 Dec 16 21:15 15 -> /dev/pty0
> lrwxrwxrwx 1 corinna vinschen 0 Dec 16 21:15 16 -> /dev/pty0
> lrwxrwxrwx 1 corinna vinschen 0 Dec 16 21:15 17 -> /dev/pty0
> lrwxrwxrwx 1 corinna vinschen 0 Dec 16 21:15 18 -> /dev/pty0
> lrwxrwxrwx 1 corinna vinschen 0 Dec 16 21:15 19 -> /dev/pty0
> 
> However, this tmpfile has been unlinked already, so it has been moved to the
> recycle bin:
> 
> $ ls -l /tmp/sh.lVQq04
> ls: /tmp/sh.lVQq04: No such file or directory
> 
> So the path in the fd subdir doesn't reflect the actual file path.
> 
> But after starting cat, cat tries to open /proc/self/fd/0 which
> is in fact the non-existing path /tmp/sh.lVQq04.  Bad luck.
> 
> In contrast to Linux the symlinks are not just faked symlinks with the
> underlying OS having direct access to the file descriptors.  The way
> it's implemented in Cygwin uses the actual file path resolution and then
> either works or fails as above.  I'm not sure how to fix that easily.
> I guess the fd/0 symlink would have to show the actual file path pointing
> to the recycle bin.  But that's often not what you want either since it
> hows a patch outside the POSIX namespace.
  shows a path

Sorry,
Corinna

-- 
Corinna Vinschen
Cygwin Maintainer

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: /dev/fd/N not synonymous with file descriptor N; it is on Linux

[Wayne Davison]

On Sun, Dec 16, 2018 at 12:29 PM Corinna Vinschen wrote:
> In contrast to Linux the symlinks are not just faked symlinks with the underlying OS having direct access to the file descriptors.

Yeah, Linux is more like a fuse where the open filehandles are used
directly on open, and the stat calls return pretend symlinks with the
lsof info.

> The way it's implemented in Cygwin uses the actual file path resolution and then either works or fails as above.

If Cygwin can't be changed into a fuse idiom, perhaps a hard-link
idiom could be used? For instance, when the /proc/$PID/fd/0 symlink is
being created, the code could try to hard-link the file to
/proc/$PID/.fd/0 first (note the dot-fd) before creating the normal
fd/0 symlink. Then, when a request came in to read /proc/$PID/fd/0, it
could check if the associated hard-link exists and read from that
instead.  Or perhaps the "symlink" info could be stored elsewhere and
the fd/0 file could be the hard-link, requiring the stat code to be
tweaked to look up the pretend-symlink info?

..wayne..

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: /dev/fd/N not synonymous with file descriptor N; it is on Linux

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1178 bytes --]

On Dec 16 13:36, Wayne Davison wrote:
> On Sun, Dec 16, 2018 at 12:29 PM Corinna Vinschen wrote:
> > In contrast to Linux the symlinks are not just faked symlinks with the underlying OS having direct access to the file descriptors.
> 
> Yeah, Linux is more like a fuse where the open filehandles are used
> directly on open, and the stat calls return pretend symlinks with the
> lsof info.
> 
> > The way it's implemented in Cygwin uses the actual file path resolution and then either works or fails as above.
> 
> If Cygwin can't be changed into a fuse idiom, perhaps a hard-link
> idiom could be used? For instance, when the /proc/$PID/fd/0 symlink is
> being created, the code could try to hard-link the file to
> /proc/$PID/.fd/0 first

Nice try, but hardlinks don't cross FS borders.  /proc is a filesystem
on its own with its own inode numbers.

I'm mulling over adding some hack to open().  It could try to recognize
the special case of opening a processes' own descriptor symlink within
/proc and then warp the open() call into dup().  No idea how tricky
or even feasible that is, though...


Corinna

-- 
Corinna Vinschen
Cygwin Maintainer

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: /dev/fd/N not synonymous with file descriptor N; it is on Linux

[Houder]

[-- Attachment #1: Type: text/plain, Size: 1603 bytes --]

On 2018-12-16 21:28, Corinna Vinschen wrote:
> On Dec 16 17:31, Houder wrote:
>> L.S.,
>> 
>> /dev/fd/N not synonymous with file descriptor N; it is on Linux
> 
> Yes, it is.  Most of the time.  Try this:
> 
> $ echo foo | cat /dev/fd/0
> 
> The problem is that some of the concepts don't work as desired:
> 
>> 64-@@ cat /dev/fd/0 <<\EOF
> 
> If you observe what happens in tcsh in this situation you see that it
> doesn't even execute cat as long as you didn't type EOF.  What you type
> is written to a tmpfile:
> 
> $ ls -l /proc/5980/fd
> total 0
> lrwxrwxrwx 1 corinna vinschen 0 Dec 16 21:15 0 -> /tmp/sh.lVQq04
> lrwxrwxrwx 1 corinna vinschen 0 Dec 16 21:15 15 -> /dev/pty0
> lrwxrwxrwx 1 corinna vinschen 0 Dec 16 21:15 16 -> /dev/pty0
> lrwxrwxrwx 1 corinna vinschen 0 Dec 16 21:15 17 -> /dev/pty0
> lrwxrwxrwx 1 corinna vinschen 0 Dec 16 21:15 18 -> /dev/pty0
> lrwxrwxrwx 1 corinna vinschen 0 Dec 16 21:15 19 -> /dev/pty0
> 
> However, this tmpfile has been unlinked already, so it has been moved 
> to the
> recycle bin:
> 
> $ ls -l /tmp/sh.lVQq04
> ls: /tmp/sh.lVQq04: No such file or directory
> 
> So the path in the fd subdir doesn't reflect the actual file path.
> 
> But after starting cat, cat tries to open /proc/self/fd/0 which
> is in fact the non-existing path /tmp/sh.lVQq04.  Bad luck.

Yes Corinna, I am aware of the above. I described it here:

     https://cygwin.com/ml/cygwin/2018-12/msg00040.html
     ( Re: Bash heredoc on FD 3 )

Sorry for NOT expressing myself more clearly.

The STC is, as I wrote, based on what I observed on Linux.

See attachment.

Regards,
Henri

[-- Attachment #2: disassembly.txt --]
[-- Type: text/plain, Size: 3201 bytes --]

...

 - recurring pattern: the file descriptor to the tmpfile is dup'ed to fd N, where N is equal
   to the one in /dev/fd/N, the device file specified to be opened by the child.

@@ strace -ff -e open,close,unlink,read,write,execve,dup2 -- bash ./here_doc_disx.sh                                                        
# here_doc_disx.sh:
# cat /dev/fd/0 <<\EOF
# Hello, world!
# EOF

execve("/usr/bin/bash", ["bash", "./here_doc_disx.sh"], 0x7ffcaa1b6388 /* 63 vars */) = 0
open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
close(3)                                = 0
open("/lib64/libtinfo.so.6", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\300\331\0\0\0\0\0\0"..., 832) = 832
close(3)                                = 0
open("/lib64/libdl.so.2", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\240\r\0\0\0\0\0\0"..., 832) = 832
close(3)                                = 0
open("/lib64/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0P\10\2\0\0\0\0\0"..., 832) = 832
close(3)                                = 0
open("/dev/tty", O_RDWR|O_NONBLOCK)     = 3
close(3)                                = 0
open("/usr/lib/locale/locale-archive", O_RDONLY|O_CLOEXEC) = 3
close(3)                                = 0
open("/usr/lib64/gconv/gconv-modules.cache", O_RDONLY) = 3
close(3)                                = 0
open("./here_doc_disx.sh", O_RDONLY)    = 3
read(3, "cat /dev/fd/0 <<\\EOF\nHello, worl"..., 80) = 39
dup2(3, 255)                            = 255
close(3)                                = 0
read(255, "cat /dev/fd/0 <<\\EOF\nHello, worl"..., 39) = 39
strace: Process 1681 attached
[pid  1681] close(255)                  = 0
[pid  1681] open("/tmp/sh-thd-932991243", O_WRONLY|O_CREAT|O_EXCL|O_TRUNC, 0600) = 3
[pid  1681] write(3, "Hello, world!\n", 14) = 14
[pid  1681] open("/tmp/sh-thd-932991243", O_RDONLY) = 4
[pid  1681] close(3)                    = 0
[pid  1681] unlink("/tmp/sh-thd-932991243") = 0
[pid  1681] dup2(4, 0)                  = 0
[pid  1681] close(4)                    = 0
[pid  1681] execve("/usr/bin/cat", ["cat", "/dev/fd/0"], 0x75ebd70e90 /* 62 vars */) = 0
[pid  1681] open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
[pid  1681] close(3)                    = 0
[pid  1681] open("/lib64/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
[pid  1681] read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0P\10\2\0\0\0\0\0"..., 832) = 832
[pid  1681] close(3)                    = 0
[pid  1681] open("/usr/lib/locale/locale-archive", O_RDONLY|O_CLOEXEC) = 3
[pid  1681] close(3)                    = 0
[pid  1681] open("/dev/fd/0", O_RDONLY) = 3
[pid  1681] read(3, "Hello, world!\n", 131072) = 14
[pid  1681] write(1, "Hello, world!\n", 14) = 14
Hello, world!
[pid  1681] read(3, "", 131072)         = 0
[pid  1681] close(3)                    = 0
[pid  1681] close(1)                    = 0
[pid  1681] close(2)                    = 0
[pid  1681] +++ exited with 0 +++
--- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=1681, si_uid=1000, si_status=0, si_utime=0, si_stime=0} ---
read(255, "", 39)                       = 0
+++ exited with 0 +++
@@ 

#=====

[-- Attachment #3: Type: text/plain, Size: 219 bytes --]


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: /dev/fd/N not synonymous with file descriptor N; it is on Linux

[Houder]

On 2018-12-16 22:55, Corinna Vinschen wrote:
[snip]

> I'm mulling over adding some hack to open().  It could try to recognize
> the special case of opening a processes' own descriptor symlink within
> /proc and then warp the open() call into dup().  No idea how tricky
> or even feasible that is, though...

That is why I wrote the following in my STC:

         // Q: does Cygwin attempt to read the /tmp directory? (an 
attempt that
         //    will fail, because the file has been unlinked)
         // it appears that reading a symlnk in /dev/fd can best be 
diverted to
         // the open file descriptor of the process ...

What I meant was, that I see no reason to modify the symlink in this
special case, but in stead of that to access the file using fd N, where
N is equal to the one in /dev/fd/N.

File descriptor N has been left open by bash and should not have been
closed as result of the exec ...

And indeed, I have _no_ clue if the above is feasible (and tricky?) in
Cygwin; otherwise I would have posted a solution.

Regards,
Henri

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: /dev/fd/N not synonymous with file descriptor N; it is on Linux

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 3600 bytes --]

On Dec 17 05:30, Houder wrote:
> On 2018-12-16 22:55, Corinna Vinschen wrote:
> [snip]
> 
> > I'm mulling over adding some hack to open().  It could try to recognize
> > the special case of opening a processes' own descriptor symlink within
> > /proc and then warp the open() call into dup().  No idea how tricky
> > or even feasible that is, though...
> 
> That is why I wrote the following in my STC:
> 
>         // Q: does Cygwin attempt to read the /tmp directory? (an attempt
> that
>         //    will fail, because the file has been unlinked)
>         // it appears that reading a symlnk in /dev/fd can best be diverted
> to
>         // the open file descriptor of the process ...
> 
> What I meant was, that I see no reason to modify the symlink in this
> special case, but in stead of that to access the file using fd N, where
> N is equal to the one in /dev/fd/N.
> 
> File descriptor N has been left open by bash and should not have been
> closed as result of the exec ...

tl;dr:

  cat tries to open /dev/fd/0.  /dev/fd/0 refers to a non-existing
  file and that's why open fails.  Cygwin doesn't have special code to
  handle the fd symlinks as refference to an actually open descriptors.

The long story:

  The descriptor hasn't been closed.  Cat still has the file open as fd 0.

  The problem is that you tell cat to open and read from /dev/fd/0,
  *not* from fd 0.  That's quite a difference.  /dev/fd/0 is just some
  arbitrary file which is given to open(2), a symlink at that.  First
  thing open(2) does is to call the symlink evaluation code.  The
  symlink is ultimately evaluated to the filename opened by the shell.

  So, cat's open(2) tries to open "/tmp/whatever".  But, as you noticed,
  that path doesn't exist anymore since it has been deleted by the shell
  before even writing to it via unlink(2).

  What happens is that unlink(2) is supposed to delete a file in use.
  Since that's not possible in Windows, the file gets renamed into the
  recycle bin and just marked for deletion.

  The fact that the filename changed is not known to the application of
  course, it just gets success reported from unlink.  Cygwin itself
  doesn't know the new name of the file either.  There just isn't a
  reason to keep track since it's going to be removed anyway at one
  point.

  And even *if* we keep track, we can't use this information since files
  marked for deletion can't be opened on Windows (ERROR_DELETE_PENDING
  or ERROR_ACCESS_DENIED, I'm not sure which).

  Here's another problem with handling /dev/fd/0 as just some descriptor
  we can call dup(2) on:

  /dev/fd/0 is just some symlink which evaluates like this:

  - /dev/fd is a symlink to /proc/self/fd
  - /proc/self is a symlink to /proc/<current_process_pid>

  So before we even get to evaluate fd 0 symlink, we're at

    /proc/<current_process_pid>/fd/0

  Here's the question:  What if you don't give /proc/self/fd/0 to
  cat, but something like /proc/<some_other_processes_pid>/fd/0?

  That's where Cygwin just fails because /proc is a process-local fake
  in our user space Cygwin DLL.  Emulating /proc is fun stuff, the full
  functionality is pretty tricky without going to great lengths like
  starting a service with SYSTEM permissions.

  Bottom line is, we could do more if we decide that running cygserver
  is a requirement.  As it is, we always strived for a setup which
  works "out of the box", without having to start services and stuff.


Corinna

-- 
Corinna Vinschen
Cygwin Maintainer

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: /dev/fd/N not synonymous with file descriptor N; it is on Linux

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 2372 bytes --]

On Dec 17 10:25, Corinna Vinschen wrote:
> On Dec 17 05:30, Houder wrote:
> > On 2018-12-16 22:55, Corinna Vinschen wrote:
> > [snip]
> > 
> > > I'm mulling over adding some hack to open().  It could try to recognize
> > > the special case of opening a processes' own descriptor symlink within
> > > /proc and then warp the open() call into dup().  No idea how tricky
> > > or even feasible that is, though...
> > 
> > That is why I wrote the following in my STC:
> > 
> >         // Q: does Cygwin attempt to read the /tmp directory? (an attempt
> > that
> >         //    will fail, because the file has been unlinked)
> >         // it appears that reading a symlnk in /dev/fd can best be diverted
> > to
> >         // the open file descriptor of the process ...
> > 
> > What I meant was, that I see no reason to modify the symlink in this
> > special case, but in stead of that to access the file using fd N, where
> > N is equal to the one in /dev/fd/N.
> > 
> > File descriptor N has been left open by bash and should not have been
> > closed as result of the exec ...
> 
> tl;dr:
> 
>   cat tries to open /dev/fd/0.  /dev/fd/0 refers to a non-existing
>   file and that's why open fails.  Cygwin doesn't have special code to
>   handle the fd symlinks as refference to an actually open descriptors.
> 
> The long story:
> 
>   The descriptor hasn't been closed.  Cat still has the file open as fd 0.
> 
>   The problem is that you tell cat to open and read from /dev/fd/0,
>   *not* from fd 0.  That's quite a difference.  /dev/fd/0 is just some
>   arbitrary file which is given to open(2), a symlink at that.  First
>   thing open(2) does is to call the symlink evaluation code.  The
>   symlink is ultimately evaluated to the filename opened by the shell.

I just tested this on Linux.  Basically, open(2) in Cygwin is equivalent
to calling readlink(2); open(2):

  readlink(incoming_filename, resolved_filename, ...);
  open (resolved_filename, ...);

If I do this on Linux with a cat-clone calling readlink before calling
open, the error returned by open(2) is the same as on Cygwin, ENOENT.

The tricky part is to find out when resolving the filename is not
required because we already have a descriptor / HANDLE and how to
proceed from there...


Corinna

-- 
Corinna Vinschen
Cygwin Maintainer

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: /dev/fd/N not synonymous with file descriptor N; it is on Linux

[Houder]

On 2018-12-17 12:26, Corinna Vinschen wrote:
> On Dec 17 10:25, Corinna Vinschen wrote:
>> On Dec 17 05:30, Houder wrote:
>> > On 2018-12-16 22:55, Corinna Vinschen wrote:
>> > [snip]
>> >
>> > > I'm mulling over adding some hack to open().  It could try to recognize
>> > > the special case of opening a processes' own descriptor symlink within
>> > > /proc and then warp the open() call into dup().  No idea how tricky
>> > > or even feasible that is, though...
>> >
>> > That is why I wrote the following in my STC:
>> >
>> >         // Q: does Cygwin attempt to read the /tmp directory? (an attempt
>> > that
>> >         //    will fail, because the file has been unlinked)
>> >         // it appears that reading a symlnk in /dev/fd can best be diverted
>> > to
>> >         // the open file descriptor of the process ...
>> >
>> > What I meant was, that I see no reason to modify the symlink in this
>> > special case, but in stead of that to access the file using fd N, where
>> > N is equal to the one in /dev/fd/N.
>> >
>> > File descriptor N has been left open by bash and should not have been
>> > closed as result of the exec ...
[snip]

> The tricky part is to find out when resolving the filename is not
> required because we already have a descriptor / HANDLE and how to
> proceed from there...

Ref: https://cygwin.com/ml/cygwin/2018-12/msg00125.html
( /dev/fd/N not synonymous with file descriptor N; it is on Linux )

For the record only:

You were/are right and I was complete wrong: open file descriptor N is
only by chance equal to /dev/fd/N ...

Neither the semantics of FreeBSD (and Solaris) -- similar to dup() --
nor those of Linux (where the opening of the deleted file results in a
new entry in the open file table) are feasible in Cygwin.

FreeBSD (Solaris) and Linux require the concept of a "directory cache"
to exist in order to make the above possible (that cache must also be
exposed to the user in some way).

FreeBSD (Solaris): based on the filename, the corresponding open file
descriptION is found; a new file descriptor to this entry in the open
file table is returned to the user ...

It appears as if the inherited open file descriptor is dup'ed, but it
is not.

Linux: based on the filename, the corresponding inode is found; a new
entry in the open file table is created; a file descriptor to the new
entry in the open file table is returned to the user ...
(this can be verified using kcmp(2) ).

The above is not possible in Cygwin. Cygwin cannot follow here.

Regards,
Henri

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: /dev/fd/N not synonymous with file descriptor N; it is on Linux

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 662 bytes --]

On Dec 16 17:31, Houder wrote:
> L.S.,
> 
> /dev/fd/N not synonymous with file descriptor N; it is on Linux
> 
> 64-@@ cat /dev/fd/0 <<\EOF
> > Hi
> > EOF
> cat: /dev/fd/0: No such file or directory
> 
> fails on Cygwin; not on Linux.
> 
> Also see:
> 
>     https://cygwin.com/ml/cygwin/2018-12/msg00028.html
>     ( Bash heredoc on FD 3 )
> 
> Based on the output of strace on Linux, I composed an STC, that duplicates
> the steps taken by bash (and cat).

This should work in the latest developer snapshot uploaded to
https://cygwin.com/snapshots/  Please give it a try.


Thanks,
Corinna

-- 
Corinna Vinschen
Cygwin Maintainer

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: /dev/fd/N not synonymous with file descriptor N; it is on Linux

[Houder]

On Sun, 6 Jan 2019 21:19:50, Corinna Vinschen  wrote:
> 
> On Dec 16 17:31, Houder wrote:
> > L.S.,
> >
> > /dev/fd/N not synonymous with file descriptor N; it is on Linux
> >
> > 64-@@ cat /dev/fd/0 <<\EOF
> > > Hi
> > > EOF
> > cat: /dev/fd/0: No such file or directory
> >
> > fails on Cygwin; not on Linux.
> >
> > Also see:
> >
> >     https://cygwin.com/ml/cygwin/2018-12/msg00028.html
> >     ( Bash heredoc on FD 3 )
> >
> > Based on the output of strace on Linux, I composed an STC, that duplicates
> > the steps taken by bash (and cat).
> 
> This should work in the latest developer snapshot uploaded to
> https://cygwin.com/snapshots/  Please give it a try.
> 
> Thanks,
> Corinna

Nice!

This solves: <program> -i /dev/fd/N N<<EOF (i.e. Steven Penny's problem)

Howver ...
(and I sure that I am not telling you anything new)

it still not the same as Linux ...

So, for the record only:

 - suppose a hacker deletes the logfile, but the file is still "help open"
   by some program
 - the logfile can be rescued on Linux, but not on Cygwin

64-@@ echo aap > aap.txt
64-@@ tail -f aap.txt
aap

.. in another terminal
64-@@ ls -l /proc/4120/fd # 4120 is the procid of "tail"
total 0
lrwxrwxrwx 1 Henri None 0 Jan 22 09:25 0 -> /dev/pty1
lrwxrwxrwx 1 Henri None 0 Jan 22 09:25 1 -> /dev/pty1
lrwxrwxrwx 1 Henri None 0 Jan 22 09:25 2 -> /dev/pty1
lrwxrwxrwx 1 Henri None 0 Jan 22 09:25 3 -> /home/Henri/redirect/aap.txt

64-@@ stat -L /proc/4120/fd/3 # shows a file of 4 bytes ...
  File: /proc/4120/fd/3
  Size: 4               Blocks: 1          IO Block: 65536  regular file
Device: 33d91880h/869865600d    Inode: 35747322042382612  Links: 1
Access: (0644/-rw-r--r--)  Uid: ( 1000/   Henri)   Gid: (  513/    None)
Access: 2019-01-22 09:24:41.978178200 +0100
Modify: 2019-01-22 09:24:41.978178200 +0100
Change: 2019-01-22 09:24:41.978178200 +0100
 Birth: 2019-01-22 09:24:41.978178200 +0100

64-@@ rm aap.txt # hacker removing the logfile

64-@@ stat -L /proc/4120/fd/3
  File: /proc/4120/fd/3
  Size: 4               Blocks: 1          IO Block: 65536  regular file
Device: 33d91880h/869865600d    Inode: 35747322042382612  Links: 0
Access: (0644/-rw-r--r--)  Uid: ( 1000/   Henri)   Gid: (  513/    None)
Access: 2019-01-22 09:24:41.978178200 +0100
Modify: 2019-01-22 09:24:41.978178200 +0100
Change: 2019-01-22 09:26:25.021040800 +0100
 Birth: 2019-01-22 09:24:41.978178200 +0100

64-@@ cp /proc/4120/fd/3 noot.txt # logfile rescue
64-@@ cat noot.txt # empty! (however, this works on Linux)
64-@@ cat /proc/4120/fd/3 # indeed, does not show contents
64-@@

=====


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: /dev/fd/N not synonymous with file descriptor N; it is on Linux

[Houder]

[-- Attachment #1: Type: text/plain, Size: 1212 bytes --]

On 2019-01-22 09:50, Houder wrote:
> On Sun, 6 Jan 2019 21:19:50, Corinna Vinschen  wrote:
>> 
>> On Dec 16 17:31, Houder wrote:
>> > L.S.,
>> >
>> > /dev/fd/N not synonymous with file descriptor N; it is on Linux
>> >
>> > 64-@@ cat /dev/fd/0 <<\EOF
>> > > Hi
>> > > EOF
>> > cat: /dev/fd/0: No such file or directory
>> >
>> > fails on Cygwin; not on Linux.
>> >
>> > Also see:
>> >
>> >     https://cygwin.com/ml/cygwin/2018-12/msg00028.html
>> >     ( Bash heredoc on FD 3 )
>> >
>> > Based on the output of strace on Linux, I composed an STC, that duplicates
>> > the steps taken by bash (and cat).
>> 
>> This should work in the latest developer snapshot uploaded to
>> https://cygwin.com/snapshots/  Please give it a try.
>> 
>> Thanks,
>> Corinna
> 
> Nice!
> 
> This solves: <program> -i /dev/fd/N N<<EOF (i.e. Steven Penny's 
> problem)
> 
> Howver ...
> (and I sure that I am not telling you anything new)
> 
> it still not the same as Linux ...
> 
> So, for the record only:

and as another example, this STC succeeds on Linux ..., but fails on 
Cygwin.

64-@@ ./stca /dev/fd/0 <<EOF
> bla
> EOF
fd1 = 0
argv[1] = /dev/fd/0
fd2 = 3
id = writefd2, errno = 13, errstr = Permission denied
64-@@

=====


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: stca.c --]
[-- Type: text/x-c; name=stca.c, Size: 6104 bytes --]

// gcc -Wall -o stca stca.c

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h> // strerror()
#include <errno.h>
#include <fcntl.h>
#include <sys/stat.h> // lstat()
#include <time.h>

/*
 A file that has been deleted, but is still "held open" by an entry in the open file table
 (i.e. the file is referred by an open file descriptor), can be opened again read-write in
 Linux, but not in Cygwin ...
 */

// NOTE: my text uses "fd 2" for a file descriptor that equals 2; and fd2 for variable fd2

static int
errExit(const char *str)
{
    printf("id = %s, errno = %d, errstr = %s\n", str, errno, strerror(errno));
    fflush(stdout);
    exit(EXIT_FAILURE);
}

static void
Usage(const char *const arg)
{
    fprintf(stderr, "Usage: %s /dev/fd/N\n", arg);
    exit(EXIT_FAILURE);
}

#if 0
#define DISPLAYSTATINFO
static void displayStatInfo(const struct stat *sb);
#endif

// invoke as follows: ./stca /dev/fd/0 <<EOF

int
main(int argc, char *argv[])
{
    if (argc < 2) {
        Usage(argv[0]);
        exit(EXIT_SUCCESS);
    }

    // @@ ./stca /dev/fd/8 8<<EOF 0<&- # fd 8 is connected to tmpfile (bash!)
    // using an argument, the stc can be told to read from fd 8 (fd 0 can be closed)
    // @@ ./stca /dev/fd/5 8<<EOF 5<&8- 0<&- # ... here fd 0 and fd 8 are closed

    // kludge
    if (strlen(argv[1]) < 9) Usage(argv[0]);
    if (strncmp(argv[1], "/dev/fd/", 8) != 0) Usage(argv[0]);
    if ( (strlen(argv[1]) > 9) ||
         ( (argv[1][8] - 0x30 < 0) || (argv[1][8] - 0x30 > 9) ) ) {
        fprintf(stderr, "N must be a single digit, please!\n");
        Usage(argv[0]);
    }
    // end kludge

    int fd1;
    fd1 = argv[1][8] - 0x30;
    printf("fd1 = %d\n", fd1);

    // using bash, /dev/fd/N refers to a file which been unlink'ed
    printf("argv[1] = %s\n", argv[1]);

    errno = 0;

    int fd2;
#if 0
    if ( (fd2 = open(argv[1], O_RDONLY)) == -1) errExit("openfd2"); // what 'cat' does
#else
#define WRITE
    if ( (fd2 = open(argv[1], O_RDWR)) == -1) errExit("openfd2");
    //if ( (fd2 = open(argv[1], O_RDWR | O_APPEND)) == -1) errExit("openfd2");
#endif
    printf("fd2 = %d\n", fd2);

#if defined(DISPLAYSTATINFO)
    struct stat sb;
    if (fstat(fd2, &sb) == -1) errExit("fstatfd2");
    displayStatInfo(&sb);
#endif

    /* if file has been opened read-write, write() will succeed in Linux,
       but fail in Cygwin
     */

#if defined(WRITE)
    if (write(fd2, "Hello, world!\n", 14) == -1) errExit("writefd2");
#endif

    // how about an lseek() first? (write() modifies the file offset)
    if (lseek(fd2, 0, SEEK_SET) == -1) errExit("lseekfd2");

    char buf[80] = { 0 }; // another kludge
    if (read(fd2, buf, sizeof(buf) ) == -1) errExit("readfd2");
    printf("buf = \\\n%s", buf);

    if (close(fd2) == -1) printf("closefd2 failed\n");
}

#if defined(DISPLAYSTATINFO)
  #if 1
static void
displayStatInfo(const struct stat *sb)
{
   printf("Link count: %ld\n", (long)sb->st_nlink);
}

  #else
static char *filePermStr(mode_t perm, int flags);

static void
displayStatInfo(const struct stat *sb)
{
   printf("File type:                ");

   // deprecated by S_ISREG(m) and c.s. ...? /usr/include/linux/stat.h
   switch (sb->st_mode & S_IFMT) {
   case S_IFBLK:  printf("block device\n");            break;
   case S_IFCHR:  printf("character device\n");        break;
   case S_IFDIR:  printf("directory\n");               break;
   case S_IFIFO:  printf("FIFO/pipe\n");               break;
   case S_IFLNK:  printf("symlink\n");                 break;
   case S_IFREG:  printf("regular file\n");            break;
   case S_IFSOCK: printf("socket\n");                  break;
   default:       printf("unknown?\n");                break;
   }

   printf("I-node number:            %ld\n", (long)sb->st_ino);

#if 0
   printf("Mode:                     %lo (octal)\n", (unsigned long)sb->st_mode);
#else
   printf("Mode:                     %lo (octal), ls(1)-style: %s\n",
                                     (unsigned long)sb->st_mode, filePermStr(sb->st_mode, 0));
#endif

   printf("Link count:               %ld\n", (long)sb->st_nlink);
   printf("Ownership:                UID=%ld   GID=%ld\n", (long)sb->st_uid, (long)sb->st_gid);

   printf("Preferred I/O block size: %ld bytes\n", (long)sb->st_blksize);
   printf("File size:                %lld bytes\n", (long long)sb->st_size);
   printf("Blocks allocated:         %lld\n", (long long)sb->st_blocks);

   printf("Last status change:       %s", ctime(&sb->st_ctime));
   printf("Last file access:         %s", ctime(&sb->st_atime));
   printf("Last file modification:   %s", ctime(&sb->st_mtime));
}

/* Return ls(1)-style string for file permissions mask */

/* Include set-user-ID, set-group-ID, and sticky bit information in
   returned string */
#define FP_SPECIAL 1

#define STR_SIZE sizeof("rwxrwxrwx")

static char *
filePermStr(mode_t perm, int flags)
{
    static char str[STR_SIZE];

    /* If FP_SPECIAL was specified, we emulate the trickery of ls(1) in
       returning set-user-ID, set-group-ID, and sticky bit information in
       the user/group/other execute fields. This is made more complex by
       the fact that the case of the character displayed for this bits
       depends on whether the corresponding execute bit is on or off. */

    snprintf(str, STR_SIZE, "%c%c%c%c%c%c%c%c%c",
        (perm & S_IRUSR) ? 'r' : '-', (perm & S_IWUSR) ? 'w' : '-',
        (perm & S_IXUSR) ?
            (((perm & S_ISUID) && (flags & FP_SPECIAL)) ? 's' : 'x') :
            (((perm & S_ISUID) && (flags & FP_SPECIAL)) ? 'S' : '-'),
        (perm & S_IRGRP) ? 'r' : '-', (perm & S_IWGRP) ? 'w' : '-',
        (perm & S_IXGRP) ?
            (((perm & S_ISGID) && (flags & FP_SPECIAL)) ? 's' : 'x') :
            (((perm & S_ISGID) && (flags & FP_SPECIAL)) ? 'S' : '-'),
        (perm & S_IROTH) ? 'r' : '-', (perm & S_IWOTH) ? 'w' : '-',
        (perm & S_IXOTH) ?
            (((perm & S_ISVTX) && (flags & FP_SPECIAL)) ? 't' : 'x') :
            (((perm & S_ISVTX) && (flags & FP_SPECIAL)) ? 'T' : '-'));

    return str;
}
  #endif
#endif

//=====

[-- Attachment #3: Type: text/plain, Size: 219 bytes --]


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: /dev/fd/N not synonymous with file descriptor N; it is on Linux

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1687 bytes --]

On Jan 22 09:50, Houder wrote:
> On Sun, 6 Jan 2019 21:19:50, Corinna Vinschen  wrote:
> > > Also see:
> > >
> > >     https://cygwin.com/ml/cygwin/2018-12/msg00028.html
> > >     ( Bash heredoc on FD 3 )
> > >
> > > Based on the output of strace on Linux, I composed an STC, that duplicates
> > > the steps taken by bash (and cat).
> > 
> > This should work in the latest developer snapshot uploaded to
> > https://cygwin.com/snapshots/  Please give it a try.
> 
> This solves: <program> -i /dev/fd/N N<<EOF (i.e. Steven Penny's problem)
> 
> Howver ...
> (and I sure that I am not telling you anything new)
> [...]
> 64-@@ rm aap.txt # hacker removing the logfile
> 
> 64-@@ stat -L /proc/4120/fd/3
>   File: /proc/4120/fd/3
>   Size: 4               Blocks: 1          IO Block: 65536  regular file
> Device: 33d91880h/869865600d    Inode: 35747322042382612  Links: 0
> Access: (0644/-rw-r--r--)  Uid: ( 1000/   Henri)   Gid: (  513/    None)
> Access: 2019-01-22 09:24:41.978178200 +0100
> Modify: 2019-01-22 09:24:41.978178200 +0100
> Change: 2019-01-22 09:26:25.021040800 +0100
>  Birth: 2019-01-22 09:24:41.978178200 +0100
> 
> 64-@@ cp /proc/4120/fd/3 noot.txt # logfile rescue
> 64-@@ cat noot.txt # empty! (however, this works on Linux)
> 64-@@ cat /proc/4120/fd/3 # indeed, does not show contents
> 64-@@

Yes, this does not work under older systems.  However, it works as
desired at least with Windows 10 1709 due to changes in the OS in
terms of handling of deleted files.

I tested this with W8.1, W10 1709 and W10 1809.  It fails on 8.1
but works fine on both W10 versions.


Corinna

-- 
Corinna Vinschen
Cygwin Maintainer

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: /dev/fd/N not synonymous with file descriptor N; it is on Linux

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 803 bytes --]

On Jan 22 09:57, Houder wrote:
> On 2019-01-22 09:50, Houder wrote:
> > On Sun, 6 Jan 2019 21:19:50, Corinna Vinschen  wrote:
> > > This should work in the latest developer snapshot uploaded to
> > > https://cygwin.com/snapshots/  Please give it a try.
> > So, for the record only:
> 
> and as another example, this STC succeeds on Linux ..., but fails on Cygwin.
> 
> 64-@@ ./stca /dev/fd/0 <<EOF
> > bla
> > EOF
> fd1 = 0
> argv[1] = /dev/fd/0
> fd2 = 3
> id = writefd2, errno = 13, errstr = Permission denied
> 64-@@

Not sure what you're testing.  This is the result for me on both,
Windows 8.1 and Windows 10 1809:

)$ ./stca /dev/fd/0 <<EOF
? bla
? EOF
fd1 = 0
argv[1] = /dev/fd/0
fd2 = 3
buf = \
Hello, world!


Corinna

-- 
Corinna Vinschen
Cygwin Maintainer

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: /dev/fd/N not synonymous with file descriptor N; it is on Linux

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1834 bytes --]

On Jan 22 10:02, Corinna Vinschen wrote:
> On Jan 22 09:50, Houder wrote:
> > On Sun, 6 Jan 2019 21:19:50, Corinna Vinschen  wrote:
> > > > Also see:
> > > >
> > > >     https://cygwin.com/ml/cygwin/2018-12/msg00028.html
> > > >     ( Bash heredoc on FD 3 )
> > > >
> > > > Based on the output of strace on Linux, I composed an STC, that duplicates
> > > > the steps taken by bash (and cat).
> > > 
> > > This should work in the latest developer snapshot uploaded to
> > > https://cygwin.com/snapshots/  Please give it a try.
> > 
> > This solves: <program> -i /dev/fd/N N<<EOF (i.e. Steven Penny's problem)
> > 
> > Howver ...
> > (and I sure that I am not telling you anything new)
> > [...]
> > 64-@@ rm aap.txt # hacker removing the logfile
> > 
> > 64-@@ stat -L /proc/4120/fd/3
> >   File: /proc/4120/fd/3
> >   Size: 4               Blocks: 1          IO Block: 65536  regular file
> > Device: 33d91880h/869865600d    Inode: 35747322042382612  Links: 0
> > Access: (0644/-rw-r--r--)  Uid: ( 1000/   Henri)   Gid: (  513/    None)
> > Access: 2019-01-22 09:24:41.978178200 +0100
> > Modify: 2019-01-22 09:24:41.978178200 +0100
> > Change: 2019-01-22 09:26:25.021040800 +0100
> >  Birth: 2019-01-22 09:24:41.978178200 +0100
> > 
> > 64-@@ cp /proc/4120/fd/3 noot.txt # logfile rescue
> > 64-@@ cat noot.txt # empty! (however, this works on Linux)
> > 64-@@ cat /proc/4120/fd/3 # indeed, does not show contents
> > 64-@@
> 
> Yes, this does not work under older systems.  However, it works as
> desired at least with Windows 10 1709 due to changes in the OS in
> terms of handling of deleted files.
> 
> I tested this with W8.1, W10 1709 and W10 1809.  It fails on 8.1

... as expected ...

> but works fine on both W10 versions.

Corinna

-- 
Corinna Vinschen
Cygwin Maintainer

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: /dev/fd/N not synonymous with file descriptor N; it is on Linux

[Houder]

On 2019-01-22 10:06, Corinna Vinschen wrote:
> On Jan 22 09:57, Houder wrote:
>> On 2019-01-22 09:50, Houder wrote:
>> > On Sun, 6 Jan 2019 21:19:50, Corinna Vinschen  wrote:
>> > > This should work in the latest developer snapshot uploaded to
>> > > https://cygwin.com/snapshots/  Please give it a try.
>> > So, for the record only:
>> 
>> and as another example, this STC succeeds on Linux ..., but fails on 
>> Cygwin.
>> 
>> 64-@@ ./stca /dev/fd/0 <<EOF
>> > bla
>> > EOF
>> fd1 = 0
>> argv[1] = /dev/fd/0
>> fd2 = 3
>> id = writefd2, errno = 13, errstr = Permission denied
>> 64-@@
> 
> Not sure what you're testing.  This is the result for me on both,
> Windows 8.1 and Windows 10 1809:

Curious! It fails (for me) on W7 ...

> Not sure what you're testing.

STC inherits a "read-only" open file descriptor from bash. On Linux
the file can be opened read-write (via procfs), because a new entry
is created in the open file table.

(opening the file read-write (via fdescfs) on FreeBSD would fail)

For this reason the output does not show what has been entered via
the here-doc.

In short, I was merely testing the semantics of Linux.

> )$ ./stca /dev/fd/0 <<EOF
> ? bla
> ? EOF
> fd1 = 0
> argv[1] = /dev/fd/0
> fd2 = 3
> buf = \
> Hello, world!
> 
> 
> Corinna

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: /dev/fd/N not synonymous with file descriptor N; it is on Linux

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 2452 bytes --]

On Jan 22 10:25, Houder wrote:
> On 2019-01-22 10:06, Corinna Vinschen wrote:
> > On Jan 22 09:57, Houder wrote:
> > > On 2019-01-22 09:50, Houder wrote:
> > > > On Sun, 6 Jan 2019 21:19:50, Corinna Vinschen  wrote:
> > > > > This should work in the latest developer snapshot uploaded to
> > > > > https://cygwin.com/snapshots/  Please give it a try.
> > > > So, for the record only:
> > > 
> > > and as another example, this STC succeeds on Linux ..., but fails on
> > > Cygwin.
> > > 
> > > 64-@@ ./stca /dev/fd/0 <<EOF
> > > > bla
> > > > EOF
> > > fd1 = 0
> > > argv[1] = /dev/fd/0
> > > fd2 = 3
> > > id = writefd2, errno = 13, errstr = Permission denied
> > > 64-@@
> > 
> > Not sure what you're testing.  This is the result for me on both,
> > Windows 8.1 and Windows 10 1809:
> 
> Curious! It fails (for me) on W7 ...

It works for me just as well on W7:

$ uname -a
CYGWIN_NT-6.1 vmbert764 2.12.0(0.333/5/3) 2019-01-21 22:47 x86_64 Cygwin
$ ./stca /dev/fd/0 <<EOF
? bla
? EOF
fd1 = 0
argv[1] = /dev/fd/0
fd2 = 3
buf = \
Hello, world!

> > Not sure what you're testing.
> 
> STC inherits a "read-only" open file descriptor from bash. On Linux
> the file can be opened read-write (via procfs), because a new entry
> is created in the open file table.
> 
> (opening the file read-write (via fdescfs) on FreeBSD would fail)
> 
> For this reason the output does not show what has been entered via
> the here-doc.
> 
> In short, I was merely testing the semantics of Linux.

Ah, ok.  This is a bit of a problem on Windows.  The code tries to
reopen the file by handle.  Under some circumstances(*) we can't reopen
the file.  In that case the code just tries to duplicate the handle.
However, a duplicated file handle can't have more permissions than the
original handle.

So if it fails for you, it seems the reopen failed and the handle
only got duplicated.  In that case, you can't gain write perms if
the original handle only got read perms.

What the code fails to do is trying to open the file by name as a last
resort.  There was a (good?) reason I didn't implement that, but I don't
remember ATM.


Corinna

(*) E.g., deleted files on systems older than Windows 10 1709,
    or files on filesystems not supporting the "reopen-by-handle"
    semantics.  However, the latter case should work at least
    for non-deleted files

-- 
Corinna Vinschen
Cygwin Maintainer

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: /dev/fd/N not synonymous with file descriptor N; it is on Linux

[Houder]

On Tue, 22 Jan 2019 10:41:57, Corinna Vinschen  wrote:
> On Jan 22 10:25, Houder wrote:
[snip]

> > Curious! It fails (for me) on W7 ...
> 
> It works for me just as well on W7:
> 
> $ uname -a
> CYGWIN_NT-6.1 vmbert764 2.12.0(0.333/5/3) 2019-01-21 22:47 x86_64 Cygwin
> $ ./stca /dev/fd/0 <<EOF
> ? bla
> ? EOF
> fd1 =3D 0
> argv[1] =3D /dev/fd/0
> fd2 =3D 3
> buf =3D \
> Hello, world!

That is odd ... (I am using the same version of Cygwin as you do, do I
not?)

64-@@ uname -a
CYGWIN_NT-6.1 Seven 2.12.0s(0.333/5/3) 2019-01-21 10:25 x86_64 Cygwin
(however, I only replaced the cygwin1.dll)

(the snapshot is apparently not the same as your version 2.12.0)

> > > Not sure what you're testing.
> >
> > STC inherits a "read-only" open file descriptor from bash. On Linux
> > the file can be opened read-write (via procfs), because a new entry
> > is created in the open file table.
> >
> > (opening the file read-write (via fdescfs) on FreeBSD would fail)
> >
> > For this reason the output does not show what has been entered via
> > the here-doc.
> >
> > In short, I was merely testing the semantics of Linux.
> 
> Ah, ok.  This is a bit of a problem on Windows.  The code tries to
> reopen the file by handle.  Under some circumstances(*) we can't reopen
> the file.  In that case the code just tries to duplicate the handle.
> However, a duplicated file handle can't have more permissions than the
> original handle.
> 
> So if it fails for you, it seems the reopen failed and the handle
> only got duplicated.  In that case, you can't gain write perms if
> the original handle only got read perms.

Understood.

> What the code fails to do is trying to open the file by name as a last
> resort.  There was a (good?) reason I didn't implement that, but I don't
> remember ATM.

Not relevant in this case, as the file has been deleted ... (i.e., in this
case, it cannot be opened by name).

Henri

=====


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: /dev/fd/N not synonymous with file descriptor N; it is on Linux

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1083 bytes --]

On Jan 22 11:20, Houder wrote:
> On Tue, 22 Jan 2019 10:41:57, Corinna Vinschen  wrote:
> > On Jan 22 10:25, Houder wrote:
> [snip]
> 
> > > Curious! It fails (for me) on W7 ...
> > 
> > It works for me just as well on W7:
> > 
> > $ uname -a
> > CYGWIN_NT-6.1 vmbert764 2.12.0(0.333/5/3) 2019-01-21 22:47 x86_64 Cygwin
> > $ ./stca /dev/fd/0 <<EOF
> > ? bla
> > ? EOF
> > fd1 =3D 0
> > argv[1] =3D /dev/fd/0
> > fd2 =3D 3
> > buf =3D \
> > Hello, world!
> 
> That is odd ... (I am using the same version of Cygwin as you do, do I
> not?)
> 
> 64-@@ uname -a
> CYGWIN_NT-6.1 Seven 2.12.0s(0.333/5/3) 2019-01-21 10:25 x86_64 Cygwin
> (however, I only replaced the cygwin1.dll)

Yes

> (the snapshot is apparently not the same as your version 2.12.0)

My version only differs by the fact that it's built without optimization
and with broken posix timer code since I'm revamping it ATM.

Maybe you should run the above shell session under strace and see if
something unusual crops up.  BLODA?


Corinna

-- 
Corinna Vinschen
Cygwin Maintainer

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: /dev/fd/N not synonymous with file descriptor N; it is on Linux

[Houder]

On Tue, 22 Jan 2019 11:39:28, Corinna Vinschen  wrote:
> 
> On Jan 22 11:20, Houder wrote:
> > On Tue, 22 Jan 2019 10:41:57, Corinna Vinschen  wrote:
> > > On Jan 22 10:25, Houder wrote:
> > [snip]
> >
> > > > Curious! It fails (for me) on W7 ...
> > >
> > > It works for me just as well on W7:
[snip]

> Maybe you should run the above shell session under strace and see if
> something unusual crops up.  BLODA?

NO BLODA.

Ok, for the record (as this is W7, i.e. pre-pre-W10 :-)

Using my original STC again: (source code included below)

 - create file (in /tmp) write-only, write "Hello, world!" to file, close fd
 - open file once more read-only
 - unlink file
 - open file, using /dev/fd/N, read-write <==== succeeds (and the handle shown by fcntl is read-write)
 - write "*****" to file (using the fd obtained in the previous line), lseek to begin of file
  - write fails w/ "Permission denied" <==== so ... the file cannot be written to?
 - read file (using the same fd)

Regards,
Henri

64-@@ uname -a
CYGWIN_NT-6.1 Seven 2.12.0s(0.333/5/3) 2019-01-21 10:25 x86_64 Cygwin

-----
From the output of strace:

----- # write ... fails
   74   25044 [main] stc 1368 write: write(4, 0x100403117, 14)
   31   25075 [main] stc 1368 seterrno_from_nt_status: /ext/build/mknetrel/src/cygwin-snapshot-20190121-1/winsup/cygwin/fhandler.cc:294 status 0xC0000022 -> windows error 5
   29   25104 [main] stc 1368 geterrno_from_win_error: windows error 5 == errno 13
   27   25131 [main] stc 1368 write: -1 = write(4, 0x100403117, 14), errno 13

i.e. the STC fails in fhandler_base::raw_write() in winsup/cygwin/fhandler.cc,
right after NtWriteFile() ...

Btw, earlier on, when /tmp/stc.txt is unlinked, an "Sharing violation" occurs:
   32   21456 [main] stc 1368 unlink_nt: Trying to delete \??\E:\Cygwin64\tmp\stc.txt, isdir = 0
----- # Sharing violation? in unlink_nt() in winsup/cygwin/syscalls.cc, after the 3rd call to NtOpenfile()
   50   21506 [main] stc 1368 unlink_nt: Sharing violation when opening \??\E:\Cygwin64\tmp\stc.txt
  443   21949 [main] stc 1368 try_to_bin: \??\E:\Cygwin64\tmp\stc.txt, return bin_status 2 ### "has been moved"?
   35   21984 [main] stc 1368 unlink_nt: \??\E:\Cygwin64\tmp\stc.txt, return status = 0x0
   28   22012 [main] stc 1368 unlink: 0 = unlink(/tmp/stc.txt)

-----
Output of the STC:
64-@@ ./stc # source code included below ...
Invoked from: /usr/bin/bash (parent process)
flagsfl2 = 0x118000
devfile = /dev/fd/3
flagsfl3 = 0x118002
/dev/fd/3: Cannot write!, id = openfd3, errno = 13, errstr = Permission denied
buf = Hello, world!
64-@@
#  this is odd: open("/dev/fd/3", O_RDWR) succeeds ... write(4, ...) does not
#  fail w/ "Bad file descriptor", but w/ "Permission denied"!

-----
If the STC is put "on hold" right before termination (before closing
the file descriptors), the "bin" shows:

64-@@ pwd
/drv/e/$RECYCLE.BIN/S-1-5-21-91509220-1575020443-2714799223-1000
64-@@ ls -al
total 6
drwx------+ 1 Henri        None            0 Jan 27 17:27 .
drwxrwx---+ 1 Henri        None            0 Jan 27 08:47 ..
-rw-r-----  1 Unknown+User Unknown+Group  14 Jan 27 17:27 .???017d00000001f91f4e494283f3b8a953
-rwx------+ 1 Henri        None          129 Jan 27 08:33 desktop.ini

This file cannot be accessed (read) ... (can be read by STC, but NOT written to!).

64-@@ icacls .
. BUILTIN\Administrators:(OI)(CI)(F)
  NT AUTHORITY\SYSTEM:(OI)(CI)(F)
  Seven\Henri:(OI)(CI)(F)
  Mandatory Label\Low Mandatory Level:(OI)(CI)(IO)(NW)

Successfully processed 1 files; Failed processing 0 files
64-@@ getfacl .
# file: .
# owner: Henri
# group: None
user::rwx
group::---
group:SYSTEM:rwx        #effective:---
group:Administrators:rwx        #effective:---
mask::---
other::---
default:user::rwx
default:group::---
default:group:SYSTEM:rwx        #effective:---
default:group:Administrators:rwx        #effective:---
default:mask::---
default:other::---

-----
source code of STC:
// gcc -Wall -o stc stc.c

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h> // strerror()
#include <errno.h>
#include <fcntl.h>

// NOTE: my text uses "fd 2" for a file descriptor that equals 2; and fd2 for variable fd2

/*
 LPI, 5.11 The /dev/fd Directory (Linux Programming Interface, Michael Kerrisk)
 ...
FIXME: No, on Linux the file is "reopened": a new open file description is created!

Cygwin/ W7
file deleted: (fd2 is opened before file deletion)
 - file can be written to via fd3 iff fd2 has been opened read-write
file NOT deleted:
 - file can be written to via fd3, even if fd2 has been opened read-only
 */

/*
 - create file (in /tmp) write-only, write "Hello, world!" to file, close fd
 - open file once more read-only (or read-write)
 - unlink file
 - open file, using /dev/fd/N (or /proc/<pid>/fd/N), read-write
 - write "*****" to file (using the fd obtained in the previous line), lseek to begin of file
 - read file (using the same fd)
 */

void
errExit(const char *str)
{
    fprintf(stderr, "id = %s, errno = %d, errstr = %s\n", str, errno, strerror(errno));
    exit(EXIT_FAILURE);
}

int
main(int argc, char *argv[])
{
#if 1
    {
        char tmp0[30] = { 0 }; char tmp1[30] = { 0 };
        if (snprintf(tmp0, sizeof(tmp0), "/proc/%u/exe", (unsigned int)getppid() ) == -1 ) errExit("sprintf-tmp0");
        //printf("tmp0 = %s\n", tmp0);
        if (strlen(tmp0) == sizeof(tmp0) ) errExit("strlen-tmp0");
        if (readlink(tmp0, tmp1, sizeof(tmp1) - 1) ==  -1) errExit("readlink-tmp1");
        //printf("tmp1 = %s\n", tmp1);
        if (strlen(tmp1) == sizeof(tmp1) ) errExit("strlen-tmp1");
        fprintf(stderr, "Invoked from: %s (parent process)\n", tmp1);
    }
#endif

    int fd1, fd2, fd3;

    errno = 0;

    // kludge (ignore error if file does not exist) ...
    if (unlink("/tmp/stc.txt") == -1) {
        if (errno != 2) errExit("unlink"); else errno = 0;
    }

    // create a tmpfile in the same way that bash would do ...
    fd1 = open("/tmp/stc.txt", O_WRONLY|O_CREAT|O_EXCL|O_TRUNC, 0600);
    if (fd1 == -1)
        errExit("openfd1");

    if (write(fd1, "Hello, world!\n", 14) == -1) errExit("writefd1");

    // close fd1 like bash would do ...
    if (close(fd1) == -1) errExit("closefd1");

    // also open this tmpfile for reading like bash would do ...
    fd2 = open("/tmp/stc.txt", O_RDONLY);
    //fd2 = open("/tmp/stc.txt", O_RDWR);
    if (fd2 == -1)
        errExit("openfd2");

#if 1
    int flagsfl2;
    if ( (flagsfl2 = fcntl(fd2, F_GETFL)) == -1) errExit("fcntlfd1-getfl2");
    fprintf(stderr, "flagsfl2 = 0x%x\n", flagsfl2);
    // Linux: /usr/include/bits/fcntl-linux.h
    // Cygwin: /usr/include/sys/_default_fcntl.h
#endif

    // delete the tmpfile like bash would do ...
    if (unlink("/tmp/stc.txt") == -1) errExit("unlink");

    // kludge: compose a string (using fd2) representing "the device file"
    // in /dev/fd (a symlnk to /proc/self/fd) for file descriptor fd2 which
    // is still open ...
    char devfile[16] = "/dev/fd/";       devfile[8] =  fd2 + 0x30; devfile[9] =  '\0';
    //char devfile[16] = "/proc/self/fd/"; devfile[14] = fd2 + 0x30; devfile[15] = '\0';

    fprintf(stderr, "devfile = %s\n", devfile);

    // open this device file; it succeeds on Linux, but fails on Cygwin (that
    // is, it failed before 20190107?)
    //fd3 = open(devfile, O_RDONLY); // this failed before 20190107?
    fd3 = open(devfile, O_RDWR); // odd: succeeds on W7 (fd2 is read-only!)

    if (fd3 == -1) { // does not happen beyond 20190106
        const char *id = "openfd3";
#if 1
	fprintf(stderr, "%s: Cannot open!, id = %s, errno = %d, errstr = %s\n",
            devfile, id, errno, strerror(errno));
        /* previously, (before 20190107?)
           it was not even possible to open the file, using either "/dev/fd/N"
           or "/proc/self/fd/N" if the target had been unlink'ed ... Now it is
           possible to open the file as indicated, but it can only be read. On
           Linux however, it is even possible to open it "read-write" ...
           (where fd3 will point to a brand-new open file handle)
         */
        errno = 0;
        char buf[16] = { 0 };
        if (read(fd2, buf, sizeof(buf) ) == -1) errExit("readfd2");
        fprintf(stderr, "buf = %s", buf);
        if (close(fd2) == -1) fprintf(stderr, "closefd2 failed\n");
        exit(EXIT_FAILURE);
#else
        errExit(id);
#endif
    }

#if 0
    {
        char buf[16] = { 0 };
        if (read(fd2, buf, sizeof(buf) ) == -1) errExit("readfd2");
        fprintf(stderr, "buf = %s", buf);
    }
#endif

#if 1
    int flagsfl3;
    if ( (flagsfl3 = fcntl(fd3, F_GETFL)) == -1) errExit("fcntlfd3-getfl3");
    fprintf(stderr, "flagsfl3 = 0x%x\n", flagsfl3);
    // Linux: /usr/include/bits/fcntl-linux.h
    // Cygwin: /usr/include/sys/_default_fcntl.h
#endif

// option: write using fd3
#if 1
    // failure unless fd2 has been opened read-write (and file has been deleted)
    // W7: fd3 will be a duplicate of fd2 in that case ...
    if (write(fd3, "*****\n", 14) == -1) {
        const char *id = "openfd3";
  #if 1
        fprintf(stderr, "%s: Cannot write!, id = %s, errno = %d, errstr = %s\n",
        devfile, id, errno, strerror(errno));
        errno = 0;
  #else
        errExit(id);
  #endif
    }

    // required because of write() -- reset file offset
    if (lseek(fd3, 0, SEEK_SET) == -1) errExit("lseekfd3");

#endif // end - option: write using fd3

    char buf[16] = { 0 };
    if (read(fd3, buf, sizeof(buf) ) == -1) errExit("readfd3");
    fprintf(stderr, "buf = %s", buf);
//sleep(30);
    if (close(fd3) == -1) fprintf(stderr, "closefd3 failed\n");
    if (close(fd2) == -1) fprintf(stderr, "closefd2 failed\n");
}

//=====


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: /dev/fd/N not synonymous with file descriptor N; it is on Linux

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 2214 bytes --]

On Jan 27 19:39, Houder wrote:
> On Tue, 22 Jan 2019 11:39:28, Corinna Vinschen  wrote:
> > 
> > On Jan 22 11:20, Houder wrote:
> > > On Tue, 22 Jan 2019 10:41:57, Corinna Vinschen  wrote:
> > > > On Jan 22 10:25, Houder wrote:
> > > [snip]
> > >
> > > > > Curious! It fails (for me) on W7 ...
> > > >
> > > > It works for me just as well on W7:
> [snip]
> 
> > Maybe you should run the above shell session under strace and see if
> > something unusual crops up.  BLODA?
> 
> NO BLODA.
> 
> Ok, for the record (as this is W7, i.e. pre-pre-W10 :-)
> 
> Using my original STC again: (source code included below)
> 
>  - create file (in /tmp) write-only, write "Hello, world!" to file, close fd
>  - open file once more read-only
>  - unlink file
>  - open file, using /dev/fd/N, read-write <==== succeeds (and the handle shown by fcntl is read-write)
>  - write "*****" to file (using the fd obtained in the previous line), lseek to begin of file
>   - write fails w/ "Permission denied" <==== so ... the file cannot be written to?

Yes, that scenario fails on W7 but works on W10 1709 and later.  Keep in
mind that the OS doesn't allow to reopen a file which has been deleted.
Cygwin tries a best effort by duplicating the handle.  A duplicated file
handle can't have more permissions than the original handle, so if the
original handle was opened for reading only, the duplicated handle can't
have write perms.

> Btw, earlier on, when /tmp/stc.txt is unlinked, an "Sharing violation" occurs:
>    32   21456 [main] stc 1368 unlink_nt: Trying to delete \??\E:\Cygwin64\tmp\stc.txt, isdir = 0
> ----- # Sharing violation? in unlink_nt() in winsup/cygwin/syscalls.cc, after the 3rd call to NtOpenfile()

Yes, that's expected.  Cygwin first tries to open the file for DELETE
only.  If that works, it can just delete the file and be done with it.

However, if there's any other open handle to the file with more perms
than just DELETE, this first call fails with STATUS_SHARING_VIOLATION.
Now Cygwin knows that the file is in-use and that the job is not just
done with deleting the file.  More effort is required.


Corinna

-- 
Corinna Vinschen
Cygwin Maintainer

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: /dev/fd/N not synonymous with file descriptor N; it is on Linux

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1730 bytes --]

On Jan 27 22:57, Corinna Vinschen wrote:
> On Jan 27 19:39, Houder wrote:
> > On Tue, 22 Jan 2019 11:39:28, Corinna Vinschen  wrote:
> > > 
> > > On Jan 22 11:20, Houder wrote:
> > > > On Tue, 22 Jan 2019 10:41:57, Corinna Vinschen  wrote:
> > > > > On Jan 22 10:25, Houder wrote:
> > > > [snip]
> > > >
> > > > > > Curious! It fails (for me) on W7 ...
> > > > >
> > > > > It works for me just as well on W7:
> > [snip]
> > 
> > > Maybe you should run the above shell session under strace and see if
> > > something unusual crops up.  BLODA?
> > 
> > NO BLODA.
> > 
> > Ok, for the record (as this is W7, i.e. pre-pre-W10 :-)
> > 
> > Using my original STC again: (source code included below)
> > 
> >  - create file (in /tmp) write-only, write "Hello, world!" to file, close fd
> >  - open file once more read-only
> >  - unlink file
> >  - open file, using /dev/fd/N, read-write <==== succeeds (and the handle shown by fcntl is read-write)
> >  - write "*****" to file (using the fd obtained in the previous line), lseek to begin of file
> >   - write fails w/ "Permission denied" <==== so ... the file cannot be written to?
> 
> Yes, that scenario fails on W7 but works on W10 1709 and later.  Keep in
> mind that the OS doesn't allow to reopen a file which has been deleted.

... on Windows 8.1 and earlier and (probably, but I can't test this) on
W10 versions prior to 1709.

> Cygwin tries a best effort by duplicating the handle.  A duplicated file
> handle can't have more permissions than the original handle, so if the
> original handle was opened for reading only, the duplicated handle can't
> have write perms.
> [etc]


Corinna

-- 
Corinna Vinschen
Cygwin Maintainer

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: /dev/fd/N not synonymous with file descriptor N; it is on Linux

[Houder]

On Sun, 27 Jan 2019 22:57:21, Corinna Vinschen  wrote:
> 
> On Jan 27 19:39, Houder wrote:
> > NO BLODA.
> >
> > Ok, for the record (as this is W7, i.e. pre-pre-W10 :-)
> >
> > Using my original STC again: (source code included below)
> >
> >  - create file (in /tmp) write-only, write "Hello, world!" to file, close
> >    fd
> >  - open file once more read-only
> >  - unlink file
> >  - open file, using /dev/fd/N, read-write <==== succeeds (and the handle
> >    shown by fcntl is read-write)
> >  - write "*****" to file (using the fd obtained in the previous line),
> >    lseek to begin of file
> >   - write fails w/ "Permission denied" <==== so ... the file cannot be
> >     written to?
> 
> Yes, that scenario fails on W7 but works on W10 1709 and later.  Keep in
> mind that the OS doesn't allow to reopen a file which has been deleted.
> Cygwin tries a best effort by duplicating the handle.  A duplicated file
> handle can't have more permissions than the original handle, so if the
> original handle was opened for reading only, the duplicated handle can't
> have write perms.
[snip]

Yes Corinna, I already got that from one of your previous replies. You gave
the same explanation here:

    https://cygwin.com/ml/cygwin/2019-01/msg00171.html
    (Date: Tue, 22 Jan 2019 10:41:57 +0100)

    "A duplicated file handle can't have more permissions than the original
     handle" (i.e what occurs on pre "Windows 10 1709" systems)

The "funny" thing is, in that same post, you showed that the STC succeeded
on your virtual W7 system ... (contradiction!).

The STC in that post executes the same scenario as above ...

 - the difference is that the first 3 steps are carried out by bash, when
   invoked as follows:

   @@ ./stca /dev/fd/N N<<EOF

Both STC's (stc.c and stca.c) fail on my W7 (Note: stc.c is the testcase
that I included in my previous post -- and the one I started this thread
with).

Mind this: I am NOT upset that the STC's (plural) fail on (my) W7. Not at
all!

As you indicated in your commit of January 8, 2019, you have "to lie your
ass off" to make things work.

However, if a file descriptor refers to a "duplicated file handle" (which
is read-only), then at least write() should fail w.

    Bad file descriptor (i.s.o. Permission denied)

Also fcntl(..., F_GETFL) should NOT show a handle w. write permission.

But I stop here. I let it rest.

But not without saying thanks to you for all the help I got from you in
the past years.

Regards,
Henri

-----
author  Corinna Vinschen <corinna@vinschen.de>
        Tue, 8 Jan 2019 20:37:43 +0000 (21:37 +0100)
committer       Corinna Vinschen <corinna@vinschen.de>
        Tue, 8 Jan 2019 20:47:28 +0000 (21:47 +0100)
commit  ec36c59f1a9349e690849e393251623f5936408c
tree    fd8b12b24bdb4fb21a4600cdd0dd14d4e91fb30d        tree
parent  0c545f3264aaaac3d02d3ef785a2e2e9d77ed03f        commit | diff
Cygwin: open: workaround reopen file w/ delete disposition set

On pre-W10 systems there's no way to reopen a file by handle if
the delete disposition is set.  We try to get around with
duplicating the handle.

 535 /* Open system call handler function. */
 536 int
 537 fhandler_base::open (int flags, mode_t mode)
 538 {
..
 694   status = NtCreateFile (&fh, access, &attr, &io, NULL, file_attributes, shared,
 695                          create_disposition, options, p, plen);
 696   /* Pre-W10, we can't open a file by handle with delete disposition
 697      set, so we have to lie our ass off. */
 698   if (get_handle () && status == STATUS_DELETE_PENDING)
 699     {
 700       BOOL ret = DuplicateHandle (GetCurrentProcess (), get_handle (),
 701                                   GetCurrentProcess (), &fh,
 702                                   access, !(flags & O_CLOEXEC), 0);
 703       if (!ret)
 704         ret = DuplicateHandle (GetCurrentProcess (), get_handle (),
 705                                GetCurrentProcess (), &fh,
 706                                0, !(flags & O_CLOEXEC),
 707                                DUPLICATE_SAME_ACCESS);
 708       if (!ret)
 709         debug_printf ("DuplicateHandle after STATUS_DELETE_PENDING, %E");
 710       else
 711         status = STATUS_SUCCESS;
 712     }

=====


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: /dev/fd/N not synonymous with file descriptor N; it is on Linux

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 2735 bytes --]

On Jan 28 15:15, Houder wrote:
> On Sun, 27 Jan 2019 22:57:21, Corinna Vinschen  wrote:
> > 
> > On Jan 27 19:39, Houder wrote:
> > > NO BLODA.
> > >
> > > Ok, for the record (as this is W7, i.e. pre-pre-W10 :-)
> > >
> > > Using my original STC again: (source code included below)
> > >
> > >  - create file (in /tmp) write-only, write "Hello, world!" to file, close
> > >    fd
> > >  - open file once more read-only
> > >  - unlink file
> > >  - open file, using /dev/fd/N, read-write <==== succeeds (and the handle
> > >    shown by fcntl is read-write)
> > >  - write "*****" to file (using the fd obtained in the previous line),
> > >    lseek to begin of file
> > >   - write fails w/ "Permission denied" <==== so ... the file cannot be
> > >     written to?
> > 
> > Yes, that scenario fails on W7 but works on W10 1709 and later.  Keep in
> > mind that the OS doesn't allow to reopen a file which has been deleted.
> > Cygwin tries a best effort by duplicating the handle.  A duplicated file
> > handle can't have more permissions than the original handle, so if the
> > original handle was opened for reading only, the duplicated handle can't
> > have write perms.
> [snip]
> 
> Yes Corinna, I already got that from one of your previous replies. You gave
> the same explanation here:
> 
>     https://cygwin.com/ml/cygwin/2019-01/msg00171.html
>     (Date: Tue, 22 Jan 2019 10:41:57 +0100)
> 
>     "A duplicated file handle can't have more permissions than the original
>      handle" (i.e what occurs on pre "Windows 10 1709" systems)
> 
> The "funny" thing is, in that same post, you showed that the STC succeeded
> on your virtual W7 system ... (contradiction!).
> 
> The STC in that post executes the same scenario as above ...
> 
>  - the difference is that the first 3 steps are carried out by bash, when
>    invoked as follows:
> 
>    @@ ./stca /dev/fd/N N<<EOF
> 
> Both STC's (stc.c and stca.c) fail on my W7 (Note: stc.c is the testcase
> that I included in my previous post -- and the one I started this thread
> with).
> 
> Mind this: I am NOT upset that the STC's (plural) fail on (my) W7. Not at
> all!

There was a minor difference when I tested it:  My shell is not bash
but tcsh, and tcsh opens the here document with different perms.

I added debug output to fhandler_base::open and on bash the ACCESS_MASK
set on the incoming handle is 0x12019F with tcsh and 0x120089 with bash.
Compare with https://msdn.microsoft.com/en-us/library/cc246802.aspx, and
you'll see that tcsh opened the file woth O_RDWR, while bash opened the
file with O_RDONLY.

I didn't notice this before, sorry!


Corinna

-- 
Corinna Vinschen
Cygwin Maintainer

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]