[Bug libstdc++/108178] New: Filesystem::copy_file can't copy from /proc on Linux machines

[Bug libstdc++/108178] New: Filesystem::copy_file can't copy from /proc on Linux machines

[faivel555 at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108178

            Bug ID: 108178
           Summary: Filesystem::copy_file can't copy from /proc on Linux
                    machines
           Product: gcc
           Version: 11.3.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: libstdc++
          Assignee: unassigned at gcc dot gnu.org
          Reporter: faivel555 at gmail dot com
  Target Milestone: ---

Hello,
When trying to copy a file from /proc folder on Linux, copy_file won't copy the
file but will generate a success status.

The problem is that copy_file passes 0 file size to sendfile() on Linux
machines as you can see here:
https://github.com/gcc-mirror/gcc/blob/master/libstdc%2B%2B-v3/src/filesystem/ops-common.h#L499 
, making sendfile() to do nothing. Since most of the files if not all files in
/proc don't have a valid file size, the copying will be successful but no files
are copied.


Thanks,
Faivel

[Bug libstdc++/108178] Filesystem::copy_file can't copy from /proc on Linux machines

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108178

--- Comment #1 from Richard Biener <rguenth at gcc dot gnu.org> ---
But that's a bug of the procfs filesystem, not of the standard library.

[Bug libstdc++/108178] Filesystem::copy_file can't copy from /proc on Linux machines

[redi at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108178

Jonathan Wakely <redi at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |NEW
   Last reconfirmed|                            |2022-12-19
     Ever confirmed|0                           |1

--- Comment #2 from Jonathan Wakely <redi at gcc dot gnu.org> ---
Yes, but we should probably add a workaround anyway.

Maybe we should not use sendfile for zero-sized files, and use fstream instead.
That will copy until EOF, without caring about the file size obtained from
stat(2). For real files that actually do have zero size that might be slightly
slower than attempting to use sendfile, but hopefully not too bad. I'll try to
benchmark it.

[Bug libstdc++/108178] Filesystem::copy_file can't copy from /proc on Linux machines

[redi at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108178

--- Comment #3 from Jonathan Wakely <redi at gcc dot gnu.org> ---
On the other hand, it's not a bug in /proc, because the size of some of those
files might vary dynamically, and the size isn't known until you try to read
it. So even if we got a size from stat, we might read a different number of
bytes once we open it, and so sendfile wouldn't copy the whole thing anyway.
And on the gripping hand, reading from /proc could be considered undefined
behaviour because files that are changing while performing std::filesystem
operations cause races and so the behaviour is not defined by the standard.

But we can make it work in practice.

[Bug libstdc++/108178] Filesystem::copy_file can't copy from /proc on Linux machines

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108178

--- Comment #4 from Richard Biener <rguenth at gcc dot gnu.org> ---
In hindsight the files in /proc should probably have been named pipes, not
regular files.

[Bug libstdc++/108178] Filesystem::copy_file can't copy from /proc on Linux machines

[redi at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108178

Jonathan Wakely <redi at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |ASSIGNED
           Assignee|unassigned at gcc dot gnu.org      |redi at gcc dot gnu.org

--- Comment #5 from Jonathan Wakely <redi at gcc dot gnu.org> ---
I have a patch for GCC 14 stage 1.

[Bug libstdc++/108178] Filesystem::copy_file can't copy from /proc on Linux machines

[redi at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108178

--- Comment #6 from Jonathan Wakely <redi at gcc dot gnu.org> ---
Patch: https://gcc.gnu.org/pipermail/gcc-patches/2023-March/614415.html

[Bug libstdc++/108178] Filesystem::copy_file can't copy from /proc on Linux machines

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108178

--- Comment #7 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The master branch has been updated by Jonathan Wakely <redi@gcc.gnu.org>:

https://gcc.gnu.org/g:07a0e108247f23fcb919c61595adae143f1ea02a

commit r14-1570-g07a0e108247f23fcb919c61595adae143f1ea02a
Author: Jonathan Wakely <jwakely@redhat.com>
Date:   Tue Mar 21 12:29:08 2023 +0000

    libstdc++: Make std::filesystem::copy_file work for procfs [PR108178]

    The size reported by stat is always zero for some special files such as
    those under /proc, which means the current copy_file implementation
    thinks there is nothing to copy. Instead of trusting the stat value, try
    to read a character from a streambuf and check for EOF.

    libstdc++-v3/ChangeLog:

            PR libstdc++/108178
            * src/filesystem/ops-common.h (do_copy_file): Check for empty
            files by trying to read a character.
            * testsuite/27_io/filesystem/operations/copy_file_108178.cc:
            New test.

[Bug libstdc++/108178] Filesystem::copy_file can't copy from /proc on Linux machines

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108178

--- Comment #8 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The releases/gcc-13 branch has been updated by Jonathan Wakely
<redi@gcc.gnu.org>:

https://gcc.gnu.org/g:c0a57904308ad0f10833f38cb207c750e5658b6b

commit r13-7420-gc0a57904308ad0f10833f38cb207c750e5658b6b
Author: Jonathan Wakely <jwakely@redhat.com>
Date:   Tue Mar 21 12:29:08 2023 +0000

    libstdc++: Make std::filesystem::copy_file work for procfs [PR108178]

    The size reported by stat is always zero for some special files such as
    those under /proc, which means the current copy_file implementation
    thinks there is nothing to copy. Instead of trusting the stat value, try
    to read a character from a streambuf and check for EOF.

    For the backport, we also need to avoid trying to use sendfile when stat
    reports a zero size, so that we use streambufs to copy the file.

    libstdc++-v3/ChangeLog:

            PR libstdc++/108178
            * src/filesystem/ops-common.h (do_copy_file): Check for empty
            files by trying to read a character.
            * testsuite/27_io/filesystem/operations/copy_file_108178.cc:
            New test.

    (cherry picked from commit 07a0e108247f23fcb919c61595adae143f1ea02a)

[Bug libstdc++/108178] Filesystem::copy_file can't copy from /proc on Linux machines

[redi at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108178

Jonathan Wakely <redi at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|---                         |11.5

[Bug libstdc++/108178] Filesystem::copy_file can't copy from /proc on Linux machines

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108178

--- Comment #9 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The releases/gcc-12 branch has been updated by Jonathan Wakely
<redi@gcc.gnu.org>:

https://gcc.gnu.org/g:f85947338197b12b77aa5eb0eb2d1b4ea7dbdd54

commit r12-9908-gf85947338197b12b77aa5eb0eb2d1b4ea7dbdd54
Author: Jonathan Wakely <jwakely@redhat.com>
Date:   Tue Mar 21 12:29:08 2023 +0000

    libstdc++: Make std::filesystem::copy_file work for procfs [PR108178]

    The size reported by stat is always zero for some special files such as
    those under /proc, which means the current copy_file implementation
    thinks there is nothing to copy. Instead of trusting the stat value, try
    to read a character from a streambuf and check for EOF.

    For the backport, we also need to avoid trying to use sendfile when stat
    reports a zero size, so that we use streambufs to copy the file.

    libstdc++-v3/ChangeLog:

            PR libstdc++/108178
            * src/filesystem/ops-common.h (do_copy_file): Check for empty
            files by trying to read a character.
            * testsuite/27_io/filesystem/operations/copy_file_108178.cc:
            New test.

    (cherry picked from commit 07a0e108247f23fcb919c61595adae143f1ea02a)

[Bug libstdc++/108178] Filesystem::copy_file can't copy from /proc on Linux machines

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108178

--- Comment #10 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The releases/gcc-11 branch has been updated by Jonathan Wakely
<redi@gcc.gnu.org>:

https://gcc.gnu.org/g:d69407149194e2a7ee20537c76acb1976c8659be

commit r11-11048-gd69407149194e2a7ee20537c76acb1976c8659be
Author: Jonathan Wakely <jwakely@redhat.com>
Date:   Tue Mar 21 12:29:08 2023 +0000

    libstdc++: Make std::filesystem::copy_file work for procfs [PR108178]

    The size reported by stat is always zero for some special files such as
    those under /proc, which means the current copy_file implementation
    thinks there is nothing to copy. Instead of trusting the stat value, try
    to read a character from a streambuf and check for EOF.

    For the backport, we also need to avoid trying to use sendfile when stat
    reports a zero size, so that we use streambufs to copy the file.

    libstdc++-v3/ChangeLog:

            PR libstdc++/108178
            * src/filesystem/ops-common.h (do_copy_file): Check for empty
            files by trying to read a character.
            * testsuite/27_io/filesystem/operations/copy_file_108178.cc:
            New test.

    (cherry picked from commit 07a0e108247f23fcb919c61595adae143f1ea02a)

[Bug libstdc++/108178] Filesystem::copy_file can't copy from /proc on Linux machines

[redi at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108178

Jonathan Wakely <redi at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|ASSIGNED                    |RESOLVED
         Resolution|---                         |FIXED

--- Comment #11 from Jonathan Wakely <redi at gcc dot gnu.org> ---
Fixed for 11.5, 12.4 and 13.2