[Bug other/115189] New: libiberty introduces UNC paths waking up binutils bug

[Bug other/115189] New: libiberty introduces UNC paths waking up binutils bug

[tomas.kalibera at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=115189

            Bug ID: 115189
           Summary: libiberty introduces UNC paths waking up binutils bug
           Product: gcc
           Version: 13.2.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: other
          Assignee: unassigned at gcc dot gnu.org
          Reporter: tomas.kalibera at gmail dot com
  Target Milestone: ---

With gcc 13.2 (and binutils 2.42) located on a mapped network drive on Windows,
it is not possible to build a trivial example below. It worked with gcc 12.3
(and binutils 2.40).

The problem is that a UNC path (\\host\share) is passed to the linker (ld.exe)
instead of the traditional drive letter (D:\directory) via which gcc is
invoked, and the linker cannot deal with it. Reverting

commit e2bb55ec3b70cf45088bb73ba9ca990129328d60
Author: niXman <i.nixman@autistici.org>
Date:   Sat Feb 11 06:18:10 2023 +0000

    libiberty: fix lrealpath on Windows NTFS symlinks

    gcc computes the wrong prefix if invoked through an NTFS
    symlink. Try to resolve it if possible.

    PR/108350

    libiberty/ChangeLog:

            * lrealpath.c (lrealpath): try to resolve symlink and
            use UNC paths where applicable.

restores the previous behavior (and I am using it as a work-around).

My understanding is (but I've not tested) that this has been fixed in binutils:

https://sourceware.org/bugzilla/show_bug.cgi?id=29533
https://sourceware.org/bugzilla/show_bug.cgi?id=31527

However, binutils with that fix haven't been released, yet. It is not
impossible that UNC paths newly produced by gcc would confuse some other tools
coming from the Unix world (though I've not seen it happen, yet).

In principle, I would tend to think that people on Windows use mapped network
drives primarily to avoid UNC paths, as their applications may not work with
them, so, ideally one should respect that and preserve the mapped network
drives in paths. I wonder if it would be possible to defensively amend
libiberty following this principle and hence reduce the amount of bugs it wakes
up in other software.

------------------


# example output from a trivial example
# using Msys2 shell, gcc is on p: mapped from //tsclient/share

env PATH=/p/x86_64-w64-mingw32.static.posix/bin:$PATH g++ -o hw hw.c
P:\x86_64-w64-mingw32.static.posix\bin/ld.exe: cannot find
//tsclient/share/x86_64-w64-mingw32.static.posix/bin/../lib/gcc/x86_64-w64-mingw32.static.posix/13.2.0/../../../../lib/crt2.o:
Invalid argument
P:\x86_64-w64-mingw32.static.posix\bin/ld.exe: cannot find
//tsclient/share/x86_64-w64-mingw32.static.posix/bin/../lib/gcc/x86_64-w64-mingw32.static.posix/13.2.0/crtbegin.o:
Invalid argument
P:\x86_64-w64-mingw32.static.posix\bin/ld.exe: cannot find -lstdc++: No such
file or directory
P:\x86_64-w64-mingw32.static.posix\bin/ld.exe: cannot find -lmingw32: No such
file or directory
...

[Bug other/115189] libiberty introduces UNC paths waking up binutils bug

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=115189

Andrew Pinski <pinskia at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
               Host|                            |*-*-mingw

--- Comment #1 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
I am not sure if there is anything to be done on the GCC side really.

[Bug other/115189] libiberty introduces UNC paths waking up binutils bug

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=115189

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
      Known to fail|                            |13.3.0
      Known to work|                            |12.3.0

--- Comment #2 from Richard Biener <rguenth at gcc dot gnu.org> ---
The question is whether there's a workaround available on the GCC side?  Of
course we can't fix older releases, like for example 13.3 what was just
released.  13.4 is a year away when the binutils fix should have been
propagated.

Is there a way to canonicalize "back" UNCs to use mapped drive letters?

[Bug other/115189] libiberty introduces UNC paths waking up binutils bug

[tomas.kalibera at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=115189

--- Comment #3 from Tomas Kalibera <tomas.kalibera at gmail dot com> ---
I hope there is a more elegant solution, but I could think of a best-effort
one. Check if the result of the normalization is a UNC path, if yes, check if
the original path started with a drive letter, if yes, modify the UNC path
replacing the \\server\share by that drive, normalize the modified path again
and if it normalizes to the same UNC path, use the version with the drive;
otherwise, use the original UNC result of normalization.

[Bug other/115189] libiberty introduces UNC paths waking up binutils bug

[gnu.org at billz dot fastmail.fm]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=115189

--- Comment #4 from Bill Zissimopoulos <gnu.org at billz dot fastmail.fm> ---
The lrealpath function on Windows uses the GetFinalPathNameByHandleW function
with the VOLUME_NAME_DOS flag.

I do not currently have access to a Windows system with dev tools, but my
recollection is that the VOLUME_NAME_DOS flag should be sufficient to correctly
find the DOS drive in most cases including network drives. Unfortunately
GetFinalPathNameByHandleW does not work with drives created with the
DefineDosDeviceW API (e.g. drives created using the SUBST utility).

How are people who stumble on this bug create their drives?

A more robust alternative to GetFinalPathNameByHandleW/VOLUME_NAME_DOS may be
the following:

- Use GetFinalPathNameByHandleW with VOLUME_NAME_NORMALIZED|VOLUME_NAME_NT.

- Enumerate all available drives for the current process using
GetLogicalDrives.

- For each logical drive use QueryDosDeviceW to see if the returned prefix
matches the NT namespace prefix returned by the earlier call to
GetFinalPathNameByHandleW.

For an example of this approach see the chromium code here:

https://chromium.googlesource.com/chromium/src/base/+/master/files/file_util_win.cc#828

(See functions NormalizeFilePath and DevicePathToDriveLetterPath.)

[Bug other/115189] libiberty introduces UNC paths waking up binutils bug

[tomas.kalibera at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=115189

--- Comment #5 from Tomas Kalibera <tomas.kalibera at gmail dot com> ---
(In reply to Bill Zissimopoulos from comment #4)

> I do not currently have access to a Windows system with dev tools, but my
> recollection is that the VOLUME_NAME_DOS flag should be sufficient to
> correctly find the DOS drive in most cases including network drives.
> Unfortunately GetFinalPathNameByHandleW does not work with drives created
> with the DefineDosDeviceW API (e.g. drives created using the SUBST utility).
> 
> How are people who stumble on this bug create their drives?

Yes, that's probably the case.

I can reproduce the problem with "subst" and with the Windows Explorer
("File/Map network drive"). I assume the is the usual way to map network
drives.

Also, e.g.

mklink /D c:\share \\tsclient\share

results in "c:\share" normalized to "\\tsclient\share" (though I expect using
directory symlinks used this way would be rare).

[Bug other/115189] libiberty introduces UNC paths waking up binutils bug

[gnu.org at billz dot fastmail.fm]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=115189

--- Comment #6 from Bill Zissimopoulos <gnu.org at billz dot fastmail.fm> ---
> I can reproduce the problem with "subst" and with the Windows Explorer
> ("File/Map network drive"). I assume the is the usual way to map network
> drives.

The usual way to map network drives is indeed "Map network drive", but
GetFinalPathNameByHandleW should work with those. I am surprised that it does
not, unless the problem you are experiencing is not related to
GetFinalPathNameByHandleW after all.

[Bug other/115189] libiberty introduces UNC paths waking up binutils bug

[tomas.kalibera at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=115189

--- Comment #7 from Tomas Kalibera <tomas.kalibera at gmail dot com> ---
(In reply to Bill Zissimopoulos from comment #6)
> > I can reproduce the problem with "subst" and with the Windows Explorer
> > ("File/Map network drive"). I assume the is the usual way to map network
> > drives.
> 
> The usual way to map network drives is indeed "Map network drive", but
> GetFinalPathNameByHandleW should work with those. I am surprised that it
> does not, unless the problem you are experiencing is not related to
> GetFinalPathNameByHandleW after all.

I am testing using lrealpath.c from libiberty (gcc 13.3), instrumented to make
sure GetFinalPathNameByHandle is called and seeing what it returned. It returns
UNC paths (\\server\share) for mapped network drives. I've tried with a regular
samba network share in addition to the RDP share, the result is the same. On
Windows 10 Pro, Version     10.0.19045 Build 19045.

[Bug other/115189] libiberty introduces UNC paths waking up binutils bug

[tomas.kalibera at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=115189

--- Comment #8 from Tomas Kalibera <tomas.kalibera at gmail dot com> ---
(In reply to Bill Zissimopoulos from comment #4)

> A more robust alternative to GetFinalPathNameByHandleW/VOLUME_NAME_DOS may
> be the following:
> 
> - Use GetFinalPathNameByHandleW with VOLUME_NAME_NORMALIZED|VOLUME_NAME_NT.
> 
> - Enumerate all available drives for the current process using
> GetLogicalDrives.
> 
> - For each logical drive use QueryDosDeviceW to see if the returned prefix
> matches the NT namespace prefix returned by the earlier call to
> GetFinalPathNameByHandleW.
> 
> For an example of this approach see the chromium code here:
> 
> https://chromium.googlesource.com/chromium/src/base/+/master/files/
> file_util_win.cc#828
> 
> (See functions NormalizeFilePath and DevicePathToDriveLetterPath.)

I am not sure how this would work. On my system, I get a number of different NT
namespace prefixes from QueryDosDevice:

(1) \??\UNC\tsclient\share
(2) \??\UNC\192.168.0.45\share
(3) \Device\LanmanRedirector\;Y:0000000001189cc1\192.168.0.45\share
(4) \Device\RdpDr\;Z:2\tsclient\share

the forms of (1),(2) are with "subst". The form of (3) is when using Explorer
on a real samba drive (also "net use"). The form of (4) is when using Explorer
on an RDP share (also "net use").

But, GetFinalPathNameByHandle() returns prefixes in a yet different form:

(A) /Device/Mup/192.168.0.45/share
(B) /Device/Mup/tsclient/share

where (A) is for a real samba drive and (B) for the RDP share.

So, simply matching the prefix (whether the result of QueryDosDevice() is a
prefix of the result of GetFinalPathNameByHandle) probably wouldn't work?