Cygwin setup writing incorrect symlinks for native

Cygwin setup writing incorrect symlinks for native

[David Allsopp]

Running Cygwin setup 2.912 with --symlink-type native (or
CYGWIN=winsymlinks:native) is not correctly translating all symlinks. A
default install has these faulty ones:

/etc/pki/tls/cert.pem ->
\??\/etc\pki\ca-trust\extracted\pem\tls-ca-bundle.pem
/etc/pki/tls/certs/ca-bundle.crt ->
\??\/etc\pki\ca-trust\extracted\pem\tls-ca-bundle.pem
/etc/pki/tls/certs/ca-bundle.trust.crt ->
\??\/etc\pki\ca-trust\extracted\openssl\ca-bundle.trust.crt
/etc/ssl/certs -> \??\/etc\pki\tls\certs
/lib/security/cacerts -> \??\/etc\pki\ca-trust\extracted\java\cacerts
/usr/share/doc/groff-1.22.4/pdf/mom-pdf.pdf ->
\??\/usr\share\doc\groff-1.22.4\examples\mom\mom-pdf.pdf

In particular, we noticed that /etc/ssl/certs being invalid breaks git.


David

Re: Cygwin setup writing incorrect symlinks for native

[Jon Turney]

On 06/01/2022 10:46, David Allsopp wrote:
> Running Cygwin setup 2.912 with --symlink-type native (or
> CYGWIN=winsymlinks:native) is not correctly translating all symlinks. A
> default install has these faulty ones:
> 
> /etc/pki/tls/cert.pem ->
> \??\/etc\pki\ca-trust\extracted\pem\tls-ca-bundle.pem
> /etc/pki/tls/certs/ca-bundle.crt ->
> \??\/etc\pki\ca-trust\extracted\pem\tls-ca-bundle.pem
> /etc/pki/tls/certs/ca-bundle.trust.crt ->
> \??\/etc\pki\ca-trust\extracted\openssl\ca-bundle.trust.crt
> /etc/ssl/certs -> \??\/etc\pki\tls\certs
> /lib/security/cacerts -> \??\/etc\pki\ca-trust\extracted\java\cacerts
> /usr/share/doc/groff-1.22.4/pdf/mom-pdf.pdf ->
> \??\/usr\share\doc\groff-1.22.4\examples\mom\mom-pdf.pdf


Thanks for reporting this.

Yes, it seems this is messed up where the target of the symlink is an 
absolute path.

I've uploaded setup 2.913 with an attempted fix.

   https://cygwin.com/setup/setup-2.913.x86_64.exe
   https://cygwin.com/setup/setup-2.913.x86.exe

Please try that (you'll have to reinstall the effected packages), and 
see if it improves things for you?

RE: Cygwin setup writing incorrect symlinks for native

[David Allsopp]

Jon Turney wrote:
> On 06/01/2022 10:46, David Allsopp wrote:
> > Running Cygwin setup 2.912 with --symlink-type native (or
> > CYGWIN=winsymlinks:native) is not correctly translating all symlinks.
> > A default install has these faulty ones:
> >
> > /etc/pki/tls/cert.pem ->
> > \??\/etc\pki\ca-trust\extracted\pem\tls-ca-bundle.pem
> > /etc/pki/tls/certs/ca-bundle.crt ->
> > \??\/etc\pki\ca-trust\extracted\pem\tls-ca-bundle.pem
> > /etc/pki/tls/certs/ca-bundle.trust.crt ->
> > \??\/etc\pki\ca-trust\extracted\openssl\ca-bundle.trust.crt
> > /etc/ssl/certs -> \??\/etc\pki\tls\certs /lib/security/cacerts ->
> > \??\/etc\pki\ca-trust\extracted\java\cacerts
> > /usr/share/doc/groff-1.22.4/pdf/mom-pdf.pdf ->
> > \??\/usr\share\doc\groff-1.22.4\examples\mom\mom-pdf.pdf
> 
> 
> Thanks for reporting this.
> 
> Yes, it seems this is messed up where the target of the symlink is an
> absolute path.
> 
> I've uploaded setup 2.913 with an attempted fix.
> 
>    https://cygwin.com/setup/setup-2.913.x86_64.exe
>    https://cygwin.com/setup/setup-2.913.x86.exe
> 
> Please try that (you'll have to reinstall the effected packages), and see
> if it improves things for you?

Thank you! This fixes it for the Cygwin side, although the symlinks are still technically not right from the Windows side. They're still showing up prefixed \??\.

It's an age since I mucked around with those functions - IIRC the API function for writing them adds the \??\ itself and doesn't expect it to have been supplied? However, when you read the reparse point you have to remember to remove it yourself because who needs consistency :o)

C:\cygwin64>dir /s | findstr ??
01/06/2022  04:30 PM    <SYMLINK>      cert.pem [\??\C:\cygwin64\etc\pki\ca-trust\extracted\pem\tls-ca-bundle.pem]
01/06/2022  04:30 PM    <SYMLINK>      ca-bundle.crt [\??\C:\cygwin64\etc\pki\ca-trust\extracted\pem\tls-ca-bundle.pem]
01/06/2022  04:30 PM    <SYMLINK>      ca-bundle.trust.crt [\??\C:\cygwin64\etc\pki\ca-trust\extracted\openssl\ca-bundle.trust.crt]
01/06/2022  04:30 PM    <SYMLINKD>     certs [\??\C:\cygwin64\etc\pki\tls\certs]
01/06/2022  04:30 PM    <SYMLINK>      cacerts [\??\C:\cygwin64\etc\pki\ca-trust\extracted\java\cacerts]
01/06/2022  04:30 PM    <SYMLINK>      mom-pdf.pdf [\??\C:\cygwin64\usr\share\doc\groff-1.22.4\examples\mom\mom-pdf.pdf]

All best,


David

Re: Cygwin setup writing incorrect symlinks for native

[Jon Turney]

On 06/01/2022 16:45, David Allsopp wrote:
> Jon Turney wrote:
>> On 06/01/2022 10:46, David Allsopp wrote:
>>> Running Cygwin setup 2.912 with --symlink-type native (or
>>> CYGWIN=winsymlinks:native) is not correctly translating all symlinks.
>>> A default install has these faulty ones:
>>>
>>> /etc/pki/tls/cert.pem ->
>>> \??\/etc\pki\ca-trust\extracted\pem\tls-ca-bundle.pem
>>> /etc/pki/tls/certs/ca-bundle.crt ->
>>> \??\/etc\pki\ca-trust\extracted\pem\tls-ca-bundle.pem
>>> /etc/pki/tls/certs/ca-bundle.trust.crt ->
>>> \??\/etc\pki\ca-trust\extracted\openssl\ca-bundle.trust.crt
>>> /etc/ssl/certs -> \??\/etc\pki\tls\certs /lib/security/cacerts ->
>>> \??\/etc\pki\ca-trust\extracted\java\cacerts
>>> /usr/share/doc/groff-1.22.4/pdf/mom-pdf.pdf ->
>>> \??\/usr\share\doc\groff-1.22.4\examples\mom\mom-pdf.pdf
>>
>>
>> Thanks for reporting this.
>>
>> Yes, it seems this is messed up where the target of the symlink is an
>> absolute path.
>>
>> I've uploaded setup 2.913 with an attempted fix.
>>
>>     https://cygwin.com/setup/setup-2.913.x86_64.exe
>>     https://cygwin.com/setup/setup-2.913.x86.exe
>>
>> Please try that (you'll have to reinstall the effected packages), and see
>> if it improves things for you?
> 
> Thank you! This fixes it for the Cygwin side, although the symlinks are still technically not right from the Windows side. They're still showing up prefixed \??\.
> 
> It's an age since I mucked around with those functions - IIRC the API function for writing them adds the \??\ itself and doesn't expect it to have been supplied? However, when you read the reparse point you have to remember to remove it yourself because who needs consistency :o)

Thanks for testing.  It seems I still didn't have this quite right and 
these symlinks just happened to work for cygwin, but not native tools.

Please try

   https://cygwin.com/setup/setup-2.914.x86_64.exe
   https://cygwin.com/setup/setup-2.914.x86.exe

> C:\cygwin64>dir /s | findstr ??
> 01/06/2022  04:30 PM    <SYMLINK>      cert.pem [\??\C:\cygwin64\etc\pki\ca-trust\extracted\pem\tls-ca-bundle.pem]
> 01/06/2022  04:30 PM    <SYMLINK>      ca-bundle.crt [\??\C:\cygwin64\etc\pki\ca-trust\extracted\pem\tls-ca-bundle.pem]
> 01/06/2022  04:30 PM    <SYMLINK>      ca-bundle.trust.crt [\??\C:\cygwin64\etc\pki\ca-trust\extracted\openssl\ca-bundle.trust.crt]
> 01/06/2022  04:30 PM    <SYMLINKD>     certs [\??\C:\cygwin64\etc\pki\tls\certs]
> 01/06/2022  04:30 PM    <SYMLINK>      cacerts [\??\C:\cygwin64\etc\pki\ca-trust\extracted\java\cacerts]
> 01/06/2022  04:30 PM    <SYMLINK>      mom-pdf.pdf [\??\C:\cygwin64\usr\share\doc\groff-1.22.4\examples\mom\mom-pdf.pdf]

RE: Cygwin setup writing incorrect symlinks for native

[David Allsopp]

Jon Turney wrote:
> On 06/01/2022 16:45, David Allsopp wrote:
> > Jon Turney wrote:
> >> On 06/01/2022 10:46, David Allsopp wrote:
> >>> Running Cygwin setup 2.912 with --symlink-type native (or
> >>> CYGWIN=winsymlinks:native) is not correctly translating all symlinks.
> >>> A default install has these faulty ones:
> >>>
> >>> /etc/pki/tls/cert.pem ->
> >>> \??\/etc\pki\ca-trust\extracted\pem\tls-ca-bundle.pem
> >>> /etc/pki/tls/certs/ca-bundle.crt ->
> >>> \??\/etc\pki\ca-trust\extracted\pem\tls-ca-bundle.pem
> >>> /etc/pki/tls/certs/ca-bundle.trust.crt ->
> >>> \??\/etc\pki\ca-trust\extracted\openssl\ca-bundle.trust.crt
> >>> /etc/ssl/certs -> \??\/etc\pki\tls\certs /lib/security/cacerts ->
> >>> \??\/etc\pki\ca-trust\extracted\java\cacerts
> >>> /usr/share/doc/groff-1.22.4/pdf/mom-pdf.pdf ->
> >>> \??\/usr\share\doc\groff-1.22.4\examples\mom\mom-pdf.pdf
> >>
> >>
> >> Thanks for reporting this.
> >>
> >> Yes, it seems this is messed up where the target of the symlink is an
> >> absolute path.
> >>
> >> I've uploaded setup 2.913 with an attempted fix.
> >>
> >>     https://cygwin.com/setup/setup-2.913.x86_64.exe
> >>     https://cygwin.com/setup/setup-2.913.x86.exe
> >>
> >> Please try that (you'll have to reinstall the effected packages), and
> >> see if it improves things for you?
> >
> > Thank you! This fixes it for the Cygwin side, although the symlinks are
> > still technically not right from the Windows side. They're still showing
> > up prefixed \??\.
> >
> > It's an age since I mucked around with those functions - IIRC the API
> > function for writing them adds the \??\ itself and doesn't expect it
> > to have been supplied? However, when you read the reparse point you
> > have to remember to remove it yourself because who needs consistency
> > :o)
> 
> Thanks for testing.  It seems I still didn't have this quite right and
> these symlinks just happened to work for cygwin, but not native tools.
> 
> Please try
> 
>    https://cygwin.com/setup/setup-2.914.x86_64.exe
>    https://cygwin.com/setup/setup-2.914.x86.exe

I'm afraid that's just changed the \??\ to \\?\ on those symlinks.

All best,


David

Re: Cygwin setup writing incorrect symlinks for native

[Jon Turney]

On 09/01/2022 09:35, David Allsopp wrote:
> Jon Turney wrote:
>> On 06/01/2022 16:45, David Allsopp wrote:
>>> Jon Turney wrote:
>>>> On 06/01/2022 10:46, David Allsopp wrote:
>>>>> Running Cygwin setup 2.912 with --symlink-type native (or
>>>>> CYGWIN=winsymlinks:native) is not correctly translating all symlinks.
>>>>> A default install has these faulty ones:
>>>>>
>>>>> /etc/pki/tls/cert.pem ->
>>>>> \??\/etc\pki\ca-trust\extracted\pem\tls-ca-bundle.pem
>>>>> /etc/pki/tls/certs/ca-bundle.crt ->
>>>>> \??\/etc\pki\ca-trust\extracted\pem\tls-ca-bundle.pem
>>>>> /etc/pki/tls/certs/ca-bundle.trust.crt ->
>>>>> \??\/etc\pki\ca-trust\extracted\openssl\ca-bundle.trust.crt
>>>>> /etc/ssl/certs -> \??\/etc\pki\tls\certs /lib/security/cacerts ->
>>>>> \??\/etc\pki\ca-trust\extracted\java\cacerts
>>>>> /usr/share/doc/groff-1.22.4/pdf/mom-pdf.pdf ->
>>>>> \??\/usr\share\doc\groff-1.22.4\examples\mom\mom-pdf.pdf
>>>>
>>>>
[...]
>>
>> Thanks for testing.  It seems I still didn't have this quite right and
>> these symlinks just happened to work for cygwin, but not native tools.
>>
>> Please try
>>
>>     https://cygwin.com/setup/setup-2.914.x86_64.exe
>>     https://cygwin.com/setup/setup-2.914.x86.exe
> 
> I'm afraid that's just changed the \??\ to \\?\ on those symlinks.

Well, that has the advantage of being correct :)

(since it's the 'Win32 File Namespace' prefix, which 
CreateSymbolicLinkW() is documented to accept for the target filename, 
and is required when that exceeds MAX_PATH (260) characters)

In my (brief) testing, e.g. the CMD builtin 'type' is able to open 
symlinks of this form.  However, it seems there are some parts of 
Windows (e.g. DIR, File Explorer), which don't handle symlinks like that 
well.

So, I've copied the behaviour of the cygwin DLL, which avoids using that 
prefix if the target is less than MAX_PATH characters.  This seem to 
work better in those cases with typical paths (but I'd claim we're just 
working around a bug in Windows here, as things will still be broken if 
the cygwin root is a path long enough that we can't do that).

Please try:

   https://cygwin.com/setup/setup-2.915.x86_64.exe
   https://cygwin.com/setup/setup-2.915.x86.exe

RE: Cygwin setup writing incorrect symlinks for native

[David Allsopp]

Jon Turney wrote:
> On 09/01/2022 09:35, David Allsopp wrote:
> > Jon Turney wrote:
> >> On 06/01/2022 16:45, David Allsopp wrote:
> >>> Jon Turney wrote:
> >>>> On 06/01/2022 10:46, David Allsopp wrote:
> >>>>> Running Cygwin setup 2.912 with --symlink-type native (or
> >>>>> CYGWIN=winsymlinks:native) is not correctly translating all
> symlinks.
> >>>>> A default install has these faulty ones:
> >>>>>
> >>>>> /etc/pki/tls/cert.pem ->
> >>>>> \??\/etc\pki\ca-trust\extracted\pem\tls-ca-bundle.pem
> >>>>> /etc/pki/tls/certs/ca-bundle.crt ->
> >>>>> \??\/etc\pki\ca-trust\extracted\pem\tls-ca-bundle.pem
> >>>>> /etc/pki/tls/certs/ca-bundle.trust.crt ->
> >>>>> \??\/etc\pki\ca-trust\extracted\openssl\ca-bundle.trust.crt
> >>>>> /etc/ssl/certs -> \??\/etc\pki\tls\certs /lib/security/cacerts ->
> >>>>> \??\/etc\pki\ca-trust\extracted\java\cacerts
> >>>>> /usr/share/doc/groff-1.22.4/pdf/mom-pdf.pdf ->
> >>>>> \??\/usr\share\doc\groff-1.22.4\examples\mom\mom-pdf.pdf
> >>>>
> >>>>
> [...]
> >>
> >> Thanks for testing.  It seems I still didn't have this quite right
> >> and these symlinks just happened to work for cygwin, but not native
> tools.
> >>
> >> Please try
> >>
> >>     https://cygwin.com/setup/setup-2.914.x86_64.exe
> >>     https://cygwin.com/setup/setup-2.914.x86.exe
> >
> > I'm afraid that's just changed the \??\ to \\?\ on those symlinks.
> 
> Well, that has the advantage of being correct :)
> 
> (since it's the 'Win32 File Namespace' prefix, which
> CreateSymbolicLinkW() is documented to accept for the target filename, and
> is required when that exceeds MAX_PATH (260) characters)
> 
> In my (brief) testing, e.g. the CMD builtin 'type' is able to open
> symlinks of this form.  However, it seems there are some parts of Windows
> (e.g. DIR, File Explorer), which don't handle symlinks like that well.
> 
> So, I've copied the behaviour of the cygwin DLL, which avoids using that
> prefix if the target is less than MAX_PATH characters.  This seem to work
> better in those cases with typical paths (but I'd claim we're just working
> around a bug in Windows here, as things will still be broken if the cygwin
> root is a path long enough that we can't do that).
> 
> Please try:
> 
>    https://cygwin.com/setup/setup-2.915.x86_64.exe
>    https://cygwin.com/setup/setup-2.915.x86.exe

That's working (for cmd as well), thanks!


David

Re: Cygwin setup writing incorrect symlinks for native

[Jon Turney]

On 12/01/2022 16:03, David Allsopp wrote:
> Jon Turney wrote:
>>
>> Please try:
>>
>>     https://cygwin.com/setup/setup-2.915.x86_64.exe
>>     https://cygwin.com/setup/setup-2.915.x86.exe
> 
> That's working (for cmd as well), thanks!

Great!  Thanks for reporting this problem and testing my several 
attempts at fixing it :)