[PATCH 1/2] Treat Windows Store's "app execution aliases" as symbolic links

[PATCH 1/2] Treat Windows Store's "app execution aliases" as symbolic links

[Johannes Schindelin]

When the Windows Store version of Python is installed, so-called "app
execution aliases" are put into the `PATH`. These are reparse points
under the hood, with an undocumented format.

We do know a bit about this format, though, as per the excellent analysis:
https://www.tiraniddo.dev/2019/09/overview-of-windows-execution-aliases.html

	The first 4 bytes is the reparse tag, in this case it's
	0x8000001B which is documented in the Windows SDK as
	IO_REPARSE_TAG_APPEXECLINK. Unfortunately there doesn't seem to
	be a corresponding structure, but with a bit of reverse
	engineering we can work out the format is as follows:

	Version: <4 byte integer>
	Package ID: <NUL Terminated Unicode String>
	Entry Point: <NUL Terminated Unicode String>
	Executable: <NUL Terminated Unicode String>
	Application Type: <NUL Terminated Unicode String>

Let's treat them as symbolic links. For example, in this developer's
setup, this will result in the following nice output:

	$ cd $LOCALAPPDATA/Microsoft/WindowsApps/

	$ ls -l python3.exe
	lrwxrwxrwx 1 me 4096 105 Aug 23  2020 python3.exe -> '/c/Program Files/WindowsApps/PythonSoftwareFoundation.Python.3.7_3.7.2544.0_x64__qbz5n2kfra8p0/python.exe'

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 winsup/cygwin/path.cc | 24 ++++++++++++++++++++++++
 1 file changed, 24 insertions(+)

diff --git a/winsup/cygwin/path.cc b/winsup/cygwin/path.cc
index f3b9913bd0..63f377efb1 100644
--- a/winsup/cygwin/path.cc
+++ b/winsup/cygwin/path.cc
@@ -2538,6 +2538,30 @@ check_reparse_point_target (HANDLE h, bool remote, PREPARSE_DATA_BUFFER rp,
       if (check_reparse_point_string (psymbuf))
 	return PATH_SYMLINK | PATH_REP;
     }
+  else if (!remote && rp->ReparseTag == IO_REPARSE_TAG_APPEXECLINK)
+    {
+      /* App execution aliases are commonly used by Windows Store apps. */
+      WCHAR *buf = (WCHAR *)(rp->GenericReparseBuffer.DataBuffer + 4);
+      DWORD size = rp->ReparseDataLength / sizeof(WCHAR), n;
+
+      /*
+         It seems that app execution aliases have a payload of four
+	 NUL-separated wide string: package id, entry point, executable
+	 and application type. We're interested in the executable. */
+      for (int i = 0; i < 3 && size > 0; i++)
+        {
+	  n = wcsnlen (buf, size - 1);
+	  if (i == 2 && n > 0 && n < size)
+	    {
+	      RtlInitCountedUnicodeString (psymbuf, buf, n * sizeof(WCHAR));
+	      return PATH_SYMLINK | PATH_REP;
+	    }
+	  if (i == 2)
+	    break;
+	  buf += n + 1;
+	  size -= n + 1;
+	}
+    }
   else if (rp->ReparseTag == IO_REPARSE_TAG_LX_SYMLINK)
     {
       /* WSL symlink.  Problem: We have to convert the path to UTF-16 for
--
2.30.2

Re: [PATCH 1/2] Treat Windows Store's "app execution aliases" as symbolic links

[Joe Lowe]

I am skeptical about this patch (part 1), interposing appexec reparse 
point data as symlinks for cygwin applications.

The appexec reparse point data is essentially an extended attribute 
holding data that is used by CreateProcess(), more like a windows .lnk 
file or an X11 .desktop file, not like a posix symlink. M$ just chose an 
unnecessarily obtuse way to store the files data. This reminds me of old 
Macintosh zero length font files.

The useful function of the patch would seem to be as a way to display a 
portion of the data in shell directory listings for the user. I suggest 
this function is better provided by updated application code.


The patch part 2 seems entirely appropriate.


Joe L.


On 2021-03-12 07:11, Johannes Schindelin via Cygwin-patches wrote:
> When the Windows Store version of Python is installed, so-called "app
> execution aliases" are put into the `PATH`. These are reparse points
> under the hood, with an undocumented format.
> 
> We do know a bit about this format, though, as per the excellent analysis:
> https://www.tiraniddo.dev/2019/09/overview-of-windows-execution-aliases.html
> 
> 	The first 4 bytes is the reparse tag, in this case it's
> 	0x8000001B which is documented in the Windows SDK as
> 	IO_REPARSE_TAG_APPEXECLINK. Unfortunately there doesn't seem to
> 	be a corresponding structure, but with a bit of reverse
> 	engineering we can work out the format is as follows:
> 
> 	Version: <4 byte integer>
> 	Package ID: <NUL Terminated Unicode String>
> 	Entry Point: <NUL Terminated Unicode String>
> 	Executable: <NUL Terminated Unicode String>
> 	Application Type: <NUL Terminated Unicode String>
> 
> Let's treat them as symbolic links. For example, in this developer's
> setup, this will result in the following nice output:
> 
> 	$ cd $LOCALAPPDATA/Microsoft/WindowsApps/
> 
> 	$ ls -l python3.exe
> 	lrwxrwxrwx 1 me 4096 105 Aug 23  2020 python3.exe -> '/c/Program Files/WindowsApps/PythonSoftwareFoundation.Python.3.7_3.7.2544.0_x64__qbz5n2kfra8p0/python.exe'
> 
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>   winsup/cygwin/path.cc | 24 ++++++++++++++++++++++++
>   1 file changed, 24 insertions(+)
> 
> diff --git a/winsup/cygwin/path.cc b/winsup/cygwin/path.cc
> index f3b9913bd0..63f377efb1 100644
> --- a/winsup/cygwin/path.cc
> +++ b/winsup/cygwin/path.cc
> @@ -2538,6 +2538,30 @@ check_reparse_point_target (HANDLE h, bool remote, PREPARSE_DATA_BUFFER rp,
>         if (check_reparse_point_string (psymbuf))
>   	return PATH_SYMLINK | PATH_REP;
>       }
> +  else if (!remote && rp->ReparseTag == IO_REPARSE_TAG_APPEXECLINK)
> +    {
> +      /* App execution aliases are commonly used by Windows Store apps. */
> +      WCHAR *buf = (WCHAR *)(rp->GenericReparseBuffer.DataBuffer + 4);
> +      DWORD size = rp->ReparseDataLength / sizeof(WCHAR), n;
> +
> +      /*
> +         It seems that app execution aliases have a payload of four
> +	 NUL-separated wide string: package id, entry point, executable
> +	 and application type. We're interested in the executable. */
> +      for (int i = 0; i < 3 && size > 0; i++)
> +        {
> +	  n = wcsnlen (buf, size - 1);
> +	  if (i == 2 && n > 0 && n < size)
> +	    {
> +	      RtlInitCountedUnicodeString (psymbuf, buf, n * sizeof(WCHAR));
> +	      return PATH_SYMLINK | PATH_REP;
> +	    }
> +	  if (i == 2)
> +	    break;
> +	  buf += n + 1;
> +	  size -= n + 1;
> +	}
> +    }
>     else if (rp->ReparseTag == IO_REPARSE_TAG_LX_SYMLINK)
>       {
>         /* WSL symlink.  Problem: We have to convert the path to UTF-16 for
> --
> 2.30.2
> 

Re: [PATCH 1/2] Treat Windows Store's "app execution aliases" as symbolic links

[Johannes Schindelin]

Hi Joe,

On Fri, 12 Mar 2021, Joe Lowe wrote:

> I am skeptical about this patch (part 1), interposing appexec reparse point
> data as symlinks for cygwin applications.
>
> The appexec reparse point data is essentially an extended attribute holding
> data that is used by CreateProcess(), more like a windows .lnk file or an X11
> .desktop file, not like a posix symlink. M$ just chose an unnecessarily obtuse
> way to store the files data. This reminds me of old Macintosh zero length font
> files.

The obvious difference being that you cannot read those 0-length files.
And you _can_ determine the target from reading .lnk or .desktop files.

> The useful function of the patch would seem to be as a way to display a
> portion of the data in shell directory listings for the user. I suggest this
> function is better provided by updated application code.

I find your argument unconvincing.

For all practical purposes, users are likely to want to treat app
execution aliases as if they were symbolic links.

If users want to know more about the app execution alias than just the
path of the actual `.exe` (and that is a rather huge if), _then_ I would
buy your argument that it should be queried via application code.

But for the common case of reading the corresponding `.exe` or accessing
the path? Why should we follow your suggestion and keep making it really
hard for users to get to that information? I really don't get it.

Ciao,
Johannes

>
>
> The patch part 2 seems entirely appropriate.
>
>
> Joe L.
>
>
> On 2021-03-12 07:11, Johannes Schindelin via Cygwin-patches wrote:
> > When the Windows Store version of Python is installed, so-called "app
> > execution aliases" are put into the `PATH`. These are reparse points
> > under the hood, with an undocumented format.
> >
> > We do know a bit about this format, though, as per the excellent analysis:
> > https://www.tiraniddo.dev/2019/09/overview-of-windows-execution-aliases.html
> >
> >  The first 4 bytes is the reparse tag, in this case it's
> >  0x8000001B which is documented in the Windows SDK as
> >  IO_REPARSE_TAG_APPEXECLINK. Unfortunately there doesn't seem to
> >  be a corresponding structure, but with a bit of reverse
> >  engineering we can work out the format is as follows:
> >
> >  Version: <4 byte integer>
> >  Package ID: <NUL Terminated Unicode String>
> >  Entry Point: <NUL Terminated Unicode String>
> >  Executable: <NUL Terminated Unicode String>
> >  Application Type: <NUL Terminated Unicode String>
> >
> > Let's treat them as symbolic links. For example, in this developer's
> > setup, this will result in the following nice output:
> >
> >  $ cd $LOCALAPPDATA/Microsoft/WindowsApps/
> >
> >  $ ls -l python3.exe
> >  lrwxrwxrwx 1 me 4096 105 Aug 23  2020 python3.exe -> '/c/Program
> >  Files/WindowsApps/PythonSoftwareFoundation.Python.3.7_3.7.2544.0_x64__qbz5n2kfra8p0/python.exe'
> >
> > Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> > ---
> >   winsup/cygwin/path.cc | 24 ++++++++++++++++++++++++
> >   1 file changed, 24 insertions(+)
> >
> > diff --git a/winsup/cygwin/path.cc b/winsup/cygwin/path.cc
> > index f3b9913bd0..63f377efb1 100644
> > --- a/winsup/cygwin/path.cc
> > +++ b/winsup/cygwin/path.cc
> > @@ -2538,6 +2538,30 @@ check_reparse_point_target (HANDLE h, bool remote,
> > PREPARSE_DATA_BUFFER rp,
> >          if (check_reparse_point_string (psymbuf))
> >    return PATH_SYMLINK | PATH_REP;
> >       }
> > +  else if (!remote && rp->ReparseTag == IO_REPARSE_TAG_APPEXECLINK)
> > +    {
> > +      /* App execution aliases are commonly used by Windows Store apps. */
> > +      WCHAR *buf = (WCHAR *)(rp->GenericReparseBuffer.DataBuffer + 4);
> > +      DWORD size = rp->ReparseDataLength / sizeof(WCHAR), n;
> > +
> > +      /*
> > +         It seems that app execution aliases have a payload of four
> > +	 NUL-separated wide string: package id, entry point, executable
> > +	 and application type. We're interested in the executable. */
> > +      for (int i = 0; i < 3 && size > 0; i++)
> > +        {
> > +	  n = wcsnlen (buf, size - 1);
> > +	  if (i == 2 && n > 0 && n < size)
> > +	    {
> > +	      RtlInitCountedUnicodeString (psymbuf, buf, n * sizeof(WCHAR));
> > +	      return PATH_SYMLINK | PATH_REP;
> > +	    }
> > +	  if (i == 2)
> > +	    break;
> > +	  buf += n + 1;
> > +	  size -= n + 1;
> > +	}
> > +    }
> >     else if (rp->ReparseTag == IO_REPARSE_TAG_LX_SYMLINK)
> >       {
> >         /* WSL symlink.  Problem: We have to convert the path to UTF-16 for
> > --
> > 2.30.2
> >
>
>

Re: [PATCH 1/2] Treat Windows Store's "app execution aliases" as symbolic links

[Joe Lowe]

Hi Johannes,

I agree on the usefulness to the user of showing appexec target 
executable as symlink target. But I am uncertain about the effect on code.

One example: Any app that is able to archive/copy posix symlinks will 
convert the appexec to a symlink and silently drop the appexec data. 
Whether this is a significant issue depends on if most/all relevent 
store apps function the same when the executable is exec-ed directly vs 
via the appexec link.

Another example: Much code exists in the field that intentionally 
detects symlinks, dereferences, and works directly on the target. This 
may not be an issue, if most/all relevent store apps function the same 
when the executable is exec-ed directly vs via the appexec link.


Joe L.



On 3/13/2021 4:21 PM, Johannes Schindelin wrote:
> Hi Joe,
> 
> On Fri, 12 Mar 2021, Joe Lowe wrote:
> 
>> I am skeptical about this patch (part 1), interposing appexec reparse point
>> data as symlinks for cygwin applications.
>>
>> The appexec reparse point data is essentially an extended attribute holding
>> data that is used by CreateProcess(), more like a windows .lnk file or an X11
>> .desktop file, not like a posix symlink. M$ just chose an unnecessarily obtuse
>> way to store the files data. This reminds me of old Macintosh zero length font
>> files.
> 
> The obvious difference being that you cannot read those 0-length files.
> And you _can_ determine the target from reading .lnk or .desktop files.
> 
>> The useful function of the patch would seem to be as a way to display a
>> portion of the data in shell directory listings for the user. I suggest this
>> function is better provided by updated application code.
> 
> I find your argument unconvincing.
> 
> For all practical purposes, users are likely to want to treat app
> execution aliases as if they were symbolic links.
> 
> If users want to know more about the app execution alias than just the
> path of the actual `.exe` (and that is a rather huge if), _then_ I would
> buy your argument that it should be queried via application code.
> 
> But for the common case of reading the corresponding `.exe` or accessing
> the path? Why should we follow your suggestion and keep making it really
> hard for users to get to that information? I really don't get it.
> 
> Ciao,
> Johannes
> 
>>
>>
>> The patch part 2 seems entirely appropriate.
>>
>>
>> Joe L.
>>
>>
>> On 2021-03-12 07:11, Johannes Schindelin via Cygwin-patches wrote:
>>> When the Windows Store version of Python is installed, so-called "app
>>> execution aliases" are put into the `PATH`. These are reparse points
>>> under the hood, with an undocumented format.
>>>
>>> We do know a bit about this format, though, as per the excellent analysis:
>>> https://www.tiraniddo.dev/2019/09/overview-of-windows-execution-aliases.html
>>>
>>>   The first 4 bytes is the reparse tag, in this case it's
>>>   0x8000001B which is documented in the Windows SDK as
>>>   IO_REPARSE_TAG_APPEXECLINK. Unfortunately there doesn't seem to
>>>   be a corresponding structure, but with a bit of reverse
>>>   engineering we can work out the format is as follows:
>>>
>>>   Version: <4 byte integer>
>>>   Package ID: <NUL Terminated Unicode String>
>>>   Entry Point: <NUL Terminated Unicode String>
>>>   Executable: <NUL Terminated Unicode String>
>>>   Application Type: <NUL Terminated Unicode String>
>>>
>>> Let's treat them as symbolic links. For example, in this developer's
>>> setup, this will result in the following nice output:
>>>
>>>   $ cd $LOCALAPPDATA/Microsoft/WindowsApps/
>>>
>>>   $ ls -l python3.exe
>>>   lrwxrwxrwx 1 me 4096 105 Aug 23  2020 python3.exe -> '/c/Program
>>>   Files/WindowsApps/PythonSoftwareFoundation.Python.3.7_3.7.2544.0_x64__qbz5n2kfra8p0/python.exe'
>>>
>>> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
>>> ---
>>>    winsup/cygwin/path.cc | 24 ++++++++++++++++++++++++
>>>    1 file changed, 24 insertions(+)
>>>
>>> diff --git a/winsup/cygwin/path.cc b/winsup/cygwin/path.cc
>>> index f3b9913bd0..63f377efb1 100644
>>> --- a/winsup/cygwin/path.cc
>>> +++ b/winsup/cygwin/path.cc
>>> @@ -2538,6 +2538,30 @@ check_reparse_point_target (HANDLE h, bool remote,
>>> PREPARSE_DATA_BUFFER rp,
>>>           if (check_reparse_point_string (psymbuf))
>>>     return PATH_SYMLINK | PATH_REP;
>>>        }
>>> +  else if (!remote && rp->ReparseTag == IO_REPARSE_TAG_APPEXECLINK)
>>> +    {
>>> +      /* App execution aliases are commonly used by Windows Store apps. */
>>> +      WCHAR *buf = (WCHAR *)(rp->GenericReparseBuffer.DataBuffer + 4);
>>> +      DWORD size = rp->ReparseDataLength / sizeof(WCHAR), n;
>>> +
>>> +      /*
>>> +         It seems that app execution aliases have a payload of four
>>> +	 NUL-separated wide string: package id, entry point, executable
>>> +	 and application type. We're interested in the executable. */
>>> +      for (int i = 0; i < 3 && size > 0; i++)
>>> +        {
>>> +	  n = wcsnlen (buf, size - 1);
>>> +	  if (i == 2 && n > 0 && n < size)
>>> +	    {
>>> +	      RtlInitCountedUnicodeString (psymbuf, buf, n * sizeof(WCHAR));
>>> +	      return PATH_SYMLINK | PATH_REP;
>>> +	    }
>>> +	  if (i == 2)
>>> +	    break;
>>> +	  buf += n + 1;
>>> +	  size -= n + 1;
>>> +	}
>>> +    }
>>>      else if (rp->ReparseTag == IO_REPARSE_TAG_LX_SYMLINK)
>>>        {
>>>          /* WSL symlink.  Problem: We have to convert the path to UTF-16 for
>>> --
>>> 2.30.2
>>>
>>
>>

Re: [PATCH 1/2] Treat Windows Store's "app execution aliases" as symbolic links

[Johannes Schindelin]

Hi Joe,

On Sat, 13 Mar 2021, Joe Lowe wrote:

> I agree on the usefulness to the user of showing appexec target executable as
> symlink target. But I am uncertain about the effect on code.

Maybe. But I am concerned about the effect of not being able to do
anything useful with app execution aliases in the first place. I'd rather
have them represented as if they were symlinks (even if that is not 100%
accurate) than not being able to even answer the very simple (and
obvious!) question: where does this thing point to?

> One example: Any app that is able to archive/copy posix symlinks will convert
> the appexec to a symlink and silently drop the appexec data. Whether this is a
> significant issue depends on if most/all relevent store apps function the same
> when the executable is exec-ed directly vs via the appexec link.

I do have bad news for you: WSL symlinks are _already_ supported
(https://github.com/cygwin/cygwin/blob/d10d0d9b03f85/winsup/cygwin/path.cc#L2561-L2630),
and just like app execution aliases, they are not faithfully recreated.

Likewise junctions:
https://github.com/cygwin/cygwin/blob/d10d0d9b03f85/winsup/cygwin/path.cc#L2540-L2560

This suggests to me that the endeavor to have archive/copy programs based
on Cygwin's runtime work as you expect is doomed from the start.

> Another example: Much code exists in the field that intentionally detects
> symlinks, dereferences, and works directly on the target. This may not be an
> issue, if most/all relevent store apps function the same when the executable
> is exec-ed directly vs via the appexec link.

As far as I can tell, the only thing you _can_ do with an app execution
alias (apart from creating/deleting it) is to execute it. If you try to
read or write them, you get a "Permission denied".

Their intended use case seems to be the same as Debian Alternatives
(https://wiki.debian.org/DebianAlternatives) which _are_ implemented as
symbolic links.

So I still think that the best we can do with app execution aliases in
Cygwin is to display them as if they were plain old symbolic links.

Ciao,
Johannes

>
>
> Joe L.
>
>
>
> On 3/13/2021 4:21 PM, Johannes Schindelin wrote:
> > Hi Joe,
> >
> > On Fri, 12 Mar 2021, Joe Lowe wrote:
> >
> > > I am skeptical about this patch (part 1), interposing appexec reparse
> > > point
> > > data as symlinks for cygwin applications.
> > >
> > > The appexec reparse point data is essentially an extended attribute
> > > holding
> > > data that is used by CreateProcess(), more like a windows .lnk file or an
> > > X11
> > > .desktop file, not like a posix symlink. M$ just chose an unnecessarily
> > > obtuse
> > > way to store the files data. This reminds me of old Macintosh zero length
> > > font
> > > files.
> >
> > The obvious difference being that you cannot read those 0-length files.
> > And you _can_ determine the target from reading .lnk or .desktop files.
> >
> > > The useful function of the patch would seem to be as a way to display a
> > > portion of the data in shell directory listings for the user. I suggest
> > > this
> > > function is better provided by updated application code.
> >
> > I find your argument unconvincing.
> >
> > For all practical purposes, users are likely to want to treat app
> > execution aliases as if they were symbolic links.
> >
> > If users want to know more about the app execution alias than just the
> > path of the actual `.exe` (and that is a rather huge if), _then_ I would
> > buy your argument that it should be queried via application code.
> >
> > But for the common case of reading the corresponding `.exe` or accessing
> > the path? Why should we follow your suggestion and keep making it really
> > hard for users to get to that information? I really don't get it.
> >
> > Ciao,
> > Johannes
> >
> > >
> > >
> > > The patch part 2 seems entirely appropriate.
> > >
> > >
> > > Joe L.
> > >
> > >
> > > On 2021-03-12 07:11, Johannes Schindelin via Cygwin-patches wrote:
> > > > When the Windows Store version of Python is installed, so-called "app
> > > > execution aliases" are put into the `PATH`. These are reparse points
> > > > under the hood, with an undocumented format.
> > > >
> > > > We do know a bit about this format, though, as per the excellent
> > > > analysis:
> > > > https://www.tiraniddo.dev/2019/09/overview-of-windows-execution-aliases.html
> > > >
> > > >   The first 4 bytes is the reparse tag, in this case it's
> > > >   0x8000001B which is documented in the Windows SDK as
> > > >   IO_REPARSE_TAG_APPEXECLINK. Unfortunately there doesn't seem to
> > > >   be a corresponding structure, but with a bit of reverse
> > > >   engineering we can work out the format is as follows:
> > > >
> > > >   Version: <4 byte integer>
> > > >   Package ID: <NUL Terminated Unicode String>
> > > >   Entry Point: <NUL Terminated Unicode String>
> > > >   Executable: <NUL Terminated Unicode String>
> > > >   Application Type: <NUL Terminated Unicode String>
> > > >
> > > > Let's treat them as symbolic links. For example, in this developer's
> > > > setup, this will result in the following nice output:
> > > >
> > > >   $ cd $LOCALAPPDATA/Microsoft/WindowsApps/
> > > >
> > > >   $ ls -l python3.exe
> > > >   lrwxrwxrwx 1 me 4096 105 Aug 23  2020 python3.exe -> '/c/Program
> > > >   Files/WindowsApps/PythonSoftwareFoundation.Python.3.7_3.7.2544.0_x64__qbz5n2kfra8p0/python.exe'
> > > >
> > > > Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> > > > ---
> > > >    winsup/cygwin/path.cc | 24 ++++++++++++++++++++++++
> > > >    1 file changed, 24 insertions(+)
> > > >
> > > > diff --git a/winsup/cygwin/path.cc b/winsup/cygwin/path.cc
> > > > index f3b9913bd0..63f377efb1 100644
> > > > --- a/winsup/cygwin/path.cc
> > > > +++ b/winsup/cygwin/path.cc
> > > > @@ -2538,6 +2538,30 @@ check_reparse_point_target (HANDLE h, bool
> > > > remote,
> > > > PREPARSE_DATA_BUFFER rp,
> > > >           if (check_reparse_point_string (psymbuf))
> > > >     return PATH_SYMLINK | PATH_REP;
> > > >        }
> > > > +  else if (!remote && rp->ReparseTag == IO_REPARSE_TAG_APPEXECLINK)
> > > > +    {
> > > > +      /* App execution aliases are commonly used by Windows Store apps.
> > > > */
> > > > +      WCHAR *buf = (WCHAR *)(rp->GenericReparseBuffer.DataBuffer + 4);
> > > > +      DWORD size = rp->ReparseDataLength / sizeof(WCHAR), n;
> > > > +
> > > > +      /*
> > > > +         It seems that app execution aliases have a payload of four
> > > > +	 NUL-separated wide string: package id, entry point, executable
> > > > +	 and application type. We're interested in the executable. */
> > > > +      for (int i = 0; i < 3 && size > 0; i++)
> > > > +        {
> > > > +	  n = wcsnlen (buf, size - 1);
> > > > +	  if (i == 2 && n > 0 && n < size)
> > > > +	    {
> > > > +	      RtlInitCountedUnicodeString (psymbuf, buf, n * sizeof(WCHAR));
> > > > +	      return PATH_SYMLINK | PATH_REP;
> > > > +	    }
> > > > +	  if (i == 2)
> > > > +	    break;
> > > > +	  buf += n + 1;
> > > > +	  size -= n + 1;
> > > > +	}
> > > > +    }
> > > >      else if (rp->ReparseTag == IO_REPARSE_TAG_LX_SYMLINK)
> > > >        {
> > > >          /* WSL symlink.  Problem: We have to convert the path to UTF-16
> > > > for
> > > > --
> > > > 2.30.2
> > > >
> > >
> > >
>

Re: [PATCH 1/2] Treat Windows Store's "app execution aliases" as symbolic links

[Corinna Vinschen]

On Mar 13 19:41, Joe Lowe via Cygwin-patches wrote:
> Hi Johannes,
> 
> I agree on the usefulness to the user of showing appexec target executable
> as symlink target. But I am uncertain about the effect on code.
> 
> One example: Any app that is able to archive/copy posix symlinks will
> convert the appexec to a symlink and silently drop the appexec data. Whether
> this is a significant issue depends on if most/all relevent store apps
> function the same when the executable is exec-ed directly vs via the appexec
> link.

You won't get a sufficent, POSIX-like handling one way or the other.
Handling them as files will not allow correct archiving either, because
the reparse point data required to restore them correctly to work in
Windows won't be archived anyway.  There's only so much we can do in the
Windows environment.  ¯\_(ツ)_/¯

I'd just like to postpone this patch to get 3.2.0 out.  I'll add that
patch to the inevitable 3.2.1 release, given the massive testing of
3.2.0-0.1 on the Cygwin ML...


Thanks,
Corinna

Re: [PATCH 1/2] Treat Windows Store's "app execution aliases" as symbolic links

[Hans-Bernhard Bröker]

Am 15.03.2021 um 04:19 schrieb Johannes Schindelin via Cygwin-patches:
> Hi Joe,
> 
> On Sat, 13 Mar 2021, Joe Lowe wrote:
> 
>> I agree on the usefulness to the user of showing appexec target executable as
>> symlink target. But I am uncertain about the effect on code.
> 
> Maybe. But I am concerned about the effect of not being able to do
> anything useful with app execution aliases in the first place. 

That argument might hold more sway if Windows itself didn't quite so 
completely hide that information from users, too.

I found only one Windows native tool that will even show _any_ kind of 
information about these reparse points: fsutil.  That is a) only 
available as part of the highly optional WSL feature and b) only gives 
you a hexdump of the actual data, without any meaningful interpretation.

For something that Windows itself gives the "no user servicable parts 
inside" treatment to the extent it does for these reparse points, I 
rather doubt that Cygwin users really _need_ to see it.

Re: [PATCH 1/2] Treat Windows Store's "app execution aliases" as symbolic links

[Corinna Vinschen]

Hi Johannes,

I'm not opposed to treat these applinks as symlinks.  I have a
suggestion and a style nit, though.

On Mar 12 16:11, Johannes Schindelin via Cygwin-patches wrote:
> When the Windows Store version of Python is installed, so-called "app
> execution aliases" are put into the `PATH`. These are reparse points
> under the hood, with an undocumented format.
> 
> We do know a bit about this format, though, as per the excellent analysis:
> https://www.tiraniddo.dev/2019/09/overview-of-windows-execution-aliases.html
> 
> 	The first 4 bytes is the reparse tag, in this case it's
> 	0x8000001B which is documented in the Windows SDK as
> 	IO_REPARSE_TAG_APPEXECLINK. Unfortunately there doesn't seem to
> 	be a corresponding structure, but with a bit of reverse
> 	engineering we can work out the format is as follows:
> 
> 	Version: <4 byte integer>
> 	Package ID: <NUL Terminated Unicode String>
> 	Entry Point: <NUL Terminated Unicode String>
> 	Executable: <NUL Terminated Unicode String>
> 	Application Type: <NUL Terminated Unicode String>

Given we know this layout, what about introducing a matching struct,
like I did for REPARSE_LX_SYMLINK_BUFFER, for instructional purposes?

I. e.

typedef struct _REPARSE_APPEXECLINK_BUFFER
{
  DWORD ReparseTag;
  WORD  ReparseDataLength;
  WORD  Reserved;
  struct {
    DWORD Version;       /* Take member name with a grain of salt. */
    WCHAR Strings[1];	 /* Four serialized, NUL-terminated WCHAR strings:
			    - Package ID
			    - Entry Point
			    - Executable Path
			    - Application Type
			    We're only interested in the Executable Path */
  } AppExecLinkReparseBuffer;
} REPARSE_APPEXECLINK_BUFFER,*PREPARSE_APPEXECLINK_BUFFER;


> +  else if (!remote && rp->ReparseTag == IO_REPARSE_TAG_APPEXECLINK)
> +    {
> +      /* App execution aliases are commonly used by Windows Store apps. */
> +      WCHAR *buf = (WCHAR *)(rp->GenericReparseBuffer.DataBuffer + 4);

Analogue:

     PREPARSE_APPEXECLINK_BUFFER rpl = (PREPARSE_APPEXECLINK_BUFFER) rp;
     WCHAR *buf = rpl->AppExecLinkReparseBuffer.Strings;

Maybe use 'str' or 'strp' here, instead of buf?

> +      for (int i = 0; i < 3 && size > 0; i++)
> +        {
> +	  n = wcsnlen (buf, size - 1);
> +	  if (i == 2 && n > 0 && n < size)
> +	    {
> +	      RtlInitCountedUnicodeString (psymbuf, buf, n * sizeof(WCHAR));
                                                                  ^^^
                                                                  space


Thanks,
Corinna

Re: [PATCH 1/2] Treat Windows Store's "app execution aliases" as symbolic links

[Johannes Schindelin]

Hi Hans-Bernhard,

On Mon, 15 Mar 2021, Hans-Bernhard Bröker wrote:

> Am 15.03.2021 um 04:19 schrieb Johannes Schindelin via Cygwin-patches:
>
> > On Sat, 13 Mar 2021, Joe Lowe wrote:
> >
> > > I agree on the usefulness to the user of showing appexec target
> > > executable as symlink target. But I am uncertain about the effect on
> > > code.
> >
> > Maybe. But I am concerned about the effect of not being able to do
> > anything useful with app execution aliases in the first place.
>
> That argument might hold more sway if Windows itself didn't quite so
> completely hide that information from users, too.

"So completely"? It at least executes them, and it does offer you to turn
them aliases on and off (see
https://www.tenforums.com/tutorials/102096-manage-app-execution-aliases-windows-10-a.html)

Granted, the user interface has a lot of room for improvement, but if you
are dead set on finding out what, say, that `idle.exe` app execution alias
refers to, you can go to `Settings>Apps>Apps & features>App execution
aliases` and find out that it is owned by the Python 3.7 package. That
does not give you the path, but it does give you way more information than
you claimed Windows would offer to you.

> I found only one Windows native tool that will even show _any_ kind of
> information about these reparse points: fsutil.  That is a) only available as
> part of the highly optional WSL feature and b) only gives you a hexdump of the
> actual data, without any meaningful interpretation.

The `fsutil` program, contrary to your claim, is available without WSL:
https://docs.microsoft.com/en-us/windows-server/administration/windows-commands/fsutil

And yes, for under-documented reparse points, the tool gives you only a
hexdump.

One of those under-documented reparse point types is the WSL symbolic
link, which you will notice are supported in Cygwin, removing quite some
sway from your argument...

> For something that Windows itself gives the "no user servicable parts inside"
> treatment to the extent it does for these reparse points, I rather doubt that
> Cygwin users really _need_ to see it.

Well, that's funny: you are talking to one Cygwin user who needs to see
it. So I feel a bit ignored by you there.

Ciao,
Johannes

Re: [PATCH 1/2] Treat Windows Store's "app execution aliases" as symbolic links

[Hans-Bernhard Bröker]

Am 22.03.2021 um 16:22 schrieb Johannes Schindelin:
> On Mon, 15 Mar 2021, Hans-Bernhard Bröker wrote:
>> Am 15.03.2021 um 04:19 schrieb Johannes Schindelin via Cygwin-patches:

>> That argument might hold more sway if Windows itself didn't quite so
>> completely hide that information from users, too.

> "So completely"? It at least executes them, and it does offer you to turn
> them aliases on and off (see
> https://www.tenforums.com/tutorials/102096-manage-app-execution-aliases-windows-10-a.html)

That's a completely different piece of information than what you want 
Cygwin to show.

> Granted, the user interface has a lot of room for improvement, but if you
> are dead set on finding out what, say, that `idle.exe` app execution alias
> refers to, you can go to `Settings>Apps>Apps & features>App execution
> aliases` and find out that it is owned by the Python 3.7 package. 

Knowing which package that thing came from has essentially nothing to do 
with what its interpretation as a symlink would look like.  Apples and 
Oranges.

> The `fsutil` program, contrary to your claim, is available without WSL:
> https://docs.microsoft.com/en-us/windows-server/administration/windows-commands/fsutil

And that very page tells me, in a big "Notice" blurb:

> You must enable Windows Subsystem for Linux before you can run fsutil. Run the following command as Administrator in PowerShell to enable this optional feature:


> One of those under-documented reparse point types is the WSL symbolic
> link, which you will notice are supported in Cygwin, removing quite some
> sway from your argument...

I notice no such thing right now, running the currently available 
release version 3.1.7:

stat: cannot stat '//wsl$/Debian/home/hbbro/link_to_a': Input/output error

[other commands that want to show more than just the name behave 
equivalently]

Links made by WSL directly on a Windows filesystem are understood by 
Cygwin.  But that's because WSL uses Windows symlinks in that case.

Microsoft could almost certainly just have used a symlink to implement 
this rather trivial feature.  But for some reason they apparently didn't 
care to explain anywhere, they chose to wildly overcomplicate it, 
inventing a completly type of reparse point.  So for what it's worth, 
that thing _is_not_a_symlink_.  Pretending it is one is bound to cause 
more problems than it solves.

> Well, that's funny: you are talking to one Cygwin user who needs to see
> it. So I feel a bit ignored by you there.

All conclusions based on a single example are wrong.

Re: [PATCH 1/2] Treat Windows Store's "app execution aliases" as symbolic links

[Corinna Vinschen]

On Mar 22 22:54, Hans-Bernhard Bröker wrote:
> Am 22.03.2021 um 16:22 schrieb Johannes Schindelin:
> > One of those under-documented reparse point types is the WSL symbolic
> > link, which you will notice are supported in Cygwin, removing quite some
> > sway from your argument...
> 
> I notice no such thing right now, running the currently available release
> version 3.1.7:
> 
> stat: cannot stat '//wsl$/Debian/home/hbbro/link_to_a': Input/output error

What type of WSL symlink is that?  Cygwin reads WSL 1 symlinks(*) and
creates them by default in symlink(2) cals since Cygwin 3.1.5.

(*) IO_REPARSE_TAG_LX_SYMLINK 0xa000001d


Corinna

Re: [PATCH 1/2] Treat Windows Store's "app execution aliases" as symbolic links

[Hans-Bernhard Bröker]

Am 23.03.2021 um 10:30 schrieb Corinna Vinschen via Cygwin-patches:
 > On Mar 22 22:54, Hans-Bernhard Bröker wrote:
 >> Am 22.03.2021 um 16:22 schrieb Johannes Schindelin:
 >>> One of those under-documented reparse point types is the WSL symbolic
 >>> link, which you will notice are supported in Cygwin, removing quite 
some
 >>> sway from your argument...
 >>
 >> I notice no such thing right now, running the currently available 
release
 >> version 3.1.7:
 >>
 >> stat: cannot stat '//wsl$/Debian/home/hbbro/link_to_a': Input/output 
error
 >
 > What type of WSL symlink is that?

It's what WSL Debian creates when I 'ln -s' inside its own filesystem.

Windows' own "dir" command shows it as

22.03.2021  22:34    <JUNCTION>     link_to_a [...]

But it cannot do anything else with it.  Even fsutil doesn't work on 
that thing:

C:\prg\test>fsutil reparsePoint query \\wsl$\Debian\home\hbbro
Fehler:  Unzulässige Funktion.

Re: [PATCH 1/2] Treat Windows Store's "app execution aliases" as symbolic links

[Ken Brown]

On 3/24/2021 2:55 PM, Hans-Bernhard Bröker wrote:
> Am 23.03.2021 um 10:30 schrieb Corinna Vinschen via Cygwin-patches:
>  > On Mar 22 22:54, Hans-Bernhard Bröker wrote:
>  >> Am 22.03.2021 um 16:22 schrieb Johannes Schindelin:
>  >>> One of those under-documented reparse point types is the WSL symbolic
>  >>> link, which you will notice are supported in Cygwin, removing quite some
>  >>> sway from your argument...
>  >>
>  >> I notice no such thing right now, running the currently available release
>  >> version 3.1.7:
>  >>
>  >> stat: cannot stat '//wsl$/Debian/home/hbbro/link_to_a': Input/output error
>  >
>  > What type of WSL symlink is that?
> 
> It's what WSL Debian creates when I 'ln -s' inside its own filesystem.
> 
> Windows' own "dir" command shows it as
> 
> 22.03.2021  22:34    <JUNCTION>     link_to_a [...]
> 
> But it cannot do anything else with it.  Even fsutil doesn't work on that thing:
> 
> C:\prg\test>fsutil reparsePoint query \\wsl$\Debian\home\hbbro
> Fehler:  Unzulässige Funktion.

Are you running WSL1 or WSL2?  I have WSL1, and the stat command such as the one 
you tried fails in the same way as yours.  Nevertheless, a symlink created under 
WSL is indeed recognized as such by Cygwin.  I verified this as follows:

1. Within WSL,

$ ln -s foo mysymlink
$ cp -a mysymlink /mnt/c/cygwin64/tmp

2. Within Cygwin,

$ stat /tmp/mysymlink
   File: /tmp/mysymlink -> foo
   Size: 3               Blocks: 0          IO Block: 65536  symbolic link
Device: 74d6767bh/1960212091d   Inode: 25614222880728371  Links: 1
Access: (0777/lrwxrwxrwx)  Uid: (197609/  kbrown)   Gid: (197121/    None)
Access: 2021-03-24 16:25:50.729219700 -0400
Modify: 2021-03-24 16:25:50.729219700 -0400
Change: 2021-03-24 16:27:13.979376200 -0400
  Birth: 2021-03-24 16:27:13.979376200 -0400

3. I then ran the stat command under gdb with a breakpoint at 
check_reparse_point_target and verified that Cygwin recognized /tmp/mysymlink as 
a WSL symlink (IO_REPARSE_TAG_LX_SYMLINK).

Someone with WSL2 should try a similar experiment to make sure that the symlink 
representation as a reparse point hasn't changed.

As to the failure of the stat command that you tried, I suspect it is related to 
the '\\wsl$' magic rather than anything to do with the symlink itself.  If you 
run that stat command under strace, you'll see that Cygwin calls NtCreateFile 
(\??\UNC\wsl$\...), which succeeds, and then calls 
NtFsControlFile(FSCTL_GET_REPARSE_POINT), which fails with STATUS_NOT_IMPLEMENTED.

Ken

Re: [PATCH 1/2] Treat Windows Store's "app execution aliases" as symbolic links

[Hans-Bernhard Bröker]

Am 24.03.2021 um 21:58 schrieb Ken Brown:
> On 3/24/2021 2:55 PM, Hans-Bernhard Bröker wrote:
>> Am 23.03.2021 um 10:30 schrieb Corinna Vinschen via Cygwin-patches:
>>  > On Mar 22 22:54, Hans-Bernhard Bröker wrote:
>>  >> Am 22.03.2021 um 16:22 schrieb Johannes Schindelin:

>> It's what WSL Debian creates when I 'ln -s' inside its own filesystem.
>>
>> Windows' own "dir" command shows it as
>>
>> 22.03.2021  22:34    <JUNCTION>     link_to_a [...]
>>
>> But it cannot do anything else with it.  Even fsutil doesn't work on 
>> that thing:
>>
>> C:\prg\test>fsutil reparsePoint query \\wsl$\Debian\home\hbbro
>> Fehler:  Unzulässige Funktion.
> 
> Are you running WSL1 or WSL2?  

To the best of my knowledge it's WSL1.

> I have WSL1, and the stat command such as 
> the one you tried fails in the same way as yours.  Nevertheless, a 
> symlink created under WSL is indeed recognized as such by Cygwin.  I 
> verified this as follows:
> 
> 1. Within WSL,
> 
> $ ln -s foo mysymlink
> $ cp -a mysymlink /mnt/c/cygwin64/tmp

Here it gets a bit weird.  The result of that procedure depends on 
whether the link target, 'foo', exists in cygwin64/tmp prior to running 
the above commands.

If 'foo' exists, the copy of the symlink becomes a Windows native 
symlink (reparse point class 0xa000000c).  If it doesn't the copy turns 
into a reparse point of class 0xa000001d, which 'fsutil reparsepoint 
query' decodes as "name replacement", Cygwin as a (broken) symlink, and 
'dir' lists as <JUNCTION>.  In other words, a WSL symlink.

It's quite strange that copying a native WSL1 symlink from inside WSL's 
own file system out into the Windows side of things does _not_ always 
yield an identical copy.  Some layer sitting between WSL and the Windows 
file system may modify the copy in flight.

The same difference applies if, instead of copying an existing symlink, 
you just have WSL create one directly in the Windows tree, i.e.

	cd /mnt/c/cygwin64/tmp
	rm a b
	touch a
	ln -s a la
	ln -s b lb
	touch b

yields a Windows symlink for 'la', and a WSL symlink for 'lb'.