[PATCH 0/2] Handle "app execution aliases"

[PATCH 0/2] Handle "app execution aliases"

[Johannes Schindelin]

When installing e.g. Python via the Windows Store, it is common that the
`python3.exe` entry in the `PATH` is not actually an executable at all,
but an "app executaion alias" (i.e. a special class of reparse point).

These filesystem entries are presented as 0-size files, but they are not
readable, which is why Cygwin has problems to execute them, with the error
message "Permission denied".

This issue has been reported a couple of times in the Git for Windows and
in the MSYS2 project, and even in Cygwin
(https://cygwin.com/pipermail/cygwin/2020-May/244969.html, the thread
devolved into a discussion about Thunderbird vs Outlook before long,
though).

The second patch fixes that, and for good measure, the first patch teaches
Cygwin to treat these reparse points as symbolic links.

Johannes Schindelin (2):
  Treat Windows Store's "app execution aliases" as symbolic links
  Allow executing Windows Store's "app execution aliases"

 winsup/cygwin/path.cc  | 24 ++++++++++++++++++++++++
 winsup/cygwin/spawn.cc |  7 +++++++
 2 files changed, 31 insertions(+)

--
2.30.2

[PATCH v2 0/2] Handle "app execution aliases"

[Johannes Schindelin]

When installing e.g. Python via the Windows Store, it is common that the
`python3.exe` entry in the `PATH` is not actually an executable at all,
but an "app executaion alias" (i.e. a special class of reparse point).

These filesystem entries are presented as 0-size files, but they are not
readable, which is why Cygwin has problems to execute them, with the error
message "Permission denied".

This issue has been reported a couple of times in the Git for Windows and
in the MSYS2 project, and even in Cygwin
(https://cygwin.com/pipermail/cygwin/2020-May/244969.html, the thread
devolved into a discussion about Thunderbird vs Outlook before long,
though).

The second patch fixes that, and for good measure, the first patch teaches
Cygwin to treat these reparse points as symbolic links.

Changes since v1:

- Introduce and use `struct _REPARSE_APPEXECLINK_BUFFER`.

Johannes Schindelin (2):
  Treat Windows Store's "app execution aliases" as symbolic links
  Allow executing Windows Store's "app execution aliases"

 winsup/cygwin/path.cc  | 40 ++++++++++++++++++++++++++++++++++++++++
 winsup/cygwin/spawn.cc |  7 +++++++
 2 files changed, 47 insertions(+)

Range-diff against v1:
1:  218dc58e36 ! 1:  529cb4ad54 Treat Windows Store's "app execution aliases" as symbolic links
    @@ Commit message
         Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>

      ## winsup/cygwin/path.cc ##
    +@@ winsup/cygwin/path.cc: symlink_info::check_sysfile (HANDLE h)
    +   return res;
    + }
    +
    ++typedef struct _REPARSE_APPEXECLINK_BUFFER
    ++{
    ++  DWORD ReparseTag;
    ++  WORD  ReparseDataLength;
    ++  WORD  Reserved;
    ++  struct {
    ++    DWORD Version;       /* Take member name with a grain of salt. */
    ++    WCHAR Strings[1];    /* Four serialized, NUL-terminated WCHAR strings:
    ++			   - Package ID
    ++			   - Entry Point
    ++			   - Executable Path
    ++			   - Application Type
    ++			   We're only interested in the Executable Path */
    ++  } AppExecLinkReparseBuffer;
    ++} REPARSE_APPEXECLINK_BUFFER,*PREPARSE_APPEXECLINK_BUFFER;
    ++
    + static bool
    + check_reparse_point_string (PUNICODE_STRING subst)
    + {
     @@ winsup/cygwin/path.cc: check_reparse_point_target (HANDLE h, bool remote, PREPARSE_DATA_BUFFER rp,
            if (check_reparse_point_string (psymbuf))
      	return PATH_SYMLINK | PATH_REP;
    @@ winsup/cygwin/path.cc: check_reparse_point_target (HANDLE h, bool remote, PREPAR
     +  else if (!remote && rp->ReparseTag == IO_REPARSE_TAG_APPEXECLINK)
     +    {
     +      /* App execution aliases are commonly used by Windows Store apps. */
    -+      WCHAR *buf = (WCHAR *)(rp->GenericReparseBuffer.DataBuffer + 4);
    -+      DWORD size = rp->ReparseDataLength / sizeof(WCHAR), n;
    ++      PREPARSE_APPEXECLINK_BUFFER rpl = (PREPARSE_APPEXECLINK_BUFFER) rp;
    ++      WCHAR *buf = rpl->Strings;
    ++      DWORD size = rp->ReparseDataLength / sizeof (WCHAR), n;
     +
    -+      /*
    -+         It seems that app execution aliases have a payload of four
    ++      /* It seems that app execution aliases have a payload of four
     +	 NUL-separated wide string: package id, entry point, executable
     +	 and application type. We're interested in the executable. */
     +      for (int i = 0; i < 3 && size > 0; i++)
    -+        {
    ++	{
     +	  n = wcsnlen (buf, size - 1);
     +	  if (i == 2 && n > 0 && n < size)
     +	    {
    -+	      RtlInitCountedUnicodeString (psymbuf, buf, n * sizeof(WCHAR));
    ++	      RtlInitCountedUnicodeString (psymbuf, buf, n * sizeof (WCHAR));
     +	      return PATH_SYMLINK | PATH_REP;
     +	    }
     +	  if (i == 2)
2:  647dff4c7a = 2:  1c2659f902 Allow executing Windows Store's "app execution aliases"
--
2.31.0

[PATCH v2 1/2] Treat Windows Store's "app execution aliases" as symbolic links

[Johannes Schindelin]

When the Windows Store version of Python is installed, so-called "app
execution aliases" are put into the `PATH`. These are reparse points
under the hood, with an undocumented format.

We do know a bit about this format, though, as per the excellent analysis:
https://www.tiraniddo.dev/2019/09/overview-of-windows-execution-aliases.html

	The first 4 bytes is the reparse tag, in this case it's
	0x8000001B which is documented in the Windows SDK as
	IO_REPARSE_TAG_APPEXECLINK. Unfortunately there doesn't seem to
	be a corresponding structure, but with a bit of reverse
	engineering we can work out the format is as follows:

	Version: <4 byte integer>
	Package ID: <NUL Terminated Unicode String>
	Entry Point: <NUL Terminated Unicode String>
	Executable: <NUL Terminated Unicode String>
	Application Type: <NUL Terminated Unicode String>

Let's treat them as symbolic links. For example, in this developer's
setup, this will result in the following nice output:

	$ cd $LOCALAPPDATA/Microsoft/WindowsApps/

	$ ls -l python3.exe
	lrwxrwxrwx 1 me 4096 105 Aug 23  2020 python3.exe -> '/c/Program Files/WindowsApps/PythonSoftwareFoundation.Python.3.7_3.7.2544.0_x64__qbz5n2kfra8p0/python.exe'

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 winsup/cygwin/path.cc | 40 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 40 insertions(+)

diff --git a/winsup/cygwin/path.cc b/winsup/cygwin/path.cc
index f3b9913bd0..56834963a2 100644
--- a/winsup/cygwin/path.cc
+++ b/winsup/cygwin/path.cc
@@ -2439,6 +2439,22 @@ symlink_info::check_sysfile (HANDLE h)
   return res;
 }

+typedef struct _REPARSE_APPEXECLINK_BUFFER
+{
+  DWORD ReparseTag;
+  WORD  ReparseDataLength;
+  WORD  Reserved;
+  struct {
+    DWORD Version;       /* Take member name with a grain of salt. */
+    WCHAR Strings[1];    /* Four serialized, NUL-terminated WCHAR strings:
+			   - Package ID
+			   - Entry Point
+			   - Executable Path
+			   - Application Type
+			   We're only interested in the Executable Path */
+  } AppExecLinkReparseBuffer;
+} REPARSE_APPEXECLINK_BUFFER,*PREPARSE_APPEXECLINK_BUFFER;
+
 static bool
 check_reparse_point_string (PUNICODE_STRING subst)
 {
@@ -2538,6 +2554,30 @@ check_reparse_point_target (HANDLE h, bool remote, PREPARSE_DATA_BUFFER rp,
       if (check_reparse_point_string (psymbuf))
 	return PATH_SYMLINK | PATH_REP;
     }
+  else if (!remote && rp->ReparseTag == IO_REPARSE_TAG_APPEXECLINK)
+    {
+      /* App execution aliases are commonly used by Windows Store apps. */
+      PREPARSE_APPEXECLINK_BUFFER rpl = (PREPARSE_APPEXECLINK_BUFFER) rp;
+      WCHAR *buf = rpl->Strings;
+      DWORD size = rp->ReparseDataLength / sizeof (WCHAR), n;
+
+      /* It seems that app execution aliases have a payload of four
+	 NUL-separated wide string: package id, entry point, executable
+	 and application type. We're interested in the executable. */
+      for (int i = 0; i < 3 && size > 0; i++)
+	{
+	  n = wcsnlen (buf, size - 1);
+	  if (i == 2 && n > 0 && n < size)
+	    {
+	      RtlInitCountedUnicodeString (psymbuf, buf, n * sizeof (WCHAR));
+	      return PATH_SYMLINK | PATH_REP;
+	    }
+	  if (i == 2)
+	    break;
+	  buf += n + 1;
+	  size -= n + 1;
+	}
+    }
   else if (rp->ReparseTag == IO_REPARSE_TAG_LX_SYMLINK)
     {
       /* WSL symlink.  Problem: We have to convert the path to UTF-16 for
--
2.31.0

[PATCH v2 2/2] Allow executing Windows Store's "app execution aliases"

[Johannes Schindelin]

The Windows Store version of Python (and apparently other Windows Store
applications) install a special reparse point called "app execution
alias" into the user's `PATH`.

These applications can be executed without any problem, but they cannot
be read as if they were files. This trips up Cygwin's beautiful logic that
tries to determine whether we're about to execute a Cygwin executable or
not: instead of executing the application, it will fail, saying
"Permission denied".

Let's detect this situation (`NtOpenFile()` helpfully says that this
operation is not supported on this reparse point type), and simply skip
the logic: Windows Store apps are not Cygwin executables (and even if
they were, it is unlikely that they would come with a compatible
`cygwin1.dll` or `msys-2.0.dll`).

This fixes https://github.com/msys2/MSYS2-packages/issues/1943

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 winsup/cygwin/spawn.cc | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/winsup/cygwin/spawn.cc b/winsup/cygwin/spawn.cc
index af177c0f13..ea08f3662e 100644
--- a/winsup/cygwin/spawn.cc
+++ b/winsup/cygwin/spawn.cc
@@ -1166,6 +1166,13 @@ av::setup (const char *prog_arg, path_conv& real_path, const char *ext,
 			     FILE_SYNCHRONOUS_IO_NONALERT
 			     | FILE_OPEN_FOR_BACKUP_INTENT
 			     | FILE_NON_DIRECTORY_FILE);
+	if (status == STATUS_IO_REPARSE_TAG_NOT_HANDLED)
+	  {
+	    /* This is most likely an app execution alias (such as the
+	       Windows Store version of Python, i.e. not a Cygwin program */
+	    real_path.set_cygexec (false);
+	    break;
+	  }
 	if (!NT_SUCCESS (status))
 	  {
 	    /* File is not readable?  Doesn't mean it's not executable.
--
2.31.0

Re: [PATCH v2 0/2] Handle "app execution aliases"

[Corinna Vinschen]

On Mar 22 16:51, Johannes Schindelin via Cygwin-patches wrote:
> When installing e.g. Python via the Windows Store, it is common that the
> `python3.exe` entry in the `PATH` is not actually an executable at all,
> but an "app executaion alias" (i.e. a special class of reparse point).
> 
> These filesystem entries are presented as 0-size files, but they are not
> readable, which is why Cygwin has problems to execute them, with the error
> message "Permission denied".
> 
> This issue has been reported a couple of times in the Git for Windows and
> in the MSYS2 project, and even in Cygwin
> (https://cygwin.com/pipermail/cygwin/2020-May/244969.html, the thread
> devolved into a discussion about Thunderbird vs Outlook before long,
> though).
> 
> The second patch fixes that, and for good measure, the first patch teaches
> Cygwin to treat these reparse points as symbolic links.
> 
> Changes since v1:
> 
> - Introduce and use `struct _REPARSE_APPEXECLINK_BUFFER`.
> 
> Johannes Schindelin (2):
>   Treat Windows Store's "app execution aliases" as symbolic links
>   Allow executing Windows Store's "app execution aliases"
> 
>  winsup/cygwin/path.cc  | 40 ++++++++++++++++++++++++++++++++++++++++
>  winsup/cygwin/spawn.cc |  7 +++++++
>  2 files changed, 47 insertions(+)

I decided to apply this now, while we're still discussing the osf handle
problem.

Pushed with two fixes.  I prepended "Cygwin:" to the git log subject and
I patched this compile time problem:

  path.cc: In function ‘int check_reparse_point_target(HANDLE, bool, PREPARSE_DATA_BUFFER, PUNICODE_STRING)’:
  path.cc:2581:25: error: ‘struct _REPARSE_APPEXECLINK_BUFFER’ has no member named ‘Strings’
   2581 |       WCHAR *buf = rpl->Strings;
	|                         ^~~~~~~

I also added this to the release notes.


Thanks,
Corinna

Re: [PATCH v2 0/2] Handle "app execution aliases"

[Johannes Schindelin]

Hi Corinna,

On Tue, 23 Mar 2021, Corinna Vinschen wrote:

> On Mar 22 16:51, Johannes Schindelin via Cygwin-patches wrote:
> > When installing e.g. Python via the Windows Store, it is common that the
> > `python3.exe` entry in the `PATH` is not actually an executable at all,
> > but an "app executaion alias" (i.e. a special class of reparse point).
> >
> > These filesystem entries are presented as 0-size files, but they are not
> > readable, which is why Cygwin has problems to execute them, with the error
> > message "Permission denied".
> >
> > This issue has been reported a couple of times in the Git for Windows and
> > in the MSYS2 project, and even in Cygwin
> > (https://cygwin.com/pipermail/cygwin/2020-May/244969.html, the thread
> > devolved into a discussion about Thunderbird vs Outlook before long,
> > though).
> >
> > The second patch fixes that, and for good measure, the first patch teaches
> > Cygwin to treat these reparse points as symbolic links.
> >
> > Changes since v1:
> >
> > - Introduce and use `struct _REPARSE_APPEXECLINK_BUFFER`.
> >
> > Johannes Schindelin (2):
> >   Treat Windows Store's "app execution aliases" as symbolic links
> >   Allow executing Windows Store's "app execution aliases"
> >
> >  winsup/cygwin/path.cc  | 40 ++++++++++++++++++++++++++++++++++++++++
> >  winsup/cygwin/spawn.cc |  7 +++++++
> >  2 files changed, 47 insertions(+)
>
> I decided to apply this now, while we're still discussing the osf handle
> problem.
>
> Pushed with two fixes.  I prepended "Cygwin:" to the git log subject and
> I patched this compile time problem:
>
>   path.cc: In function ‘int check_reparse_point_target(HANDLE, bool, PREPARSE_DATA_BUFFER, PUNICODE_STRING)’:
>   path.cc:2581:25: error: ‘struct _REPARSE_APPEXECLINK_BUFFER’ has no member named ‘Strings’
>    2581 |       WCHAR *buf = rpl->Strings;
> 	|                         ^~~~~~~

Uh oh. Sorry for the breakage. I thought I had test-compiled it...
Apparently I didn't (or I ignored a warning or something).
>
> I also added this to the release notes.

Thank you!

Ciao,
Johannes