3.5.x regression: misquoting command line arguments from native processes

3.5.x regression: misquoting command line arguments from native processes

[David Allsopp]

I have what appears to be a regression in Cygwin 3.5.0 which, owing to
a CI system lagging behind, we've only just discovered.

In order to torture our Unicode support, OCaml's Windows CI compiles
its sources in C:\projects\🐫реализация-mingw64 (that's a directory
under C:\projects with the camel emoji U+1F42A followed, I am told, by
the word "implementation" in Russian then with the name of the "port"
of OCaml appended in ASCII).

AppVeyor updated to Cygwin 3.5.1 at the weekend and now, during the
build, we end up with the following kind of error seen in, say,
https://ci.appveyor.com/project/avsm/ocaml/builds/49538801#L711. What
is happening here is that Cygwin's make has passed
C:\projects\🐫реализация-mingw64\byte\bin\flexdll_mingw64.o to a
non-Cygwin program (flexlink.exe) which has called back to Cygwin's
x86_64-w64-mingw32-gcc ultimately via CreateProcess and which has then
failed.

I have reduced this chain to the following program, which takes
argv[1] and calls Cygwin's stat program using _wsystem with argv[1]
"-quoted:

#include <stdlib.h>
#include <string.h>

int wmain(int argc, wchar_t **argv) {
  int len = 2 + 6 + wcslen(argv[1]);
  wchar_t *cmd = (wchar_t *)malloc(len * sizeof(wchar_t));
  wcscpy(cmd, L"stat ");
  cmd[5] = L'"';
  cmd[6] = 0;
  wcscat(cmd, argv[1]);
  cmd[len - 1] = L'"';
  cmd[len] = 0;
  _wsystem(cmd);
}

Compile with x86_64-w64-mingw32-gcc -municode -DUNICODE -D_UNICODE and
then see it works on Cygwin 3.4.10:

$ ./t.exe 'C:\Devel\🐫реализация-mingw64\flexdll\flexdll_mingw64.o'
  File: C:\Devel\🐫реализация-mingw64\flexdll\flexdll_mingw64.o
  Size: 11441           Blocks: 12         IO Block: 65536  regular file
Device: 35525,39577     Inode: 3659174698707368  Links: 1
Access: (0644/-rw-r--r--)  Uid: (1049719/     DRA)   Gid: (1049089/Domain Users)
Access: 2024-04-03 16:43:36.782037800 +0100
Modify: 2024-04-03 14:09:08.012955600 +0100
Change: 2024-04-03 14:09:08.012955600 +0100
 Birth: 2024-04-03 14:09:08.006955500 +0100

but fails on 3.5.1+ (including the latest 3.6.0-0.109 snapshot):

$ ./t.exe 'C:\Devel\🐫реализация-mingw64\flexdll\flexdll_mingw64.o'
stat: cannot stat
'"C:\Devel\'$'\360\237\220\253''реализация-mingw64\flexdll\flexdll_mingw64.o':
No such file or directory

Note the stray double-quote at the start of the error. This appears to
be down to the presence of the emoji, rather than the Cyrillic
characters, this is fine in both:

$ ./t.exe 'C:\Devel\реализация\flexdll\flexdll_mingw64.o'
  File: C:\Devel\реализация\flexdll\flexdll_mingw64.o
  Size: 11441           Blocks: 12         IO Block: 65536  regular file
Device: 35525,39577     Inode: 16607023627376791  Links: 1
Access: (0644/-rw-r--r--)  Uid: (1049719/     DRA)   Gid: (1049089/Domain Users)
Access: 2024-04-03 16:48:43.003742000 +0100
Modify: 2024-04-03 14:09:08.012955600 +0100
Change: 2024-04-03 16:32:35.648039600 +0100
 Birth: 2024-04-03 16:32:35.647539400 +0100

All best,


David

Re: 3.5.x regression: misquoting command line arguments from native processes

[Corinna Vinschen]

Hi David,

On Apr  3 16:53, David Allsopp via Cygwin wrote:
> I have what appears to be a regression in Cygwin 3.5.0 which, owing to
> a CI system lagging behind, we've only just discovered.
> [...]
> $ ./t.exe 'C:\Devel\🐫реализация-mingw64\flexdll\flexdll_mingw64.o'
> stat: cannot stat
> '"C:\Devel\'$'\360\237\220\253''реализация-mingw64\flexdll\flexdll_mingw64.o':
> No such file or directory

Thanks a lot for the STC!

I think I fixed that for 3.5.4.  I pushed a patch and the test release
cygwin-3.6.0-0.115.g579064bf4d40 is just building and should be ready
for testing in an hour or two.

Please give it a try.


Thanks,
Corinna

Re: 3.5.x regression: misquoting command line arguments from native processes

[Corinna Vinschen]

Hi David,

On Apr  9 22:38, Corinna Vinschen via Cygwin wrote:
> On Apr  3 16:53, David Allsopp via Cygwin wrote:
> > I have what appears to be a regression in Cygwin 3.5.0 which, owing to
> > a CI system lagging behind, we've only just discovered.
> > [...]
> > $ ./t.exe 'C:\Devel\🐫реализация-mingw64\flexdll\flexdll_mingw64.o'
> > stat: cannot stat
> > '"C:\Devel\'$'\360\237\220\253''реализация-mingw64\flexdll\flexdll_mingw64.o':
> > No such file or directory
> 
> Thanks a lot for the STC!
> 
> I think I fixed that for 3.5.4.  I pushed a patch and the test release
> cygwin-3.6.0-0.115.g579064bf4d40 is just building and should be ready
> for testing in an hour or two.
> 
> Please give it a try.

Sorry for nagging, but do you have some feedback, be it bad or good?


Thanks,
Corinna

Re: 3.5.x regression: misquoting command line arguments from native processes

[David Allsopp]

Hi Corinna,

> On Apr  9 22:38, Corinna Vinschen via Cygwin wrote:
> > On Apr  3 16:53, David Allsopp via Cygwin wrote:
> > > I have what appears to be a regression in Cygwin 3.5.0 which, owing to
> > > a CI system lagging behind, we've only just discovered.
> > > [...]
> > > $ ./t.exe 'C:\Devel\🐫реализация-mingw64\flexdll\flexdll_mingw64.o'
> > > stat: cannot stat
> > > '"C:\Devel\'$'\360\237\220\253''реализация-mingw64\flexdll\flexdll_mingw64.o':
> > > No such file or directory
> >
> > Thanks a lot for the STC!
> >
> > I think I fixed that for 3.5.4.  I pushed a patch and the test release
> > cygwin-3.6.0-0.115.g579064bf4d40 is just building and should be ready
> > for testing in an hour or two.
> >
> > Please give it a try.
>
> Sorry for nagging, but do you have some feedback, be it bad or good?

Sorry for having needed nagging! It does indeed fix it, thank you -
our smoke-test Bactrian camels can be restored 🐫

All best,


David

Re: 3.5.x regression: misquoting command line arguments from native processes

[Corinna Vinschen]

Hi David,

On Apr 20 08:43, David Allsopp via Cygwin wrote:
> Hi Corinna,
> 
> > On Apr  9 22:38, Corinna Vinschen via Cygwin wrote:
> > > On Apr  3 16:53, David Allsopp via Cygwin wrote:
> > > > I have what appears to be a regression in Cygwin 3.5.0 which, owing to
> > > > a CI system lagging behind, we've only just discovered.
> > > > [...]
> > > > $ ./t.exe 'C:\Devel\🐫реализация-mingw64\flexdll\flexdll_mingw64.o'
> > > > stat: cannot stat
> > > > '"C:\Devel\'$'\360\237\220\253''реализация-mingw64\flexdll\flexdll_mingw64.o':
> > > > No such file or directory
> > >
> > > Thanks a lot for the STC!
> > >
> > > I think I fixed that for 3.5.4.  I pushed a patch and the test release
> > > cygwin-3.6.0-0.115.g579064bf4d40 is just building and should be ready
> > > for testing in an hour or two.
> > >
> > > Please give it a try.
> >
> > Sorry for nagging, but do you have some feedback, be it bad or good?
> 
> Sorry for having needed nagging! It does indeed fix it, thank you -
> our smoke-test Bactrian camels can be restored 🐫

Great, nice to read this. I'll keep my camel testcase around, just in
case :)


Thanks,
Corinna