Process argument passing problems

Process argument passing problems

[reiter.christoph]

Hey everyone,

I have to following problem (using cmd.exe):

C:\cygwin64\home\xy>C:\cygwin64\bin\echo.exe -DFOO=\"BAR\" -DBAR=\"FOO\"
-DFOO=\BAR" -DBAR="FOO"

I would have expected:

-DFOO="BAR" -DBAR="FOO"

Context: I'm executing various cygwin programs through (native) Python and
passing C macros to gcc like 'gcc -DFOO=\"BAR\"' fails because of this.

More background: https://github.com/msys2/MINGW-packages/issues/3548


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Process argument passing problems

[Hans-Bernhard Bröker]

Am 21.06.2019 um 17:19 schrieb reiter.christoph@gmail.com:
> Hey everyone,
> 
> I have to following problem (using cmd.exe):
> 
> C:\cygwin64\home\xy>C:\cygwin64\bin\echo.exe -DFOO=\"BAR\" -DBAR=\"FOO\"
> -DFOO=\BAR" -DBAR="FOO"
> 
> I would have expected:
> 
> -DFOO="BAR" -DBAR="FOO"

Expecting cmd.exe to parse command lines in anything resembling a sane
way is, unfortunately, an exercise in futility.

Microsoft allowed blanks in filenames with hardly any regard for the
concepts of command lines and their parsing.  We're still fighting with
the repercussions now, 25 years later.

> Context: I'm executing various cygwin programs through (native) Python and
> passing C macros to gcc like 'gcc -DFOO=\"BAR\"' fails because of this.

Are you sure Python even goes through cmd.exe to pass those command
lines?  If it doesn't, figuring out what cmd.exe does may be even less
fruitful than it already is.

> More background: https://github.com/msys2/MINGW-packages/issues/3548

Please note that that's about MSYS2, which, while closely related to
Cygwin, is not really the topic of this mailing list.

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Process argument passing problems

[Brian Inglis]

On 2019-06-21 14:15, Hans-Bernhard Bröker wrote:
> Am 21.06.2019 um 17:19 schrieb reiter.christoph@gmail.com:
>> I have to following problem (using cmd.exe):
>> C:\cygwin64\home\xy>C:\cygwin64\bin\echo.exe -DFOO=\"BAR\" -DBAR=\"FOO\"
>> -DFOO=\BAR" -DBAR="FOO"
>> I would have expected:
>> -DFOO="BAR" -DBAR="FOO"
> Expecting cmd.exe to parse command lines in anything resembling a sane
> way is, unfortunately, an exercise in futility.
> Microsoft allowed blanks in filenames with hardly any regard for the
> concepts of command lines and their parsing.  We're still fighting with
> the repercussions now, 25 years later.
>> Context: I'm executing various cygwin programs through (native) Python and
>> passing C macros to gcc like 'gcc -DFOO=\"BAR\"' fails because of this.
> Are you sure Python even goes through cmd.exe to pass those command
> lines?  If it doesn't, figuring out what cmd.exe does may be even less
> fruitful than it already is.
>> More background: https://github.com/msys2/MINGW-packages/issues/3548
> Please note that that's about MSYS2, which, while closely related to
> Cygwin, is not really the topic of this mailing list.

Not only are you running from cmd, you are running Cygwin /bin/echo, which does
additional command line parsing when not invoked from Cygwin.
Please note that the cmd escape char is "^" caret/uparrow not backslash "\" so
you might have more luck trying e.g.
	> \cygwin64\bin\echo ^"-DFOO=\"BAR\" -DBAR=\"FOO\"
	-DFOO="BAR" -DBAR="FOO"
or
	> \cygwin64\bin\echo ^"-DFOO=\^"BAR\^" -DBAR=\^"FOO\^"
	-DFOO="BAR" -DBAR="FOO"
depending on what you are trying to pass from where to what.

All quotes and backslashes are stripped by shells while building the calling
argv array, so you should only need a backslash or quotes to pass args with
spaces from cmd to Cygwin programs, unless programs require quotes around option
values, which gcc shouldn't.

You will have to read up on the details of MSYS2 and Python command line parsing
and how they pass command line arguments to WinExec(), ProcessCreate(), or
ShellExecute() to run programs.

Using Cygwin shells and python would make this a whole lot easier!

-- 
Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada

This email may be disturbing to some readers as it contains
too much technical detail. Reader discretion is advised.

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Process argument passing problems

[reiter.christoph]

On Fri, 2019-06-21 at 22:15 +0200, Hans-Bernhard Bröker wrote:
> Am 21.06.2019 um 17:19 schrieb reiter.christoph@gmail.com:
> > Hey everyone,
> > 
> > I have to following problem (using cmd.exe):
> > 
> > C:\cygwin64\home\xy>C:\cygwin64\bin\echo.exe -DFOO=\"BAR\" -DBAR=\"FOO\"
> > -DFOO=\BAR" -DBAR="FOO"
> > 
> > I would have expected:
> > 
> > -DFOO="BAR" -DBAR="FOO"
> 
> Expecting cmd.exe to parse command lines in anything resembling a sane
> way is, unfortunately, an exercise in futility.
> 
> Microsoft allowed blanks in filenames with hardly any regard for the
> concepts of command lines and their parsing.  We're still fighting with
> the repercussions now, 25 years later.

I'm actually using ProcessCreate() and calling cygwin gcc. The echo.exe example
seemed to show the exact same problem which is why I thought it might be easier
to reproduce for others.

> > Context: I'm executing various cygwin programs through (native) Python and
> > passing C macros to gcc like 'gcc -DFOO=\"BAR\"' fails because of this.
> 
> Are you sure Python even goes through cmd.exe to pass those command
> lines?  If it doesn't, figuring out what cmd.exe does may be even less
> fruitful than it already is.

Not, it doesn't use cmd.exe. It uses subprocess.list2cmdline to convert the
argument list to something suitable for ProcessCreate() and then calls
ProcessCreate.

>>> subprocess.list2cmdline(['-DFOO="BAR"'])
'-DFOO=\\"BAR\\"'
...

> > More background: https://github.com/msys2/MINGW-packages/issues/3548
> 
> Please note that that's about MSYS2, which, while closely related to
> Cygwin, is not really the topic of this mailing list.

Feel free to ignore it if doesn't help. The bug shows how I came to the
conclusion that it might be a cygwin bug, which made me post here.

Thanks!


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Process argument passing problems

[reiter.christoph]

On Fri, 2019-06-21 at 15:13 -0600, Brian Inglis wrote:
> [...]
> Using Cygwin shells and python would make this a whole lot easier!

Thanks for all the extra info! I'll try to come up with a better example not
involving cmd.exe :)


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Process argument passing problems

[reiter.christoph]

On Fri, 2019-06-21 at 17:19 +0200, reiter.christoph@gmail.com wrote:
> Hey everyone,
> 
> I have to following problem (using cmd.exe):
> 

I've now converted the example to a C program (compiles with mingw and msvc):

https://gist.github.com/lazka/3d828f9ed3bdf6b981eeb5edca65c657

The output is included. In case I missed any potential problems, please tell me!


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple