Arguments parsing in cygwin executables

Arguments parsing in cygwin executables

[Sasha Unknown]

Hello.

==Preamble==
Some time ago I noticed that cygwin executables (e.g. bash, tar, echo
& so on) handle specially *, {} and some other symbols in
command-line, even when being invoked not from shell (e.g.
programmatic invocation or cmd.exe). After some googling, I found
CYGWIN=noglob setting, which fixed the issue. After enabling it I
hoped that cygwin executables will start parsing command-line in a
standard for Windows executables way (I am not talking about path
translation, only about handling special characters and splitting
command string into argv array).

==Main==
It revealed that even with CYGWIN=noglob, cygwin executables parse
command line differently from other windows executables. (Again, I
underline: I'm talking about invocation from cmd.exe or programmatic
invocation, not invocation from bash.) Concretely, the 3rd and 4th
test-cases from here fail:
https://msdn.microsoft.com/en-us/library/windows/desktop/17w5ykft(v=vs.85).aspx.
(BTW, ironically, with CYGWIN=glob only 3rd test-case fails.)

==Questions==
So, questions are:
1. Is this behavior intentional, or is it bug?
1a. If intentional: Maybe there is a way to force cygwin executables
to perform command-line parsing in windows-canonical way (i.e.
CommandLineToArgv-like)? (I am talking about splitting command string
into argv array, not about path translation.)
2. In any case, could you point me to part of cygwin sources which is
responsible for this? (In case of intentionality -- to understand what
behavior I'm now forced to adopt to, in case of bug -- to possibly aid
fixing.)

==Test cases==
1. I compiled code from the link above with g++.
2. And written simple simple script in bash:
> #!/bin/bash
> for a in "$@"; do echo "$((++i)): $a"; done
Both have same behavior, which differs from canonical windows parsing.

==Additional==
What I'm trying to do?
I am trying to find proper way of passing parameters to cygwin
executables from non-cygwin application (which doesn't even know that
it's dealing with cywgin).
I suppose that if one wants *nix-typical parsing, he will use some
shell (e.g. bash), or at least CYGWIN=glob.

BTW, in CYGWIN=glob mode, curly braces are handled wrongly
(c:\cygwin\echo.exe {aa} should return {aa}, not aa; because no , or
.. inside {}).

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Arguments parsing in cygwin executables

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 3759 bytes --]

Hi Sasha,

On Jun 17 20:56, Sasha Unknown wrote:
> Hello.
> 
> ==Preamble==
> Some time ago I noticed that cygwin executables (e.g. bash, tar, echo
> & so on) handle specially *, {} and some other symbols in
> command-line, even when being invoked not from shell (e.g.
> programmatic invocation or cmd.exe). After some googling, I found
> CYGWIN=noglob setting, which fixed the issue. After enabling it I
> hoped that cygwin executables will start parsing command-line in a
> standard for Windows executables way (I am not talking about path
> translation, only about handling special characters and splitting
> command string into argv array).
> 
> ==Main==
> It revealed that even with CYGWIN=noglob, cygwin executables parse
> command line differently from other windows executables. (Again, I
> underline: I'm talking about invocation from cmd.exe or programmatic
> invocation, not invocation from bash.) Concretely, the 3rd and 4th
> test-cases from here fail:
> https://msdn.microsoft.com/en-us/library/windows/desktop/17w5ykft(v=vs.85).aspx.
> (BTW, ironically, with CYGWIN=glob only 3rd test-case fails.)

Uhm...I just tried it myself and independently of the CYGWIN=glob
setting only the 3rd test case fails for me.  Test case 4 works fine.

> ==Questions==
> So, questions are:
> 1. Is this behavior intentional, or is it bug?

The differences in argv processing when called from a non-Cygwin parent
process have been discussed a couple of times in the last years, but I
don't think there's a consensus if that's a bug or a feature.  The 
function hasn't seen any noticable changes since the year 2000, though,
and any behavioral change *might* introduce backward incompatibilities
with existing scripts...

> 1a. If intentional: Maybe there is a way to force cygwin executables
> to perform command-line parsing in windows-canonical way (i.e.
> CommandLineToArgv-like)? (I am talking about splitting command string
> into argv array, not about path translation.)
> 2. In any case, could you point me to part of cygwin sources which is
> responsible for this? (In case of intentionality -- to understand what
> behavior I'm now forced to adopt to, in case of bug -- to possibly aid
> fixing.)

https://sourceware.org/git/gitweb.cgi?p=newlib-cygwin.git;a=blob;f=winsup/cygwin/dcrt0.cc;h=738860d0ea92ef575755b9e1752a5c53c3ccaa97;hb=HEAD

Look for function build_argv.

If you're willing to take a stab at it to fix the aforementioned
test case 3, I'm willing to include it.  As for how to contribute,
see https://cygwin.com/contrib.html

Just two points:

- If the patch changes more than 10 lines, we need a copyright
  assignment.  See https://cygwin.com/contrib.html, there's a standard
  copyright assignment form you can simply send as signed PDF by mail to
  the address given therein.

- Please make sure to implement it so that we can switch back to the old
  behaviour by checking some global bool variable ("bool old_argv" or so).
  I'll then help adding the required code to allow doing that via the
  CYGWIN environment variable for the (hopefully) rare cases which
  require the old behaviour.

> BTW, in CYGWIN=glob mode, curly braces are handled wrongly
> (c:\cygwin\echo.exe {aa} should return {aa}, not aa; because no , or
> .. inside {}).

I don't think so.  GLOB_BRACE globbing is meant to do brance globbing
just as csh does.  This doesn't require a comma or anything else within
the braces.  Try this in tcsh or even bash.  The underlying code is
(almost) stock FreeBSD code, btw.


Thanks,
Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: Arguments parsing in cygwin executables

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 4211 bytes --]

On Jun 19 10:44, Corinna Vinschen wrote:
> Hi Sasha,
> 
> On Jun 17 20:56, Sasha Unknown wrote:
> > Hello.
> > 
> > ==Preamble==
> > Some time ago I noticed that cygwin executables (e.g. bash, tar, echo
> > & so on) handle specially *, {} and some other symbols in
> > command-line, even when being invoked not from shell (e.g.
> > programmatic invocation or cmd.exe). After some googling, I found
> > CYGWIN=noglob setting, which fixed the issue. After enabling it I
> > hoped that cygwin executables will start parsing command-line in a
> > standard for Windows executables way (I am not talking about path
> > translation, only about handling special characters and splitting
> > command string into argv array).
> > 
> > ==Main==
> > It revealed that even with CYGWIN=noglob, cygwin executables parse
> > command line differently from other windows executables. (Again, I
> > underline: I'm talking about invocation from cmd.exe or programmatic
> > invocation, not invocation from bash.) Concretely, the 3rd and 4th
> > test-cases from here fail:
> > https://msdn.microsoft.com/en-us/library/windows/desktop/17w5ykft(v=vs.85).aspx.
> > (BTW, ironically, with CYGWIN=glob only 3rd test-case fails.)
> 
> Uhm...I just tried it myself and independently of the CYGWIN=glob
> setting only the 3rd test case fails for me.  Test case 4 works fine.
> 
> > ==Questions==
> > So, questions are:
> > 1. Is this behavior intentional, or is it bug?
> 
> The differences in argv processing when called from a non-Cygwin parent
> process have been discussed a couple of times in the last years, but I
> don't think there's a consensus if that's a bug or a feature.  The 
> function hasn't seen any noticable changes since the year 2000, though,
> and any behavioral change *might* introduce backward incompatibilities
> with existing scripts...
> 
> > 1a. If intentional: Maybe there is a way to force cygwin executables
> > to perform command-line parsing in windows-canonical way (i.e.
> > CommandLineToArgv-like)? (I am talking about splitting command string
> > into argv array, not about path translation.)
> > 2. In any case, could you point me to part of cygwin sources which is
> > responsible for this? (In case of intentionality -- to understand what
> > behavior I'm now forced to adopt to, in case of bug -- to possibly aid
> > fixing.)
> 
> https://sourceware.org/git/gitweb.cgi?p=newlib-cygwin.git;a=blob;f=winsup/cygwin/dcrt0.cc;h=738860d0ea92ef575755b9e1752a5c53c3ccaa97;hb=HEAD
> 
> Look for function build_argv.
> 
> If you're willing to take a stab at it to fix the aforementioned
> test case 3, I'm willing to include it.  As for how to contribute,
> see https://cygwin.com/contrib.html
> 
> Just two points:
> 
> - If the patch changes more than 10 lines, we need a copyright
>   assignment.  See https://cygwin.com/contrib.html, there's a standard
>   copyright assignment form you can simply send as signed PDF by mail to
>   the address given therein.
> 
> - Please make sure to implement it so that we can switch back to the old
>   behaviour by checking some global bool variable ("bool old_argv" or so).
>   I'll then help adding the required code to allow doing that via the
>   CYGWIN environment variable for the (hopefully) rare cases which
>   require the old behaviour.
> 
> > BTW, in CYGWIN=glob mode, curly braces are handled wrongly
> > (c:\cygwin\echo.exe {aa} should return {aa}, not aa; because no , or
> > .. inside {}).
> 
> I don't think so.  GLOB_BRACE globbing is meant to do brance globbing
> just as csh does.  This doesn't require a comma or anything else within
> the braces.  Try this in tcsh or even bash.  The underlying code is

No, not in bash.  I see what you mean.  Bash handles that differently
than [t]csh.  However, csh is the obvious role model for this behaviour,
so if I had to choose who's right and who's wrong, csh (and thus the
glob original BSD code) is the clear winner.

> (almost) stock FreeBSD code, btw.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: Arguments parsing in cygwin executables

[Andrey Repin]

Greetings, Corinna Vinschen!

>> > BTW, in CYGWIN=glob mode, curly braces are handled wrongly
>> > (c:\cygwin\echo.exe {aa} should return {aa}, not aa; because no , or
>> > .. inside {}).
>> 
>> I don't think so.  GLOB_BRACE globbing is meant to do brance globbing
>> just as csh does.  This doesn't require a comma or anything else within
>> the braces.  Try this in tcsh or even bash.  The underlying code is

> No, not in bash.  I see what you mean.  Bash handles that differently
> than [t]csh.  However, csh is the obvious role model for this behaviour,
> so if I had to choose who's right and who's wrong, csh (and thus the
> glob original BSD code) is the clear winner.

I would also argue that bash handling is wrong.
But it is historical, and unlikely to change, I guess.

>> (almost) stock FreeBSD code, btw.


-- 
With best regards,
Andrey Repin
Friday, June 19, 2015 13:32:45

Sorry for my terrible english...


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Arguments parsing in cygwin executables

[Sasha Unknown]

Hello.

[Sorry if duplicate, previous message was sent from wrong address, so
not sure if it reached.]

==Preamble==
Some time ago I noticed that cygwin executables (e.g. bash, tar, echo
& so on) handle specially *, {} and some other symbols in
command-line, even when being invoked not from shell (e.g.
programmatic invocation or cmd.exe). After some googling, I found
CYGWIN=noglob setting, which fixed the issue. After enabling it I
hoped that cygwin executables will start parsing command-line in a
standard for Windows executables way (I am not talking about path
translation, only about handling special characters and splitting
command string into argv array).

==Main==
It revealed that even with CYGWIN=noglob, cygwin executables parse
command line differently from other windows executables. (Again, I
underline: I'm talking about invocation from cmd.exe or programmatic
invocation, not invocation from bash.) Concretely, the 3rd and 4th
test-cases from here fail:
https://msdn.microsoft.com/en-us/library/windows/desktop/17w5ykft(v=vs.85).aspx.
(BTW, ironically, with CYGWIN=glob only 3rd test-case fails.)

==Questions==
So, questions are:
1. Is this behavior intentional, or is it bug?
1a. If intentional: Maybe there is a way to force cygwin executables
to perform command-line parsing in windows-canonical way (i.e.
CommandLineToArgv-like)? (I am talking about splitting command string
into argv array, not about path translation.)
2. In any case, could you point me to part of cygwin sources which is
responsible for this? (In case of intentionality -- to understand what
behavior I'm now forced to adopt to, in case of bug -- to possibly aid
fixing.)

==Test cases==
1. I compiled code from the link above with g++.
2. And written simple simple script in bash:
> #!/bin/bash
> for a in "$@"; do echo "$((++i)): $a"; done
(to be called like this: c:\cygwin\bin\bash.exe c:\script.sh args-to-test)
Both have same behavior, which differs from canonical windows parsing.

==Additional==
What I'm trying to do?
I am trying to find proper way of passing parameters to cygwin
executables from non-cygwin application (which doesn't even know that
it's dealing with cywgin).
I suppose that if one wants *nix-typical parsing, he will use some
shell (e.g. bash), or at least CYGWIN=glob.

BTW, in CYGWIN=glob mode, curly braces are handled wrongly
(c:\cygwin\echo.exe {aa} should return {aa}, not aa; because no , or
.. inside {}).

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple