[PATCH] Fix parameter passing containing quote/equal to Windows batch command

[PATCH] Fix parameter passing containing quote/equal to Windows batch command

[Daniel Dai]

We notice one issue when running a Windows batch command inside
cygwin. Here is one example.

Simple batch file:
a.bat:
echo %1

Run it under cygwin:
./a.bat a=b
a

./a.bat "a=b"
a

If we pass additional \"
./a.bat "\"a=b\""
"\"a

There seems no way to pass a=b into bat.

Attach quote.patch contains a fix. It does two things:
1. If the parameter contains a equal sign, automatically add quote
(similar to space, tab, new line, quote cygwin already do)
2. If the parameter is already quoted, don't quote again

Patch:
Index: cygwin/winf.cc
==============================
=====================================
RCS file: /cvs/src/src/winsup/cygwin/winf.cc,v
retrieving revision 1.10
diff -u -p -r1.10 winf.cc
--- cygwin/winf.cc      19 Jun 2013 16:00:43 -0000      1.10
+++ cygwin/winf.cc      20 Jan 2014 00:50:15 -0000
@@ -72,11 +72,16 @@ linebuf::fromargv (av& newargv, const ch
     {
       char *p = NULL;
       const char *a;
+      boolean enclosed_with_quote = false;

       a = i ? newargv[i] : (char *) real_path;
       int len = strlen (a);
-      if (len != 0 && !strpbrk (a, " \t\n\r\""))
-       add (a, len);
+      if (len != 0 && a[0] == '\"' && a[len-1] == '\"') {
+        enclosed_with_quote = true;
+      }
+      if (enclosed_with_quote || (len != 0 && !strpbrk (a, " \t\n\r\"="))) {
+        add (a, len);
+      }
       else
        {
          add ("\"", 1);

Thanks,
Daniel

Re: [PATCH] Fix parameter passing containing quote/equal to Windows batch command

[Max Polk]

On 1/20/2014 1:02 AM, Daniel Dai wrote:
> We notice one issue when running a Windows batch command inside
> cygwin. Here is one example.
>
> Simple batch file:
> a.bat:
> echo %1
>
> Run it under cygwin:
> ./a.bat a=b
> a
>
> ./a.bat "a=b"
> a
>
> If we pass additional \"
> ./a.bat "\"a=b\""
> "\"a
>
> There seems no way to pass a=b into bat.

This is how batch files work, and likely not a problem with Cygwin:
http://support.microsoft.com/kb/35938
Excerpt: "it is not possible to include an equal sign as an argument to 
a batch file"

Be careful to note that cmd.exe and .bat files naturally split a=b into 
two arguments and strip out the equals sign:

(Run from cmd.exe)
C:\>Argecho.bat a=b
FIRST a
SECOND b
THIRD

I did notice that adding double quotes (in cmd.exe) will make will it 
arrive as one argument, and note that the double quotes are still there:

(Run from cmd.exe)
C:\>Argecho.bat "a=b"
FIRST "a=b"
SECOND
THIRD

There is a problem getting Cygwin the above test case, however.

The test script was:
C:\>type Argecho.bat
@echo off
echo FIRST %1
echo SECOND %2
echo THIRD %3

When run from Cygwin bash, and you force the double quotes by 
surrounding double quotes "a=b" with single quotes '"a=b"', you seem to 
get too *many* quotes in the batch file:

(Run from bash, the batch file behaves correctly as if run from cmd.exe)
$ Argecho.bat a=b
FIRST a
SECOND b
THIRD

(Run from bash, same as above since bash removes the double quotes prior 
to passing to program):
$ Argecho.bat "a=b"
FIRST a
SECOND b
THIRD

(Run from bash, this is what is surprising double surrounded with single)
$ Argecho.bat '"a=b"'
FIRST "\"a
SECOND b\""
THIRD

It seems that only the final test case above doesn't behave as expected.

Re: [PATCH] Fix parameter passing containing quote/equal to Windows batch command

[Daniel Dai]

Max,
Thanks for your reply.

Yes, the double quote issue can be reproducible from command line, but
not the equal sign.

Let's demonstrate the equal sign issue with a C program:

#include "unistd.h"

int main(int argc, char** argv) {
    execl("a.bat", "a.bat", "a=b");
    return 0;
}

The intention of the program is to pass "a=b" as a single argument.
However, compile and run it under cygwin, I get:
$ cc -o myprog myprog.c
$ ./myprog
a

I run the program with Visual Studio, I get "a=b".

The way Windows/Unix handles parameter containing equal sign is
different. IMHO, Cygwin should be the place to fill this semantic gap.

Thanks,
Daniel



On Mon, Jan 20, 2014 at 4:30 PM, Max Polk <maxpolk@gmail.com> wrote:
> On 1/20/2014 1:02 AM, Daniel Dai wrote:
>>
>> We notice one issue when running a Windows batch command inside
>> cygwin. Here is one example.
>>
>> Simple batch file:
>> a.bat:
>> echo %1
>>
>> Run it under cygwin:
>> ./a.bat a=b
>> a
>>
>> ./a.bat "a=b"
>> a
>>
>> If we pass additional \"
>> ./a.bat "\"a=b\""
>> "\"a
>>
>> There seems no way to pass a=b into bat.
>
>
> This is how batch files work, and likely not a problem with Cygwin:
> http://support.microsoft.com/kb/35938
> Excerpt: "it is not possible to include an equal sign as an argument to a
> batch file"
>
> Be careful to note that cmd.exe and .bat files naturally split a=b into two
> arguments and strip out the equals sign:
>
> (Run from cmd.exe)
> C:\>Argecho.bat a=b
> FIRST a
> SECOND b
> THIRD
>
> I did notice that adding double quotes (in cmd.exe) will make will it arrive
> as one argument, and note that the double quotes are still there:
>
> (Run from cmd.exe)
> C:\>Argecho.bat "a=b"
> FIRST "a=b"
> SECOND
> THIRD
>
> There is a problem getting Cygwin the above test case, however.
>
> The test script was:
> C:\>type Argecho.bat
> @echo off
> echo FIRST %1
> echo SECOND %2
> echo THIRD %3
>
> When run from Cygwin bash, and you force the double quotes by surrounding
> double quotes "a=b" with single quotes '"a=b"', you seem to get too *many*
> quotes in the batch file:
>
> (Run from bash, the batch file behaves correctly as if run from cmd.exe)
> $ Argecho.bat a=b
> FIRST a
> SECOND b
> THIRD
>
> (Run from bash, same as above since bash removes the double quotes prior to
> passing to program):
> $ Argecho.bat "a=b"
> FIRST a
> SECOND b
> THIRD
>
> (Run from bash, this is what is surprising double surrounded with single)
> $ Argecho.bat '"a=b"'
> FIRST "\"a
> SECOND b\""
> THIRD
>
> It seems that only the final test case above doesn't behave as expected.

Re: [PATCH] Fix parameter passing containing quote/equal to Windows batch command

[Václav Zeman]

On 21 January 2014 01:30, Max Polk wrote:
> On 1/20/2014 1:02 AM, Daniel Dai wrote:
>>
>> We notice one issue when running a Windows batch command inside
>> cygwin. Here is one example.
>>
>> Simple batch file:
>> a.bat:
>> echo %1
>>
>> Run it under cygwin:
>> ./a.bat a=b
>> a
>>
>> ./a.bat "a=b"
>> a
>>
>> If we pass additional \"
>> ./a.bat "\"a=b\""
>> "\"a
>>
>> There seems no way to pass a=b into bat.
>
>
> This is how batch files work, and likely not a problem with Cygwin:
> http://support.microsoft.com/kb/35938
> Excerpt: "it is not possible to include an equal sign as an argument to a
> batch file"
>
> Be careful to note that cmd.exe and .bat files naturally split a=b into two
> arguments and strip out the equals sign:
>
> (Run from cmd.exe)
> C:\>Argecho.bat a=b
> FIRST a
> SECOND b
> THIRD
>
> I did notice that adding double quotes (in cmd.exe) will make will it arrive
> as one argument, and note that the double quotes are still there:
>
> (Run from cmd.exe)
> C:\>Argecho.bat "a=b"
> FIRST "a=b"
> SECOND
> THIRD
>
> There is a problem getting Cygwin the above test case, however.
>
> The test script was:
> C:\>type Argecho.bat
> @echo off
> echo FIRST %1
> echo SECOND %2
> echo THIRD %3
>
> When run from Cygwin bash, and you force the double quotes by surrounding
> double quotes "a=b" with single quotes '"a=b"', you seem to get too *many*
> quotes in the batch file:
>
> (Run from bash, the batch file behaves correctly as if run from cmd.exe)
> $ Argecho.bat a=b
> FIRST a
> SECOND b
> THIRD
>
> (Run from bash, same as above since bash removes the double quotes prior to
> passing to program):
> $ Argecho.bat "a=b"
> FIRST a
> SECOND b
> THIRD
>
> (Run from bash, this is what is surprising double surrounded with single)
> $ Argecho.bat '"a=b"'
> FIRST "\"a
> SECOND b\""
> THIRD
>
> It seems that only the final test case above doesn't behave as expected.

Beware! The way CMD.exe handles command line and parameters is
different/incompatible from the way C (MSVCRT) application handles
them. Do not confuse the two. You cannot prove/show anything about how
C application handles arguments by using CMD.exe as a show case.

-- 
VZ

Re: [PATCH] Fix parameter passing containing quote/equal to Windows batch command

[Daniel Dai]

Thanks Václav!

You are right, I tried on exe file, things works fine on exe, but not
bat file. Seems some cleanup is done before we running into main
function of exe.

So how do we want to fix bat? The rules in my patch seems apply even for exe:
1. if parameter is already quoted, don't quote again
2. if parameter contains equal, quote it

It probably duplicate some logic with exe bootstrap code, but I don't
see any harm.

Another way to fix is to apply the rule only we see a bat file. That
should solve my problem as well.

Thanks,
Daniel





On Tue, Jan 21, 2014 at 4:31 AM, Václav Zeman <vhaisman@gmail.com> wrote:
> On 21 January 2014 01:30, Max Polk wrote:
>> On 1/20/2014 1:02 AM, Daniel Dai wrote:
>>>
>>> We notice one issue when running a Windows batch command inside
>>> cygwin. Here is one example.
>>>
>>> Simple batch file:
>>> a.bat:
>>> echo %1
>>>
>>> Run it under cygwin:
>>> ./a.bat a=b
>>> a
>>>
>>> ./a.bat "a=b"
>>> a
>>>
>>> If we pass additional \"
>>> ./a.bat "\"a=b\""
>>> "\"a
>>>
>>> There seems no way to pass a=b into bat.
>>
>>
>> This is how batch files work, and likely not a problem with Cygwin:
>> http://support.microsoft.com/kb/35938
>> Excerpt: "it is not possible to include an equal sign as an argument to a
>> batch file"
>>
>> Be careful to note that cmd.exe and .bat files naturally split a=b into two
>> arguments and strip out the equals sign:
>>
>> (Run from cmd.exe)
>> C:\>Argecho.bat a=b
>> FIRST a
>> SECOND b
>> THIRD
>>
>> I did notice that adding double quotes (in cmd.exe) will make will it arrive
>> as one argument, and note that the double quotes are still there:
>>
>> (Run from cmd.exe)
>> C:\>Argecho.bat "a=b"
>> FIRST "a=b"
>> SECOND
>> THIRD
>>
>> There is a problem getting Cygwin the above test case, however.
>>
>> The test script was:
>> C:\>type Argecho.bat
>> @echo off
>> echo FIRST %1
>> echo SECOND %2
>> echo THIRD %3
>>
>> When run from Cygwin bash, and you force the double quotes by surrounding
>> double quotes "a=b" with single quotes '"a=b"', you seem to get too *many*
>> quotes in the batch file:
>>
>> (Run from bash, the batch file behaves correctly as if run from cmd.exe)
>> $ Argecho.bat a=b
>> FIRST a
>> SECOND b
>> THIRD
>>
>> (Run from bash, same as above since bash removes the double quotes prior to
>> passing to program):
>> $ Argecho.bat "a=b"
>> FIRST a
>> SECOND b
>> THIRD
>>
>> (Run from bash, this is what is surprising double surrounded with single)
>> $ Argecho.bat '"a=b"'
>> FIRST "\"a
>> SECOND b\""
>> THIRD
>>
>> It seems that only the final test case above doesn't behave as expected.
>
> Beware! The way CMD.exe handles command line and parameters is
> different/incompatible from the way C (MSVCRT) application handles
> them. Do not confuse the two. You cannot prove/show anything about how
> C application handles arguments by using CMD.exe as a show case.
>
> --
> VZ

Re: [PATCH] Fix parameter passing containing quote/equal to Windows batch command

[Max Polk]

On 1/21/2014 4:27 AM, Daniel Dai wrote:
> Max,
> Thanks for your reply.
>
> Yes, the double quote issue can be reproducible from command line, but
> not the equal sign.
>
> Let's demonstrate the equal sign issue with a C program:
>
> #include "unistd.h"
>
> int main(int argc, char** argv) {
>      execl("a.bat", "a.bat", "a=b");
>      return 0;
> }
>
> The intention of the program is to pass "a=b" as a single argument.
> However, compile and run it under cygwin, I get:
> $ cc -o myprog myprog.c
> $ ./myprog
> a
>
> I run the program with Visual Studio, I get "a=b".
>
> The way Windows/Unix handles parameter containing equal sign is
> different. IMHO, Cygwin should be the place to fill this semantic gap.

In Visual Studio, you don't get "a=b", you get the same results, a and b 
are separate arguments.

(Run from cmd.exe window)
C:\path>BatchTest.exe

C:\path>FIRST a
SECOND b
THIRD

(Run from bash.exe)
$ BatchTest.exe
$ FIRST a
SECOND b
THIRD

Source code to Visual Studio project:

#include "stdafx.h"
#include <process.h>

int _tmain(int argc, _TCHAR* argv[])
{
     _execl ("Argecho.bat", "Argecho.bat", "a=b", NULL);
}

Script for Argecho.bat:

@echo off
echo FIRST %1
echo SECOND %2
echo THIRD %3

Cygwin looks consistent with both Visual Studio compiled app calling a 
batch file and the command line calling the batch file.  It looks the 
same both ways.

Re: [PATCH] Fix parameter passing containing quote/equal to Windows batch command

[Daniel Dai]

Max,
Yes, you are right. I did a "echo %*" mistakenly in my bat script on Windows.

In terms of consistency, this one does makes a difference (parameter
contains space):

int main(int argc, char** argv) {
    execl("a.bat", "a.bat", "a b");
    return 0;
}

On cygwin, $1=="a b", on Windows however, I get %1==a. Cygwin quote
the parameters containing space automatically.

Anyway, I am not bothered by inconsistent behavior between
Windows/Cygwin. What I really care about is fixing the quote issue,
otherwise I have no way to pass parameter with equal sign to bat.

Thanks,
Daniel



On Tue, Jan 21, 2014 at 5:15 PM, Max Polk <maxpolk@gmail.com> wrote:
> On 1/21/2014 4:27 AM, Daniel Dai wrote:
>>
>> Max,
>> Thanks for your reply.
>>
>> Yes, the double quote issue can be reproducible from command line, but
>> not the equal sign.
>>
>> Let's demonstrate the equal sign issue with a C program:
>>
>> #include "unistd.h"
>>
>> int main(int argc, char** argv) {
>>      execl("a.bat", "a.bat", "a=b");
>>      return 0;
>> }
>>
>> The intention of the program is to pass "a=b" as a single argument.
>> However, compile and run it under cygwin, I get:
>> $ cc -o myprog myprog.c
>> $ ./myprog
>> a
>>
>> I run the program with Visual Studio, I get "a=b".
>>
>> The way Windows/Unix handles parameter containing equal sign is
>> different. IMHO, Cygwin should be the place to fill this semantic gap.
>
>
> In Visual Studio, you don't get "a=b", you get the same results, a and b are
> separate arguments.
>
> (Run from cmd.exe window)
> C:\path>BatchTest.exe
>
> C:\path>FIRST a
> SECOND b
> THIRD
>
> (Run from bash.exe)
> $ BatchTest.exe
> $ FIRST a
> SECOND b
> THIRD
>
> Source code to Visual Studio project:
>
> #include "stdafx.h"
> #include <process.h>
>
> int _tmain(int argc, _TCHAR* argv[])
> {
>     _execl ("Argecho.bat", "Argecho.bat", "a=b", NULL);
> }
>
> Script for Argecho.bat:
>
>
> @echo off
> echo FIRST %1
> echo SECOND %2
> echo THIRD %3
>
> Cygwin looks consistent with both Visual Studio compiled app calling a batch
> file and the command line calling the batch file.  It looks the same both
> ways.
>

Re: [PATCH] Fix parameter passing containing quote/equal to Windows batch command

[Christopher Faylor]

On Sun, Jan 19, 2014 at 10:02:56PM -0800, Daniel Dai wrote:
>We notice one issue when running a Windows batch command inside
>cygwin. Here is one example.
>
>Simple batch file:
>a.bat:
>echo %1
>
>Run it under cygwin:
>./a.bat a=b
>a
>
>./a.bat "a=b"
>a
>
>If we pass additional \"
>./a.bat "\"a=b\""
>"\"a
>
>There seems no way to pass a=b into bat.
>
>Attach quote.patch contains a fix. It does two things:
>1. If the parameter contains a equal sign, automatically add quote
>(similar to space, tab, new line, quote cygwin already do)
>2. If the parameter is already quoted, don't quote again

I don't understand the 2) part of the patch.  If the parameter contains
a quote then the quote needs to be transmitted to the subprocess.
That's what is happening right now.  That allows:

"\"foo bar\""

to be seen by the subprocess as "foo bar", quotes and all.

I'm going to add '=' to the list of special characters but I don't
see the need for the rest of that patch.

(Sorry for the delay in responding.  A hard drive failure wiped out
my cygwin cross-compil stuff.  Amazingly enough, I have recent
backups)

cgf

Re: [PATCH] Fix parameter passing containing quote/equal to Windows batch command

[Daniel Dai]

Hi, Christopher,

The current logic is: if the parameter contains quote, then put a
quote around the parameter (winf.cc:78). However, if the quote is in
the beginning/end, cygwin will still quote it, and thus double quoted
parameter (such as ""a=b"").

In the previous example, I want to pass equal sign into a bat file, I
know Windows bat will not see equal sign unless it is surrender by
quote, so I have to quote it (that's not true now with auto quote for
equal sign you just checked in). Then winf.cc:78 will detect the
quote, and put another quote around, so what Windows bat see is a
double quoted parameter. And surprisingly in this case, bat file does
not treat ""a=b"" as a single parameter, %1 becomes ""a, %2 becomes
b"". Anyway, my point for #2 is, if the parameter is quoted, it
usually means user want the executable see the parameter inside the
quote, it should be treated differently than quote in the middle of
the parameter, we shall not add another quote around it in this
scenario.

And thanks for checkin #1.

Thanks,
Daniel

Re: [PATCH] Fix parameter passing containing quote/equal to Windows batch command

[Christopher Faylor]

On Fri, Jan 24, 2014 at 03:14:30PM -0800, Daniel Dai wrote:
>Hi, Christopher,
>
>The current logic is: if the parameter contains quote, then put a
>quote around the parameter (winf.cc:78). However, if the quote is in
>the beginning/end, cygwin will still quote it, and thus double quoted
>parameter (such as ""a=b"").

That is as intended.  It doesn't matter where the quote is.  This is an
argv list.  Quotes don't mean anything in a UNIX argv list.  They do
need to be quoted for Windows though.

So, if there is a quote at the beginning of argv[7], then the process
should see a quote in argv[7].

If I say pass "\"a=b\"", the subprocess should diligently report "a=b"
quotes and all.  That is what it does now after my change.

AFAICT, this is all working as it should.

cgf

Re: [PATCH] Fix parameter passing containing quote/equal to Windows batch command

[Daniel Dai]

Yes, that's the Unix behavior. However, bat file will eat the quote.
If you pass "a=b" to bat, you will only get a=b. That's why I can
quote the parameters containing equal sign. But anyway, if cygwin
automatically quote equal sign, I don't need to quote the parameter.
That should enough to solve my problem. Thanks Christopher!