* [bug] globify dospath reacts poorly with escaped double quotes @ 2019-10-08 9:05 Mingye Wang 2019-10-08 20:09 ` Brian Inglis 2019-10-10 13:13 ` Ken Brown 0 siblings, 2 replies; 5+ messages in thread From: Mingye Wang @ 2019-10-08 9:05 UTC (permalink / raw) To: cygwin Hi, This bug is inherited from early versions of Cygwin. It's so old that MSYS2 has this problem too. There is no way of conveying a double quote in an argument once globify() decides it has seen a dospath. Neither the `\"` nor `""` work, because they are both unified to `\"` in quoted() and turned into a `\\` pattern in globify(). This is problematic for programmers trying to write a routine to reliably escape an argument for the Cygwin command-line. A way to patch the problem is with a lookahead in globify(): if (dos_spec && *s == '\\') { /**/p++ = '\\'; /**/if (s[1] == '"' && s[2]) { /****/*p = *++s; /****/continue; /**/} } *p = *s; [Apologies for the formatting; the gmail web editor hates leading spaces.] (Note: The backslash thing has always been different from the MSCRT handling, which only transforms backslashes followed by a double quote. But this is fine as long as we are internally consistent. Well... is it documented anywhere?) -- Regards, Mingye Wang -- Mingye Wang (Artoria2e5) -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [bug] globify dospath reacts poorly with escaped double quotes 2019-10-08 9:05 [bug] globify dospath reacts poorly with escaped double quotes Mingye Wang @ 2019-10-08 20:09 ` Brian Inglis 2019-10-10 16:37 ` Brian Inglis 2019-10-10 13:13 ` Ken Brown 1 sibling, 1 reply; 5+ messages in thread From: Brian Inglis @ 2019-10-08 20:09 UTC (permalink / raw) To: cygwin On 2019-10-08 03:05, Mingye Wang wrote: > This bug is inherited from early versions of Cygwin. It's so old that > MSYS2 has this problem too. Probably not a bug then but a feature for Cygwin. Msys2 is another system with different goals, using GNU toolchains to build native Windows programs, not a platform for running POSIX applications like Cygwin, and OT for this list. > There is no way of conveying a double quote in an argument once > globify() decides it has seen a dospath. Neither the `\"` nor `""` > work, because they are both unified to `\"` in quoted() and turned > into a `\\` pattern in globify(). Msys2 tools have to make their own arrangements if they support native Windows paths. Personally I found when I used to use DOS and Windows tools, it was easier using slashes instead of backslashes as directory separators, as most interfaces did not care, including most DOS & Windows APIs. > This is problematic for programmers trying to write a routine to > reliably escape an argument for the Cygwin command-line. Backslash escaping and switching enclosing quotes on shell command lines works reliably to pass any arguments into Cygwin programs, as do the various shell command line parameter and wildcard expansions. Passing special characters into arguments interpreted by other programs requires additional care. > A way to patch the problem is with a lookahead in globify(): > > if (dos_spec && *s == '\\') { > /**/p++ = '\\'; > /**/if (s[1] == '"' && s[2]) { > /****/*p = *++s; > /****/continue; > /**/} > } > *p = *s; > > [Apologies for the formatting; the gmail web editor hates leading spaces.] > > (Note: The backslash thing has always been different from the MSCRT > handling, which only transforms backslashes followed by a double > quote. But this is fine as long as we are internally consistent. > Well... is it documented anywhere?) Support of DOS paths is inconsistent in Cygwin utilities and may not work: use cygpath, or the low level API, to convert DOS to POSIX paths before passing to Cygwin programs, or functions. Backslash should only be used to escape command line characters with special meaning to the shell, or escapes in strings in other languages. Any other use should specify what kinds or arguments ypu are trying to handle, how you are getting your arguments in, and passing them to globify. Invoking Cygwin programs from other Cygwin programs is best done using the exec or spawn functions with (unescaped, unquoted) arguments in varargs arg lists or arrays. Invoking Windows programs works best when done from a cmd wrapper, but anything involving any Windows command line requires work to generalize. See previous recent posts for what is required. Avoid system() and similar calls if possible, as they will then go thru an additional shell layer. -- Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada This email may be disturbing to some readers as it contains too much technical detail. Reader discretion is advised. -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [bug] globify dospath reacts poorly with escaped double quotes 2019-10-08 20:09 ` Brian Inglis @ 2019-10-10 16:37 ` Brian Inglis 0 siblings, 0 replies; 5+ messages in thread From: Brian Inglis @ 2019-10-10 16:37 UTC (permalink / raw) To: cygwin On 2019-10-10 02:30, Mingye Wang wrote: > On Tue, 8 Oct 2019 14:09:18 -0600, Brian Inglis wrote: >> On 2019-10-08 03:05, Mingye Wang wrote: >> >> This bug is inherited from early versions of Cygwin. It's so old that >>> MSYS2 has this problem too. >> >> Probably not a bug then but a feature for Cygwin. >> Msys2 is another system with different goals, using GNU toolchains to build >> native Windows programs, not a platform for running POSIX applications like >> Cygwin, and OT for this list. >> >>> There is no way of conveying a double quote in an argument once >>> globify() decides it has seen a dospath. Neither the `\"` nor `""` >>> work, because they are both unified to `\"` in quoted() and turned >>> into a `\\` pattern in globify(). >> >> Msys2 tools have to make their own arrangements if they support native Windows >> paths. >> Personally I found when I used to use DOS and Windows tools, it was easier using >> slashes instead of backslashes as directory separators, as most interfaces did >> not care, including most DOS & Windows APIs. [please Reply to List, or Reply to All and send only to the mailing list address, to maintain email thread references] > Use the source, Luke. Globbing on the cmdline, as we all know, is something > you only care about when you are *not* passing in arguments from a Unix-type > cmdline, and build_argv in dcrt0.cc does *only* run globify if the parent > process is *not* Cygwin-powered. We need to understand your use case(s) and issue(s), and you to explain those to us. Please provide at least test vectors showing what you are trying to get, what you are providing, what you are getting, the difference, and why you think it should be different. You can often get code formatted for web mail by using gist or similar editors or other markdown environments and paste the text as formatted into the web mail compose window. You can also attach as text a file or extract showing the file(s) and context(s) where you create the content, and/or a simple test script or program to do so, and/or where you think the problem resides, or a patch as at least a context diff, preferably git format-patch or send-email output. If you send a patch, please also show how your test vectors, plus a variety of more normal usage, work before and after your patch is applied. >>> This is problematic for programmers trying to write a routine to >>> reliably escape an argument for the Cygwin command-line. >> >> Backslash escaping and switching enclosing quotes on shell command lines works >> reliably to pass any arguments into Cygwin programs, as do the various shell >> command line parameter and wildcard expansions. > See above. Where? Have you considered bypassing command lines using sockets, pipes, shared memory segments, memory mapped temporary files, etc. Have you considered trying to pass argument and environment vectors directly to the Cygwin program, as would be done by an invoking Cygwin program? Have you tried using single quotes around problematic arguments, quoted as required to pass this thru from the calling environment, which if you're calling a program directly, should not require any escaping or quoting, or could include whatever other escaping or quoting you need to pass the info transparently, perhaps via an interop wrapper program or script? >> Passing special characters into arguments interpreted by other programs requires >> additional care. >> >>> A way to patch the problem is with a lookahead in globify(): >>> >>> if (dos_spec && *s == '\\') { >>> /**/p++ = '\\'; >>> /**/if (s[1] == '"' && s[2]) { >>> /****/*p = *++s; >>> /****/continue; >>> /**/} >>> } >>> *p = *s; >>> >>> [Apologies for the formatting; the gmail web editor hates leading spaces.] >>> >>> (Note: The backslash thing has always been different from the MSCRT >>> handling, which only transforms backslashes followed by a double >>> quote. But this is fine as long as we are internally consistent. >>> Well... is it documented anywhere?) >> >> Support of DOS paths is inconsistent in Cygwin utilities and may not work: use >> cygpath, or the low level API, to convert DOS to POSIX paths before passing to >> Cygwin programs, or functions. >> >> Backslash should only be used to escape command line characters with special >> meaning to the shell, or escapes in strings in other languages. >> >> Any other use should specify what kinds or arguments ypu are trying to handle, >> how you are getting your arguments in, and passing them to globify. >> >> Invoking Cygwin programs from other Cygwin programs is best done using the exec >> or spawn functions with (unescaped, unquoted) arguments in varargs arg lists or >> arrays. >> >> Invoking Windows programs works best when done from a cmd wrapper, but anything >> involving any Windows command line requires work to generalize. >> See previous recent posts for what is required. >> >> Avoid system() and similar calls if possible, as they will then go thru an >> additional shell layer. -- Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada This email may be disturbing to some readers as it contains too much technical detail. Reader discretion is advised. -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [bug] globify dospath reacts poorly with escaped double quotes 2019-10-08 9:05 [bug] globify dospath reacts poorly with escaped double quotes Mingye Wang 2019-10-08 20:09 ` Brian Inglis @ 2019-10-10 13:13 ` Ken Brown 1 sibling, 0 replies; 5+ messages in thread From: Ken Brown @ 2019-10-10 13:13 UTC (permalink / raw) To: cygwin On 10/8/2019 5:05 AM, Mingye Wang wrote: > Hi, > > This bug is inherited from early versions of Cygwin. It's so old that > MSYS2 has this problem too. > > There is no way of conveying a double quote in an argument once > globify() decides it has seen a dospath. Neither the `\"` nor `""` > work, because they are both unified to `\"` in quoted() and turned > into a `\\` pattern in globify(). > > This is problematic for programmers trying to write a routine to > reliably escape an argument for the Cygwin command-line. > > A way to patch the problem is with a lookahead in globify(): > > if (dos_spec && *s == '\\') { > /**/p++ = '\\'; > /**/if (s[1] == '"' && s[2]) { > /****/*p = *++s; > /****/continue; > /**/} > } > *p = *s; > > [Apologies for the formatting; the gmail web editor hates leading spaces.] > > (Note: The backslash thing has always been different from the MSCRT > handling, which only transforms backslashes followed by a double > quote. But this is fine as long as we are internally consistent. > Well... is it documented anywhere?) I don't know enough about Windows command-line quoting to comment on the substance of this. But it would help those who do if you would send your patch to the cygwin-patches mailing list using git format-patch and git send-email. And please use the formatting conventions of the surrounding code. For example, if (dos_spec && *s == '\\') { [...] } Thanks. Ken -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [bug] globify dospath reacts poorly with escaped double quotes @ 2019-10-10 8:30 Mingye Wang 0 siblings, 0 replies; 5+ messages in thread From: Mingye Wang @ 2019-10-10 8:30 UTC (permalink / raw) To: cygwin On Tue, 8 Oct 2019 14:09:18 -0600, Brian Inglis wrote: > Msys2 tools have to make their own arrangements if they support native Windows > paths. > Personally I found when I used to use DOS and Windows tools, it was easier using > slashes instead of backslashes as directory separators, as most interfaces did > not care, including most DOS & Windows APIs. Use the source, Luke. Globbing on the cmdline, as we all know, is something you only care about when you are *not* passing in arguments from a Unix-type cmdline, and build_argv in dcrt0.cc does *only* run globify if the parent process is *not* Cygwin-powered. > Backslash escaping and switching enclosing quotes on shell command lines... See above. -- Mingye Wang (Artoria2e5) -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2019-10-10 16:37 UTC | newest] Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2019-10-08 9:05 [bug] globify dospath reacts poorly with escaped double quotes Mingye Wang 2019-10-08 20:09 ` Brian Inglis 2019-10-10 16:37 ` Brian Inglis 2019-10-10 13:13 ` Ken Brown 2019-10-10 8:30 Mingye Wang
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).