From: Andrey Repin <anrdaemon@yandex.ru>
To: "Jérôme Froissart" <software@froissart.eu>, cygwin@cygwin.com
Subject: Re: Unconsistent command-line parsing in case of UTF-8 quoted arguments
Date: Sun, 4 Oct 2020 14:18:09 +0300 [thread overview]
Message-ID: <634821436.20201004141809@yandex.ru> (raw)
In-Reply-To: <CAFC9CLCtfMORMxAK6==jdwY5ZbX6jWwo+JCfDwM3njgvGduf0w@mail.gmail.com>
Greetings, Jérôme Froissart!
> By discussing a merge request on another project [1], I think
> billziss-gh found a weirdness in the way Cygwin parses the command
> line arguments when non-ASCII characters come into play.
> EXPECTED BEHAVIOUR:
> cygwin should parse the following command line
> binary.exe --non-ascii "charaçtérs" --ascii "nothing-fancy-here"
> as
> argv = ["binary.exe",
> "--non-ascii",
> "chara\xXX\xXXt\xXX\xXXrs",
> "--ascii",
> "nothing-fancy-here"]
> // \xXX\xXX being the UTF-8 encoding of the special characters,
> but this does not really matter here
> before calling main()
> ACTUAL BEHAVIOUR:
> it parses it as
> argv = ["binary.exe",
> "--non-ascii",
> "\"chara\xXX\xXXt\xXX\xXXrs\"", // mind the unstripped
> quotes here...
> "--ascii",
> "nothing-fancy-here" // ...but not here
> ]
> It looks that words containing UTF-8 characters are not properly
> stripped when they are surrounded by quotes, unlinke ASCII words.
> More examples and a better description is available at [1] (thanks to
> billziss-gh for his analysis, much more thorough than mine)
> For the record, we wrote a work-around in our specific program, but
> handling this issue in Cygwin might be a better way to solve it.
> [1]: https://github.com/billziss-gh/sshfs-win/pull/208 (Checking for
> quotes around non-ascii usernames passed by Windows)
> Thanks for your help! In case you didn't have time, please tell me
> where to look at, and I might try to fix it myself and send a patch
> proposal if that is easy enough (I have never read Cygwin's code yet).
This seems like the Cygwin command was launched from a non-Cygwin terminal or
from a terminal where locale was not set to UNICODE.
Please provide the results of "locale" command right before running your test
binary.
--
With best regards,
Andrey Repin
Sunday, October 4, 2020 14:16:17
Sorry for my terrible english...
next prev parent reply other threads:[~2020-10-04 11:20 UTC|newest]
Thread overview: 18+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-10-02 21:40 Jérôme Froissart
2020-10-03 2:22 ` Doug Henderson
2020-10-04 11:18 ` Andrey Repin [this message]
2020-10-06 21:36 ` Jérôme Froissart
2020-10-07 1:10 ` Andrey Repin
2020-10-07 22:21 ` Jérôme Froissart
2020-10-11 18:55 ` Andrey Repin
2020-10-07 2:20 ` Brian Inglis
2020-10-07 5:17 ` Thomas Wolff
2020-10-07 23:32 ` Brian Inglis
2020-10-08 0:59 ` Eliot Moss
2020-10-08 6:22 ` Brian Inglis
2020-10-13 16:30 ` Kaz Kylheku (Cygwin)
2020-10-14 21:47 ` Jérôme Froissart
2020-10-14 22:14 ` Jérôme Froissart
2020-10-15 5:14 ` UTF-8 quoted args passed to program include quotes when run from cmd Brian Inglis
2020-10-19 2:32 ` Unconsistent command-line parsing in case of UTF-8 quoted arguments Kaz Kylheku (Cygwin)
2020-10-13 17:34 ` Brian Inglis
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=634821436.20201004141809@yandex.ru \
--to=anrdaemon@yandex.ru \
--cc=cygwin@cygwin.com \
--cc=software@froissart.eu \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).