* [PATCH] Cygwin: pty: Add workaround for ISO-2022 and ISCII in convert_mb_str(). @ 2020-09-11 10:54 Takashi Yano 2020-09-11 12:08 ` Corinna Vinschen 0 siblings, 1 reply; 13+ messages in thread From: Takashi Yano @ 2020-09-11 10:54 UTC (permalink / raw) To: cygwin-patches - In convert_mb_str(), exclude ISO-2022 and ISCII from the processing for the case that the multibyte char is splitted in the middle. The reason is as follows. * ISO-2022 is too complicated to handle correctly. * Not sure what to do with ISCII. --- winsup/cygwin/fhandler_tty.cc | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/winsup/cygwin/fhandler_tty.cc b/winsup/cygwin/fhandler_tty.cc index 37d033bbe..ee5c6a90a 100644 --- a/winsup/cygwin/fhandler_tty.cc +++ b/winsup/cygwin/fhandler_tty.cc @@ -117,6 +117,9 @@ CreateProcessW_Hooked return CreateProcessW_Orig (n, c, pa, ta, inh, f, e, d, si, pi); } +#define IS_ISO_2022(x) ( (x) >= 50220 && (x) <= 50229 ) +#define IS_ISCII(x) ( (x) >= 57002 && (x) <= 57011 ) + static void convert_mb_str (UINT cp_to, char *ptr_to, size_t *len_to, UINT cp_from, const char *ptr_from, size_t len_from, @@ -126,8 +129,10 @@ convert_mb_str (UINT cp_to, char *ptr_to, size_t *len_to, tmp_pathbuf tp; wchar_t *wbuf = tp.w_get (); int wlen = 0; - if (cp_from == CP_UTF7) - /* MB_ERR_INVALID_CHARS does not work properly for UTF-7. + if (cp_from == CP_UTF7 || IS_ISO_2022 (cp_from) || IS_ISCII (cp_from)) + /* - MB_ERR_INVALID_CHARS does not work properly for UTF-7. + - ISO-2022 is too complicated to handle correctly. + - FIXME: Not sure what to do for ISCII. Therefore, just convert string without checking */ wlen = MultiByteToWideChar (cp_from, 0, ptr_from, len_from, wbuf, NT_MAX_PATH); -- 2.28.0 ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH] Cygwin: pty: Add workaround for ISO-2022 and ISCII in convert_mb_str(). 2020-09-11 10:54 [PATCH] Cygwin: pty: Add workaround for ISO-2022 and ISCII in convert_mb_str() Takashi Yano @ 2020-09-11 12:08 ` Corinna Vinschen 2020-09-11 12:35 ` Takashi Yano 0 siblings, 1 reply; 13+ messages in thread From: Corinna Vinschen @ 2020-09-11 12:08 UTC (permalink / raw) To: cygwin-patches On Sep 11 19:54, Takashi Yano via Cygwin-patches wrote: > - In convert_mb_str(), exclude ISO-2022 and ISCII from the processing > for the case that the multibyte char is splitted in the middle. > The reason is as follows. > * ISO-2022 is too complicated to handle correctly. > * Not sure what to do with ISCII. > --- > winsup/cygwin/fhandler_tty.cc | 9 +++++++-- > 1 file changed, 7 insertions(+), 2 deletions(-) > > diff --git a/winsup/cygwin/fhandler_tty.cc b/winsup/cygwin/fhandler_tty.cc > index 37d033bbe..ee5c6a90a 100644 > --- a/winsup/cygwin/fhandler_tty.cc > +++ b/winsup/cygwin/fhandler_tty.cc > @@ -117,6 +117,9 @@ CreateProcessW_Hooked > return CreateProcessW_Orig (n, c, pa, ta, inh, f, e, d, si, pi); > } > > +#define IS_ISO_2022(x) ( (x) >= 50220 && (x) <= 50229 ) > +#define IS_ISCII(x) ( (x) >= 57002 && (x) <= 57011 ) > + > static void > convert_mb_str (UINT cp_to, char *ptr_to, size_t *len_to, > UINT cp_from, const char *ptr_from, size_t len_from, > @@ -126,8 +129,10 @@ convert_mb_str (UINT cp_to, char *ptr_to, size_t *len_to, > tmp_pathbuf tp; > wchar_t *wbuf = tp.w_get (); > int wlen = 0; > - if (cp_from == CP_UTF7) > - /* MB_ERR_INVALID_CHARS does not work properly for UTF-7. > + if (cp_from == CP_UTF7 || IS_ISO_2022 (cp_from) || IS_ISCII (cp_from)) > + /* - MB_ERR_INVALID_CHARS does not work properly for UTF-7. > + - ISO-2022 is too complicated to handle correctly. > + - FIXME: Not sure what to do for ISCII. > Therefore, just convert string without checking */ > wlen = MultiByteToWideChar (cp_from, 0, ptr_from, len_from, > wbuf, NT_MAX_PATH); > -- > 2.28.0 I'd prefer to not handle them at all. We just don't support these charsets, same as JIS, EBCDIC, you name it, which are not ASCII compatible. Let's please just drop any handling for these weird or outdated codepages. Thanks, Corinna ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH] Cygwin: pty: Add workaround for ISO-2022 and ISCII in convert_mb_str(). 2020-09-11 12:08 ` Corinna Vinschen @ 2020-09-11 12:35 ` Takashi Yano 2020-09-11 14:06 ` Corinna Vinschen 0 siblings, 1 reply; 13+ messages in thread From: Takashi Yano @ 2020-09-11 12:35 UTC (permalink / raw) To: cygwin-patches Hi Corinna, On Fri, 11 Sep 2020 14:08:40 +0200 Corinna Vinschen wrote: > On Sep 11 19:54, Takashi Yano via Cygwin-patches wrote: > > - In convert_mb_str(), exclude ISO-2022 and ISCII from the processing > > for the case that the multibyte char is splitted in the middle. > > The reason is as follows. > > * ISO-2022 is too complicated to handle correctly. > > * Not sure what to do with ISCII. > > --- > > winsup/cygwin/fhandler_tty.cc | 9 +++++++-- > > 1 file changed, 7 insertions(+), 2 deletions(-) > > > > diff --git a/winsup/cygwin/fhandler_tty.cc b/winsup/cygwin/fhandler_tty.cc > > index 37d033bbe..ee5c6a90a 100644 > > --- a/winsup/cygwin/fhandler_tty.cc > > +++ b/winsup/cygwin/fhandler_tty.cc > > @@ -117,6 +117,9 @@ CreateProcessW_Hooked > > return CreateProcessW_Orig (n, c, pa, ta, inh, f, e, d, si, pi); > > } > > > > +#define IS_ISO_2022(x) ( (x) >= 50220 && (x) <= 50229 ) > > +#define IS_ISCII(x) ( (x) >= 57002 && (x) <= 57011 ) > > + > > static void > > convert_mb_str (UINT cp_to, char *ptr_to, size_t *len_to, > > UINT cp_from, const char *ptr_from, size_t len_from, > > @@ -126,8 +129,10 @@ convert_mb_str (UINT cp_to, char *ptr_to, size_t *len_to, > > tmp_pathbuf tp; > > wchar_t *wbuf = tp.w_get (); > > int wlen = 0; > > - if (cp_from == CP_UTF7) > > - /* MB_ERR_INVALID_CHARS does not work properly for UTF-7. > > + if (cp_from == CP_UTF7 || IS_ISO_2022 (cp_from) || IS_ISCII (cp_from)) > > + /* - MB_ERR_INVALID_CHARS does not work properly for UTF-7. > > + - ISO-2022 is too complicated to handle correctly. > > + - FIXME: Not sure what to do for ISCII. > > Therefore, just convert string without checking */ > > wlen = MultiByteToWideChar (cp_from, 0, ptr_from, len_from, > > wbuf, NT_MAX_PATH); > > -- > > 2.28.0 > > I'd prefer to not handle them at all. We just don't support these > charsets, same as JIS, EBCDIC, you name it, which are not ASCII > compatible. Let's please just drop any handling for these weird > or outdated codepages. What do you mean by "just drop any handling"? Do you mean remove following if block? > > + if (cp_from == CP_UTF7 || IS_ISO_2022 (cp_from) || IS_ISCII (cp_from)) > > + /* - MB_ERR_INVALID_CHARS does not work properly for UTF-7. > > + - ISO-2022 is too complicated to handle correctly. > > + - FIXME: Not sure what to do for ISCII. > > Therefore, just convert string without checking */ > > wlen = MultiByteToWideChar (cp_from, 0, ptr_from, len_from, > > wbuf, NT_MAX_PATH); In this case, the conversion for ISO-2022, ISCII and UTF-7 will not be done correctly. Or skip charset conversion if the codepage is EBCDIC, ISO-2022 or ISCII? What should we do for UTF-7? What should happen if user or apps chage codepage to one of them? -- Takashi Yano <takashi.yano@nifty.ne.jp> ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH] Cygwin: pty: Add workaround for ISO-2022 and ISCII in convert_mb_str(). 2020-09-11 12:35 ` Takashi Yano @ 2020-09-11 14:06 ` Corinna Vinschen 2020-09-11 15:10 ` Thomas Wolff 2020-09-11 16:05 ` Takashi Yano 0 siblings, 2 replies; 13+ messages in thread From: Corinna Vinschen @ 2020-09-11 14:06 UTC (permalink / raw) To: cygwin-patches On Sep 11 21:35, Takashi Yano via Cygwin-patches wrote: > Hi Corinna, > > On Fri, 11 Sep 2020 14:08:40 +0200 > Corinna Vinschen wrote: > > On Sep 11 19:54, Takashi Yano via Cygwin-patches wrote: > > > - In convert_mb_str(), exclude ISO-2022 and ISCII from the processing > > > for the case that the multibyte char is splitted in the middle. > > > The reason is as follows. > > > * ISO-2022 is too complicated to handle correctly. > > > * Not sure what to do with ISCII. > > > --- > > > winsup/cygwin/fhandler_tty.cc | 9 +++++++-- > > > 1 file changed, 7 insertions(+), 2 deletions(-) > > > > > > diff --git a/winsup/cygwin/fhandler_tty.cc b/winsup/cygwin/fhandler_tty.cc > > > index 37d033bbe..ee5c6a90a 100644 > > > --- a/winsup/cygwin/fhandler_tty.cc > > > +++ b/winsup/cygwin/fhandler_tty.cc > > > @@ -117,6 +117,9 @@ CreateProcessW_Hooked > > > return CreateProcessW_Orig (n, c, pa, ta, inh, f, e, d, si, pi); > > > } > > > > > > +#define IS_ISO_2022(x) ( (x) >= 50220 && (x) <= 50229 ) > > > +#define IS_ISCII(x) ( (x) >= 57002 && (x) <= 57011 ) > > > + > > > static void > > > convert_mb_str (UINT cp_to, char *ptr_to, size_t *len_to, > > > UINT cp_from, const char *ptr_from, size_t len_from, > > > @@ -126,8 +129,10 @@ convert_mb_str (UINT cp_to, char *ptr_to, size_t *len_to, > > > tmp_pathbuf tp; > > > wchar_t *wbuf = tp.w_get (); > > > int wlen = 0; > > > - if (cp_from == CP_UTF7) > > > - /* MB_ERR_INVALID_CHARS does not work properly for UTF-7. > > > + if (cp_from == CP_UTF7 || IS_ISO_2022 (cp_from) || IS_ISCII (cp_from)) > > > + /* - MB_ERR_INVALID_CHARS does not work properly for UTF-7. > > > + - ISO-2022 is too complicated to handle correctly. > > > + - FIXME: Not sure what to do for ISCII. > > > Therefore, just convert string without checking */ > > > wlen = MultiByteToWideChar (cp_from, 0, ptr_from, len_from, > > > wbuf, NT_MAX_PATH); > > > -- > > > 2.28.0 > > > > I'd prefer to not handle them at all. We just don't support these > > charsets, same as JIS, EBCDIC, you name it, which are not ASCII > > compatible. Let's please just drop any handling for these weird > > or outdated codepages. > > What do you mean by "just drop any handling"? > > Do you mean remove following if block? > > > + if (cp_from == CP_UTF7 || IS_ISO_2022 (cp_from) || IS_ISCII (cp_from)) > > > + /* - MB_ERR_INVALID_CHARS does not work properly for UTF-7. > > > + - ISO-2022 is too complicated to handle correctly. > > > + - FIXME: Not sure what to do for ISCII. > > > Therefore, just convert string without checking */ > > > wlen = MultiByteToWideChar (cp_from, 0, ptr_from, len_from, > > > wbuf, NT_MAX_PATH); > In this case, the conversion for ISO-2022, ISCII and UTF-7 will > not be done correctly. > > Or skip charset conversion if the codepage is EBCDIC, ISO-2022 > or ISCII? What should we do for UTF-7? Nothing, just like for any other of these weird charsets. Cygwin never supported any charset which wasn't at least ASCII compatible in the 0 <= x <= 127 range. Just ignore them and the possibility that a user chooses them for fun. > What should happen if user or apps chage codepage to one of them? Garbage output, I guess. We shouldn't really care. Corinna ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH] Cygwin: pty: Add workaround for ISO-2022 and ISCII in convert_mb_str(). 2020-09-11 14:06 ` Corinna Vinschen @ 2020-09-11 15:10 ` Thomas Wolff 2020-09-11 15:18 ` Thomas Wolff 2020-09-11 16:05 ` Takashi Yano 1 sibling, 1 reply; 13+ messages in thread From: Thomas Wolff @ 2020-09-11 15:10 UTC (permalink / raw) To: cygwin-patches Am 11.09.2020 um 16:06 schrieb Corinna Vinschen: > On Sep 11 21:35, Takashi Yano via Cygwin-patches wrote: >> Hi Corinna, >> >> On Fri, 11 Sep 2020 14:08:40 +0200 >> Corinna Vinschen wrote: >>> On Sep 11 19:54, Takashi Yano via Cygwin-patches wrote: >>>> - In convert_mb_str(), exclude ISO-2022 and ISCII from the processing >>>> for the case that the multibyte char is splitted in the middle. >>>> The reason is as follows. >>>> * ISO-2022 is too complicated to handle correctly. >>>> * Not sure what to do with ISCII. >>>> --- >>>> winsup/cygwin/fhandler_tty.cc | 9 +++++++-- >>>> 1 file changed, 7 insertions(+), 2 deletions(-) >>>> >>>> diff --git a/winsup/cygwin/fhandler_tty.cc b/winsup/cygwin/fhandler_tty.cc >>>> index 37d033bbe..ee5c6a90a 100644 >>>> --- a/winsup/cygwin/fhandler_tty.cc >>>> +++ b/winsup/cygwin/fhandler_tty.cc >>>> @@ -117,6 +117,9 @@ CreateProcessW_Hooked >>>> return CreateProcessW_Orig (n, c, pa, ta, inh, f, e, d, si, pi); >>>> } >>>> >>>> +#define IS_ISO_2022(x) ( (x) >= 50220 && (x) <= 50229 ) >>>> +#define IS_ISCII(x) ( (x) >= 57002 && (x) <= 57011 ) >>>> + >>>> static void >>>> convert_mb_str (UINT cp_to, char *ptr_to, size_t *len_to, >>>> UINT cp_from, const char *ptr_from, size_t len_from, >>>> @@ -126,8 +129,10 @@ convert_mb_str (UINT cp_to, char *ptr_to, size_t *len_to, >>>> tmp_pathbuf tp; >>>> wchar_t *wbuf = tp.w_get (); >>>> int wlen = 0; >>>> - if (cp_from == CP_UTF7) >>>> - /* MB_ERR_INVALID_CHARS does not work properly for UTF-7. >>>> + if (cp_from == CP_UTF7 || IS_ISO_2022 (cp_from) || IS_ISCII (cp_from)) >>>> + /* - MB_ERR_INVALID_CHARS does not work properly for UTF-7. >>>> + - ISO-2022 is too complicated to handle correctly. >>>> + - FIXME: Not sure what to do for ISCII. >>>> Therefore, just convert string without checking */ >>>> wlen = MultiByteToWideChar (cp_from, 0, ptr_from, len_from, >>>> wbuf, NT_MAX_PATH); >>>> -- >>>> 2.28.0 >>> I'd prefer to not handle them at all. We just don't support these >>> charsets, same as JIS, EBCDIC, you name it, which are not ASCII >>> compatible. Let's please just drop any handling for these weird >>> or outdated codepages. >> What do you mean by "just drop any handling"? >> >> Do you mean remove following if block? >>>> + if (cp_from == CP_UTF7 || IS_ISO_2022 (cp_from) || IS_ISCII (cp_from)) >>>> + /* - MB_ERR_INVALID_CHARS does not work properly for UTF-7. >>>> + - ISO-2022 is too complicated to handle correctly. >>>> + - FIXME: Not sure what to do for ISCII. >>>> Therefore, just convert string without checking */ >>>> wlen = MultiByteToWideChar (cp_from, 0, ptr_from, len_from, >>>> wbuf, NT_MAX_PATH); >> In this case, the conversion for ISO-2022, ISCII and UTF-7 will >> not be done correctly. >> >> Or skip charset conversion if the codepage is EBCDIC, ISO-2022 >> or ISCII? What should we do for UTF-7? > Nothing, just like for any other of these weird charsets. Cygwin never > supported any charset which wasn't at least ASCII compatible in the > 0 <= x <= 127 range. Actually, in Shift-JIS (CP932, supported via locale ja_JP.sjis), 0x5C is ¥ :/ > Just ignore them and the possibility that a > user chooses them for fun. > >> What should happen if user or apps chage codepage to one of them? > Garbage output, I guess. We shouldn't really care. > > > Corinna ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH] Cygwin: pty: Add workaround for ISO-2022 and ISCII in convert_mb_str(). 2020-09-11 15:10 ` Thomas Wolff @ 2020-09-11 15:18 ` Thomas Wolff 0 siblings, 0 replies; 13+ messages in thread From: Thomas Wolff @ 2020-09-11 15:18 UTC (permalink / raw) To: cygwin-patches Am 11.09.2020 um 17:10 schrieb Thomas Wolff: > Am 11.09.2020 um 16:06 schrieb Corinna Vinschen: >> On Sep 11 21:35, Takashi Yano via Cygwin-patches wrote: >>> Hi Corinna, >>> >>> On Fri, 11 Sep 2020 14:08:40 +0200 >>> Corinna Vinschen wrote: >>>> On Sep 11 19:54, Takashi Yano via Cygwin-patches wrote: >>>>> - In convert_mb_str(), exclude ISO-2022 and ISCII from the processing >>>>> for the case that the multibyte char is splitted in the middle. >>>>> The reason is as follows. >>>>> * ISO-2022 is too complicated to handle correctly. >>>>> * Not sure what to do with ISCII. >>>>> --- >>>>> winsup/cygwin/fhandler_tty.cc | 9 +++++++-- >>>>> 1 file changed, 7 insertions(+), 2 deletions(-) >>>>> >>>>> diff --git a/winsup/cygwin/fhandler_tty.cc >>>>> b/winsup/cygwin/fhandler_tty.cc >>>>> index 37d033bbe..ee5c6a90a 100644 >>>>> --- a/winsup/cygwin/fhandler_tty.cc >>>>> +++ b/winsup/cygwin/fhandler_tty.cc >>>>> @@ -117,6 +117,9 @@ CreateProcessW_Hooked >>>>> return CreateProcessW_Orig (n, c, pa, ta, inh, f, e, d, si, pi); >>>>> } >>>>> +#define IS_ISO_2022(x) ( (x) >= 50220 && (x) <= 50229 ) >>>>> +#define IS_ISCII(x) ( (x) >= 57002 && (x) <= 57011 ) >>>>> + >>>>> static void >>>>> convert_mb_str (UINT cp_to, char *ptr_to, size_t *len_to, >>>>> UINT cp_from, const char *ptr_from, size_t len_from, >>>>> @@ -126,8 +129,10 @@ convert_mb_str (UINT cp_to, char *ptr_to, >>>>> size_t *len_to, >>>>> tmp_pathbuf tp; >>>>> wchar_t *wbuf = tp.w_get (); >>>>> int wlen = 0; >>>>> - if (cp_from == CP_UTF7) >>>>> - /* MB_ERR_INVALID_CHARS does not work properly for UTF-7. >>>>> + if (cp_from == CP_UTF7 || IS_ISO_2022 (cp_from) || IS_ISCII >>>>> (cp_from)) >>>>> + /* - MB_ERR_INVALID_CHARS does not work properly for UTF-7. >>>>> + - ISO-2022 is too complicated to handle correctly. >>>>> + - FIXME: Not sure what to do for ISCII. >>>>> Therefore, just convert string without checking */ >>>>> wlen = MultiByteToWideChar (cp_from, 0, ptr_from, len_from, >>>>> wbuf, NT_MAX_PATH); >>>>> -- >>>>> 2.28.0 >>>> I'd prefer to not handle them at all. We just don't support these >>>> charsets, same as JIS, EBCDIC, you name it, which are not ASCII >>>> compatible. Let's please just drop any handling for these weird >>>> or outdated codepages. >>> What do you mean by "just drop any handling"? >>> >>> Do you mean remove following if block? >>>>> + if (cp_from == CP_UTF7 || IS_ISO_2022 (cp_from) || IS_ISCII >>>>> (cp_from)) >>>>> + /* - MB_ERR_INVALID_CHARS does not work properly for UTF-7. >>>>> + - ISO-2022 is too complicated to handle correctly. >>>>> + - FIXME: Not sure what to do for ISCII. >>>>> Therefore, just convert string without checking */ >>>>> wlen = MultiByteToWideChar (cp_from, 0, ptr_from, len_from, >>>>> wbuf, NT_MAX_PATH); >>> In this case, the conversion for ISO-2022, ISCII and UTF-7 will >>> not be done correctly. >>> >>> Or skip charset conversion if the codepage is EBCDIC, ISO-2022 >>> or ISCII? What should we do for UTF-7? >> Nothing, just like for any other of these weird charsets. Cygwin never >> supported any charset which wasn't at least ASCII compatible in the >> 0 <= x <= 127 range. > Actually, in Shift-JIS (CP932, supported via locale ja_JP.sjis), 0x5C > is ¥ :/ ... or maybe not, as explained in https://en.wikipedia.org/wiki/Code_page_932_(Microsoft_Windows)#Single-byte_character_differences. Terrible. >> Just ignore them and the possibility that a >> user chooses them for fun. >> >>> What should happen if user or apps chage codepage to one of them? >> Garbage output, I guess. We shouldn't really care. >> >> >> Corinna > ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH] Cygwin: pty: Add workaround for ISO-2022 and ISCII in convert_mb_str(). 2020-09-11 14:06 ` Corinna Vinschen 2020-09-11 15:10 ` Thomas Wolff @ 2020-09-11 16:05 ` Takashi Yano 2020-09-11 17:38 ` Takashi Yano 2020-09-11 18:14 ` Corinna Vinschen 1 sibling, 2 replies; 13+ messages in thread From: Takashi Yano @ 2020-09-11 16:05 UTC (permalink / raw) To: cygwin-patches [-- Attachment #1: Type: text/plain, Size: 4070 bytes --] On Fri, 11 Sep 2020 16:06:01 +0200 Corinna Vinschen wrote: > On Sep 11 21:35, Takashi Yano via Cygwin-patches wrote: > > Hi Corinna, > > > > On Fri, 11 Sep 2020 14:08:40 +0200 > > Corinna Vinschen wrote: > > > On Sep 11 19:54, Takashi Yano via Cygwin-patches wrote: > > > > - In convert_mb_str(), exclude ISO-2022 and ISCII from the processing > > > > for the case that the multibyte char is splitted in the middle. > > > > The reason is as follows. > > > > * ISO-2022 is too complicated to handle correctly. > > > > * Not sure what to do with ISCII. > > > > --- > > > > winsup/cygwin/fhandler_tty.cc | 9 +++++++-- > > > > 1 file changed, 7 insertions(+), 2 deletions(-) > > > > > > > > diff --git a/winsup/cygwin/fhandler_tty.cc b/winsup/cygwin/fhandler_tty.cc > > > > index 37d033bbe..ee5c6a90a 100644 > > > > --- a/winsup/cygwin/fhandler_tty.cc > > > > +++ b/winsup/cygwin/fhandler_tty.cc > > > > @@ -117,6 +117,9 @@ CreateProcessW_Hooked > > > > return CreateProcessW_Orig (n, c, pa, ta, inh, f, e, d, si, pi); > > > > } > > > > > > > > +#define IS_ISO_2022(x) ( (x) >= 50220 && (x) <= 50229 ) > > > > +#define IS_ISCII(x) ( (x) >= 57002 && (x) <= 57011 ) > > > > + > > > > static void > > > > convert_mb_str (UINT cp_to, char *ptr_to, size_t *len_to, > > > > UINT cp_from, const char *ptr_from, size_t len_from, > > > > @@ -126,8 +129,10 @@ convert_mb_str (UINT cp_to, char *ptr_to, size_t *len_to, > > > > tmp_pathbuf tp; > > > > wchar_t *wbuf = tp.w_get (); > > > > int wlen = 0; > > > > - if (cp_from == CP_UTF7) > > > > - /* MB_ERR_INVALID_CHARS does not work properly for UTF-7. > > > > + if (cp_from == CP_UTF7 || IS_ISO_2022 (cp_from) || IS_ISCII (cp_from)) > > > > + /* - MB_ERR_INVALID_CHARS does not work properly for UTF-7. > > > > + - ISO-2022 is too complicated to handle correctly. > > > > + - FIXME: Not sure what to do for ISCII. > > > > Therefore, just convert string without checking */ > > > > wlen = MultiByteToWideChar (cp_from, 0, ptr_from, len_from, > > > > wbuf, NT_MAX_PATH); > > > > -- > > > > 2.28.0 > > > > > > I'd prefer to not handle them at all. We just don't support these > > > charsets, same as JIS, EBCDIC, you name it, which are not ASCII > > > compatible. Let's please just drop any handling for these weird > > > or outdated codepages. > > > > What do you mean by "just drop any handling"? > > > > Do you mean remove following if block? > > > > + if (cp_from == CP_UTF7 || IS_ISO_2022 (cp_from) || IS_ISCII (cp_from)) > > > > + /* - MB_ERR_INVALID_CHARS does not work properly for UTF-7. > > > > + - ISO-2022 is too complicated to handle correctly. > > > > + - FIXME: Not sure what to do for ISCII. > > > > Therefore, just convert string without checking */ > > > > wlen = MultiByteToWideChar (cp_from, 0, ptr_from, len_from, > > > > wbuf, NT_MAX_PATH); > > In this case, the conversion for ISO-2022, ISCII and UTF-7 will > > not be done correctly. > > > > Or skip charset conversion if the codepage is EBCDIC, ISO-2022 > > or ISCII? What should we do for UTF-7? > > Nothing, just like for any other of these weird charsets. Cygwin never > supported any charset which wasn't at least ASCII compatible in the > 0 <= x <= 127 range. Just ignore them and the possibility that a > user chooses them for fun. > > > What should happen if user or apps chage codepage to one of them? > > Garbage output, I guess. We shouldn't really care. Do you mean a patch attached? Please try: (1) Open mintty with "env CYGWIN=disable_pcon mintty". (2) Start cmd.exe in that mintty. (3) Try chcp such as 37 (EBCDIC), 65000 (UTF-7), 50220 (ISO-2022), and 57002 (ISCII). (4) Execute dir or some other commands in cmd.exe. For 65000, 50220 adn 57002, even the prompt will be broken. Are the results as you expected? If pseudo console is enabled, all the above are work without problem. With the previous patch, the results was sane even if pseudo console is disabled. -- Takashi Yano <takashi.yano@nifty.ne.jp> [-- Attachment #2: 0001-Cygwin-pty-Drop-handling-for-UTF-7-in-convert_mb_str.patch --] [-- Type: application/octet-stream, Size: 3764 bytes --] From 7d26f4da2969425f7c3cc8e968a256d388b7e58a Mon Sep 17 00:00:00 2001 From: Takashi Yano <takashi.yano@nifty.ne.jp> Date: Sat, 12 Sep 2020 00:37:26 +0900 Subject: [PATCH] Cygwin: pty: Drop handling for UTF-7 in convert_mb_str(). - Charset conversion for UTF-7, ISO-2022 and ISCII, which are not supported in cygwin, does not work properly as a result. At the expense of the above, the code has been simplified a bit. --- winsup/cygwin/fhandler_tty.cc | 86 ++++++++++++++++------------------- 1 file changed, 38 insertions(+), 48 deletions(-) diff --git a/winsup/cygwin/fhandler_tty.cc b/winsup/cygwin/fhandler_tty.cc index 95b28c3da..8910af1e7 100644 --- a/winsup/cygwin/fhandler_tty.cc +++ b/winsup/cygwin/fhandler_tty.cc @@ -122,58 +122,48 @@ convert_mb_str (UINT cp_to, char *ptr_to, size_t *len_to, UINT cp_from, const char *ptr_from, size_t len_from, mbstate_t *mbp) { - size_t nlen; tmp_pathbuf tp; wchar_t *wbuf = tp.w_get (); int wlen = 0; - if (cp_from == CP_UTF7) - /* MB_ERR_INVALID_CHARS does not work properly for UTF-7. - Therefore, just convert string without checking */ - wlen = MultiByteToWideChar (cp_from, 0, ptr_from, len_from, - wbuf, NT_MAX_PATH); - else - { - char *tmpbuf = tp.c_get (); - memcpy (tmpbuf, mbp->__value.__wchb, mbp->__count); - if (mbp->__count + len_from > NT_MAX_PATH) - len_from = NT_MAX_PATH - mbp->__count; - memcpy (tmpbuf + mbp->__count, ptr_from, len_from); - int total_len = mbp->__count + len_from; - mbp->__count = 0; - int mblen = 0; - for (const char *p = tmpbuf; p < tmpbuf + total_len; p += mblen) - /* Max bytes in multibyte char is 4. */ - for (mblen = 1; mblen <= 4; mblen ++) - { - /* Try conversion */ - int l = MultiByteToWideChar (cp_from, MB_ERR_INVALID_CHARS, - p, mblen, - wbuf + wlen, NT_MAX_PATH - wlen); - if (l) - { /* Conversion Success */ - wlen += l; - break; - } - else if (mblen == 4) - { /* Conversion Fail */ - l = MultiByteToWideChar (cp_from, 0, p, 1, - wbuf + wlen, NT_MAX_PATH - wlen); - wlen += l; - mblen = 1; - break; - } - else if (p + mblen == tmpbuf + total_len) - { /* Multibyte char incomplete */ - memcpy (mbp->__value.__wchb, p, mblen); - mbp->__count = mblen; - break; - } - /* Retry conversion with extended length */ + char *tmpbuf = tp.c_get (); + memcpy (tmpbuf, mbp->__value.__wchb, mbp->__count); + if (mbp->__count + len_from > NT_MAX_PATH) + len_from = NT_MAX_PATH - mbp->__count; + memcpy (tmpbuf + mbp->__count, ptr_from, len_from); + int total_len = mbp->__count + len_from; + mbp->__count = 0; + int mblen = 0; + for (const char *p = tmpbuf; p < tmpbuf + total_len; p += mblen) + /* Max bytes in multibyte char supported is 4. */ + for (mblen = 1; mblen <= 4; mblen ++) + { + /* Try conversion */ + int l = MultiByteToWideChar (cp_from, MB_ERR_INVALID_CHARS, + p, mblen, + wbuf + wlen, NT_MAX_PATH - wlen); + if (l) + { /* Conversion Success */ + wlen += l; + break; } - } - nlen = WideCharToMultiByte (cp_to, 0, wbuf, wlen, - ptr_to, *len_to, NULL, NULL); - *len_to = nlen; + else if (mblen == 4) + { /* Conversion Fail */ + l = MultiByteToWideChar (cp_from, 0, p, 1, + wbuf + wlen, NT_MAX_PATH - wlen); + wlen += l; + mblen = 1; + break; + } + else if (p + mblen == tmpbuf + total_len) + { /* Multibyte char incomplete */ + memcpy (mbp->__value.__wchb, p, mblen); + mbp->__count = mblen; + break; + } + /* Retry conversion with extended length */ + } + *len_to = WideCharToMultiByte (cp_to, 0, wbuf, wlen, + ptr_to, *len_to, NULL, NULL); } static bool -- 2.28.0 ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH] Cygwin: pty: Add workaround for ISO-2022 and ISCII in convert_mb_str(). 2020-09-11 16:05 ` Takashi Yano @ 2020-09-11 17:38 ` Takashi Yano 2020-09-11 18:37 ` Takashi Yano 2020-09-11 18:14 ` Corinna Vinschen 1 sibling, 1 reply; 13+ messages in thread From: Takashi Yano @ 2020-09-11 17:38 UTC (permalink / raw) To: cygwin-patches [-- Attachment #1: Type: text/plain, Size: 4436 bytes --] On Sat, 12 Sep 2020 01:05:04 +0900 Takashi Yano via Cygwin-patches <cygwin-patches@cygwin.com> wrote: > On Fri, 11 Sep 2020 16:06:01 +0200 > Corinna Vinschen wrote: > > On Sep 11 21:35, Takashi Yano via Cygwin-patches wrote: > > > Hi Corinna, > > > > > > On Fri, 11 Sep 2020 14:08:40 +0200 > > > Corinna Vinschen wrote: > > > > On Sep 11 19:54, Takashi Yano via Cygwin-patches wrote: > > > > > - In convert_mb_str(), exclude ISO-2022 and ISCII from the processing > > > > > for the case that the multibyte char is splitted in the middle. > > > > > The reason is as follows. > > > > > * ISO-2022 is too complicated to handle correctly. > > > > > * Not sure what to do with ISCII. > > > > > --- > > > > > winsup/cygwin/fhandler_tty.cc | 9 +++++++-- > > > > > 1 file changed, 7 insertions(+), 2 deletions(-) > > > > > > > > > > diff --git a/winsup/cygwin/fhandler_tty.cc b/winsup/cygwin/fhandler_tty.cc > > > > > index 37d033bbe..ee5c6a90a 100644 > > > > > --- a/winsup/cygwin/fhandler_tty.cc > > > > > +++ b/winsup/cygwin/fhandler_tty.cc > > > > > @@ -117,6 +117,9 @@ CreateProcessW_Hooked > > > > > return CreateProcessW_Orig (n, c, pa, ta, inh, f, e, d, si, pi); > > > > > } > > > > > > > > > > +#define IS_ISO_2022(x) ( (x) >= 50220 && (x) <= 50229 ) > > > > > +#define IS_ISCII(x) ( (x) >= 57002 && (x) <= 57011 ) > > > > > + > > > > > static void > > > > > convert_mb_str (UINT cp_to, char *ptr_to, size_t *len_to, > > > > > UINT cp_from, const char *ptr_from, size_t len_from, > > > > > @@ -126,8 +129,10 @@ convert_mb_str (UINT cp_to, char *ptr_to, size_t *len_to, > > > > > tmp_pathbuf tp; > > > > > wchar_t *wbuf = tp.w_get (); > > > > > int wlen = 0; > > > > > - if (cp_from == CP_UTF7) > > > > > - /* MB_ERR_INVALID_CHARS does not work properly for UTF-7. > > > > > + if (cp_from == CP_UTF7 || IS_ISO_2022 (cp_from) || IS_ISCII (cp_from)) > > > > > + /* - MB_ERR_INVALID_CHARS does not work properly for UTF-7. > > > > > + - ISO-2022 is too complicated to handle correctly. > > > > > + - FIXME: Not sure what to do for ISCII. > > > > > Therefore, just convert string without checking */ > > > > > wlen = MultiByteToWideChar (cp_from, 0, ptr_from, len_from, > > > > > wbuf, NT_MAX_PATH); > > > > > -- > > > > > 2.28.0 > > > > > > > > I'd prefer to not handle them at all. We just don't support these > > > > charsets, same as JIS, EBCDIC, you name it, which are not ASCII > > > > compatible. Let's please just drop any handling for these weird > > > > or outdated codepages. > > > > > > What do you mean by "just drop any handling"? > > > > > > Do you mean remove following if block? > > > > > + if (cp_from == CP_UTF7 || IS_ISO_2022 (cp_from) || IS_ISCII (cp_from)) > > > > > + /* - MB_ERR_INVALID_CHARS does not work properly for UTF-7. > > > > > + - ISO-2022 is too complicated to handle correctly. > > > > > + - FIXME: Not sure what to do for ISCII. > > > > > Therefore, just convert string without checking */ > > > > > wlen = MultiByteToWideChar (cp_from, 0, ptr_from, len_from, > > > > > wbuf, NT_MAX_PATH); > > > In this case, the conversion for ISO-2022, ISCII and UTF-7 will > > > not be done correctly. > > > > > > Or skip charset conversion if the codepage is EBCDIC, ISO-2022 > > > or ISCII? What should we do for UTF-7? > > > > Nothing, just like for any other of these weird charsets. Cygwin never > > supported any charset which wasn't at least ASCII compatible in the > > 0 <= x <= 127 range. Just ignore them and the possibility that a > > user chooses them for fun. > > > > > What should happen if user or apps chage codepage to one of them? > > > > Garbage output, I guess. We shouldn't really care. > > Do you mean a patch attached? > > Please try: > (1) Open mintty with "env CYGWIN=disable_pcon mintty". > (2) Start cmd.exe in that mintty. > (3) Try chcp such as > 37 (EBCDIC), > 65000 (UTF-7), > 50220 (ISO-2022), > and 57002 (ISCII). > (4) Execute dir or some other commands in cmd.exe. > > For 65000, 50220 adn 57002, even the prompt will be broken. > Are the results as you expected? > > If pseudo console is enabled, all the above are work without > problem. With the previous patch, the results was sane even > if pseudo console is disabled. How about the patch attached? I think this is safer than previous patch. -- Takashi Yano <takashi.yano@nifty.ne.jp> [-- Attachment #2: 0001-Cygwin-pty-Skip-multibyte-char-boundary-check-condit.patch --] [-- Type: application/octet-stream, Size: 1770 bytes --] From be6d20abf3027ccf24c549c58a7e1d05c1ea4dbd Mon Sep 17 00:00:00 2001 From: Takashi Yano <takashi.yano@nifty.ne.jp> Date: Sat, 12 Sep 2020 02:29:21 +0900 Subject: [PATCH] Cygwin: pty: Skip multibyte char boundary check conditionally. - For charset in which MB_ERR_INVALID_CHARS does not work properly, skip multibyte char boundary check in convert_mb_str(). --- winsup/cygwin/fhandler_tty.cc | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/winsup/cygwin/fhandler_tty.cc b/winsup/cygwin/fhandler_tty.cc index 95b28c3da..50c36f645 100644 --- a/winsup/cygwin/fhandler_tty.cc +++ b/winsup/cygwin/fhandler_tty.cc @@ -122,13 +122,15 @@ convert_mb_str (UINT cp_to, char *ptr_to, size_t *len_to, UINT cp_from, const char *ptr_from, size_t len_from, mbstate_t *mbp) { - size_t nlen; + bool check_valid = false; + if (MultiByteToWideChar (cp_from, MB_ERR_INVALID_CHARS, "A", 1, NULL, 0)) + check_valid = true; tmp_pathbuf tp; wchar_t *wbuf = tp.w_get (); int wlen = 0; - if (cp_from == CP_UTF7) - /* MB_ERR_INVALID_CHARS does not work properly for UTF-7. - Therefore, just convert string without checking */ + if (!check_valid) + /* If MB_ERR_INVALID_CHARS does not work properly, + just convert string without checking */ wlen = MultiByteToWideChar (cp_from, 0, ptr_from, len_from, wbuf, NT_MAX_PATH); else @@ -171,9 +173,8 @@ convert_mb_str (UINT cp_to, char *ptr_to, size_t *len_to, /* Retry conversion with extended length */ } } - nlen = WideCharToMultiByte (cp_to, 0, wbuf, wlen, - ptr_to, *len_to, NULL, NULL); - *len_to = nlen; + *len_to = WideCharToMultiByte (cp_to, 0, wbuf, wlen, + ptr_to, *len_to, NULL, NULL); } static bool -- 2.28.0 ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH] Cygwin: pty: Add workaround for ISO-2022 and ISCII in convert_mb_str(). 2020-09-11 17:38 ` Takashi Yano @ 2020-09-11 18:37 ` Takashi Yano 2020-09-11 18:57 ` Corinna Vinschen 0 siblings, 1 reply; 13+ messages in thread From: Takashi Yano @ 2020-09-11 18:37 UTC (permalink / raw) To: cygwin-patches On Sat, 12 Sep 2020 02:38:43 +0900 Takashi Yano via Cygwin-patches <cygwin-patches@cygwin.com> wrote: > On Sat, 12 Sep 2020 01:05:04 +0900 > Takashi Yano via Cygwin-patches <cygwin-patches@cygwin.com> wrote: > > On Fri, 11 Sep 2020 16:06:01 +0200 > > Corinna Vinschen wrote: > > > On Sep 11 21:35, Takashi Yano via Cygwin-patches wrote: > > > > Hi Corinna, > > > > > > > > On Fri, 11 Sep 2020 14:08:40 +0200 > > > > Corinna Vinschen wrote: > > > > > On Sep 11 19:54, Takashi Yano via Cygwin-patches wrote: > > > > > > - In convert_mb_str(), exclude ISO-2022 and ISCII from the processing > > > > > > for the case that the multibyte char is splitted in the middle. > > > > > > The reason is as follows. > > > > > > * ISO-2022 is too complicated to handle correctly. > > > > > > * Not sure what to do with ISCII. > > > > > > --- > > > > > > winsup/cygwin/fhandler_tty.cc | 9 +++++++-- > > > > > > 1 file changed, 7 insertions(+), 2 deletions(-) > > > > > > > > > > > > diff --git a/winsup/cygwin/fhandler_tty.cc b/winsup/cygwin/fhandler_tty.cc > > > > > > index 37d033bbe..ee5c6a90a 100644 > > > > > > --- a/winsup/cygwin/fhandler_tty.cc > > > > > > +++ b/winsup/cygwin/fhandler_tty.cc > > > > > > @@ -117,6 +117,9 @@ CreateProcessW_Hooked > > > > > > return CreateProcessW_Orig (n, c, pa, ta, inh, f, e, d, si, pi); > > > > > > } > > > > > > > > > > > > +#define IS_ISO_2022(x) ( (x) >= 50220 && (x) <= 50229 ) > > > > > > +#define IS_ISCII(x) ( (x) >= 57002 && (x) <= 57011 ) > > > > > > + > > > > > > static void > > > > > > convert_mb_str (UINT cp_to, char *ptr_to, size_t *len_to, > > > > > > UINT cp_from, const char *ptr_from, size_t len_from, > > > > > > @@ -126,8 +129,10 @@ convert_mb_str (UINT cp_to, char *ptr_to, size_t *len_to, > > > > > > tmp_pathbuf tp; > > > > > > wchar_t *wbuf = tp.w_get (); > > > > > > int wlen = 0; > > > > > > - if (cp_from == CP_UTF7) > > > > > > - /* MB_ERR_INVALID_CHARS does not work properly for UTF-7. > > > > > > + if (cp_from == CP_UTF7 || IS_ISO_2022 (cp_from) || IS_ISCII (cp_from)) > > > > > > + /* - MB_ERR_INVALID_CHARS does not work properly for UTF-7. > > > > > > + - ISO-2022 is too complicated to handle correctly. > > > > > > + - FIXME: Not sure what to do for ISCII. > > > > > > Therefore, just convert string without checking */ > > > > > > wlen = MultiByteToWideChar (cp_from, 0, ptr_from, len_from, > > > > > > wbuf, NT_MAX_PATH); > > > > > > -- > > > > > > 2.28.0 > > > > > > > > > > I'd prefer to not handle them at all. We just don't support these > > > > > charsets, same as JIS, EBCDIC, you name it, which are not ASCII > > > > > compatible. Let's please just drop any handling for these weird > > > > > or outdated codepages. > > > > > > > > What do you mean by "just drop any handling"? > > > > > > > > Do you mean remove following if block? > > > > > > + if (cp_from == CP_UTF7 || IS_ISO_2022 (cp_from) || IS_ISCII (cp_from)) > > > > > > + /* - MB_ERR_INVALID_CHARS does not work properly for UTF-7. > > > > > > + - ISO-2022 is too complicated to handle correctly. > > > > > > + - FIXME: Not sure what to do for ISCII. > > > > > > Therefore, just convert string without checking */ > > > > > > wlen = MultiByteToWideChar (cp_from, 0, ptr_from, len_from, > > > > > > wbuf, NT_MAX_PATH); > > > > In this case, the conversion for ISO-2022, ISCII and UTF-7 will > > > > not be done correctly. > > > > > > > > Or skip charset conversion if the codepage is EBCDIC, ISO-2022 > > > > or ISCII? What should we do for UTF-7? > > > > > > Nothing, just like for any other of these weird charsets. Cygwin never > > > supported any charset which wasn't at least ASCII compatible in the > > > 0 <= x <= 127 range. Just ignore them and the possibility that a > > > user chooses them for fun. > > > > > > > What should happen if user or apps chage codepage to one of them? > > > > > > Garbage output, I guess. We shouldn't really care. > > > > Do you mean a patch attached? > > > > Please try: > > (1) Open mintty with "env CYGWIN=disable_pcon mintty". > > (2) Start cmd.exe in that mintty. > > (3) Try chcp such as > > 37 (EBCDIC), > > 65000 (UTF-7), > > 50220 (ISO-2022), > > and 57002 (ISCII). > > (4) Execute dir or some other commands in cmd.exe. > > > > For 65000, 50220 adn 57002, even the prompt will be broken. > > Are the results as you expected? > > > > If pseudo console is enabled, all the above are work without > > problem. With the previous patch, the results was sane even > > if pseudo console is disabled. > > How about the patch attached? > I think this is safer than previous patch. I have revised this patch to fit current git head, and submit to cygwin-patches@cygwin.com. -- Takashi Yano <takashi.yano@nifty.ne.jp> ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH] Cygwin: pty: Add workaround for ISO-2022 and ISCII in convert_mb_str(). 2020-09-11 18:37 ` Takashi Yano @ 2020-09-11 18:57 ` Corinna Vinschen 2020-09-11 19:11 ` Takashi Yano 0 siblings, 1 reply; 13+ messages in thread From: Corinna Vinschen @ 2020-09-11 18:57 UTC (permalink / raw) To: cygwin-patches On Sep 12 03:37, Takashi Yano via Cygwin-patches wrote: > On Sat, 12 Sep 2020 02:38:43 +0900 > Takashi Yano via Cygwin-patches <cygwin-patches@cygwin.com> wrote: > > How about the patch attached? > > I think this is safer than previous patch. > > I have revised this patch to fit current git head, and submit > to cygwin-patches@cygwin.com. Thanks, but I didn't apply this one because it doesn't really make sense to go to the extra effort to support outdated and incompatible codepages we don't handle as Cygwin codeset at all. IMHO it's not worth to call another MBTWC just to check if a codepage supports the MB_ERR_INVALID_CHARS flag. Corinna ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH] Cygwin: pty: Add workaround for ISO-2022 and ISCII in convert_mb_str(). 2020-09-11 18:57 ` Corinna Vinschen @ 2020-09-11 19:11 ` Takashi Yano 2020-10-13 11:44 ` Corinna Vinschen 0 siblings, 1 reply; 13+ messages in thread From: Takashi Yano @ 2020-09-11 19:11 UTC (permalink / raw) To: cygwin-patches On Fri, 11 Sep 2020 20:57:06 +0200 Corinna Vinschen wrote: > On Sep 12 03:37, Takashi Yano via Cygwin-patches wrote: > > On Sat, 12 Sep 2020 02:38:43 +0900 > > Takashi Yano via Cygwin-patches <cygwin-patches@cygwin.com> wrote: > > > How about the patch attached? > > > I think this is safer than previous patch. > > > > I have revised this patch to fit current git head, and submit > > to cygwin-patches@cygwin.com. > > Thanks, but I didn't apply this one because it doesn't really make sense > to go to the extra effort to support outdated and incompatible codepages > we don't handle as Cygwin codeset at all. IMHO it's not worth to call > another MBTWC just to check if a codepage supports the MB_ERR_INVALID_CHARS > flag. I have checked which codepage does not support MB_ERR_INVALID_CHARS. The result is as follows. 42 50220 50221 50222 50225 50227 50229 52936 57002 57003 57004 57005 57006 57007 57008 57009 57010 57011 65000 If all of these are not worth for everyone, I agree with you. -- Takashi Yano <takashi.yano@nifty.ne.jp> ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH] Cygwin: pty: Add workaround for ISO-2022 and ISCII in convert_mb_str(). 2020-09-11 19:11 ` Takashi Yano @ 2020-10-13 11:44 ` Corinna Vinschen 0 siblings, 0 replies; 13+ messages in thread From: Corinna Vinschen @ 2020-10-13 11:44 UTC (permalink / raw) To: cygwin-patches On Sep 12 04:11, Takashi Yano via Cygwin-patches wrote: > On Fri, 11 Sep 2020 20:57:06 +0200 > Corinna Vinschen wrote: > > On Sep 12 03:37, Takashi Yano via Cygwin-patches wrote: > > > On Sat, 12 Sep 2020 02:38:43 +0900 > > > Takashi Yano via Cygwin-patches <cygwin-patches@cygwin.com> wrote: > > > > How about the patch attached? > > > > I think this is safer than previous patch. > > > > > > I have revised this patch to fit current git head, and submit > > > to cygwin-patches@cygwin.com. > > > > Thanks, but I didn't apply this one because it doesn't really make sense > > to go to the extra effort to support outdated and incompatible codepages > > we don't handle as Cygwin codeset at all. IMHO it's not worth to call > > another MBTWC just to check if a codepage supports the MB_ERR_INVALID_CHARS > > flag. > > I have checked which codepage does not support MB_ERR_INVALID_CHARS. > The result is as follows. > > 42 > 50220 > 50221 > 50222 > 50225 > 50227 > 50229 > 52936 > 57002 > 57003 > 57004 > 57005 > 57006 > 57007 > 57008 > 57009 > 57010 > 57011 > 65000 Yup, these are documented on MSDN: https://docs.microsoft.com/en-us/windows/win32/api/stringapiset/nf-stringapiset-widechartomultibyte > If all of these are not worth for everyone, I agree with you. I think we can skip those. Corinna ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH] Cygwin: pty: Add workaround for ISO-2022 and ISCII in convert_mb_str(). 2020-09-11 16:05 ` Takashi Yano 2020-09-11 17:38 ` Takashi Yano @ 2020-09-11 18:14 ` Corinna Vinschen 1 sibling, 0 replies; 13+ messages in thread From: Corinna Vinschen @ 2020-09-11 18:14 UTC (permalink / raw) To: cygwin-patches On Sep 12 01:05, Takashi Yano via Cygwin-patches wrote: > On Fri, 11 Sep 2020 16:06:01 +0200 > Corinna Vinschen wrote: > > On Sep 11 21:35, Takashi Yano via Cygwin-patches wrote: > > > What do you mean by "just drop any handling"? > > > > > > Do you mean remove following if block? > > > > > + if (cp_from == CP_UTF7 || IS_ISO_2022 (cp_from) || IS_ISCII (cp_from)) > > > > > + /* - MB_ERR_INVALID_CHARS does not work properly for UTF-7. > > > > > + - ISO-2022 is too complicated to handle correctly. > > > > > + - FIXME: Not sure what to do for ISCII. > > > > > Therefore, just convert string without checking */ > > > > > wlen = MultiByteToWideChar (cp_from, 0, ptr_from, len_from, > > > > > wbuf, NT_MAX_PATH); > > > In this case, the conversion for ISO-2022, ISCII and UTF-7 will > > > not be done correctly. > > > > > > Or skip charset conversion if the codepage is EBCDIC, ISO-2022 > > > or ISCII? What should we do for UTF-7? > > > > Nothing, just like for any other of these weird charsets. Cygwin never > > supported any charset which wasn't at least ASCII compatible in the > > 0 <= x <= 127 range. Just ignore them and the possibility that a > > user chooses them for fun. > > > > > What should happen if user or apps chage codepage to one of them? > > > > Garbage output, I guess. We shouldn't really care. > > Do you mean a patch attached? Yes. I pushed it. We should really not care for them. Thanks, Corinna ^ permalink raw reply [flat|nested] 13+ messages in thread
end of thread, other threads:[~2020-10-13 11:44 UTC | newest] Thread overview: 13+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2020-09-11 10:54 [PATCH] Cygwin: pty: Add workaround for ISO-2022 and ISCII in convert_mb_str() Takashi Yano 2020-09-11 12:08 ` Corinna Vinschen 2020-09-11 12:35 ` Takashi Yano 2020-09-11 14:06 ` Corinna Vinschen 2020-09-11 15:10 ` Thomas Wolff 2020-09-11 15:18 ` Thomas Wolff 2020-09-11 16:05 ` Takashi Yano 2020-09-11 17:38 ` Takashi Yano 2020-09-11 18:37 ` Takashi Yano 2020-09-11 18:57 ` Corinna Vinschen 2020-09-11 19:11 ` Takashi Yano 2020-10-13 11:44 ` Corinna Vinschen 2020-09-11 18:14 ` Corinna Vinschen
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).