* [ANNOUNCEMENT] Updated: libreadline7-7.0.3-3, libreadline-devel-7.0.3-3 @ 2017-02-13 19:53 Eric Blake (cygwin) 2017-03-25 0:46 ` Steven Penny 0 siblings, 1 reply; 37+ messages in thread From: Eric Blake (cygwin) @ 2017-02-13 19:53 UTC (permalink / raw) To: cygwin A new release of readline, 7.0.3-3, has been uploaded and will soon reach a mirror near you. The previous version is now 7.0.1-2 (which was experimental, but never current; but the only difference from 7.0.1-1 was handling of pselect which is now fixed in cygwin 2.7.0-1). NEWS: ===== This is a rebuild to fold in two new official upstream patches, and to build against newer cygwin1.dll that fixes handling of alt-numkeypad extended character entry in a windows console. Remember, you must not have any bash or /bin/sh instances running when you upgrade the readline package. This release requires cygwin-2.7.0-1 or later. See also the upstream documentation in /usr/share/doc/readline/. DESCRIPTION: ============ The readline library will read a line from the terminal and return it, allowing the user to edit the line with emacs or vi editing keys. It also allows a history feature, for editing previous entries, making command line interfaces easier-to-use and more intuitive. libreadline7 provides the .dlls needed for readline and history expansion for dynamic linking in other programs, including bash and gdb; it is required for a minimal cygwin installation. libreadline-devel provides the documentation and the static libraries required for static linking; you should only need it if you plan on compiling an application that links with -lreadline or -lhistory. UPDATE: ======= To update your installation, click on the "Install Cygwin now" link on the http://cygwin.com/ web page. This downloads setup.exe to your system. Save it and run setup, answer the questions and pick up 'libreadline7' in the 'Base' category (it should already be selected), or 'libreadline-devel' in the 'Devel' category. DOWNLOAD: ========= Note that downloads from cygwin.com aren't allowed due to bandwidth limitations. This means that you will need to find a mirror which has this update, please choose the one nearest to you: http://cygwin.com/mirrors.html QUESTIONS: ========== If you want to make a point or ask a question the Cygwin mailing list is the appropriate place. -- Eric Blake volunteer cygwin readline package maintainer For more details on this list (including unsubscription), see: http://sourceware.org/lists.html -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [ANNOUNCEMENT] Updated: libreadline7-7.0.3-3, libreadline-devel-7.0.3-3 2017-02-13 19:53 [ANNOUNCEMENT] Updated: libreadline7-7.0.3-3, libreadline-devel-7.0.3-3 Eric Blake (cygwin) @ 2017-03-25 0:46 ` Steven Penny 2017-04-13 19:07 ` [ANNOUNCEMENT] Updated: libreadline7-7.0.3-3 Steven Penny 0 siblings, 1 reply; 37+ messages in thread From: Steven Penny @ 2017-03-25 0:46 UTC (permalink / raw) To: cygwin On Mon, 13 Feb 2017 13:49:40, "Eric Blake (cygwin)" wrote: > A new release of readline, 7.0.3-3, has been uploaded and will soon > reach a mirror near you. The previous version is now 7.0.1-2 (which was > experimental, but never current; but the only difference from 7.0.1-1 > was handling of pselect which is now fixed in cygwin 2.7.0-1). Since we have determined that this: http://cygwin.com/ml/cygwin/2017-02/msg00031.html is a Readline problem (and Bash problem by nature of dependency), I am reposting under the correct thread. The error linked above occurs when UTF-8 (chcp.com 65001) is set. This can be worked around by using "chcp.com 437", but UTF-8 is needed with mingw32 programs: http://cygwin.com/ml/cygwin/2017-03/msg00193.html -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [ANNOUNCEMENT] Updated: libreadline7-7.0.3-3 2017-03-25 0:46 ` Steven Penny @ 2017-04-13 19:07 ` Steven Penny 2017-04-14 8:27 ` Eric Blake 0 siblings, 1 reply; 37+ messages in thread From: Steven Penny @ 2017-04-13 19:07 UTC (permalink / raw) To: cygwin On Fri, 24 Mar 2017 17:28:11, Steven Penny wrote: > On Mon, 13 Feb 2017 13:49:40, "Eric Blake (cygwin)" wrote: > > A new release of readline, 7.0.3-3, has been uploaded and will soon > > reach a mirror near you. The previous version is now 7.0.1-2 (which was > > experimental, but never current; but the only difference from 7.0.1-1 > > was handling of pselect which is now fixed in cygwin 2.7.0-1). > Since we have determined that this: > http://cygwin.com/ml/cygwin/2017-02/msg00031.html > is a Readline problem (and Bash problem by nature of dependency), I am > reposting under the correct thread. The error linked above occurs when UTF-8 > (chcp.com 65001) is set. This can be worked around by using "chcp.com 437", > but UTF-8 is needed with mingw32 programs: http://cygwin.com/ml/cygwin/2017-03/msg00312.html Reposting this since it is a new month, and Eric has yet to respond. -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [ANNOUNCEMENT] Updated: libreadline7-7.0.3-3 2017-04-13 19:07 ` [ANNOUNCEMENT] Updated: libreadline7-7.0.3-3 Steven Penny @ 2017-04-14 8:27 ` Eric Blake 2017-04-15 10:59 ` Steven Penny 0 siblings, 1 reply; 37+ messages in thread From: Eric Blake @ 2017-04-14 8:27 UTC (permalink / raw) To: cygwin [-- Attachment #1.1: Type: text/plain, Size: 1445 bytes --] On 04/13/2017 01:34 PM, Steven Penny wrote: > On Fri, 24 Mar 2017 17:28:11, Steven Penny wrote: >> On Mon, 13 Feb 2017 13:49:40, "Eric Blake (cygwin)" wrote: >> > A new release of readline, 7.0.3-3, has been uploaded and will soon >> > reach a mirror near you. The previous version is now 7.0.1-2 (which >> was >> > experimental, but never current; but the only difference from 7.0.1-1 >> > was handling of pselect which is now fixed in cygwin 2.7.0-1). >> Since we have determined that this: >> http://cygwin.com/ml/cygwin/2017-02/msg00031.html >> is a Readline problem (and Bash problem by nature of dependency), I am >> reposting under the correct thread. The error linked above occurs when >> UTF-8 >> (chcp.com 65001) is set. This can be worked around by using "chcp.com >> 437", >> but UTF-8 is needed with mingw32 programs: > > http://cygwin.com/ml/cygwin/2017-03/msg00312.html > > Reposting this since it is a new month, and Eric has yet to respond. Is it still a problem with pselect, where rebuilding with the same configuration as 7.0.1-2 fixes things? I'm really not sure how to even go about debugging this one, and it's not my highest priority at the moment (I've got coreutils 8.27 to build for cygwin, and autoconf 2.70 to release upstream). So any help is welcome. -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 604 bytes --] ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [ANNOUNCEMENT] Updated: libreadline7-7.0.3-3 2017-04-14 8:27 ` Eric Blake @ 2017-04-15 10:59 ` Steven Penny 2017-05-15 18:21 ` Eric Blake 2017-07-27 21:37 ` Eric Blake 0 siblings, 2 replies; 37+ messages in thread From: Steven Penny @ 2017-04-15 10:59 UTC (permalink / raw) To: cygwin On Thu, 13 Apr 2017 13:48:04, Eric Blake wrote: > Is it still a problem with pselect, where rebuilding with the same > configuration as 7.0.1-2 fixes things? I'm really not sure how to even > go about debugging this one, and it's not my highest priority at the > moment (I've got coreutils 8.27 to build for cygwin, and autoconf 2.70 > to release upstream). So any help is welcome. Ok. I have not gone through the whole commit, as it is huge: http://cygwin.com/ml/cygwin/2017-01/msg00204.html but I did find something. Using: git checkout readline-7.0-alpha~1 for the last good commit and: git checkout readline-7.0-alpha for the first bad commit, I found that the change to the "rl_insert" function in "text.c" breaks pasting and Alt codes with "chcp.com 65001". Can you work with this? http://git.savannah.gnu.org/cgit/readline.git/tree/text.c?h=readline-7.0-alpha#n891 --- a/text.c +++ b/text.c @@ -71,6 +71,8 @@ static int _rl_char_search_callback PARAMS((_rl_callback_generic_arg *)); rl_insert_text. Text blocks larger than this are divided. */ #define TEXT_COUNT_MAX 1024 +int _rl_optimize_typeahead = 1; /* rl_insert tries to read typeahead */ + /* **************************************************************** */ /* */ /* Insert and Delete */ @@ -890,8 +892,42 @@ int rl_insert (count, c) int count, c; { - return (rl_insert_mode == RL_IM_INSERT ? _rl_insert_char (count, c) - : _rl_overwrite_char (count, c)); + int r, n, x; + + r = (rl_insert_mode == RL_IM_INSERT) ? _rl_insert_char (count, c) : _rl_overwrite_char (count, c); + + /* XXX -- attempt to batch-insert pending input that maps to self-insert */ + x = 0; + n = (unsigned short)-2; + while (_rl_optimize_typeahead && + (RL_ISSTATE (RL_STATE_INPUTPENDING|RL_STATE_MACROINPUT) == 0) && + _rl_pushed_input_available () == 0 && + _rl_input_queued (0) && + (n = rl_read_key ()) > 0 && + _rl_keymap[(unsigned char)n].type == ISFUNC && + _rl_keymap[(unsigned char)n].function == rl_insert) + { + r = (rl_insert_mode == RL_IM_INSERT) ? _rl_insert_char (1, n) : _rl_overwrite_char (1, n); + /* _rl_insert_char keeps its own set of pending characters to compose a + complete multibyte character, and only returns 1 if it sees a character + that's part of a multibyte character but too short to complete one. We + can try to read another character in the hopes that we will get the + next one or just punt. Right now we try to read another character. + We don't want to call rl_insert_next if _rl_insert_char has already + stored the character in the pending_bytes array because that will + result in doubled input. */ + n = (unsigned short)-2; + x++; /* count of bytes of typeahead read, currently unused */ + if (r == 1) /* read partial multibyte character */ + continue; + if (rl_done || r != 0) + break; + } + + if (n != (unsigned short)-2) /* -2 = sentinel value for having inserted N */ + r = rl_execute_next (n); + + return r; } -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [ANNOUNCEMENT] Updated: libreadline7-7.0.3-3 2017-04-15 10:59 ` Steven Penny @ 2017-05-15 18:21 ` Eric Blake 2017-05-15 20:46 ` Chet Ramey 2017-07-27 21:37 ` Eric Blake 1 sibling, 1 reply; 37+ messages in thread From: Eric Blake @ 2017-05-15 18:21 UTC (permalink / raw) To: cygwin, bug-bash, Steven Penny [-- Attachment #1.1: Type: text/plain, Size: 5542 bytes --] On 04/14/2017 11:33 PM, Steven Penny wrote: > On Thu, 13 Apr 2017 13:48:04, Eric Blake wrote: Sorry for my delay in noticing this. >> Is it still a problem with pselect, where rebuilding with the same >> configuration as 7.0.1-2 fixes things? I'm really not sure how to even >> go about debugging this one, and it's not my highest priority at the >> moment (I've got coreutils 8.27 to build for cygwin, and autoconf 2.70 >> to release upstream). So any help is welcome. > > Ok. I have not gone through the whole commit, as it is huge: > > http://cygwin.com/ml/cygwin/2017-01/msg00204.html > > but I did find something. Using: > > git checkout readline-7.0-alpha~1 > > for the last good commit and: > > git checkout readline-7.0-alpha > > for the first bad commit, I found that the change to the "rl_insert" > function in > "text.c" breaks pasting and Alt codes with "chcp.com 65001". Can you > work with > this? > > http://git.savannah.gnu.org/cgit/readline.git/tree/text.c?h=readline-7.0-alpha#n891 It's code I'm not familiar with, so I'm adding upstream bug-bash in the hopes that Chet might have an answer to why this code was changed, and if he is aware that the change may have broken things on Cygwin. > > > --- a/text.c > +++ b/text.c > @@ -71,6 +71,8 @@ static int _rl_char_search_callback > PARAMS((_rl_callback_generic_arg *)); > rl_insert_text. Text blocks larger than this are divided. */ > #define TEXT_COUNT_MAX 1024 > > +int _rl_optimize_typeahead = 1; /* rl_insert tries to read typeahead */ > + > /* **************************************************************** */ > /* */ > /* Insert and Delete */ > @@ -890,8 +892,42 @@ int > rl_insert (count, c) > int count, c; > { > - return (rl_insert_mode == RL_IM_INSERT ? _rl_insert_char (count, c) > - : _rl_overwrite_char (count, c)); > + int r, n, x; > + > + r = (rl_insert_mode == RL_IM_INSERT) ? _rl_insert_char (count, c) : > _rl_overwrite_char (count, c); > + > + /* XXX -- attempt to batch-insert pending input that maps to > self-insert */ > + x = 0; > + n = (unsigned short)-2; > + while (_rl_optimize_typeahead && > + (RL_ISSTATE (RL_STATE_INPUTPENDING|RL_STATE_MACROINPUT) == 0) && > + _rl_pushed_input_available () == 0 && > + _rl_input_queued (0) && > + (n = rl_read_key ()) > 0 && > + _rl_keymap[(unsigned char)n].type == ISFUNC && > + _rl_keymap[(unsigned char)n].function == rl_insert) Looking at JUST this line, I am also reminded that Cygwin dll handling is weird. For example, when building bash for Cygwin, I have to add the following (currently-downstream-only, but maybe I should propose it upstream) patch to bashline.c: +#if __CYGWIN__ +# ifdef __x86_64__ +# define IMP(x) __imp_##x +# else +# define IMP(x) _imp__##x +# endif +#else +# define IMP(x) x +#endif @@ -498,11 +513,12 @@ initialize_readline () kseq[0] = CTRL('J'); kseq[1] = '\0'; func = rl_function_of_keyseq (kseq, emacs_meta_keymap, (int *)NULL); - if (func == rl_vi_editing_mode) + extern rl_command_func_t *IMP(rl_vi_editing_mode); + if (func == rl_vi_editing_mode || func == IMP(rl_vi_editing_mode)) rl_unbind_key_in_map (CTRL('J'), emacs_meta_keymap); and similar. That is, anywhere that bash refers to an exported readline function pointer, checking for equality on Cygwin works only if I check both the original function name AND the __imp_rl_* function name, based on how importing functions works with dlls (perhaps that's a gcc bug that gcc doesn't automatically perform BOTH checks under the hood when looking for C function pointer compatibility, but I'm not enough of a compiler expert to know WHY it is needed, just that it solved issues that didn't work otherwise). I wonder if the comparison against rl_insert is incomplete, and needs to also check against __imp_rl_insert? > + { > + r = (rl_insert_mode == RL_IM_INSERT) ? _rl_insert_char (1, n) : > _rl_overwrite_char (1, n); > + /* _rl_insert_char keeps its own set of pending characters to > compose a > + complete multibyte character, and only returns 1 if it sees a > character > + that's part of a multibyte character but too short to complete > one. We > + can try to read another character in the hopes that we will get the > + next one or just punt. Right now we try to read another character. > + We don't want to call rl_insert_next if _rl_insert_char has already > + stored the character in the pending_bytes array because that will > + result in doubled input. */ > + n = (unsigned short)-2; > + x++; /* count of bytes of typeahead read, currently unused */ > + if (r == 1) /* read partial multibyte character */ > + continue; > + if (rl_done || r != 0) > + break; > + } > + > + if (n != (unsigned short)-2) /* -2 = sentinel value for having > inserted N */ > + r = rl_execute_next (n); > + > + return r; > } > > > -- > Problem reports: http://cygwin.com/problems.html > FAQ: http://cygwin.com/faq/ > Documentation: http://cygwin.com/docs.html > Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple > > -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 604 bytes --] ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [ANNOUNCEMENT] Updated: libreadline7-7.0.3-3 2017-05-15 18:21 ` Eric Blake @ 2017-05-15 20:46 ` Chet Ramey 2017-06-18 15:53 ` Steven Penny 2017-07-04 21:53 ` Steven Penny 0 siblings, 2 replies; 37+ messages in thread From: Chet Ramey @ 2017-05-15 20:46 UTC (permalink / raw) To: Eric Blake, cygwin, bug-bash, Steven Penny; +Cc: chet.ramey [-- Attachment #1.1: Type: text/plain, Size: 1342 bytes --] On 5/15/17 2:19 PM, Eric Blake wrote: >> git checkout readline-7.0-alpha >> >> for the first bad commit, I found that the change to the "rl_insert" >> function in >> "text.c" breaks pasting and Alt codes with "chcp.com 65001". Can you >> work with >> this? >> >> http://git.savannah.gnu.org/cgit/readline.git/tree/text.c?h=readline-7.0-alpha#n891 > > It's code I'm not familiar with, so I'm adding upstream bug-bash in the > hopes that Chet might have an answer to why this code was changed, and > if he is aware that the change may have broken things on Cygwin. It was inspired by the discussion starting with http://lists.gnu.org/archive/html/bug-readline/2015-05/msg00007.html The idea is to optimize pasted input using the assumption that it will be mostly composed of characters that map to self-insert, and you can batch read those characters and perform one display update. The way to test whether or not a character will be inserted into the editing buffer is to see whether or not it maps directly to self-insert. If that's the problem, there will have to be a cygwin-specific fix; it works elsewhere. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRU chet@case.edu http://cnswww.cns.cwru.edu/~chet/ [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 264 bytes --] ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [ANNOUNCEMENT] Updated: libreadline7-7.0.3-3 2017-05-15 20:46 ` Chet Ramey @ 2017-06-18 15:53 ` Steven Penny 2017-07-04 21:53 ` Steven Penny 1 sibling, 0 replies; 37+ messages in thread From: Steven Penny @ 2017-06-18 15:53 UTC (permalink / raw) To: cygwin On Mon, 15 May 2017 15:59:48, Chet Ramey wrote: > It was inspired by the discussion starting with > > http://lists.gnu.org/archive/html/bug-readline/2015-05/msg00007.html > > The idea is to optimize pasted input using the assumption that it will be > mostly composed of characters that map to self-insert, and you can batch > read those characters and perform one display update. > > The way to test whether or not a character will be inserted into the > editing buffer is to see whether or not it maps directly to self-insert. > If that's the problem, there will have to be a cygwin-specific fix; it > works elsewhere. http://cygwin.com/ml/cygwin/2017-05/msg00238.html Bumping this thread because it is a new month. Also of note, it is now over 4 months since my initial report, and the errant version of readline has not been patched or rolled back: http://cygwin.com/ml/cygwin/2017-02/msg00014.html and I did post the potential problem code back in April: http://cygwin.com/ml/cygwin/2017-04/msg00176.html -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [ANNOUNCEMENT] Updated: libreadline7-7.0.3-3 2017-05-15 20:46 ` Chet Ramey 2017-06-18 15:53 ` Steven Penny @ 2017-07-04 21:53 ` Steven Penny 1 sibling, 0 replies; 37+ messages in thread From: Steven Penny @ 2017-07-04 21:53 UTC (permalink / raw) To: cygwin On Mon, 15 May 2017 15:59:48, Chet Ramey wrote: > It was inspired by the discussion starting with > > http://lists.gnu.org/archive/html/bug-readline/2015-05/msg00007.html > > The idea is to optimize pasted input using the assumption that it will be > mostly composed of characters that map to self-insert, and you can batch > read those characters and perform one display update. > > The way to test whether or not a character will be inserted into the > editing buffer is to see whether or not it maps directly to self-insert. > If that's the problem, there will have to be a cygwin-specific fix; it > works elsewhere. http://cygwin.com/ml/cygwin/2017-05/msg00238.html Bumping this thread because it is a new month. Also of note, it is now over 5 months since my initial report, and the errant version of readline has not been patched or rolled back: http://cygwin.com/ml/cygwin/2017-02/msg00014.html and I did post the potential problem code back in April: http://cygwin.com/ml/cygwin/2017-04/msg00176.html -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [ANNOUNCEMENT] Updated: libreadline7-7.0.3-3 2017-04-15 10:59 ` Steven Penny 2017-05-15 18:21 ` Eric Blake @ 2017-07-27 21:37 ` Eric Blake 2017-07-27 21:39 ` Eric Blake 2017-07-27 21:46 ` Steven Penny 1 sibling, 2 replies; 37+ messages in thread From: Eric Blake @ 2017-07-27 21:37 UTC (permalink / raw) To: cygwin, Steven Penny [-- Attachment #1.1: Type: text/plain, Size: 2944 bytes --] On 04/14/2017 11:33 PM, Steven Penny wrote: > On Thu, 13 Apr 2017 13:48:04, Eric Blake wrote: >> Is it still a problem with pselect, where rebuilding with the same >> configuration as 7.0.1-2 fixes things? I've got some time today to look at building readline, but for the life of me, I can't figure out what I'm supposed to be debugging. You have so many emails saying "see this earlier URL" that I am lost in what you are saying is wrong or how to reproduce it. I'm currently testing with: bash 4.4.12-3 cygwin 2.8.2-1 libreadline7 7.0.3-3 (or self-built) > I'm really not sure how to even >> go about debugging this one, and it's not my highest priority at the >> moment (I've got coreutils 8.27 to build for cygwin, and autoconf 2.70 >> to release upstream). So any help is welcome. > > Ok. I have not gone through the whole commit, as it is huge: > > http://cygwin.com/ml/cygwin/2017-01/msg00204.html > > but I did find something. Using: > > git checkout readline-7.0-alpha~1 > > for the last good commit and: > > git checkout readline-7.0-alpha > > for the first bad commit, I found that the change to the "rl_insert" > function in > "text.c" breaks pasting and Alt codes with "chcp.com 65001". Can you > work with > this? Thanks again for trying to narrow things down. I have recompiled readline locally with optimizations turned off (so it's easier for me to see what's going on), and am set up to run gdb on bash with a given readline executable installed. If you have really narrowed the problem to rl_insert(), that's at least something I can investigate. But where I'm stuck now is what works for you and what you think is wrong. Is this something where I can start bash under mintty, or do I have to start under cmd? Right there, I already see a difference with the two environments. Starting from cmd, I did: c:\cygwin\bin> od -tx1 <Alt-num2-num3-num4><Enter><Ctrl-d> which displayed Ω 00000000 ce a9 0z 0000003 so I did indeed insert GREEK CAPITAL LETTER OMEGA U+03A9. But trying the same thing under a bash session in minty shows: ê 0000000 c3 aa 0a 0000003 so that is not the same character. I'm not sure if a code page change is supposed to alter what I see. So I'm back to cmd to try and debug things. Next, I tried: c:\cygwin\bin> .\dash <alt-num2-num3-num4> and again got Ω; pressing <enter> complains that ./dash: 1: Ω: not found However, when I try: c:\cygwin\bin> .\bash --norc <alt-num2-num3-num4> the display shows :\251 and hitting <Enter> wipes out that display without doing anything. So I _think_ I'm running into the problem you're describing, but want to make sure, since it is different based on whether I started bash from cmd or from mintty. -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 619 bytes --] ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [ANNOUNCEMENT] Updated: libreadline7-7.0.3-3 2017-07-27 21:37 ` Eric Blake @ 2017-07-27 21:39 ` Eric Blake 2017-07-27 21:46 ` Steven Penny 1 sibling, 0 replies; 37+ messages in thread From: Eric Blake @ 2017-07-27 21:39 UTC (permalink / raw) To: cygwin [-- Attachment #1.1: Type: text/plain, Size: 3564 bytes --] On 07/27/2017 12:08 PM, Eric Blake wrote: > So I'm back to cmd to try and debug things. Next, I tried: > > c:\cygwin\bin> .\dash > <alt-num2-num3-num4> > > and again got Ω; pressing <enter> complains that ./dash: 1: Ω: not found To double check things, I started .\dash, typed 'echo $$', then in a second terminal, typed 'gdb --pid XXX' with the dash pid. (gdb) b read (gdb) b select (gdb) c then in the first window, typed <a><enter> to get dash back to its input loop and the second window hit a breakpoint in read. But that didn't get me very far: Thread 1 hit Breakpoint 1, read (fd=0, ptr=0x41b540 <basebuf>, len=1024) at /usr/src/debug/cygwin-2.8.2-1/winsup/cygwin/syscalls.cc:1118 1118 { (gdb) fin Run till exit from #0 read (fd=0, ptr=0x41b540 <basebuf>, len=1024) at /usr/src/debug/cygwin-2.8.2-1/winsup/cygwin/syscalls.cc:1118 [New Thread 628.0x70c] readline: readline_callback_read_char() called with no handler! Aborted (core dumped) Urgh - gdb uses readline, so debugging readline with gdb may prove harder than planned if I don't time things right. A second time around, and instead of using fin, I stepped through: 1118 { (gdb) n ... (gdb) n 1139 cfd->read (ptr, len); (gdb) n [New Thread 736.0x960] at the point of the new thread, I typed <alt-num2-num3-num4><enter> in the first terminal, which let the read return, and the buffer contents are correct: 1140 res = len; (gdb) p len $1 = 3 (gdb) p/x ((char*)ptr)[0] $2 = 0xce (gdb) p/x ((char*)ptr)[1] $3 = 0xa9 (gdb) p ((char*)ptr)[2] $4 = '\n' so whatever dash did, it read a solid block of input from the terminal; from there, I quit debugging - obviously dash is not doing things piecemeal, and manages to replay the same output as it just read in input (when you aren't trying hard to be interactive, life is easy). > > However, when I try: > > c:\cygwin\bin> .\bash --norc > <alt-num2-num3-num4> > > the display shows :\251 Repeating the gdb attach trick, I'm able to catch bash at this breakpoint, even without hitting <enter>, just by typing <a>: Thread 1 hit Breakpoint 1, read (fd=0, ptr=0x28c013, len=1) at /usr/src/debug/cygwin-2.8.2-1/winsup/cygwin/syscalls.cc:1118 Notice a difference? dash had the terminal set up in line-oriented mode, and blindly reads until EOL or until len=1024 is exhausted; bash has the terminal set up in byte-oriented mode, and is only reading len=1 at a time. So when entering a UTF-8 character to dash, the whole character lands in the buffer at once, while under bash (presumably, as I haven't debugged that far yet), bash must reconstruct the Unicode characters from the individual bytes. Stepping through the breakpoints on <alt-num2-num3-num4> sees 0xce on the first read, and 0xa9 on the second. But, in between the two read breakpoints, the first terminal displayed ':'. So the input is still making it correctly INTO readline; but being munged on the way to output; and it very much looks like readline's fault rather than cygwin's. I'm still trying to put breakpoints in the right places (the call stack at read() points to rl_getc(), from rl_read_key(), from readline_internal_char()...), but this is at least to let you know how I'm tackling the issue, in case it helps someone else spot a solution faster than me by starting from the same information. -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 619 bytes --] ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [ANNOUNCEMENT] Updated: libreadline7-7.0.3-3 2017-07-27 21:37 ` Eric Blake 2017-07-27 21:39 ` Eric Blake @ 2017-07-27 21:46 ` Steven Penny 2017-07-28 8:28 ` Eric Blake 1 sibling, 1 reply; 37+ messages in thread From: Steven Penny @ 2017-07-27 21:46 UTC (permalink / raw) To: cygwin On Thu, 27 Jul 2017 12:08:53, Eric Blake wrote: > I've got some time today to look at building readline, but for the life > of me, I can't figure out what I'm supposed to be debugging. You have > so many emails saying "see this earlier URL" that I am lost in what you > are saying is wrong or how to reproduce it. Thanks for this. Between your 2 emails, youve put a lot on the table. Instead of getting overwhelmed, I will just start my side of the convo by replaying the problem. Then if you need more from me I am happy to help. So, here is an example problem using LATIN SMALL LETTER O WITH DIAERESIS' (U+00F6): $ chcp.com 65001 Active code page: 65001 - Alt 148 outputs nothing - Alt 0246 outputs nothing - Pasting this character does not work $ chcp.com 437 Active code page: 437 - Alt 148 works - Alt 0246 works - Pasting works http://cygwin.com/ml/cygwin/2017-02/msg00014.html Now you might say, why not just use codepage 437? Which is exactly what Corinna did say: http://cygwin.com/ml/cygwin/2017-03/msg00193.html -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [ANNOUNCEMENT] Updated: libreadline7-7.0.3-3 2017-07-27 21:46 ` Steven Penny @ 2017-07-28 8:28 ` Eric Blake 2017-07-28 14:55 ` Steven Penny 0 siblings, 1 reply; 37+ messages in thread From: Eric Blake @ 2017-07-28 8:28 UTC (permalink / raw) To: cygwin [-- Attachment #1.1: Type: text/plain, Size: 5520 bytes --] On 07/27/2017 01:56 PM, Steven Penny wrote: > On Thu, 27 Jul 2017 12:08:53, Eric Blake wrote: >> I've got some time today to look at building readline, but for the life >> of me, I can't figure out what I'm supposed to be debugging. You have >> so many emails saying "see this earlier URL" that I am lost in what you >> are saying is wrong or how to reproduce it. > > Thanks for this. Between your 2 emails, youve put a lot on the table. > Instead > of getting overwhelmed, I will just start my side of the convo by > replaying the > problem. Then if you need more from me I am happy to help. So, here is an > example problem using LATIN SMALL LETTER O WITH DIAERESIS' (U+00F6): > > $ chcp.com 65001 I still don't know your environment (it's really hard to reproduce issues if I don't know the steps to reproduce them). This looks like a bash prompt, but are you running bash inside mintty, or directly in a cmd window? When I first open a mintty window to get bash, I see: $ chcp.com Active code page: 437 and in that environment, typing <alt-1-4-8> displays nothing, but hitting <enter> then displays: -bash: $'\302\224': command not found which maps to \xc2\x94; I can confirm that with 'od -tx1'. Trying <alt-0-2-4-6> gives a different character (¦), as \xc2\xa6. When I then do $chcp.com 65001 Active code page 65001 I don't see any change in behavior. But if I first open a cmd window, with NO bash in the mix, I see: c:\cygwin\bin> chcp Active code page: 437 where both <alt-1-4-8> and <alt-0-2-4-6> output ö, and where 'od -tx1' confirms both sequences produce \xc3\xb6. Then switching code pages: c:\cygwin\bin> chcp 65001 Active code page: 65001 directly typing <alt-0-2-4-6> prints nothing, while 'od -tx1' still shows that it received \xc3\xb6. I have no idea how alt- sequences are mapped to code points (it is not as trivial as a conversion of base to get either the Unicode code-point of 0x96 or to the UTF-8 encoding), but it appears that the input within cmd is the same, while the choice of code page determines what the output will be. I also have no idea why the alt- sequences produce different inputs under cmd than under mintty. So knowing WHAT environment you are using is VITAL to me understanding the results you are seeing. At any rate, I definitely know that U+00F6 is encoded as \xc3\xb6 in UTF-8 (I confirmed that on Linux, with echo $'\xc3\xb6'). I _don't_ know what it is encoded as in Windows code page 437 or 65001. But a quick google later, and I see that for code page 437 (https://en.wikipedia.org/wiki/Code_page_437), ö is at codepoint 0x94 (decimal 148, octal 0224); meanwhile, 0xf6 is equal to decimal 246. Aha - maybe that explains the two alt- sequences under codepage 437: without a leading zero, you are typing the decimal position which looks up the character from the current code page; WITH a leading zero you are directly requesting the decimal encoding of a Unicode character. And trying some other sequences, I note that õ (LATIN SMALL LETTER O WITH TILDE' (U+00F5)) is not part of code page 437; so there is nothing I can type without a leading 0 to print one; conversely, trying <alt-0-2-4-5> which requests the same unicode character displays merely 'o' (apparently U+006f), which, when you lack o-with-tilde, is a reasonable fallback compared to printing nothing at all. Either way, the character requested by the alt-sequence in the cmd window is then transformed by Cygwin into the appropriate UTF-8 input for the tty stdin of the Cygwin child process. Hmm; repeating those sequences under 'od -tx1', when I try <alt-0-2-4-5>, I see something interesting: the moment I press 5 (while still holding alt), the display prints [G; then releasing alt prints o; the transcription is then 0000000 1b 1b 5b 47 c3 b5 0a which is ESC ESC [ G (hmm - that's the ANSI terminal escape sequence for moving to column 0), followed by the actual Unicode õ, before my ending newline. No idea why that is leaking through to Cygwin to pick up as input. Is windows trying to beep at me to tell me my Unicode request doesn't exist in the current code page? Except that beep is Ctrl-G (U+0007). But when I switch to code page 65001, wikipedia redirects me to UTF-8. So in that code page, presumably all ALT sequences represent themselves, whether or not there is a leading 0? No, experimentation shows otherwise: <alt-2> shows nothing (and not the smiley face from codepage 437); while <alt-0-2> shows ^B (where ctrl-b really is code point 2). I have no idea WHAT sequence would thus give you ö. > Now you might say, why not just use codepage 437? Which is exactly what > Corinna > did say: > > http://cygwin.com/ml/cygwin/2017-03/msg00193.html Well, obviously, the code page matters to cmd; and I have no idea what alt- sequences do (or are supposed to do) under mintty. So there may STILL be some lingering craziness on what Cygwin itself should do when it recognizes an alt- sequence coming in (if cygwin translates from the current code page to Unicode, where the current code page definitely affects which character is desired); and that's _in addition_ to what appears to be the craziness in bash when reconstructing the UTF-8 sequence for omega Ω as mentioned in my other mail. -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 619 bytes --] ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [ANNOUNCEMENT] Updated: libreadline7-7.0.3-3 2017-07-28 8:28 ` Eric Blake @ 2017-07-28 14:55 ` Steven Penny 2017-07-28 18:31 ` Eric Blake 0 siblings, 1 reply; 37+ messages in thread From: Steven Penny @ 2017-07-28 14:55 UTC (permalink / raw) To: cygwin On Thu, 27 Jul 2017 16:37:45, Eric Blake wrote: > I still don't know your environment (it's really hard to reproduce > issues if I don't know the steps to reproduce them). This looks like a > bash prompt, but are you running bash inside mintty, or directly in a > cmd window? 1. Clean Cygwin install 2. Start Cygwin.bat 3. My previous email http://cygwin.com/ml/cygwin/2017-07/msg00388.html -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [ANNOUNCEMENT] Updated: libreadline7-7.0.3-3 2017-07-28 14:55 ` Steven Penny @ 2017-07-28 18:31 ` Eric Blake 2017-07-28 18:39 ` Steven Penny 0 siblings, 1 reply; 37+ messages in thread From: Eric Blake @ 2017-07-28 18:31 UTC (permalink / raw) To: cygwin [-- Attachment #1.1: Type: text/plain, Size: 1997 bytes --] On 07/27/2017 09:43 PM, Steven Penny wrote: > On Thu, 27 Jul 2017 16:37:45, Eric Blake wrote: >> I still don't know your environment (it's really hard to reproduce >> issues if I don't know the steps to reproduce them). This looks like a >> bash prompt, but are you running bash inside mintty, or directly in a >> cmd window? > > 1. Clean Cygwin install > > 2. Start Cygwin.bat As in /etc/defaults/Cygwin.bat installed by the base-files package? It's short: @echo off setlocal enableextensions set TERM= cd /d "%~dp0bin" && .\bash --login -i so if you are already in a cmd window, and typing cygwin.bat, then it is the same as if you are directly starting bash from cmd. By the way, I didn't know cygwin.bat was still around (I had to go hunting for it). Ages ago, that use to be the target of the shortcut installed to the desktop, but we switched quite a while ago to having the target point directly to mintty instead of the .bat file (since mintty is a LOT friendlier than running inside cmd). Also, what does 'chcp.com' say prior to you executing cygwin.bat, and are you then trying to call chcp.com WHILE bash is running? I have no idea if cygwin is even aware/amenable of live code page changes while running inside a cmd window; for now, I'm first trying to focus on the odd behavior in bash when the code page is constant. > > 3. My previous email http://cygwin.com/ml/cygwin/2017-07/msg00388.html Making me chase URLs doesn't help as much as a single mail, with a step-by-step reproduction of what you did vs. what you expected, so that I can refer to a single window rather than multiple browser tabs when trying to follow those same steps. I know it's not always easy to do that (and to some extent, my debugging is also spread out), so it's not the end of the world; but it is food for thought. -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 619 bytes --] ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [ANNOUNCEMENT] Updated: libreadline7-7.0.3-3 2017-07-28 18:31 ` Eric Blake @ 2017-07-28 18:39 ` Steven Penny 2017-07-28 23:55 ` Eric Blake 2017-07-31 18:36 ` Corinna Vinschen 0 siblings, 2 replies; 37+ messages in thread From: Steven Penny @ 2017-07-28 18:39 UTC (permalink / raw) To: cygwin On Fri, 28 Jul 2017 06:41:08, Eric Blake wrote: > As in /etc/defaults/Cygwin.bat installed by the base-files package? As in "C:\cygwin64\Cygwin.bat" that can be found after a regular install of Cygwin > It's short: > > @echo off > setlocal enableextensions > set TERM=3D > cd /d "%~dp0bin" && .\bash --login -i Uh, no? Mine looks like this. Again this is the file installed by Cygwin, not something I have home brewed: @echo off C: chdir C:\cygwin64\bin bash --login -i > so if you are already in a cmd window, and typing cygwin.bat, then it is > the same as if you are directly starting bash from cmd. I am not doing this, I am just using Windows explorer to go to "C:\cygwin64", then double clicking "Cygwin.bat" > By the way, I didn't know cygwin.bat was still around (I had to go > hunting for it). Ages ago, that use to be the target of the shortcut > installed to the desktop, but we switched quite a while ago to having > the target point directly to mintty instead of the .bat file (since > mintty is a LOT friendlier than running inside cmd). I dont want mintty. As you know mintty adds another "layer" to the situation, another process. If I was using mintty I would not have discovered this problem in the first place. Some might say good, but no. It is important that even launching Cygwin via Cygwin.bat/bash.exe works in a sane manner; and that it not sacrifice features that even cmd.exe has, which is the current situation. > Also, what does 'chcp.com' say prior to you executing cygwin.bat Prior to "Cygwin.bat", it says 65001, because I feel that is the proper default for Windows. I set it like this: REG ADD 'HKCU\Console' /v CodePage /t REG_DWORD /d 65001 /f > are you then trying to call chcp.com WHILE bash is running? I have no idea if > cygwin is even aware/amenable of live code page changes while running inside a > cmd window Yes, However I do not think this is germain to the conversation, as I have found you can change it "live" without issue. > Making me chase URLs doesn't help as much as a single mail, with a > step-by-step reproduction of what you did vs. what you expected, so that > I can refer to a single window rather than multiple browser tabs when > trying to follow those same steps. Cmon man, give me a break here, I am trying. I have been posting on this issue for months, and Id rather not keep regurgitation the same stuff over and over. At any rate, here is the text from that link, LATIN SMALL LETTER O WITH DIAERESIS' (U+00F6): $ chcp.com 65001 Active code page: 65001 - Alt 148 outputs nothing - Alt 0246 outputs nothing - Pasting this character does not work $ chcp.com 437 Active code page: 437 - Alt 148 works - Alt 0246 works - Pasting works and here is why 65001 is needed at all: $ cat omega.c #include <stdio.h> int main() { printf("Ω\n"); } $ x86_64-pc-cygwin-gcc -o cygwin.exe omega.c $ x86_64-w64-mingw32-gcc -o mingw32.exe omega.c $ chcp.com 65001 Active code page: 65001 $ ./cygwin.exe Ω $ ./mingw32.exe Ω $ chcp.com 437 Active code page: 437 $ ./cygwin.exe Ω $ ./mingw32.exe â¬â -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [ANNOUNCEMENT] Updated: libreadline7-7.0.3-3 2017-07-28 18:39 ` Steven Penny @ 2017-07-28 23:55 ` Eric Blake 2017-07-29 1:55 ` Cygwin.bat (was: [ANNOUNCEMENT] Updated: libreadline7-7.0.3-3) Achim Gratz ` (2 more replies) 2017-07-31 18:36 ` Corinna Vinschen 1 sibling, 3 replies; 37+ messages in thread From: Eric Blake @ 2017-07-28 23:55 UTC (permalink / raw) To: cygwin [-- Attachment #1.1: Type: text/plain, Size: 8012 bytes --] On 07/28/2017 07:09 AM, Steven Penny wrote: > On Fri, 28 Jul 2017 06:41:08, Eric Blake wrote: >> As in /etc/defaults/Cygwin.bat installed by the base-files package? > > As in "C:\cygwin64\Cygwin.bat" that can be found after a regular install of > Cygwin > Oh, that one doesn't show up under 'cygcheck -p', so it must be created by setup.exe. >> It's short: >> >> @echo off >> setlocal enableextensions >> set TERM=3D >> cd /d "%~dp0bin" && .\bash --login -i > > Uh, no? Mine looks like this. Again this is the file installed by > Cygwin, not > something I have home brewed: > > @echo off > C: > chdir C:\cygwin64\bin > bash --login -i Odd that there are two different flavors of the file - but the point remains that they are doing pretty much the same thing: starting bash. > >> so if you are already in a cmd window, and typing cygwin.bat, then it is >> the same as if you are directly starting bash from cmd. > > I am not doing this, I am just using Windows explorer to go to > "C:\cygwin64", > then double clicking "Cygwin.bat" Yay! That means you ARE running cygwin in a cmd window (and NOT a mintty window) - because that's what double-clicking the .bat file does. Okay, I'm a bit more confident that I'm at least in the right environment to reproduce what you are seeing. I still think the desktop shortcut pointing to mintty is a nicer environment than running bash in a cmd window, but that's orthogonal. > >> By the way, I didn't know cygwin.bat was still around (I had to go >> hunting for it). Ages ago, that use to be the target of the shortcut >> installed to the desktop, but we switched quite a while ago to having >> the target point directly to mintty instead of the .bat file (since >> mintty is a LOT friendlier than running inside cmd). > > I dont want mintty. As you know mintty adds another "layer" to the > situation, > another process. If I was using mintty I would not have discovered this > problem > in the first place. Some might say good, but no. It is important that even > launching Cygwin via Cygwin.bat/bash.exe works in a sane manner; and > that it not > sacrifice features that even cmd.exe has, which is the current situation. I agree that running under cmd should work as best as possible, but cmd is severely limited, and interactions with mintty get debugged faster than interactions with cmd. > >> Also, what does 'chcp.com' say prior to you executing cygwin.bat > > Prior to "Cygwin.bat", it says 65001, because I feel that is the proper > default > for Windows. I set it like this: > > REG ADD 'HKCU\Console' /v CodePage /t REG_DWORD /d 65001 /f Ah, so I see 437 by default on my setup, but you've made a system-wide tweak. So PART of your issue is that cygwin1.dll might not be paying attention to your tweak (since the code page DOES affect what alt- sequences produce when using cmd) - it is highly possible that cygwin1.dll still needs improvements with regards to translating alt- sequences according to the current code page (when codepage 437 is active, alt-[1-9]-NNN works according to the current page; while alt-0-NNN according to Unicode code point - insofar as the code point is representable in the current code page). But my other mail pointed out that when using a plain cmd window (no cygwin in the mix), I'm not entirely sure how the alt- sequences work for code page 65001 in the first place, to know if cygwin is even interpreting things correctly. > >> are you then trying to call chcp.com WHILE bash is running? I have no >> idea if >> cygwin is even aware/amenable of live code page changes while running >> inside a >> cmd window > > Yes, However I do not think this is germain to the conversation, as I > have found > you can change it "live" without issue. Well, it IS relevant if it means that live changes to the code page SHOULD affect cygwin1.dll behavior dynamically. It's not relevant to whether bash itself is mishandling alt- sequences in general (my debugging so far says that with code page 437, typing alt-1-4-8 produces \xc3\xb6 (the correct UTF-8 encoding of ö, which is decimal code 148 in code-page 437), but that bash's parser loop (in readline) is reading only one byte at a time, and tries to treat \xc3 as 'meta-n' and triggers its incremental-search functionality, then treats \xb6 as an incomplete character to be searched for; rather than realizing that the two bytes belong together as a multibyte character. That part of readline MIGHT be related to using pselect() (where configuring pselect out of the equation takes a different code path to learn if more data needs to be read before starting to act on the data). I still haven't figured out why readline is breaking the input or how to fix it. >> Making me chase URLs doesn't help as much as a single mail, with a >> step-by-step reproduction of what you did vs. what you expected, so that >> I can refer to a single window rather than multiple browser tabs when >> trying to follow those same steps. > > Cmon man, give me a break here, I am trying. I have been posting on this > issue > for months, and Id rather not keep regurgitation the same stuff over and > over. I know, you ARE helping. But I'm also saying that some forms of help are easier than others ("look at this link" is less helpful than "here are the steps"). And it doesn't help that my free time for cygwin is sporadic. > At any rate, here is the text from that link, > LATIN SMALL LETTER O WITH DIAERESIS' (U+00F6): > > $ chcp.com 65001 > Active code page: 65001 > > - Alt 148 outputs nothing > - Alt 0246 outputs nothing But that's true even in a bare cmd window. So it's hard to say what it is SUPPOSED to do. I'm trying to debug bash, not cmd's alt- behavior. > - Pasting this character does not work Pasting is different from alt- sequences, but I haven't played with that yet, because pasting in cmd windows is a pain compared to middle-click pasting in mintty (get the alt- stuff working first, and the pasting may magically work). > > $ chcp.com 437 > Active code page: 437 > > - Alt 148 works > - Alt 0246 works > - Pasting works Wait, it works for you in bash? It wasn't working for me. > > and here is why 65001 is needed at all: > > $ cat omega.c > #include <stdio.h> > int main() { > printf("Ω\n"); Bad form to assume your compiler is using a particular code set (but presumably you write that file in a UTF-8 editor); better would be: printf("\xce\xa9") which is unambiguously the UTF-8 bytes for U+03a9 (or even "\u03a9", if your C compiler is new enough). > } > > $ x86_64-pc-cygwin-gcc -o cygwin.exe omega.c > $ x86_64-w64-mingw32-gcc -o mingw32.exe omega.c > $ chcp.com 65001 > Active code page: 65001 > > $ ./cygwin.exe > Ω > > $ ./mingw32.exe > Ω > That makes sense: code page 65001 is the Unicode page, so ideally all UTF-8 sequences should hit the terminal as their appropriate Unicode glyphs. > $ chcp.com 437 > Active code page: 437 > > $ ./cygwin.exe > Ω Cygwin may be at fault here: it is outputting a Unicode glyph even though the current code page is not Unicode. But I'm also not ruling out weirdness in cmd (it's not open source, so I can't prove who is at fault, after all). > > $ ./mingw32.exe > ╬⌐ This is the correct two-character (not multi-byte) sequence to output according to the code page 437 glyph table (in that code page, \xce is the character u+256C, and \xa9 is the character u+2310). Spitting out UTF-8 bytes to a unibyte locale (which is what code page 437 is) is generally going to produce mojibake. Which is why using a unibyte locale is annoying. -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 619 bytes --] ^ permalink raw reply [flat|nested] 37+ messages in thread
* Cygwin.bat (was: [ANNOUNCEMENT] Updated: libreadline7-7.0.3-3) 2017-07-28 23:55 ` Eric Blake @ 2017-07-29 1:55 ` Achim Gratz 2017-07-29 2:48 ` [ANNOUNCEMENT] Updated: libreadline7-7.0.3-3 Doug Henderson 2017-07-29 8:45 ` Steven Penny 2 siblings, 0 replies; 37+ messages in thread From: Achim Gratz @ 2017-07-29 1:55 UTC (permalink / raw) To: cygwin Eric Blake writes: >> As in "C:\cygwin64\Cygwin.bat" that can be found after a regular install of >> Cygwin >> > > Oh, that one doesn't show up under 'cygcheck -p', so it must be created > by setup.exe. It's created by the postinstall script of base-files. The source for it lives in /etc/defaults/Cygwin.bat in case you're wondering. >>> It's short: >>> >>> @echo off >>> setlocal enableextensions >>> set TERM=3D >>> cd /d "%~dp0bin" && .\bash --login -i >> >> Uh, no? Mine looks like this. Again this is the file installed by >> Cygwin, not >> something I have home brewed: >> >> @echo off >> C: >> chdir C:\cygwin64\bin >> bash --login -i > > Odd that there are two different flavors of the file - but the point > remains that they are doing pretty much the same thing: starting bash. You have the test version of base-files installed, Steven has the release version. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ SD adaptations for Waldorf Q V3.00R3 and Q+ V3.54R2: http://Synth.Stromeko.net/Downloads.html#WaldorfSDada -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [ANNOUNCEMENT] Updated: libreadline7-7.0.3-3 2017-07-28 23:55 ` Eric Blake 2017-07-29 1:55 ` Cygwin.bat (was: [ANNOUNCEMENT] Updated: libreadline7-7.0.3-3) Achim Gratz @ 2017-07-29 2:48 ` Doug Henderson 2017-07-29 4:23 ` Steven Penny 2017-07-29 8:45 ` Steven Penny 2 siblings, 1 reply; 37+ messages in thread From: Doug Henderson @ 2017-07-29 2:48 UTC (permalink / raw) To: cygwin On 28 July 2017 at 08:54, Eric Blake wrote: > On 07/28/2017 07:09 AM, Steven Penny wrote: >> On Fri, 28 Jul 2017 06:41:08, Eric Blake wrote: >> $ chcp.com 65001 >> Active code page: 65001 >> >> $ ./cygwin.exe >> Ω >> >> $ ./mingw32.exe >> Ω >> >> $ chcp.com 437 >> Active code page: 437 >> >> $ ./cygwin.exe >> Ω IIRC, cygwin was changed some time ago (in the last year?) to always emit unicode or UTF-8 to the console (i.e, it calls a different Win32 API function). Again, IIRC, it used to sometimes use one API call and sometimes the other to write UTF-8 and unencoded bytes, depending on something, perhaps the code page? I think it was written up in a cygwin1.dll update announcement. It looks like the mingw32 cross-compiler c runtime has not made that change. That's the Microsoft supplied CRT, I believe. In any case, this problem has nothing to do with libreadline of any version, nor the current version of cygwin1.dll. It is possible that this lies in the domain of mingw32, which suggests that it may be off-topic here or at least needs a new subject line. HTH Doug -- Doug Henderson, Calgary, Alberta, Canada - from gmail.com -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [ANNOUNCEMENT] Updated: libreadline7-7.0.3-3 2017-07-29 2:48 ` [ANNOUNCEMENT] Updated: libreadline7-7.0.3-3 Doug Henderson @ 2017-07-29 4:23 ` Steven Penny 2017-07-30 18:38 ` Doug Henderson 0 siblings, 1 reply; 37+ messages in thread From: Steven Penny @ 2017-07-29 4:23 UTC (permalink / raw) To: cygwin On Fri, 28 Jul 2017 12:13:49, Doug Henderson wrote: > In any case, this problem has nothing to do with libreadline of any > version, nor the current version of cygwin1.dll. It is possible that > this lies in the domain of mingw32, which suggests that it may be > off-topic here or at least needs a new subject line. Thats a pretty bold statement to which you have provided no backup. I am going to just flat out ignore it, and I suggest others do the same. You cant come into a 5 month thread with a flippant remark and no links and expect to be taken seriously. Even Chet is involved with this now. You want to come back with some actual links/references, then I am open to listen. Otherwise step down sir. -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [ANNOUNCEMENT] Updated: libreadline7-7.0.3-3 2017-07-29 4:23 ` Steven Penny @ 2017-07-30 18:38 ` Doug Henderson 2017-07-30 19:54 ` Steven Penny 0 siblings, 1 reply; 37+ messages in thread From: Doug Henderson @ 2017-07-30 18:38 UTC (permalink / raw) To: cygwin >On 28 July 2017 at 12:31, Steven Penny wrote: >> On Fri, 28 Jul 2017 12:13:49, Doug Henderson wrote: >> In any case, this problem has nothing to do with libreadline of any >> version, nor the current version of cygwin1.dll. It is possible that >> this lies in the domain of mingw32, which suggests that it may be >> off-topic here or at least needs a new subject line. > You want to come back with some actual links/references, then I am open to > listen. Otherwise step down sir. One of these may be the commit that I recall. https://github.com/cygwin/cygwin/commit/ef007184874ead6f288e432eb23bfc76bf65929d https://github.com/cygwin/cygwin/commit/5a3496c3e3c159e6cfb4879f5adae1092927483f https://github.com/cygwin/cygwin/commit/7630e384623f778b88be8926e2c2a2e6fcbd9008 I gave up trying to trying read C++ about 15 years ago, so all I understood was the English parts. Also the following speak for themselves. $ ldd omega_cygwin.exe ntdll.dll => /cygdrive/c/Windows/SYSTEM32/ntdll.dll (0x77450000) kernel32.dll => /cygdrive/c/Windows/system32/kernel32.dll (0x77330000) KERNELBASE.dll => /cygdrive/c/Windows/system32/KERNELBASE.dll (0x7fefd3d0000) cygwin1.dll => /usr/bin/cygwin1.dll (0x180040000) $ ldd omega_mingw32.exe ntdll.dll => /cygdrive/c/Windows/SYSTEM32/ntdll.dll (0x77450000) kernel32.dll => /cygdrive/c/Windows/system32/kernel32.dll (0x77330000) KERNELBASE.dll => /cygdrive/c/Windows/system32/KERNELBASE.dll (0x7fefd3d0000) msvcrt.dll => /cygdrive/c/Windows/system32/msvcrt.dll (0x7feff150000) $ ldd $(which bash) /usr/bin/bash: ntdll.dll => /cygdrive/c/Windows/SYSTEM32/ntdll.dll (0x77450000) kernel32.dll => /cygdrive/c/Windows/system32/kernel32.dll (0x77330000) KERNELBASE.dll => /cygdrive/c/Windows/system32/KERNELBASE.dll (0x7fefd3d0000) cygwin1.dll => /usr/bin/cygwin1.dll (0x180040000) cygiconv-2.dll => /usr/bin/cygiconv-2.dll (0x3e9d90000) cygintl-8.dll => /usr/bin/cygintl-8.dll (0x3e45c0000) cygreadline7.dll => /usr/bin/cygreadline7.dll (0x3e2050000) cygncursesw-10.dll => /usr/bin/cygncursesw-10.dll (0x3e3520000) $ ldd $(which dash) /usr/bin/dash: ntdll.dll => /cygdrive/c/Windows/SYSTEM32/ntdll.dll (0x77450000) kernel32.dll => /cygdrive/c/Windows/system32/kernel32.dll (0x77330000) KERNELBASE.dll => /cygdrive/c/Windows/system32/KERNELBASE.dll (0x7fefd3d0000) cygwin1.dll => /usr/bin/cygwin1.dll (0x180040000) HTH Doug -- Doug Henderson, Calgary, Alberta, Canada - from gmail.com -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [ANNOUNCEMENT] Updated: libreadline7-7.0.3-3 2017-07-30 18:38 ` Doug Henderson @ 2017-07-30 19:54 ` Steven Penny 0 siblings, 0 replies; 37+ messages in thread From: Steven Penny @ 2017-07-30 19:54 UTC (permalink / raw) To: cygwin On Fri, 28 Jul 2017 19:54:27, Doug Henderson wrote: > One of these may be the commit that I recall. > > https://github.com/cygwin/cygwin/commit/ef007184874ead6f288e432eb23bfc76bf65929d > > https://github.com/cygwin/cygwin/commit/5a3496c3e3c159e6cfb4879f5adae1092927483f > > https://github.com/cygwin/cygwin/commit/7630e384623f778b88be8926e2c2a2e6fcbd9008 I looked at those 3, and I am not seeing something that would cause this. Hell: http://github.com/cygwin/cygwin/commit/ef00718 is basically changing the variable names. Would be nice to have Corinna weigh in though. > I gave up trying to trying read C++ about 15 years ago, so all I > understood was the English parts. Right, so you are posting links to code that by your own admission you dont understand. > Also the following speak for themselves. > > $ ldd omega_cygwin.exe > $ ldd omega_mingw32.exe > $ ldd $(which bash) > $ ldd $(which dash) The core of the issue here is proper UTF-8 input with Cygwin. This used to be possible before the current version of readline. Before that, if you had the proper font, and the proper codepage, Cygwin would accept UTF-8 input and could handle UTF-8 output. Now at least the input part of it is broken. The point of my example omega.c was to answer Corinnas question, which was essentially, "why use 65001 codepage". The answer is because you have to, if you want sane UTF-8 input/output across cygwin/mingw32 builds. Your DLL dumps here serve about as little purpose as the GitHub links before, Doug, unless you are willing to repro the issue as Eric has, you are not really adding anything to the conversation. You are distracting from it. -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [ANNOUNCEMENT] Updated: libreadline7-7.0.3-3 2017-07-28 23:55 ` Eric Blake 2017-07-29 1:55 ` Cygwin.bat (was: [ANNOUNCEMENT] Updated: libreadline7-7.0.3-3) Achim Gratz 2017-07-29 2:48 ` [ANNOUNCEMENT] Updated: libreadline7-7.0.3-3 Doug Henderson @ 2017-07-29 8:45 ` Steven Penny 2017-07-29 10:08 ` Eric Blake 2 siblings, 1 reply; 37+ messages in thread From: Steven Penny @ 2017-07-29 8:45 UTC (permalink / raw) To: cygwin On Fri, 28 Jul 2017 09:54:59, Eric Blake wrote: > > LATIN SMALL LETTER O WITH DIAERESIS' (U+00F6): > >=20 > > $ chcp.com 65001 > > Active code page: 65001 > >=20 > > - Alt 148 outputs nothing > > - Alt 0246 outputs nothing > > But that's true even in a bare cmd window. So it's hard to say what it > is SUPPOSED to do. I'm trying to debug bash, not cmd's alt- behavior. I thought you were gaslighting me for a second, but upon testing this with a pristine virtual machine, I see you are right. The reason is that a new Windows install (and probably your environment) has "Raster Fonts" set under "Defaults > Font > Font". Changing this to "Consolas" or "Lucida Console" will allow proper behavior under 65001. -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [ANNOUNCEMENT] Updated: libreadline7-7.0.3-3 2017-07-29 8:45 ` Steven Penny @ 2017-07-29 10:08 ` Eric Blake 0 siblings, 0 replies; 37+ messages in thread From: Eric Blake @ 2017-07-29 10:08 UTC (permalink / raw) To: cygwin [-- Attachment #1.1: Type: text/plain, Size: 1282 bytes --] On 07/28/2017 01:39 PM, Steven Penny wrote: > On Fri, 28 Jul 2017 09:54:59, Eric Blake wrote: >> > LATIN SMALL LETTER O WITH DIAERESIS' (U+00F6): >> >=20 >> > $ chcp.com 65001 >> > Active code page: 65001 >> >=20 >> > - Alt 148 outputs nothing >> > - Alt 0246 outputs nothing >> >> But that's true even in a bare cmd window. So it's hard to say what it >> is SUPPOSED to do. I'm trying to debug bash, not cmd's alt- behavior. > > I thought you were gaslighting me for a second, but upon testing this > with a > pristine virtual machine, I see you are right. And yes, I'm testing in a fairly pristine install (about 1 week old) > > The reason is that a new Windows install (and probably your environment) > has > "Raster Fonts" set under "Defaults > Font > Font". Changing this to > "Consolas" > or "Lucida Console" will allow proper behavior under 65001. Aha - that's another key piece of information: the choice of font in use by the display determines which characters can be rendered where; the default font is limited in scope, but picking a font with more characters allows more things to be displayed. -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 619 bytes --] ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [ANNOUNCEMENT] Updated: libreadline7-7.0.3-3 2017-07-28 18:39 ` Steven Penny 2017-07-28 23:55 ` Eric Blake @ 2017-07-31 18:36 ` Corinna Vinschen 2017-07-31 20:01 ` Steven Penny 2017-07-31 20:05 ` David Macek 1 sibling, 2 replies; 37+ messages in thread From: Corinna Vinschen @ 2017-07-31 18:36 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 1250 bytes --] On Jul 28 05:09, Steven Penny wrote: > $ chcp.com 65001 > Active code page: 65001 > > - Alt 148 outputs nothing > - Alt 0246 outputs nothing > - Pasting this character does not work Well, it works for me. I tested this in tcsh, bash and od on Windows 10. While at it I found a bug in Windows. Try this: chcp 65001 Alt 0246 inserts ö press backspace See a line of sqare blocks or whatever replacement char the font uses. What tcsh does in this case is, it sends a sequence "\b\033[K" to the terminal which means, backspace, then clear up to the end of line. Cleaning the line is done by running the function FillConsoleOutputCharacterA with an ASCII space as fill character. And this works fine... unless the console is switched to CP 65001. In this case you *have* to use FillConsoleOutputCharacterW with a wide characeter space instead. I fixed that in Cygwin. But, anyway, I don't know where the difference is between your and my setup, but as I said, it does not depend on the codepage if the Alt+Numpad stuff works for me. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [ANNOUNCEMENT] Updated: libreadline7-7.0.3-3 2017-07-31 18:36 ` Corinna Vinschen @ 2017-07-31 20:01 ` Steven Penny 2017-07-31 20:05 ` David Macek 1 sibling, 0 replies; 37+ messages in thread From: Steven Penny @ 2017-07-31 20:01 UTC (permalink / raw) To: cygwin On Mon, 31 Jul 2017 11:48:12, Corinna Vinschen wrote: > On Jul 28 05:09, Steven Penny wrote: > > $ chcp.com 65001 > > Active code page: 65001 > >=20 > > - Alt 148 outputs nothing > > - Alt 0246 outputs nothing > > - Pasting this character does not work > > Well, it works for me. I tested this in tcsh, bash and od on > Windows 10. I am using Windows 7, which I do not think is an unreasonable thing to do, considering it is still at 49% market share: http://netmarketshare.com/operating-system-market-share.aspx Can you please retest this with Windows 7? If in fact this is something that just wont work with Windows 7 I will deal with that. However I find that hard to believe, as I have have tested different things that could have caused the problem and discovered it was the new version of readline, not my operating system. -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [ANNOUNCEMENT] Updated: libreadline7-7.0.3-3 2017-07-31 18:36 ` Corinna Vinschen 2017-07-31 20:01 ` Steven Penny @ 2017-07-31 20:05 ` David Macek 2017-07-31 21:13 ` David Macek [not found] ` <20170731200146.GD18950@calimero.vinschen.de> 1 sibling, 2 replies; 37+ messages in thread From: David Macek @ 2017-07-31 20:05 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 936 bytes --] On 31. 7. 2017 11:48, Corinna Vinschen wrote: > Well, it works for me. I tested this in tcsh, bash and od on > Windows 10. I tested on Windows 2012 R2 (8.1 equivalent) and I can confirm Steven's findings. Tried with an older installation and then once more after a complete update (`uname -a`s below). - CYGWIN_NT-6.3 computername 2.8.0(0.309/5/3) 2017-04-01 20:47 x86_64 Cygwin - CYGWIN_NT-6.3 computername 2.8.2(0.313/5/3) 2017-07-12 10:58 x86_64 Cygwin >> - Alt 148 outputs nothing >> - Alt 0246 outputs nothing >> - Pasting this character does not work I ran Cygwin from a clean command window (`cmd /d`) using `chcp ...` and `bin\bash -li`. The ö's don't appear in bash with CP 65001. Other combinations (bash with CP 437, cmd with either CP) allow ö's to appear. Try the legacy console (conhost v1), if you haven't already. Maybe it will show the same behavior even on Windows 10. -- David Macek [-- Attachment #2: S/MIME Cryptographic Signature --] [-- Type: application/pkcs7-signature, Size: 3715 bytes --] ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [ANNOUNCEMENT] Updated: libreadline7-7.0.3-3 2017-07-31 20:05 ` David Macek @ 2017-07-31 21:13 ` David Macek [not found] ` <20170731200146.GD18950@calimero.vinschen.de> 1 sibling, 0 replies; 37+ messages in thread From: David Macek @ 2017-07-31 21:13 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 1114 bytes --] On 31. 7. 2017 14:36, David Macek wrote: > On 31. 7. 2017 11:48, Corinna Vinschen wrote: >> Well, it works for me. I tested this in tcsh, bash and od on >> Windows 10. > > I tested on Windows 2012 R2 (8.1 equivalent) and I can confirm Steven's findings. Tried with an older installation and then once more after a complete update (`uname -a`s below). > > - CYGWIN_NT-6.3 computername 2.8.0(0.309/5/3) 2017-04-01 20:47 x86_64 Cygwin > - CYGWIN_NT-6.3 computername 2.8.2(0.313/5/3) 2017-07-12 10:58 x86_64 Cygwin > >>> - Alt 148 outputs nothing >>> - Alt 0246 outputs nothing >>> - Pasting this character does not work > > I ran Cygwin from a clean command window (`cmd /d`) using `chcp ...` and `bin\bash -li`. The ö's don't appear in bash with CP 65001. Other combinations (bash with CP 437, cmd with either CP) allow ö's to appear. > > Try the legacy console (conhost v1), if you haven't already. Maybe it will show the same behavior even on Windows 10. Oh, and after reading through the whole thread, I realize I should add that I've set Consolas as my font. -- David Macek [-- Attachment #2: S/MIME Cryptographic Signature --] [-- Type: application/pkcs7-signature, Size: 3715 bytes --] ^ permalink raw reply [flat|nested] 37+ messages in thread
[parent not found: <20170731200146.GD18950@calimero.vinschen.de>]
[parent not found: <20170731211327.GG18950@calimero.vinschen.de>]
* Re: [ANNOUNCEMENT] Updated: libreadline7-7.0.3-3 [not found] ` <20170731211327.GG18950@calimero.vinschen.de> @ 2017-08-01 0:56 ` Steven Penny 2017-08-01 8:45 ` Corinna Vinschen 2017-08-01 7:22 ` David Macek 1 sibling, 1 reply; 37+ messages in thread From: Steven Penny @ 2017-08-01 0:56 UTC (permalink / raw) To: cygwin On Mon, 31 Jul 2017 23:13:27, Corinna Vinschen wrote: > Ahhhhh, no, no, no. I found the problem. When using CP 65001 on > pre-W10, you *must not* use the ANSI functions PeekConsoleInputA and > ReadConsoleInputA, but select() was still using them as it did so for > ages. > > Now using the UNICODE functions PeekConsoleInputW and ReadConsoleInputW, > select is doing the right thing even with CP 65001. YOU DID IT. As I reported here: http://cygwin.com/ml/cygwin/2017-02/msg00031.html this has actually been a problem since libreadline7-7.0.1 (Dec 2016): http://cygwin.com/ml/cygwin/2016-12/msg00091.html with libreadline7-6.3.8 being the last working version until now. THANK YOU CORINNA. -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [ANNOUNCEMENT] Updated: libreadline7-7.0.3-3 2017-08-01 0:56 ` Steven Penny @ 2017-08-01 8:45 ` Corinna Vinschen 2017-08-01 14:48 ` Corinna Vinschen 0 siblings, 1 reply; 37+ messages in thread From: Corinna Vinschen @ 2017-08-01 8:45 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 997 bytes --] On Jul 31 17:56, Steven Penny wrote: > On Mon, 31 Jul 2017 23:13:27, Corinna Vinschen wrote: > > Ahhhhh, no, no, no. I found the problem. When using CP 65001 on > > pre-W10, you *must not* use the ANSI functions PeekConsoleInputA and > > ReadConsoleInputA, but select() was still using them as it did so for > > ages. > > > > Now using the UNICODE functions PeekConsoleInputW and ReadConsoleInputW, > > select is doing the right thing even with CP 65001. > > YOU DID IT. As I reported here: > > http://cygwin.com/ml/cygwin/2017-02/msg00031.html > > this has actually been a problem since libreadline7-7.0.1 (Dec 2016): > > http://cygwin.com/ml/cygwin/2016-12/msg00091.html > > with libreadline7-6.3.8 being the last working version until now. > > THANK YOU CORINNA. Thanks for testing and confirming. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [ANNOUNCEMENT] Updated: libreadline7-7.0.3-3 2017-08-01 8:45 ` Corinna Vinschen @ 2017-08-01 14:48 ` Corinna Vinschen 2017-08-01 18:20 ` Steven Penny 0 siblings, 1 reply; 37+ messages in thread From: Corinna Vinschen @ 2017-08-01 14:48 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 1309 bytes --] On Aug 1 10:45, Corinna Vinschen wrote: > On Jul 31 17:56, Steven Penny wrote: > > On Mon, 31 Jul 2017 23:13:27, Corinna Vinschen wrote: > > > Ahhhhh, no, no, no. I found the problem. When using CP 65001 on > > > pre-W10, you *must not* use the ANSI functions PeekConsoleInputA and > > > ReadConsoleInputA, but select() was still using them as it did so for > > > ages. > > > > > > Now using the UNICODE functions PeekConsoleInputW and ReadConsoleInputW, > > > select is doing the right thing even with CP 65001. > > > > YOU DID IT. As I reported here: > > > > http://cygwin.com/ml/cygwin/2017-02/msg00031.html > > > > this has actually been a problem since libreadline7-7.0.1 (Dec 2016): > > > > http://cygwin.com/ml/cygwin/2016-12/msg00091.html > > > > with libreadline7-6.3.8 being the last working version until now. > > > > THANK YOU CORINNA. > > Thanks for testing and confirming. New developer's snapshot uploaded to https://cygwin.com/snapshots/ David found a weird behaviour in terms of NumLock. Now the Alt Numpad key character input works independently of the NumLock state, just as in CMD. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [ANNOUNCEMENT] Updated: libreadline7-7.0.3-3 2017-08-01 14:48 ` Corinna Vinschen @ 2017-08-01 18:20 ` Steven Penny 2017-08-01 18:54 ` Achim Gratz 0 siblings, 1 reply; 37+ messages in thread From: Steven Penny @ 2017-08-01 18:20 UTC (permalink / raw) To: cygwin On Tue, 1 Aug 2017 16:48:30, Corinna Vinschen wrote: > David found a weird behaviour in terms of NumLock. Now the Alt > Numpad key character input works independently of the NumLock state, > just as in CMD. Could you link to Davids post? I do not see it for this month or July, and I dont seem to have trouble with numlock on or off, even with July 31 version. -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [ANNOUNCEMENT] Updated: libreadline7-7.0.3-3 2017-08-01 18:20 ` Steven Penny @ 2017-08-01 18:54 ` Achim Gratz 2017-08-01 19:02 ` Eric Blake 0 siblings, 1 reply; 37+ messages in thread From: Achim Gratz @ 2017-08-01 18:54 UTC (permalink / raw) To: cygwin Steven Penny writes: > Could you link to Davids post? I do not see it for this month or July, and I > dont seem to have trouble with numlock on or off, even with July 31 version. She can't, this was a discussion on IRC. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ SD adaptation for Waldorf rackAttack V1.04R1: http://Synth.Stromeko.net/Downloads.html#WaldorfSDada -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [ANNOUNCEMENT] Updated: libreadline7-7.0.3-3 2017-08-01 18:54 ` Achim Gratz @ 2017-08-01 19:02 ` Eric Blake 0 siblings, 0 replies; 37+ messages in thread From: Eric Blake @ 2017-08-01 19:02 UTC (permalink / raw) To: cygwin [-- Attachment #1.1: Type: text/plain, Size: 1287 bytes --] On 08/01/2017 01:54 PM, Achim Gratz wrote: > Steven Penny writes: >> Could you link to Davids post? I do not see it for this month or July, and I >> dont seem to have trouble with numlock on or off, even with July 31 version. > > She can't, this was a discussion on IRC. The gist of the conversation and today's patch: when numlock is on, 'alt-5' behaves as expected; but when numlock is off, 'alt-5' was generating two keypresses at once (the escape sequence for ALT+keypad5 as on Linux when you release keypad5, AND the windows behavior of injecting a codepoint after releasing alt). Corinna's patch means you can no longer bind alt-keypad5 to do something magic (this is because the cmd behavior is that alt-keypad sequences work regardless of numlock state); but doesn't stop you from binding ctrl-keypad5 or shift-keypad5. Yes, it means one place where cygwin can't completely emulate Linux; but the thought on IRC was that more users are probably familiar with windows alt-sequence codepoint generation than they are with Linux alt-keypress bindings, and that less than 1% of the population would even notice the difference. -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 619 bytes --] ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [ANNOUNCEMENT] Updated: libreadline7-7.0.3-3 [not found] ` <20170731211327.GG18950@calimero.vinschen.de> 2017-08-01 0:56 ` Steven Penny @ 2017-08-01 7:22 ` David Macek 2017-08-01 8:46 ` Corinna Vinschen 1 sibling, 1 reply; 37+ messages in thread From: David Macek @ 2017-08-01 7:22 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 523 bytes --] On 31. 7. 2017 23:13, Corinna Vinschen wrote: > Ahhhhh, no, no, no. I found the problem. When using CP 65001 on > pre-W10, you *must not* use the ANSI functions PeekConsoleInputA and > ReadConsoleInputA, but select() was still using them as it did so for > ages. > > Now using the UNICODE functions PeekConsoleInputW and ReadConsoleInputW, > select is doing the right thing even with CP 65001. Awesome. Confirming that the new snapshot allows ö to be input in plain bash with CP 65001. -- David Macek [-- Attachment #2: S/MIME Cryptographic Signature --] [-- Type: application/pkcs7-signature, Size: 3715 bytes --] ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [ANNOUNCEMENT] Updated: libreadline7-7.0.3-3 2017-08-01 7:22 ` David Macek @ 2017-08-01 8:46 ` Corinna Vinschen 0 siblings, 0 replies; 37+ messages in thread From: Corinna Vinschen @ 2017-08-01 8:46 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 759 bytes --] On Aug 1 09:22, David Macek wrote: > On 31. 7. 2017 23:13, Corinna Vinschen wrote: > > Ahhhhh, no, no, no. I found the problem. When using CP 65001 on > > pre-W10, you *must not* use the ANSI functions PeekConsoleInputA and > > ReadConsoleInputA, but select() was still using them as it did so for > > ages. > > > > Now using the UNICODE functions PeekConsoleInputW and ReadConsoleInputW, > > select is doing the right thing even with CP 65001. > > Awesome. Confirming that the new snapshot allows ö to be input in plain bash with CP 65001. Thanks for testing and confirming. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 37+ messages in thread
[parent not found: <160b3569-d448-1898-3dcd-b7133a772527@SystematicSw.ab.ca>]
* Re: [ANNOUNCEMENT] Updated: libreadline7-7.0.3-3 [not found] ` <160b3569-d448-1898-3dcd-b7133a772527@SystematicSw.ab.ca> @ 2017-08-01 8:44 ` Corinna Vinschen 0 siblings, 0 replies; 37+ messages in thread From: Corinna Vinschen @ 2017-08-01 8:44 UTC (permalink / raw) To: cygwin [-- Attachment #1: Type: text/plain, Size: 746 bytes --] On Jul 31 15:18, Brian Inglis wrote: > On 2017-07-31 14:01, Corinna Vinschen wrote: > > On Jul 31 14:36, David Macek wrote: > >> Try the legacy console (conhost v1), if you haven't already. Maybe it > >> will show the same behavior even on Windows 10. > > I'm using the Windows console by starting "cmd". I don't know if > > there's another console version available or how to start it. > > Cmd/System menu/Properties|Defaults/"Command Prompt"|Console Window > Properties dialogue/Options tab/[ ] Use legacy console (requires > relaunch) checkbox at bottom. Thanks! Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 37+ messages in thread
end of thread, other threads:[~2017-08-01 19:02 UTC | newest] Thread overview: 37+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2017-02-13 19:53 [ANNOUNCEMENT] Updated: libreadline7-7.0.3-3, libreadline-devel-7.0.3-3 Eric Blake (cygwin) 2017-03-25 0:46 ` Steven Penny 2017-04-13 19:07 ` [ANNOUNCEMENT] Updated: libreadline7-7.0.3-3 Steven Penny 2017-04-14 8:27 ` Eric Blake 2017-04-15 10:59 ` Steven Penny 2017-05-15 18:21 ` Eric Blake 2017-05-15 20:46 ` Chet Ramey 2017-06-18 15:53 ` Steven Penny 2017-07-04 21:53 ` Steven Penny 2017-07-27 21:37 ` Eric Blake 2017-07-27 21:39 ` Eric Blake 2017-07-27 21:46 ` Steven Penny 2017-07-28 8:28 ` Eric Blake 2017-07-28 14:55 ` Steven Penny 2017-07-28 18:31 ` Eric Blake 2017-07-28 18:39 ` Steven Penny 2017-07-28 23:55 ` Eric Blake 2017-07-29 1:55 ` Cygwin.bat (was: [ANNOUNCEMENT] Updated: libreadline7-7.0.3-3) Achim Gratz 2017-07-29 2:48 ` [ANNOUNCEMENT] Updated: libreadline7-7.0.3-3 Doug Henderson 2017-07-29 4:23 ` Steven Penny 2017-07-30 18:38 ` Doug Henderson 2017-07-30 19:54 ` Steven Penny 2017-07-29 8:45 ` Steven Penny 2017-07-29 10:08 ` Eric Blake 2017-07-31 18:36 ` Corinna Vinschen 2017-07-31 20:01 ` Steven Penny 2017-07-31 20:05 ` David Macek 2017-07-31 21:13 ` David Macek [not found] ` <20170731200146.GD18950@calimero.vinschen.de> [not found] ` <20170731211327.GG18950@calimero.vinschen.de> 2017-08-01 0:56 ` Steven Penny 2017-08-01 8:45 ` Corinna Vinschen 2017-08-01 14:48 ` Corinna Vinschen 2017-08-01 18:20 ` Steven Penny 2017-08-01 18:54 ` Achim Gratz 2017-08-01 19:02 ` Eric Blake 2017-08-01 7:22 ` David Macek 2017-08-01 8:46 ` Corinna Vinschen [not found] ` <160b3569-d448-1898-3dcd-b7133a772527@SystematicSw.ab.ca> 2017-08-01 8:44 ` Corinna Vinschen
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).