From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: by sourceware.org (Postfix, from userid 7868) id 4E0973858D35; Tue, 16 Nov 2021 14:22:37 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 4E0973858D35 Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: Takashi Yano To: cygwin-cvs@sourceware.org Subject: [newlib-cygwin/cygwin-3_3-branch] Cygwin: console: Handle Unicode surrogate pairs. X-Act-Checkin: newlib-cygwin X-Git-Author: Johannes Schindelin X-Git-Refname: refs/heads/cygwin-3_3-branch X-Git-Oldrev: 282f40e4697b00a786d4b63f94e744b2e8a221b3 X-Git-Newrev: e6ed90c8f79b2eff732ecace02b0bc5440f18b29 Message-Id: <20211116142237.4E0973858D35@sourceware.org> Date: Tue, 16 Nov 2021 14:22:37 +0000 (GMT) X-BeenThere: cygwin-cvs@cygwin.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Cygwin core component git logs List-Unsubscribe: , List-Archive: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Nov 2021 14:22:37 -0000 https://sourceware.org/git/gitweb.cgi?p=newlib-cygwin.git;h=e6ed90c8f79b2eff732ecace02b0bc5440f18b29 commit e6ed90c8f79b2eff732ecace02b0bc5440f18b29 Author: Johannes Schindelin Date: Tue Nov 16 11:26:10 2021 +0100 Cygwin: console: Handle Unicode surrogate pairs. When running Cygwin's Bash in the Windows Terminal (see https://docs.microsoft.com/en-us/windows/terminal/ for details), Cygwin is receiving keyboard input in the form of UTF-16 characters. UTF-16 has that awkward challenge that it cannot map the full Unicode range, and to make up for it, there are the ranges U+D800-U+DBFF and U+DC00-U+DFFF which are illegal except when they come in a pair encoding for Unicode characters beyond U+FFFF. Cygwin does not handle such surrogate pairs correctly at the moment, as can be seen e.g. when running Cygwin's Bash in the Windows Terminal and then inserting an emoji (e.g. via Windows + , which opens an emoji picker on recent Windows versions): Instead of showing an emoji, this shows the infamous question mark in a black triangle, i.e. the invalid Unicode character. Let's special-case surrogate pairs in this scenario. This fixes https://github.com/git-for-windows/git/issues/3281 Signed-off-by: Johannes Schindelin Diff: --- winsup/cygwin/fhandler_console.cc | 17 ++++++++++++++++- winsup/cygwin/release/3.3.3 | 6 ++++++ 2 files changed, 22 insertions(+), 1 deletion(-) diff --git a/winsup/cygwin/fhandler_console.cc b/winsup/cygwin/fhandler_console.cc index 2e754a132..a7e9723bd 100644 --- a/winsup/cygwin/fhandler_console.cc +++ b/winsup/cygwin/fhandler_console.cc @@ -919,7 +919,22 @@ fhandler_console::process_input_message (void) } else { - nread = con.con_to_str (tmp + 1, 59, unicode_char); + WCHAR second = unicode_char >= 0xd800 && unicode_char <= 0xdbff + && i + 1 < total_read ? + input_rec[i + 1].Event.KeyEvent.uChar.UnicodeChar : 0; + + if (second < 0xdc00 || second > 0xdfff) + { + nread = con.con_to_str (tmp + 1, 59, unicode_char); + } + else + { + /* handle surrogate pairs */ + WCHAR pair[2] = { unicode_char, second }; + nread = sys_wcstombs (tmp + 1, 59, pair, 2); + i++; + } + /* Determine if the keystroke is modified by META. The tricky part is to distinguish whether the right Alt key should be recognized as Alt, or as AltGr. */ diff --git a/winsup/cygwin/release/3.3.3 b/winsup/cygwin/release/3.3.3 index 1eb25e2fc..c1e8cefbd 100644 --- a/winsup/cygwin/release/3.3.3 +++ b/winsup/cygwin/release/3.3.3 @@ -16,3 +16,9 @@ Bug Fixes - Fix long-standing problem that new files don't get created with the FILE_ATTRIBUTE_ARCHIVE DOS attribute set. Addresses: https://cygwin.com/pipermail/cygwin/2021-November/249909.html + +- Handle Unicode surrogate pairs in console. Cygwin console does not + handle surrogate pairs correctly at the moment. Fix issue that + running bash in Windows Terminal and inserting an emoji does not + work as expected. + Addresses: https://github.com/git-for-windows/git/issues/3281