[newlib-cygwin/cygwin-3_5-branch] Cygwin: console: Disable cons_master_thread in win32-input-mode

[newlib-cygwin/cygwin-3_5-branch] Cygwin: console: Disable cons_master_thread in win32-input-mode

[Takashi Yano]

https://sourceware.org/git/gitweb.cgi?p=newlib-cygwin.git;h=68a14b66ff2dd6e71497a3a9583253d17e7e790e

commit 68a14b66ff2dd6e71497a3a9583253d17e7e790e
Author: Takashi Yano <takashi.yano@nifty.ne.jp>
Date:   Sat Aug 31 18:32:00 2024 +0900

    Cygwin: console: Disable cons_master_thread in win32-input-mode
    
    When win32-input-mode (which is supported by Windows Termainal) is
    set by "\033[?9001h", cons_master_thread does not work properly and
    consumes larger and larger memory space. This is because sending
    event by WriteConsoleInput() is translated into the sequence that
    is used by win32-input-mode. Due to this behaviour, write-back
    of the INPUT_RECORDs does not work as expected. With this patch,
    cons_master_thread is disabled on win32-input-mode where the signal
    keys such as Ctrl-C, Ctrl-Z etc. never comes.
    
    Addresses: https://cygwin.com/pipermail/cygwin/2024-August/256380.html
    Fixes: ff4440fcf768 ("Cygwin: console: Introduce new thread which handles input signal.")
    Reported-by: Adamyg Mob <adamyg.mob@gmail.com>
    Signed-off-by: Takashi Yano <takashi.yano@nifty.ne.jp>
    (cherry picked from commit 84d77e5918e18170c393407d477140fcf5d3e432)

Diff:
---
 winsup/cygwin/fhandler/console.cc       | 8 ++++++--
 winsup/cygwin/local_includes/fhandler.h | 2 ++
 winsup/cygwin/release/3.5.5             | 6 ++++++
 3 files changed, 14 insertions(+), 2 deletions(-)

diff --git a/winsup/cygwin/fhandler/console.cc b/winsup/cygwin/fhandler/console.cc
index d4c3f1020..dc43cd9f5 100644
--- a/winsup/cygwin/fhandler/console.cc
+++ b/winsup/cygwin/fhandler/console.cc
@@ -818,6 +818,7 @@ fhandler_console::set_input_mode (tty::cons_mode m, const termios *t,
   GetConsoleMode (p->input_handle, &oflags);
   DWORD flags = oflags
     & (ENABLE_EXTENDED_FLAGS | ENABLE_INSERT_MODE | ENABLE_QUICK_EDIT_MODE);
+  con.curr_input_mode = m;
   switch (m)
     {
     case tty::restore:
@@ -867,6 +868,7 @@ fhandler_console::set_output_mode (tty::cons_mode m, const termios *t,
   if (con.orig_virtual_terminal_processing_mode)
     flags |= ENABLE_VIRTUAL_TERMINAL_PROCESSING;
   WaitForSingleObject (p->output_mutex, mutex_timeout);
+  con.curr_output_mode = m;
   switch (m)
     {
     case tty::restore:
@@ -1109,12 +1111,12 @@ fhandler_console::bg_check (int sig, bool dontsignal)
   /* Setting-up console mode for cygwin app. This is necessary if the
      cygwin app and other non-cygwin apps are started simultaneously
      in the same process group. */
-  if (sig == SIGTTIN)
+  if (sig == SIGTTIN && con.curr_input_mode != tty::cygwin)
     {
       set_input_mode (tty::cygwin, &tc ()->ti, get_handle_set ());
       set_disable_master_thread (false, this);
     }
-  if (sig == SIGTTOU)
+  if (sig == SIGTTOU && con.curr_output_mode != tty::cygwin)
     set_output_mode (tty::cygwin, &tc ()->ti, get_handle_set ());
 
   return fhandler_termios::bg_check (sig, dontsignal);
@@ -2921,6 +2923,8 @@ fhandler_console::char_command (char c)
 		    }
 		  if (con.args[i] == 1) /* DECCKM */
 		    con.cursor_key_app_mode = (c == 'h');
+		  if (con.args[i] == 9001) /* win32-input-mode (https://github.com/microsoft/terminal/blob/main/doc/specs/%234999%20-%20Improved%20keyboard%20handling%20in%20Conpty.md) */
+		    set_disable_master_thread (c == 'h', this);
 		}
 	      /* Call fix_tab_position() if screen has been alternated. */
 	      if (need_fix_tab_position)
diff --git a/winsup/cygwin/local_includes/fhandler.h b/winsup/cygwin/local_includes/fhandler.h
index 3d819f49b..10bc9c7ec 100644
--- a/winsup/cygwin/local_includes/fhandler.h
+++ b/winsup/cygwin/local_includes/fhandler.h
@@ -2160,6 +2160,8 @@ class dev_console
   char *cons_rapoi;
   bool cursor_key_app_mode;
   bool disable_master_thread;
+  tty::cons_mode curr_input_mode;
+  tty::cons_mode curr_output_mode;
   bool master_thread_suspended;
   int num_processed; /* Number of input events in the current input buffer
 			already processed by cons_master_thread(). */
diff --git a/winsup/cygwin/release/3.5.5 b/winsup/cygwin/release/3.5.5
new file mode 100644
index 000000000..a98687c26
--- /dev/null
+++ b/winsup/cygwin/release/3.5.5
@@ -0,0 +1,6 @@
+Fixes:
+------
+
+- Fix undesired behaviour of console master thread in win32-input-mode
+  which is supported by Windows Termainal.
+  Addresses: https://cygwin.com/pipermail/cygwin/2024-August/256380.html