Long window title escape sequence breaks console output

Long window title escape sequence breaks console output

[Tomi Belan]

When a program prints the "\e]0;xxx\a" escape sequence to change the
window title, but the window title string is longer than approx. 256
characters, it breaks the terminal. It will start ignoring any console
output until approximately the next "\e" escape sequence, such as a
shorter title change.

Test script:

#!/bin/bash
printf '(1) setting short title\n'
printf '\e]0;short title\a'
printf '(2) short title was set\n'
sleep 1
printf '(3) setting long title\n'
printf '\e]0;long title long title long title long title long title
long title long title long title long title long title long title long
title long title long title long title long title long title long
title long title long title long title long title long title long
title long title\a'
printf '(4) long title was set\n'
sleep 1
printf '(5) setting short title\n'
printf '\e]0;short title\a'
printf '(6) short title was set\n'

Bad output (alternative 1):

(1) setting short title
(2) short title was set
(3) setting long title
(6) short title was set

Bad output (alternative 2):

(1) setting short title
(2) short title was set
(3) setting long title
0;short title(6) short title was set

Notice that lines (4) and (5) are just completely gone.

I reproduced the bug in all these scenarios:

a) Run cmd.exe (either directly, or in Windows Terminal, or in
Wezterm). Run "c:\cygwin\bin\bash.exe -i -l" to start cygwin. Run
"bash test.sh" in cygwin.
b) Run Windows Terminal with a MSYS2 profile
(https://www.msys2.org/docs/terminals/). Run "bash test.sh" in msys2.
c) Run cmd.exe (either directly, or in Windows Terminal, or in
Wezterm). Run "c:\msys64\usr\bin\ssh.exe some_linux_server". Run "bash
test.sh" on the remote side.

But the bug does NOT happen in these scenarios:

d) Run cygwin in Mintty with the "Cygwin64 Terminal" start menu
shortcut. Run "bash test.sh" in cygwin.
e) Run MSYS2 in Mintty with the "MSYS2 MSYS" start menu shortcut. Run
"bash test.sh" in msys2.
f) Run MSYS2 in Mintty with the "MSYS2 MSYS" start menu shortcut. Run
"ssh some_linux_server". Run "bash test.sh" on the remote side.
g) Run cmd.exe (either directly, or in Windows Terminal, or in
Wezterm). Run "ssh some_linux_server" (using Windows 10's built-in ssh
command C:\Windows\System32\OpenSSH\ssh.exe). Run "bash test.sh" on
the remote side.
h) Run "wezterm.exe ssh some_linux_server" to use Wezterm's built in
ssh client. Run "bash test.sh" on the remote side.

The bug ONLY happens if you use a non-mintty terminal AND any
Cygwin-based exe is involved (a bash.exe or ssh.exe from Cygwin or
MSYS2). It confused me for a long time because I didn't know if it was
a terminal bug, a ConPty bug, a MSYS2 bug, or a bash bug. But when I
discovered the msys2 ssh.exe does not act the same as the win10
ssh.exe, it finally pointed me in the right direction. It is in fact a
bug in Cygwin, specifically winsup.

I think I also managed to track down the exact root cause. "strace
--mask=0xffffff bash test.sh" was helpful for that, but be careful
because strace's own output can also get eaten.

For non-pty stdout (not running in Mintty), winsup parses the text for
escape sequences so that it may convert them to legacy win32 console
API calls if needed. When in the "gettitle" state, winpty appends all
characters to "wpbuf". [1] When (*src < ' '), it ends the "gettitle"
state. For legacy consoles it could call SetConsoleTitleW [2], but in
my case on Windows 10 it just flushes wpbuf to the output [3] and lets
Windows (ConPty, I think?) parse it again [4]. The bug is that wpbuf
has a limited size of 256 and any more characters are ignored [5]. So
what happens is that Cygwin truncates the "\e]0;long title...long
title\a" escape sequence and doesn't print the final \a. This makes
the terminal (or maybe ConPty?) think it's still reading a window
title, and that's why it doesn't print lines (4) and (5). It recovers
when it sees the next escape sequence.

[1] http://www.cygwin.com/git?p=newlib-cygwin.git;a=blob;f=winsup/cygwin/fhandler_console.cc;hb=fdbd1539329ad669606767ab5a63a16f825b4c45#l3564
[2] http://www.cygwin.com/git?p=newlib-cygwin.git;a=blob;f=winsup/cygwin/fhandler_console.cc;hb=fdbd1539329ad669606767ab5a63a16f825b4c45#l3792
[3] http://www.cygwin.com/git?p=newlib-cygwin.git;a=blob;f=winsup/cygwin/fhandler_console.cc;hb=fdbd1539329ad669606767ab5a63a16f825b4c45#l3569
[4] https://docs.microsoft.com/en-us/windows/console/console-virtual-terminal-sequences#window-title
[5] http://www.cygwin.com/git?p=newlib-cygwin.git;a=blob;f=winsup/cygwin/fhandler_console.cc;hb=fdbd1539329ad669606767ab5a63a16f825b4c45#l79

This theory explains why it worked in Mintty (it uses the
fhandler_pty_slave code path), why it affects both Cygwin and MSYS2
(they're both Cygwin-based), and why 256 characters is the threshold
(WPBUF_LEN) rather than 1024 (TITLESIZE) or unlimited.

I hope you enjoyed the bug report. Let me know if you have any questions.

Tomi

Re: Long window title escape sequence breaks console output

[Takashi Yano]

On Fri, 17 Jun 2022 15:38:57 +0200
Tomi Belan wrote:
> When a program prints the "\e]0;xxx\a" escape sequence to change the
> window title, but the window title string is longer than approx. 256
> characters, it breaks the terminal. It will start ignoring any console
> output until approximately the next "\e" escape sequence, such as a
> shorter title change.
> 
> Test script:
> 
> #!/bin/bash
> printf '(1) setting short title\n'
> printf '\e]0;short title\a'
> printf '(2) short title was set\n'
> sleep 1
> printf '(3) setting long title\n'
> printf '\e]0;long title long title long title long title long title
> long title long title long title long title long title long title long
> title long title long title long title long title long title long
> title long title long title long title long title long title long
> title long title\a'
> printf '(4) long title was set\n'
> sleep 1
> printf '(5) setting short title\n'
> printf '\e]0;short title\a'
> printf '(6) short title was set\n'
> 
> Bad output (alternative 1):
> 
> (1) setting short title
> (2) short title was set
> (3) setting long title
> (6) short title was set
> 
> Bad output (alternative 2):
> 
> (1) setting short title
> (2) short title was set
> (3) setting long title
> 0;short title(6) short title was set
> 
> Notice that lines (4) and (5) are just completely gone.
> 
> I reproduced the bug in all these scenarios:
> 
> a) Run cmd.exe (either directly, or in Windows Terminal, or in
> Wezterm). Run "c:\cygwin\bin\bash.exe -i -l" to start cygwin. Run
> "bash test.sh" in cygwin.
> b) Run Windows Terminal with a MSYS2 profile
> (https://www.msys2.org/docs/terminals/). Run "bash test.sh" in msys2.
> c) Run cmd.exe (either directly, or in Windows Terminal, or in
> Wezterm). Run "c:\msys64\usr\bin\ssh.exe some_linux_server". Run "bash
> test.sh" on the remote side.
> 
> But the bug does NOT happen in these scenarios:
> 
> d) Run cygwin in Mintty with the "Cygwin64 Terminal" start menu
> shortcut. Run "bash test.sh" in cygwin.
> e) Run MSYS2 in Mintty with the "MSYS2 MSYS" start menu shortcut. Run
> "bash test.sh" in msys2.
> f) Run MSYS2 in Mintty with the "MSYS2 MSYS" start menu shortcut. Run
> "ssh some_linux_server". Run "bash test.sh" on the remote side.
> g) Run cmd.exe (either directly, or in Windows Terminal, or in
> Wezterm). Run "ssh some_linux_server" (using Windows 10's built-in ssh
> command C:\Windows\System32\OpenSSH\ssh.exe). Run "bash test.sh" on
> the remote side.
> h) Run "wezterm.exe ssh some_linux_server" to use Wezterm's built in
> ssh client. Run "bash test.sh" on the remote side.
> 
> The bug ONLY happens if you use a non-mintty terminal AND any
> Cygwin-based exe is involved (a bash.exe or ssh.exe from Cygwin or
> MSYS2). It confused me for a long time because I didn't know if it was
> a terminal bug, a ConPty bug, a MSYS2 bug, or a bash bug. But when I
> discovered the msys2 ssh.exe does not act the same as the win10
> ssh.exe, it finally pointed me in the right direction. It is in fact a
> bug in Cygwin, specifically winsup.
> 
> I think I also managed to track down the exact root cause. "strace
> --mask=0xffffff bash test.sh" was helpful for that, but be careful
> because strace's own output can also get eaten.
> 
> For non-pty stdout (not running in Mintty), winsup parses the text for
> escape sequences so that it may convert them to legacy win32 console
> API calls if needed. When in the "gettitle" state, winpty appends all
> characters to "wpbuf". [1] When (*src < ' '), it ends the "gettitle"
> state. For legacy consoles it could call SetConsoleTitleW [2], but in
> my case on Windows 10 it just flushes wpbuf to the output [3] and lets
> Windows (ConPty, I think?) parse it again [4]. The bug is that wpbuf
> has a limited size of 256 and any more characters are ignored [5]. So
> what happens is that Cygwin truncates the "\e]0;long title...long
> title\a" escape sequence and doesn't print the final \a. This makes
> the terminal (or maybe ConPty?) think it's still reading a window
> title, and that's why it doesn't print lines (4) and (5). It recovers
> when it sees the next escape sequence.
> 
> [1] http://www.cygwin.com/git?p=newlib-cygwin.git;a=blob;f=winsup/cygwin/fhandler_console.cc;hb=fdbd1539329ad669606767ab5a63a16f825b4c45#l3564
> [2] http://www.cygwin.com/git?p=newlib-cygwin.git;a=blob;f=winsup/cygwin/fhandler_console.cc;hb=fdbd1539329ad669606767ab5a63a16f825b4c45#l3792
> [3] http://www.cygwin.com/git?p=newlib-cygwin.git;a=blob;f=winsup/cygwin/fhandler_console.cc;hb=fdbd1539329ad669606767ab5a63a16f825b4c45#l3569
> [4] https://docs.microsoft.com/en-us/windows/console/console-virtual-terminal-sequences#window-title
> [5] http://www.cygwin.com/git?p=newlib-cygwin.git;a=blob;f=winsup/cygwin/fhandler_console.cc;hb=fdbd1539329ad669606767ab5a63a16f825b4c45#l79
> 
> This theory explains why it worked in Mintty (it uses the
> fhandler_pty_slave code path), why it affects both Cygwin and MSYS2
> (they're both Cygwin-based), and why 256 characters is the threshold
> (WPBUF_LEN) rather than 1024 (TITLESIZE) or unlimited.
> 
> I hope you enjoyed the bug report. Let me know if you have any questions.

Thanks for the report. What situation do you assume where
such a long title needs to be set?

-- 
Takashi Yano <takashi.yano@nifty.ne.jp>

Re: Long window title escape sequence breaks console output

[Tomi Belan]

On Fri, Jun 17, 2022 at 3:38 PM Tomi Belan <tomi.belan@gmail.com> wrote:
> I hope you enjoyed the bug report. Let me know if you have any questions.
>
> Tomi

Takashi Yano wrote:
> Thanks for the report. What situation do you assume where
> such a long title needs to be set?

I experienced the bug because my shell is configured to print the
currently running command in the window title. When I ran a command
longer than 256 characters, it looked like the command didn't run. It
actually did run, but ssh.exe ate its output.

Example: https://tldp.org/HOWTO/Xterm-Title-5.html (but I'm not using zsh)

I know this isn't a common configuration, but I still hope you'll fix
it. Since cygwin just uses mailing lists instead of a real persistent
bug tracker, I'm worried about the bug being forgotten forever if it's
not fixed soon.

Truncating the title to 256 WITH the final '\a' is a suboptimal but
acceptable fix. A better idea might be: in non-legacy terminals, don't
even switch from "normal" to "gettitle", just print everything as it
arrives.

I'm not subscribed to the mailing list so I'm manually sending this as
a reply to my first email. Sorry for the mess. Please CC me on future
replies.

Re: Long window title escape sequence breaks console output

[Takashi Yano]

On Sat, 18 Jun 2022 17:56:05 +0200
Tomi Belan wrote:
> On Fri, Jun 17, 2022 at 3:38 PM Tomi Belan <tomi.belan@gmail.com> wrote:
> > I hope you enjoyed the bug report. Let me know if you have any questions.
> >
> > Tomi
> 
> Takashi Yano wrote:
> > Thanks for the report. What situation do you assume where
> > such a long title needs to be set?
> 
> I experienced the bug because my shell is configured to print the
> currently running command in the window title. When I ran a command
> longer than 256 characters, it looked like the command didn't run. It
> actually did run, but ssh.exe ate its output.
> 
> Example: https://tldp.org/HOWTO/Xterm-Title-5.html (but I'm not using zsh)
> 
> I know this isn't a common configuration, but I still hope you'll fix
> it. Since cygwin just uses mailing lists instead of a real persistent
> bug tracker, I'm worried about the bug being forgotten forever if it's
> not fixed soon.
> 
> Truncating the title to 256 WITH the final '\a' is a suboptimal but
> acceptable fix. A better idea might be: in non-legacy terminals, don't
> even switch from "normal" to "gettitle", just print everything as it
> arrives.
> 
> I'm not subscribed to the mailing list so I'm manually sending this as
> a reply to my first email. Sorry for the mess. Please CC me on future
> replies.

Fixed.
https://cygwin.com/pipermail/cygwin-patches/2022q2/011939.html

Thanks.

-- 
Takashi Yano <takashi.yano@nifty.ne.jp>

Re: Long window title escape sequence breaks console output

[Tomi Belan]

On Sun, Jun 19, 2022 at 6:16 AM Takashi Yano <takashi.yano@nifty.ne.jp> wrote:
> Fixed.
> https://cygwin.com/pipermail/cygwin-patches/2022q2/011939.html
>
> Thanks.
>
> --
> Takashi Yano <takashi.yano@nifty.ne.jp>

I'm glad to hear that. Thank you for your work on Cygwin!