Unexpected behavior using arrow keys on the terminal

Unexpected behavior using arrow keys on the terminal

[George Prekas]

[-- Attachment #1: Type: text/plain, Size: 2074 bytes --]

Using Cygwin 1.7.32, mintty 1.1.3 and OpenSSH_6.7p1 I am getting unexpected behavior regarding the use of arrow keys on the terminal. You can reproduce the behavior by doing the following:

ssh linux
cd /usr/src/linux/tools/perf
make
cd ~
/usr/src/linux/tools/perf/perf record echo 42
/usr/src/linux/tools/perf/perf report

Pressing UP or DOWN should highlight one of the rows displayed. You can verify expected behavior by using either PuTTY or native Linux.

Observation #1: You can fix perf's behavior by applying perf.patch (attached).

Observation #2: Using Wireshark, I've observed that when I ssh to a host and press UP or DOWN on my terminal 3 packets are transmitted from the client. PuTTY on the other hand transmits only 1 packet (larger in size).

Observation #3: I wrote the program test.c (attached). If I run it and press UP or DOWN:
* on Windows from cmd.exe it says "Read 3 bytes. First is 27."
* on Linux it says "Read 3 bytes. First is 27."
* on Linux via PuTTY it says "Read 3 bytes. First is 27."
* on Windows from mintty.exe it says "Read 1 bytes. First is 27. Read 1 bytes. First is 91. Read 1 bytes. First is 65."

My understanding is that the unexpected behavior occurs because Cygwin sends the UP/DOWN sequence one byte at a time. Specifically:

* winsup\cygwin\fhandler_tty.cc @ fhandler_pty_master::write
    This is the function called by the write system call invoked by mintty. Here len = 3. line_edit is invoked 3 times.
* winsup\cygwin\fhandler_termios.cc @ fhandler_termios::line_edit
    This is called by the previous and it calls accept_input.
* winsup\cygwin\fhandler_tty.cc @ fhandler_pty_master::accept_input
    This does the actual WriteFile to the pipe.

I would have provided a patch to fix the problem, but I am not sure I completely understand the semantics of the above mentioned methods.
Regards, 
George Prekas 

Dipl. Electrical and Computer Engineer, National Technical University of Athens

P.S. To be honest, I am not sure how much unexpected is this behavior. I just thought it would be good to have a bug report for it.

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: perf.patch --]
[-- Type: text/x-diff, Size: 418 bytes --]

diff --git a/tools/perf/ui/tui/setup.c b/tools/perf/ui/tui/setup.c
index 2f61256..135ed69 100644
--- a/tools/perf/ui/tui/setup.c
+++ b/tools/perf/ui/tui/setup.c
@@ -79,7 +79,7 @@ int ui__getch(int delay_secs)
 	FD_ZERO(&read_set);
 	FD_SET(0, &read_set);
 	timeout.tv_sec = 0;
-	timeout.tv_usec = 20;
+	timeout.tv_usec = 500;
         err = select(1, &read_set, NULL, NULL, &timeout);
 	if (err == 0)
 		return K_ESC;

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #3: test.c --]
[-- Type: text/x-csrc, Size: 305 bytes --]

#include <stdio.h>
#include <termios.h>

int main() {
  struct termios tio;
  char buf[16];
  int len;

  tcgetattr(0,&tio);
  tio.c_lflag &=(~ICANON & ~ECHO);
  tcsetattr(0,TCSANOW,&tio);

  while (1) {
    len = read(0, buf, sizeof(buf));
    printf("Read %d bytes. First is %d.\n", len, buf[0]);
  }
}

Re: Unexpected behavior using arrow keys on the terminal

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 2695 bytes --]

Hi George,

On Nov 12 10:35, George Prekas wrote:
> Using Cygwin 1.7.32, mintty 1.1.3 and OpenSSH_6.7p1 I am getting
> unexpected behavior regarding the use of arrow keys on the terminal.
> You can reproduce the behavior by doing the following:
> 
> ssh linux
> cd /usr/src/linux/tools/perf
> make
> cd ~
> /usr/src/linux/tools/perf/perf record echo 42
> /usr/src/linux/tools/perf/perf report
> 
> Pressing UP or DOWN should highlight one of the rows displayed. You
> can verify expected behavior by using either PuTTY or native Linux.
> 
> Observation #1: You can fix perf's behavior by applying perf.patch
> (attached).
> 
> Observation #2: Using Wireshark, I've observed that when I ssh to a
> host and press UP or DOWN on my terminal 3 packets are transmitted
> from the client. PuTTY on the other hand transmits only 1 packet
> (larger in size).
> 
> Observation #3: I wrote the program test.c (attached). If I run it and
> press UP or DOWN:
> * on Windows from cmd.exe it says "Read 3 bytes. First is 27."
> * on Linux it says "Read 3 bytes. First is 27."
> * on Linux via PuTTY it says "Read 3 bytes. First is 27."
> * on Windows from mintty.exe it says "Read 1 bytes. First is 27. Read
>   1 bytes. First is 91. Read 1 bytes. First is 65."
> 
> My understanding is that the unexpected behavior occurs because Cygwin
> sends the UP/DOWN sequence one byte at a time. Specifically:
> 
> * winsup\cygwin\fhandler_tty.cc @ fhandler_pty_master::write
>     This is the function called by the write system call invoked by
>     mintty. Here len = 3. line_edit is invoked 3 times.
> * winsup\cygwin\fhandler_termios.cc @ fhandler_termios::line_edit
>     This is called by the previous and it calls accept_input.
> * winsup\cygwin\fhandler_tty.cc @ fhandler_pty_master::accept_input
>     This does the actual WriteFile to the pipe.
> 
> I would have provided a patch to fix the problem, but I am not sure I
> completely understand the semantics of the above mentioned methods.

I have to admit that I'm not quite sure either.  In theory I'd say that
the perf tool is making some invalid assumption here.  You can't rely on
a set of bytes sent from any input source to be always sent or received
as a single package, unless you're working with a transport guaranteing
this.

Your analyzes of the underlying mechanism in Cygwin is correct, though.
Despite what I said above, I take a look into this and perhaps I can fix
the code to send more than 1 byte at a time.  


Thanks,
Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: Unexpected behavior using arrow keys on the terminal

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 3943 bytes --]

On Nov 12 16:46, Corinna Vinschen wrote:
> Hi George,
> 
> On Nov 12 10:35, George Prekas wrote:
> > Using Cygwin 1.7.32, mintty 1.1.3 and OpenSSH_6.7p1 I am getting
> > unexpected behavior regarding the use of arrow keys on the terminal.
> > You can reproduce the behavior by doing the following:
> > 
> > ssh linux
> > cd /usr/src/linux/tools/perf
> > make
> > cd ~
> > /usr/src/linux/tools/perf/perf record echo 42
> > /usr/src/linux/tools/perf/perf report
> > 
> > Pressing UP or DOWN should highlight one of the rows displayed. You
> > can verify expected behavior by using either PuTTY or native Linux.
> > 
> > Observation #1: You can fix perf's behavior by applying perf.patch
> > (attached).
> > 
> > Observation #2: Using Wireshark, I've observed that when I ssh to a
> > host and press UP or DOWN on my terminal 3 packets are transmitted
> > from the client. PuTTY on the other hand transmits only 1 packet
> > (larger in size).
> > 
> > Observation #3: I wrote the program test.c (attached). If I run it and
> > press UP or DOWN:
> > * on Windows from cmd.exe it says "Read 3 bytes. First is 27."
> > * on Linux it says "Read 3 bytes. First is 27."
> > * on Linux via PuTTY it says "Read 3 bytes. First is 27."
> > * on Windows from mintty.exe it says "Read 1 bytes. First is 27. Read
> >   1 bytes. First is 91. Read 1 bytes. First is 65."
> > 
> > My understanding is that the unexpected behavior occurs because Cygwin
> > sends the UP/DOWN sequence one byte at a time. Specifically:
> > 
> > * winsup\cygwin\fhandler_tty.cc @ fhandler_pty_master::write
> >     This is the function called by the write system call invoked by
> >     mintty. Here len = 3. line_edit is invoked 3 times.
> > * winsup\cygwin\fhandler_termios.cc @ fhandler_termios::line_edit
> >     This is called by the previous and it calls accept_input.
> > * winsup\cygwin\fhandler_tty.cc @ fhandler_pty_master::accept_input
> >     This does the actual WriteFile to the pipe.
> > 
> > I would have provided a patch to fix the problem, but I am not sure I
> > completely understand the semantics of the above mentioned methods.
> 
> I have to admit that I'm not quite sure either.  In theory I'd say that
> the perf tool is making some invalid assumption here.  You can't rely on
> a set of bytes sent from any input source to be always sent or received
> as a single package, unless you're working with a transport guaranteing
> this.
> 
> Your analyzes of the underlying mechanism in Cygwin is correct, though.
> Despite what I said above, I take a look into this and perhaps I can fix
> the code to send more than 1 byte at a time.  

In fact, both sides of the pipe were writing/reading in single byte
chunks.

The slave read side did something weird:  Before reading, it always
checked if the number of bytes available in the pipe is > VMIN.  If so,
it only read VMIN bytes.  Since VMIN is 1 by default, it always reads
only single bytes, since changing VMIN is only done rather seldomly by
applications.
This code snippet is actually 12 years old.  Much of the code
surrounding it changed considerably, just this snippet persisted.  I
disabled it now since it really looks like a remnant from the past, and
it certainly doesn't reflect how it works on other OSes.

On the master write side I applied a possible, but definitely
experimental fix, which solves this issue by writing in 32 byte chunks
in non-canonical mode.  This should (mostly) circumvent the above effect,
but it doesn't quite guarantee it.

With luck, this change also speeds up tty output by reducing the # of
calls to WriteFile/ReadFile.

I uploaded a new snapshot to https://cygwin.com/snapshots, the one
from today.  Please give it a try.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: Unexpected behavior using arrow keys on the terminal

[George Prekas]

Hi Corinna,

Your snapshot works! I verified that perf utility works. I also verified 
through Wireshark that only one packet is sent when I press UP or DOWN 
(the old behavior was that 3 packets were sent).

Thank you very much for your effort and prompt reply!

George


On Thursday, November 13, 2014 9:10 PM, Corinna Vinschen <corinna-cygwin@cygwin.com> wrote:



On Nov 12 16:46, Corinna Vinschen wrote:
> Hi George,
> 
> On Nov 12 10:35, George Prekas wrote:
> > Using Cygwin 1.7.32, mintty 1.1.3 and OpenSSH_6.7p1 I am getting
> > unexpected behavior regarding the use of arrow keys on the terminal.
> > You can reproduce the behavior by doing the following:
> > 
> > ssh linux
> > cd /usr/src/linux/tools/perf
> > make
> > cd ~
> > /usr/src/linux/tools/perf/perf record echo 42
> > /usr/src/linux/tools/perf/perf report
> > 
> > Pressing UP or DOWN should highlight one of the rows displayed. You
> > can verify expected behavior by using either PuTTY or native Linux.
> > 
> > Observation #1: You can fix perf's behavior by applying perf.patch
> > (attached).
> > 
> > Observation #2: Using Wireshark, I've observed that when I ssh to a
> > host and press UP or DOWN on my terminal 3 packets are transmitted
> > from the client. PuTTY on the other hand transmits only 1 packet
> > (larger in size).
> > 
> > Observation #3: I wrote the program test.c (attached). If I run it and
> > press UP or DOWN:
> > * on Windows from cmd.exe it says "Read 3 bytes. First is 27."
> > * on Linux it says "Read 3 bytes. First is 27."
> > * on Linux via PuTTY it says "Read 3 bytes. First is 27."
> > * on Windows from mintty.exe it says "Read 1 bytes. First is 27. Read
> >   1 bytes. First is 91. Read 1 bytes. First is 65."
> > 
> > My understanding is that the unexpected behavior occurs because Cygwin
> > sends the UP/DOWN sequence one byte at a time. Specifically:
> > 
> > * winsup\cygwin\fhandler_tty.cc @ fhandler_pty_master::write
> >     This is the function called by the write system call invoked by
> >     mintty. Here len = 3. line_edit is invoked 3 times.
> > * winsup\cygwin\fhandler_termios.cc @ fhandler_termios::line_edit
> >     This is called by the previous and it calls accept_input.
> > * winsup\cygwin\fhandler_tty.cc @ fhandler_pty_master::accept_input
> >     This does the actual WriteFile to the pipe.
> > 
> > I would have provided a patch to fix the problem, but I am not sure I
> > completely understand the semantics of the above mentioned methods.
> 
> I have to admit that I'm not quite sure either.  In theory I'd say that
> the perf tool is making some invalid assumption here.  You can't rely on
> a set of bytes sent from any input source to be always sent or received
> as a single package, unless you're working with a transport guaranteing
> this.
> 
> Your analyzes of the underlying mechanism in Cygwin is correct, though.
> Despite what I said above, I take a look into this and perhaps I can fix
> the code to send more than 1 byte at a time.  

In fact, both sides of the pipe were writing/reading in single byte
chunks.

The slave read side did something weird:  Before reading, it always
checked if the number of bytes available in the pipe is > VMIN.  If so,
it only read VMIN bytes.  Since VMIN is 1 by default, it always reads
only single bytes, since changing VMIN is only done rather seldomly by
applications.
This code snippet is actually 12 years old.  Much of the code
surrounding it changed considerably, just this snippet persisted.  I
disabled it now since it really looks like a remnant from the past, and
it certainly doesn't reflect how it works on other OSes.

On the master write side I applied a possible, but definitely
experimental fix, which solves this issue by writing in 32 byte chunks
in non-canonical mode.  This should (mostly) circumvent the above effect,
but it doesn't quite guarantee it.

With luck, this change also speeds up tty output by reducing the # of
calls to WriteFile/ReadFile.

I uploaded a new snapshot to https://cygwin.com/snapshots, the one
from today.  Please give it a try.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

Re: Unexpected behavior using arrow keys on the terminal

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 509 bytes --]

On Nov 18 17:44, George Prekas wrote:
> Hi Corinna,
> 
> Your snapshot works! I verified that perf utility works. I also verified 
> through Wireshark that only one packet is sent when I press UP or DOWN 
> (the old behavior was that 3 packets were sent).
> 
> Thank you very much for your effort and prompt reply!

Thanks for testing!


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]