* 3.1.x: Mangled input/output when calling non-cygwin programs
@ 2020-02-01 18:00 m0viefreak
2020-02-02 9:59 ` Takashi Yano
0 siblings, 1 reply; 6+ messages in thread
From: m0viefreak @ 2020-02-01 18:00 UTC (permalink / raw)
To: cygwin
Hi,
Since Cygwin 3.1's pseudo console support was introduced I've run into
lots of issues with non-cygwin programs.
1) Mangled output: See screenshot [1].
As you can see, at what it seems random places, the output is broken.
The above can easily be reproduced using Apache Maven in any project,
e.g. using a sample project created using [2].
2) Terminal width: For some reason, non-native programs do not use the
full width of the terminal (Mintty and TERM=xterm-256color). Can also
be seen in [1] (blue highlight) and reproduced e.g. using Maven with
a relatively small window.
3) Typed characters while non-cygwin program is running are lost.
Usually, when typing characters while a native program executes,
which does *not* read from standard input, after the program exits,
all these typed characters are put into the prompt.
When executing a non-native program, such as, again, "mvn clean",
which does not read from stdin, typed characters are completely lost.
This is very frustrating, since I am often quicker at tying the
next command, while the previous command is still doing something.
4) Importance of TERM: When connecting to my Cygwin installation using
SSH using Putty, TERM=putty-256color is set. When executing non-
native program from that session using that TERM, output is broken
even worse than in 1), and also keybindings are broken. I need to
force TERM=xterm-256color for those invocations.
Shouldn't this be handled transparently and enforced automatically by
the pcon code?
Note 1: This is tested using the latest Snapshot 2020-01-31 18:10:26 UTC
from https://cygwin.com/snapshots/.
Note 2: It seems that the problems can be solved by setting
CYGWIN=disable_pcon but it feels like there is some bug in the
PTY code (not in the non-native program being called).
[1] https://images2.imgbox.com/dc/2d/MQwSH9S4_o.png
[2]
http://maven.apache.org/guides/getting-started/index.html#How_do_I_make_my_first_Maven_project
--
Problem reports: http://cygwin.com/problems.html
FAQ: http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: 3.1.x: Mangled input/output when calling non-cygwin programs
2020-02-01 18:00 3.1.x: Mangled input/output when calling non-cygwin programs m0viefreak
@ 2020-02-02 9:59 ` Takashi Yano
2020-02-02 12:18 ` m0viefreak
2020-02-09 13:42 ` Takashi Yano
0 siblings, 2 replies; 6+ messages in thread
From: Takashi Yano @ 2020-02-02 9:59 UTC (permalink / raw)
To: cygwin
On Sat, 1 Feb 2020 19:00:04 +0100
m0viefreak wrote:
> Since Cygwin 3.1's pseudo console support was introduced I've run into
> lots of issues with non-cygwin programs.
Thanks for the report.
> 1) Mangled output: See screenshot [1].
> As you can see, at what it seems random places, the output is broken.
> The above can easily be reproduced using Apache Maven in any project,
> e.g. using a sample project created using [2].
>
> 2) Terminal width: For some reason, non-native programs do not use the
> full width of the terminal (Mintty and TERM=xterm-256color). Can also
> be seen in [1] (blue highlight) and reproduced e.g. using Maven with
> a relatively small window.
I looked into this problem, but Maven is something weird.
I don't think it uses termcap or terminfo, but it behaves
differently depending on TERM.
If you run
env TERM=cygwin mvn <something>
in mintty, problem 1) and 2) does not occur.
> 3) Typed characters while non-cygwin program is running are lost.
> Usually, when typing characters while a native program executes,
> which does *not* read from standard input, after the program exits,
> all these typed characters are put into the prompt.
> When executing a non-native program, such as, again, "mvn clean",
> which does not read from stdin, typed characters are completely lost.
> This is very frustrating, since I am often quicker at tying the
> next command, while the previous command is still doing something.
I know the cause of this problem. Let me consider.
> 4) Importance of TERM: When connecting to my Cygwin installation using
> SSH using Putty, TERM=putty-256color is set. When executing non-
> native program from that session using that TERM, output is broken
> even worse than in 1), and also keybindings are broken. I need to
> force TERM=xterm-256color for those invocations.
> Shouldn't this be handled transparently and enforced automatically by
> the pcon code?
Which keybinding is broken? In my environment, arrow keys and function
keys work as expected in cmd.exe and windows-native vim.
My configuration of putty is:
Terminal
+ Keybord
+ The Function keys and keypad: Xterm R6
The PTY codes do not reffer to TERM environment, so cygwin1.dll itself
should not behave differently depending on TERM.
--
Takashi Yano <takashi.yano@nifty.ne.jp>
--
Problem reports: http://cygwin.com/problems.html
FAQ: http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: 3.1.x: Mangled input/output when calling non-cygwin programs
2020-02-02 9:59 ` Takashi Yano
@ 2020-02-02 12:18 ` m0viefreak
2020-02-02 19:01 ` Takashi Yano
2020-02-09 13:42 ` Takashi Yano
1 sibling, 1 reply; 6+ messages in thread
From: m0viefreak @ 2020-02-02 12:18 UTC (permalink / raw)
To: cygwin
On 02.02.2020 10:59, Takashi Yano wrote:
> On Sat, 1 Feb 2020 19:00:04 +0100
> m0viefreak wrote:
>> Since Cygwin 3.1's pseudo console support was introduced I've run into
>> lots of issues with non-cygwin programs.
>
> Thanks for the report.
>
>> 1) Mangled output: See screenshot [1].
>> As you can see, at what it seems random places, the output is broken.
>> The above can easily be reproduced using Apache Maven in any project,
>> e.g. using a sample project created using [2].
>>
>> 2) Terminal width: For some reason, non-native programs do not use the
>> full width of the terminal (Mintty and TERM=xterm-256color). Can also
>> be seen in [1] (blue highlight) and reproduced e.g. using Maven with
>> a relatively small window.
>
> I looked into this problem, but Maven is something weird.
> I don't think it uses termcap or terminfo, but it behaves
> differently depending on TERM.
>
> If you run
> env TERM=cygwin mvn <something>
> in mintty, problem 1) and 2) does not occur.
Interesting, this solves the mangled output. But it somewhat contradicts
that TERM should not matter (see below).
>> 3) Typed characters while non-cygwin program is running are lost.
>> Usually, when typing characters while a native program executes,
>> which does *not* read from standard input, after the program exits,
>> all these typed characters are put into the prompt.
>> When executing a non-native program, such as, again, "mvn clean",
>> which does not read from stdin, typed characters are completely lost.
>> This is very frustrating, since I am often quicker at tying the
>> next command, while the previous command is still doing something.
>
> I know the cause of this problem. Let me consider.
That would be great to have the same experience as without pcon.
>> 4) Importance of TERM: When connecting to my Cygwin installation using
>> SSH using Putty, TERM=putty-256color is set. When executing non-
>> native program from that session using that TERM, output is broken
>> even worse than in 1), and also keybindings are broken. I need to
>> force TERM=xterm-256color for those invocations.
>> Shouldn't this be handled transparently and enforced automatically by
>> the pcon code?
>
> Which keybinding is broken? In my environment, arrow keys and function
> keys work as expected in cmd.exe and windows-native vim.
HOME/END keys were broken if the native app brought up an interactive
prompt.
> My configuration of putty is:
> Terminal
> + Keybord
> + The Function keys and keypad: Xterm R6
>
> The PTY codes do not reffer to TERM environment, so cygwin1.dll itself
> should not behave differently depending on TERM.
It seems that the cygwin1.dll does not care about TERM, but if a
non-native application *uses* TERM for whatever reason things start to
break. If indeed 'cygwin' is the right terminfo to be used, should the
PTY code then somehow set TERM=cygwin in the non-native process? But
what if the none-native program doesn't even know the 'cygwin' terminfo?
One more thing (maybe also related): When calling a non-native program
that prints very sinle long line (e.g. just 'mvn' without args) that
wraps around the terminal, and then copying that multi-line output from
mintty, the result differs:
- disable_pcon: A single line is copied
- default: Multiple lines are copied (like in block selection mode)
--
Problem reports: http://cygwin.com/problems.html
FAQ: http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: 3.1.x: Mangled input/output when calling non-cygwin programs
2020-02-02 12:18 ` m0viefreak
@ 2020-02-02 19:01 ` Takashi Yano
2020-02-03 14:20 ` Takashi Yano
0 siblings, 1 reply; 6+ messages in thread
From: Takashi Yano @ 2020-02-02 19:01 UTC (permalink / raw)
To: cygwin
On Sun, 2 Feb 2020 13:18:03 +0100
m0viefreak wrote:
> On 02.02.2020 10:59, Takashi Yano wrote:
> >> 4) Importance of TERM: When connecting to my Cygwin installation using
> >> SSH using Putty, TERM=putty-256color is set. When executing non-
> >> native program from that session using that TERM, output is broken
> >> even worse than in 1), and also keybindings are broken. I need to
> >> force TERM=xterm-256color for those invocations.
> >> Shouldn't this be handled transparently and enforced automatically by
> >> the pcon code?
> >
> > Which keybinding is broken? In my environment, arrow keys and function
> > keys work as expected in cmd.exe and windows-native vim.
>
> HOME/END keys were broken if the native app brought up an interactive
> prompt.
In my environment, HOME/END keys are working as expected in cmd.exe
and windows-native vim in putty even if TERM=putty-256color.
I guess Maven changes its behaviour depending on TERM.
> It seems that the cygwin1.dll does not care about TERM, but if a
> non-native application *uses* TERM for whatever reason things start to
> break. If indeed 'cygwin' is the right terminfo to be used, should the
> PTY code then somehow set TERM=cygwin in the non-native process? But
> what if the none-native program doesn't even know the 'cygwin' terminfo?
This make sens only if apps use TERM *right way*. In other words,
if apps uses TERM in the manner of termcap or terminfo. I don't
think Maven does that.
Now I am looking into jansi library which Maven uses, and found
it sets IS_CYGWIN flag according to TERM environmen. It seems
that IS_CYGWIN is set to false if TERM=cygwin. In this case,
jansi uses SetConsoleTextAttribute() call to set text color
rather than escape sequences such as ESC[31m. Ohterwise, escape
sequence is used.
See:
https://github.com/fusesource/jansi/blob/master/jansi/src/main/java/org/fusesource/jansi/AnsiConsole.java
New cygwin provide pure windows console for native apps, so
escape sequence is not handled correctly. This is the cause
of the problem 1) and 2). It may be a problem that TERM is
set even though pure windows console does not have the
capability which TERM environment declares. For the one
thought, it can make sense to unset TERM before execute
native-console apps. However, this does not solve the issue
of Maven. If Maven (jansi) handles TERM *correctly*, this
can be a solution.
> One more thing (maybe also related): When calling a non-native program
> that prints very sinle long line (e.g. just 'mvn' without args) that
> wraps around the terminal, and then copying that multi-line output from
> mintty, the result differs:
> - disable_pcon: A single line is copied
> - default: Multiple lines are copied (like in block selection mode)
This is because pseudo console insert "\r\n" at the each wrapped
point. This is the internal bhaviour of pseudo console, so we
cannot control it.
--
Takashi Yano <takashi.yano@nifty.ne.jp>
--
Problem reports: http://cygwin.com/problems.html
FAQ: http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: 3.1.x: Mangled input/output when calling non-cygwin programs
2020-02-02 19:01 ` Takashi Yano
@ 2020-02-03 14:20 ` Takashi Yano
0 siblings, 0 replies; 6+ messages in thread
From: Takashi Yano @ 2020-02-03 14:20 UTC (permalink / raw)
To: cygwin
On Mon, 3 Feb 2020 04:00:33 +0900
Takashi Yano wrote:
> On Sun, 2 Feb 2020 13:18:03 +0100
> m0viefreak wrote:
> > It seems that the cygwin1.dll does not care about TERM, but if a
> > non-native application *uses* TERM for whatever reason things start to
> > break. If indeed 'cygwin' is the right terminfo to be used, should the
> > PTY code then somehow set TERM=cygwin in the non-native process? But
> > what if the none-native program doesn't even know the 'cygwin' terminfo?
>
> This make sens only if apps use TERM *right way*. In other words,
> if apps uses TERM in the manner of termcap or terminfo. I don't
> think Maven does that.
>
> Now I am looking into jansi library which Maven uses, and found
> it sets IS_CYGWIN flag according to TERM environmen. It seems
> that IS_CYGWIN is set to false if TERM=cygwin. In this case,
> jansi uses SetConsoleTextAttribute() call to set text color
> rather than escape sequences such as ESC[31m. Ohterwise, escape
> sequence is used.
>
> See:
> https://github.com/fusesource/jansi/blob/master/jansi/src/main/java/org/fusesource/jansi/AnsiConsole.java
>
> New cygwin provide pure windows console for native apps, so
> escape sequence is not handled correctly. This is the cause
> of the problem 1) and 2). It may be a problem that TERM is
> set even though pure windows console does not have the
> capability which TERM environment declares. For the one
> thought, it can make sense to unset TERM before execute
> native-console apps. However, this does not solve the issue
> of Maven. If Maven (jansi) handles TERM *correctly*, this
> can be a solution.
As for the problem 1) and 2), I think Maven (jansi) should be
fixed rather than cygwin. I have applied the following patch
to jansi, and confirmed this solves the problem 1) and 2).
diff --git a/jansi/src/main/java/org/fusesource/jansi/AnsiConsole.java b/jansi/src/main/java/org/fusesource/jansi/AnsiConsole.java
index 628cad3..a5400c5 100644
--- a/jansi/src/main/java/org/fusesource/jansi/AnsiConsole.java
+++ b/jansi/src/main/java/org/fusesource/jansi/AnsiConsole.java
@@ -19,6 +19,10 @@ import static org.fusesource.jansi.internal.CLibrary.STDERR_FILENO;
import static org.fusesource.jansi.internal.CLibrary.STDOUT_FILENO;
import static org.fusesource.jansi.internal.CLibrary.isatty;
+import static org.fusesource.jansi.internal.Kernel32.GetConsoleMode;
+import static org.fusesource.jansi.internal.Kernel32.GetStdHandle;
+import static org.fusesource.jansi.internal.Kernel32.STD_OUTPUT_HANDLE;
+
import java.io.FilterOutputStream;
import java.io.IOException;
import java.io.OutputStream;
@@ -54,10 +58,11 @@ public class AnsiConsole {
*/
static final boolean IS_CON_EMU_ANSI = "ON".equals(System.getenv("ConEmuANSI"));
+ static private int[] mode = {0};
static final boolean IS_CYGWIN = IS_WINDOWS
&& System.getenv("PWD") != null
&& System.getenv("PWD").startsWith("/")
- && !"cygwin".equals(System.getenv("TERM"));
+ && GetConsoleMode(GetStdHandle(STD_OUTPUT_HANDLE), mode) == 0;
static final boolean IS_MINGW_XTERM = IS_WINDOWS
&& System.getenv("MSYSTEM") != null
--
Takashi Yano <takashi.yano@nifty.ne.jp>
--
Problem reports: http://cygwin.com/problems.html
FAQ: http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: 3.1.x: Mangled input/output when calling non-cygwin programs
2020-02-02 9:59 ` Takashi Yano
2020-02-02 12:18 ` m0viefreak
@ 2020-02-09 13:42 ` Takashi Yano
1 sibling, 0 replies; 6+ messages in thread
From: Takashi Yano @ 2020-02-09 13:42 UTC (permalink / raw)
To: cygwin
On Sun, 2 Feb 2020 18:59:09 +0900
Takashi Yano wrote:
> On Sat, 1 Feb 2020 19:00:04 +0100
> m0viefreak wrote:
> > 3) Typed characters while non-cygwin program is running are lost.
> > Usually, when typing characters while a native program executes,
> > which does *not* read from standard input, after the program exits,
> > all these typed characters are put into the prompt.
> > When executing a non-native program, such as, again, "mvn clean",
> > which does not read from stdin, typed characters are completely lost.
> > This is very frustrating, since I am often quicker at tying the
> > next command, while the previous command is still doing something.
>
> I know the cause of this problem. Let me consider.
I will submit a patch for this issue shortly.
--
Takashi Yano <takashi.yano@nifty.ne.jp>
--
Problem reports: http://cygwin.com/problems.html
FAQ: http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2020-02-09 13:42 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-02-01 18:00 3.1.x: Mangled input/output when calling non-cygwin programs m0viefreak
2020-02-02 9:59 ` Takashi Yano
2020-02-02 12:18 ` m0viefreak
2020-02-02 19:01 ` Takashi Yano
2020-02-03 14:20 ` Takashi Yano
2020-02-09 13:42 ` Takashi Yano
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).