Re: [PATCH] Strange behavior of cmd.exe when hammered with clear screen operations from Cygwin program.

Re: [PATCH] Strange behavior of cmd.exe when hammered with clear screen operations from Cygwin program.

[Kaz Kylheku]

On 28.07.2016 00:08, Corinna Vinschen wrote:
> Thanks for testing.  I'll check in the patch as is for now, but I'd
> still be interested to fix the aforementioned empty lines problem,
> even if just for completeness.  If you have an idea, feel free to
> propose.

Hi Corinna,

The blank lines above the cursor position, as well as any
that are below, are entering into the history simply because they
are part of the window content being saved. Any non-blank lines
in their place would be expected to be saved, and no processing
takes place to distinguish blank from non-blank.

One way to address this is to basically scan the entire window
portion of the buffer for trailing blank lines and trim the buffer
to exclude them. Then scroll by the amount of remaining lines
lines, to usher them into the history.

Finally, restore the cursor position.

Or, perhaps, two or more trailing blank lines in the window could be
condensed down to one which does get saved, so the separation
is not misleadingly condensed down to nothing.

However, there are possible arguments against trimming blank
lines which are above the cursor position. Say the window
contains this:

    +---
    | nonblank
    | blank
    | blank
    | > [] cursor
    | blank
    | blank
    | blank
    +---

It's uncontroversial to obliterate the three blank lines below the
cursor. However the blank lines between the nonblank line and
the cursor could be semantic; the user might want them in the
scrollback history "as is".

The leading nonblank line is a red herring; even if all the
lines above the cursor are blank, the reasoning applies.



--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: [PATCH] Strange behavior of cmd.exe when hammered with clear screen operations from Cygwin program.

[Kaz Kylheku]

On 28.07.2016 00:08, Corinna Vinschen wrote:
> Thanks for testing.  I'll check in the patch as is for now, but I'd
> still be interested to fix the aforementioned empty lines problem,
> even if just for completeness.  If you have an idea, feel free to
> propose.

Hi Corinna,

The blank lines above the cursor position, as well as any
that are below, are entering into the history simply because they
are part of the window content being saved. Any non-blank lines
in their place would be expected to be saved, and no processing
takes place to distinguish blank from non-blank.

One way to address this is to basically scan the entire window
portion of the buffer for trailing blank lines and trim the buffer
to exclude them. Then scroll by the amount of remaining lines
lines, to usher them into the history.

Finally, restore the cursor position.

Or, perhaps, two or more trailing blank lines in the window could be
condensed down to one which does get saved, so the separation
is not misleadingly condensed down to nothing.

However, there are possible arguments against trimming blank
lines which are above the cursor position. Say the window
contains this:

    +---
    | nonblank
    | blank
    | blank
    | > [] cursor
    | blank
    | blank
    | blank
    +---

It's uncontroversial to obliterate the three blank lines below the
cursor. However the blank lines between the nonblank line and
the cursor could be semantic; the user might want them in the
scrollback history "as is".

The leading nonblank line is a red herring; even if all the
lines above the cursor are blank, the reasoning applies.



--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: [PATCH] Strange behavior of cmd.exe when hammered with clear screen operations from Cygwin program.

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 3678 bytes --]

On Jul 28 09:36, Kaz Kylheku wrote:
> On 28.07.2016 00:08, Corinna Vinschen wrote:
> > Thanks for testing.  I'll check in the patch as is for now, but I'd
> > still be interested to fix the aforementioned empty lines problem,
> > even if just for completeness.  If you have an idea, feel free to
> > propose.
> 
> Hi Corinna,
> 
> The blank lines above the cursor position, as well as any
> that are below, are entering into the history simply because they
> are part of the window content being saved. Any non-blank lines
> in their place would be expected to be saved, and no processing
> takes place to distinguish blank from non-blank.
> 
> One way to address this is to basically scan the entire window
> portion of the buffer for trailing blank lines and trim the buffer
> to exclude them. Then scroll by the amount of remaining lines
> lines, to usher them into the history.
> 
> Finally, restore the cursor position.
> 
> Or, perhaps, two or more trailing blank lines in the window could be
> condensed down to one which does get saved, so the separation
> is not misleadingly condensed down to nothing.
> 
> However, there are possible arguments against trimming blank
> lines which are above the cursor position. Say the window
> contains this:
> 
>    +---
>    | nonblank
>    | blank
>    | blank
>    | > [] cursor
>    | blank
>    | blank
>    | blank
>    +---
> 
> It's uncontroversial to obliterate the three blank lines below the
> cursor. However the blank lines between the nonblank line and
> the cursor could be semantic; the user might want them in the
> scrollback history "as is".
> 
> The leading nonblank line is a red herring; even if all the
> lines above the cursor are blank, the reasoning applies.

The situation I was thinking about is a bit different.  Consider an only
partially filled console buffer.  In contrast to your typical UNIX
terminal application the buffer doesn't grow.  It's a fixed size and the
scroll bar reflects this buffer size right from the start.  So the user
can scroll the console windows down into the not yet used buffer area,
i.e.:

Before:

   buffer
   +---
   | nonblank
   | nonblank
   | blank
   | blank
   | > [] cursor
   | blank
   | blank      window
   | blank      ---+
   | blank         |
   | blank         |
   | blank      ---+
   | blank
   +---

Performing the clear screen in this situation, the cursor will be
repositioned to the first line in the current window and the blank lines
between the old cursor position and the new cursor position

After (now):

   buffer
   +---
   | nonblank
   | nonblank
   | blank
   | blank
   | > 
   | blank
   | blank      window
   | blank      ---+
   | > [] cursor   |
   | blank         |
   | blank      ---+
   | blank
   +---

However, the former cursor position in the buffer is still known
at clear screen time.  It's stored in dev_console::dwEnd.  So what
I was thinking about is to check the above situation and then, rather
then to just keep it as is, move the console window  and the cursor
position up to the line right after the former cursor position, i.e:

After (desired?):

   buffer
   +---
   | nonblank
   | nonblank
   | blank
   | blank      window
   | >          ---+
   | > [] cursor   |
   | blank         |
   | blank      ---+
   | blank
   | blank
   | blank
   | blank
   +---

Does that make sense or am I still missing something?


Thanks,
Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [PATCH] Strange behavior of cmd.exe when hammered with clear screen operations from Cygwin program.

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1169 bytes --]

On Jul 28 21:51, Corinna Vinschen wrote:
> The situation I was thinking about is a bit different.  Consider an only
> partially filled console buffer.  In contrast to your typical UNIX
> terminal application the buffer doesn't grow.  It's a fixed size and the
> scroll bar reflects this buffer size right from the start.  So the user
> can scroll the console windows down into the not yet used buffer area,
> i.e.:
> 
> Before:
> 
>    buffer
>    +---
>    | nonblank
>    | nonblank
>    | blank
>    | blank
>    | > [] cursor
>    | blank
>    | blank      window
>    | blank      ---+
>    | blank         |
>    | blank         |
>    | blank      ---+
>    | blank
>    +---
> 
> Performing the clear screen in this situation, the cursor will be
> repositioned to the first line in the current window and the blank lines
> between the old cursor position and the new cursor position

... will invariably become part of the buffer.  Sorry, I lost track
mid-sentence.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [PATCH] Strange behavior of cmd.exe when hammered with clear screen operations from Cygwin program.

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1017 bytes --]

On Jul 28 21:51, Corinna Vinschen wrote:
> However, the former cursor position in the buffer is still known
> at clear screen time.  It's stored in dev_console::dwEnd.  So what
> I was thinking about is to check the above situation and then, rather
> then to just keep it as is, move the console window  and the cursor
> position up to the line right after the former cursor position, i.e:
> 
> After (desired?):
> 
>    buffer
>    +---
>    | nonblank
>    | nonblank
>    | blank
>    | blank      window
>    | >          ---+
>    | > [] cursor   |
>    | blank         |
>    | blank      ---+
>    | blank
>    | blank
>    | blank
>    | blank
>    +---
> 
> Does that make sense or am I still missing something?

I applied a patch to perform this action.  It's in the latest
2.6.0-0.5 test release I just announced.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [PATCH] Strange behavior of cmd.exe when hammered with clear screen operations from Cygwin program.

[Kaz Kylheku]

On 29.07.2016 03:39, Corinna Vinschen wrote:
> On Jul 28 21:51, Corinna Vinschen wrote:
>> However, the former cursor position in the buffer is still known
>> at clear screen time.  It's stored in dev_console::dwEnd.  So what
>> I was thinking about is to check the above situation and then, rather
>> then to just keep it as is, move the console window  and the cursor
>> position up to the line right after the former cursor position, i.e:
>> 
>> After (desired?):
>> 
>>    buffer
>>    +---
>>    | nonblank
>>    | nonblank
>>    | blank
>>    | blank      window
>>    | >          ---+
>>    | > [] cursor   |
>>    | blank         |
>>    | blank      ---+
>>    | blank
>>    | blank
>>    | blank
>>    | blank
>>    +---
>> 
>> Does that make sense or am I still missing something?
> 
> I applied a patch to perform this action.  It's in the latest
> 2.6.0-0.5 test release I just announced.

I've done some interactive testing with this using
the interpreter for a Lisp dialect. I would evaluate
this expression to generate a 5 second delay and then
a clear screen VT100 sequence:

   (progn (usleep 5000000) (put-string "\e[2J"))

during this time, I would scroll the buffer somewhere.

I also tested with a cursor position somewhere in the
middle of the window, having issued:

   (put-string "\e[12H")

The programming language details don't matter; we
could do this with bash echo $'\e...' and sleep 5.

In terms of the final appearance of the terminal window,
the best results are obtained using the naive newline
ejection method! The screen is clear, and the cursor is at
the proper *window* position. (That's what we care about
in VT100-land; we don't care about Windows console buffer
semantics, and the position in the abstract buffer; position
means screen position.)

Your first patch has the effect that the physical screen position
is not always restored. The window is clear, but the cursor
is sometimes at the bottom, even though it was somewhere
in the middle of the window before the operation. In the
abstract buffer coordinates, it is correct; it's just that
the window viewport is not all the way down.

With the third patch, I've run into behavior in which the
display isn't cleared at all if the clear is issued
in a scrolled-back state.

The naive newline ejection, in spite of its flaw of adding
more blank lines to the history than one would like,
achieves the best visual results in all cases because
the viewport jumps to the bottom of the buffer when the
newlines are being emitted, and then the cursor position
is restored in that viewport position.

Cheers ...

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: [PATCH] Strange behavior of cmd.exe when hammered with clear screen operations from Cygwin program.

[Kaz Kylheku]

On 29.07.2016 03:39, Corinna Vinschen wrote:
> On Jul 28 21:51, Corinna Vinschen wrote:
>> However, the former cursor position in the buffer is still known
>> at clear screen time.  It's stored in dev_console::dwEnd.  So what
>> I was thinking about is to check the above situation and then, rather
>> then to just keep it as is, move the console window  and the cursor
>> position up to the line right after the former cursor position, i.e:
>> 
>> After (desired?):
>> 
>>    buffer
>>    +---
>>    | nonblank
>>    | nonblank
>>    | blank
>>    | blank      window
>>    | >          ---+
>>    | > [] cursor   |
>>    | blank         |
>>    | blank      ---+
>>    | blank
>>    | blank
>>    | blank
>>    | blank
>>    +---
>> 
>> Does that make sense or am I still missing something?
> 
> I applied a patch to perform this action.  It's in the latest
> 2.6.0-0.5 test release I just announced.

I've done some interactive testing with this using
the interpreter for a Lisp dialect. I would evaluate
this expression to generate a 5 second delay and then
a clear screen VT100 sequence:

   (progn (usleep 5000000) (put-string "\e[2J"))

during this time, I would scroll the buffer somewhere.

I also tested with a cursor position somewhere in the
middle of the window, having issued:

   (put-string "\e[12H")

The programming language details don't matter; we
could do this with bash echo $'\e...' and sleep 5.

In terms of the final appearance of the terminal window,
the best results are obtained using the naive newline
ejection method! The screen is clear, and the cursor is at
the proper *window* position. (That's what we care about
in VT100-land; we don't care about Windows console buffer
semantics, and the position in the abstract buffer; position
means screen position.)

Your first patch has the effect that the physical screen position
is not always restored. The window is clear, but the cursor
is sometimes at the bottom, even though it was somewhere
in the middle of the window before the operation. In the
abstract buffer coordinates, it is correct; it's just that
the window viewport is not all the way down.

With the third patch, I've run into behavior in which the
display isn't cleared at all if the clear is issued
in a scrolled-back state.

The naive newline ejection, in spite of its flaw of adding
more blank lines to the history than one would like,
achieves the best visual results in all cases because
the viewport jumps to the bottom of the buffer when the
newlines are being emitted, and then the cursor position
is restored in that viewport position.

Cheers ...

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: [PATCH] Strange behavior of cmd.exe when hammered with clear screen operations from Cygwin program.

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1503 bytes --]

On Jul 29 08:59, Kaz Kylheku wrote:
> On 29.07.2016 03:39, Corinna Vinschen wrote:
> > I applied a patch to perform this action.  It's in the latest
> > 2.6.0-0.5 test release I just announced.
> [...]
> I've done some interactive testing with this using
> the interpreter for a Lisp dialect. I would evaluate
> this expression to generate a 5 second delay and then
> a clear screen VT100 sequence:
> 
>   (progn (usleep 5000000) (put-string "\e[2J"))
> 
> during this time, I would scroll the buffer somewhere.
> 
> I also tested with a cursor position somewhere in the
> middle of the window, having issued:
> 
>   (put-string "\e[12H")
> 
> The programming language details don't matter; we
> could do this with bash echo $'\e...' and sleep 5.
> [...]
> With the third patch, I've run into behavior in which the
> display isn't cleared at all if the clear is issued
> in a scrolled-back state.

I can't reproduce this.  If I don't click wildly on the scroll bat at
the time the clear screen action takes place (so I move the window right
after clear screen), the cursor is positioned at the top of the screen,
at the end of the buffer.  So, how would I reproduce your observation so
that all window positioning is guaranteed to take place *before* the
clear screen action and still see the broken output?


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [PATCH] Strange behavior of cmd.exe when hammered with clear screen operations from Cygwin program.

[Kaz Kylheku]

On 01.08.2016 01:51, Corinna Vinschen wrote:
> On Jul 29 08:59, Kaz Kylheku wrote:
>> On 29.07.2016 03:39, Corinna Vinschen wrote:
>> > I applied a patch to perform this action.  It's in the latest
>> > 2.6.0-0.5 test release I just announced.
>> [...]
>> I've done some interactive testing with this using
>> the interpreter for a Lisp dialect. I would evaluate
>> this expression to generate a 5 second delay and then
>> a clear screen VT100 sequence:
>> 
>>   (progn (usleep 5000000) (put-string "\e[2J"))
>> 
>> during this time, I would scroll the buffer somewhere.
>> 
>> I also tested with a cursor position somewhere in the
>> middle of the window, having issued:
>> 
>>   (put-string "\e[12H")
>> 
>> The programming language details don't matter; we
>> could do this with bash echo $'\e...' and sleep 5.
>> [...]
>> With the third patch, I've run into behavior in which the
>> display isn't cleared at all if the clear is issued
>> in a scrolled-back state.
> 
> I can't reproduce this.  If I don't click wildly on the scroll bat at
> the time the clear screen action takes place (so I move the window 
> right
> after clear screen), the cursor is positioned at the top of the screen,
> at the end of the buffer.  So, how would I reproduce your observation 
> so
> that all window positioning is guaranteed to take place *before* the
> clear screen action and still see the broken output?

Hi Corinna,

I have figured out how to reproduce it. There are two
necessary conditions. No, three:

1. You must scroll the display all the way to the top
    as far as you can before the clear screen is issued.

2. The scrollback history must be full. I.e., this
    won't happen in a fresh cmd.exe window. First, print
    thousands of lines of stuff to make the buffer "tall".
    This is why  it only started happening to me after some
    amount of testing; I had filled the buffer.

3. It's probably a necessary condition that there is additional
    output immediately after the clear screen (such as the
    programming language prompt being printed), because
    the console scrolls to bottom on any output.

I'm not sure if these are sufficient, but they seem to be
on my end.

This is perhaps some kind of race condition between the scrolling
issued by cmd.exe, and the scrolling issued by the console API.
Or maybe not a race, as such, but some backlog-processing logic.
Perhaps the console simply drops scroll actions that it hasn't
had time to perform. The API-requested scroll is thrown in the
bit bucket, superseded by the scroll-to-bottom on output.


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: [PATCH] Strange behavior of cmd.exe when hammered with clear screen operations from Cygwin program.

[Kaz Kylheku]

On 01.08.2016 01:51, Corinna Vinschen wrote:
> On Jul 29 08:59, Kaz Kylheku wrote:
>> On 29.07.2016 03:39, Corinna Vinschen wrote:
>> > I applied a patch to perform this action.  It's in the latest
>> > 2.6.0-0.5 test release I just announced.
>> [...]
>> I've done some interactive testing with this using
>> the interpreter for a Lisp dialect. I would evaluate
>> this expression to generate a 5 second delay and then
>> a clear screen VT100 sequence:
>> 
>>   (progn (usleep 5000000) (put-string "\e[2J"))
>> 
>> during this time, I would scroll the buffer somewhere.
>> 
>> I also tested with a cursor position somewhere in the
>> middle of the window, having issued:
>> 
>>   (put-string "\e[12H")
>> 
>> The programming language details don't matter; we
>> could do this with bash echo $'\e...' and sleep 5.
>> [...]
>> With the third patch, I've run into behavior in which the
>> display isn't cleared at all if the clear is issued
>> in a scrolled-back state.
> 
> I can't reproduce this.  If I don't click wildly on the scroll bat at
> the time the clear screen action takes place (so I move the window 
> right
> after clear screen), the cursor is positioned at the top of the screen,
> at the end of the buffer.  So, how would I reproduce your observation 
> so
> that all window positioning is guaranteed to take place *before* the
> clear screen action and still see the broken output?

Hi Corinna,

I have figured out how to reproduce it. There are two
necessary conditions. No, three:

1. You must scroll the display all the way to the top
    as far as you can before the clear screen is issued.

2. The scrollback history must be full. I.e., this
    won't happen in a fresh cmd.exe window. First, print
    thousands of lines of stuff to make the buffer "tall".
    This is why  it only started happening to me after some
    amount of testing; I had filled the buffer.

3. It's probably a necessary condition that there is additional
    output immediately after the clear screen (such as the
    programming language prompt being printed), because
    the console scrolls to bottom on any output.

I'm not sure if these are sufficient, but they seem to be
on my end.

This is perhaps some kind of race condition between the scrolling
issued by cmd.exe, and the scrolling issued by the console API.
Or maybe not a race, as such, but some backlog-processing logic.
Perhaps the console simply drops scroll actions that it hasn't
had time to perform. The API-requested scroll is thrown in the
bit bucket, superseded by the scroll-to-bottom on output.


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: [PATCH] Strange behavior of cmd.exe when hammered with clear screen operations from Cygwin program.

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 3117 bytes --]

On Aug  2 23:19, Kaz Kylheku wrote:
> On 01.08.2016 01:51, Corinna Vinschen wrote:
> > On Jul 29 08:59, Kaz Kylheku wrote:
> > > On 29.07.2016 03:39, Corinna Vinschen wrote:
> > > > I applied a patch to perform this action.  It's in the latest
> > > > 2.6.0-0.5 test release I just announced.
> > > [...]
> > > I've done some interactive testing with this using
> > > the interpreter for a Lisp dialect. I would evaluate
> > > this expression to generate a 5 second delay and then
> > > a clear screen VT100 sequence:
> > > 
> > >   (progn (usleep 5000000) (put-string "\e[2J"))
> > > 
> > > during this time, I would scroll the buffer somewhere.
> > > 
> > > I also tested with a cursor position somewhere in the
> > > middle of the window, having issued:
> > > 
> > >   (put-string "\e[12H")
> > > 
> > > The programming language details don't matter; we
> > > could do this with bash echo $'\e...' and sleep 5.
> > > [...]
> > > With the third patch, I've run into behavior in which the
> > > display isn't cleared at all if the clear is issued
> > > in a scrolled-back state.
> > 
> > I can't reproduce this.  If I don't click wildly on the scroll bat at
> > the time the clear screen action takes place (so I move the window right
> > after clear screen), the cursor is positioned at the top of the screen,
> > at the end of the buffer.  So, how would I reproduce your observation so
> > that all window positioning is guaranteed to take place *before* the
> > clear screen action and still see the broken output?
> 
> Hi Corinna,
> 
> I have figured out how to reproduce it. There are two
> necessary conditions. No, three:
> 
> 1. You must scroll the display all the way to the top
>    as far as you can before the clear screen is issued.
> 
> 2. The scrollback history must be full. I.e., this
>    won't happen in a fresh cmd.exe window. First, print
>    thousands of lines of stuff to make the buffer "tall".
>    This is why  it only started happening to me after some
>    amount of testing; I had filled the buffer.
> 
> 3. It's probably a necessary condition that there is additional
>    output immediately after the clear screen (such as the
>    programming language prompt being printed), because
>    the console scrolls to bottom on any output.
> 
> I'm not sure if these are sufficient, but they seem to be
> on my end.

I could reproduce this now and I think I have fixed it.

There's still a problem with the position of the scrollbars in certain
scenarios where they are off by about one window size.  I added a FIXME
comment.

I have to admit I'm very puzzled that neither SetConsoleWindowInfo
nor the subsequent SetConsoleCursorPosition automatically set the
scrollbars correctly.  That's what I would expect, naively.

If anybody knows how to fix the scrollbar position in a Windows console
without moving the window contents around, please help.


Thanks,
Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [PATCH] Strange behavior of cmd.exe when hammered with clear screen operations from Cygwin program.

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 2269 bytes --]

On Jul 27 13:36, Kaz Kylheku wrote:
> On 27.07.2016 11:45, Corinna Vinschen wrote:
> > Btw, since you're apparently building your own Cygwin DLL I thought just
> > sending the patch for testing is sufficient.  If you prefer to get a
> > developer snapshot with this patch, I can do that, too.  Just say the
> > word.
> 
> The patch looks good and in many cases avoids padding the history with
> blank lines, compared to the naive newline emission approach.
> 
> Usually, the clear screen operation is preceded by a homing of the cursor.
> I.e., in VT100 language:  ESC [ H, ESC [ 2 J.  In this case, your approach
> seems to nicely trim the console data at that line. Rather, that is to
> say, when we scroll back, there aren't any blank lines, since none were
> added; the clear was done as a pure scrolling operation without any
> padding data.
> 
> Blank lines are only generated if the cursor is in the middle of the
> buffer. Since the cursor position is restored after the scrolling,
> that then adds blank lines between the scrolled-away material and the
> cursor position. And if the clear is repeated at that position,
> more blank lines are added for each repetition. Who cares about that case,
> though.

I saw that and disregarded it as not overly important.  On second
thought there should be a matching scrolling operation for this case,
too, which does not waste buffer with empty lines.

> Other implementations behave the same way, if not worse.
> 
> Case in point: Gnome Terminal (3.4.1.1) behaves like my naive \n
> repetition patch! Every issue of ESC [ 2 J adds a window height's
> worth of blank lines to the history,  regardless of the position
> of the cursor!  That is what I compared to as a "gold standard",
> so I was okay with it.

I checked against xfce4-terminal and konsole and both are rather
disappointing...

Thanks for testing.  I'll check in the patch as is for now, but I'd
still be interested to fix the aforementioned empty lines problem,
even if just for completeness.  If you have an idea, feel free to
propose.


Thanks,
Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [PATCH] Strange behavior of cmd.exe when hammered with clear screen operations from Cygwin program.

[Kaz Kylheku]

On 27.07.2016 11:45, Corinna Vinschen wrote:
> Btw, since you're apparently building your own Cygwin DLL I thought 
> just
> sending the patch for testing is sufficient.  If you prefer to get a
> developer snapshot with this patch, I can do that, too.  Just say the 
> word.

The patch looks good and in many cases avoids padding the history with
blank lines, compared to the naive newline emission approach.

Usually, the clear screen operation is preceded by a homing of the 
cursor.
I.e., in VT100 language:  ESC [ H, ESC [ 2 J.  In this case, your 
approach
seems to nicely trim the console data at that line. Rather, that is to
say, when we scroll back, there aren't any blank lines, since none were
added; the clear was done as a pure scrolling operation without any
padding data.

Blank lines are only generated if the cursor is in the middle of the
buffer. Since the cursor position is restored after the scrolling,
that then adds blank lines between the scrolled-away material and the
cursor position. And if the clear is repeated at that position,
more blank lines are added for each repetition. Who cares about that 
case,
though.

Other implementations behave the same way, if not worse.

Case in point: Gnome Terminal (3.4.1.1) behaves like my naive \n
repetition patch! Every issue of ESC [ 2 J adds a window height's
worth of blank lines to the history,  regardless of the position
of the cursor!  That is what I compared to as a "gold standard",
so I was okay with it.

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: [PATCH] Strange behavior of cmd.exe when hammered with clear screen operations from Cygwin program.

[Kaz Kylheku]

On 27.07.2016 11:45, Corinna Vinschen wrote:
> Btw, since you're apparently building your own Cygwin DLL I thought 
> just
> sending the patch for testing is sufficient.  If you prefer to get a
> developer snapshot with this patch, I can do that, too.  Just say the 
> word.

The patch looks good and in many cases avoids padding the history with
blank lines, compared to the naive newline emission approach.

Usually, the clear screen operation is preceded by a homing of the 
cursor.
I.e., in VT100 language:  ESC [ H, ESC [ 2 J.  In this case, your 
approach
seems to nicely trim the console data at that line. Rather, that is to
say, when we scroll back, there aren't any blank lines, since none were
added; the clear was done as a pure scrolling operation without any
padding data.

Blank lines are only generated if the cursor is in the middle of the
buffer. Since the cursor position is restored after the scrolling,
that then adds blank lines between the scrolled-away material and the
cursor position. And if the clear is repeated at that position,
more blank lines are added for each repetition. Who cares about that 
case,
though.

Other implementations behave the same way, if not worse.

Case in point: Gnome Terminal (3.4.1.1) behaves like my naive \n
repetition patch! Every issue of ESC [ 2 J adds a window height's
worth of blank lines to the history,  regardless of the position
of the cursor!  That is what I compared to as a "gold standard",
so I was okay with it.

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: [PATCH] Strange behavior of cmd.exe when hammered with clear screen operations from Cygwin program.

[Corinna Vinschen]

[-- Attachment #1.1: Type: text/plain, Size: 1797 bytes --]

Hi Kaz,

On Jul 27 08:40, Kaz Kylheku wrote:
> On 27.07.2016 07:35, Kaz Kylheku wrote:
> > On 27.07.2016 07:31, Kaz Kylheku wrote:
> > > The patch below seems to solves it for me:
> > 
> > Oops, no it doesn't. Stay tuned. :)
> 
> This is a bit more like it:
> 
> From: Kaz Kylheku <kaz@kylheku.com>
> Date: Wed, 27 Jul 2016 07:49:54 -0700
> Subject: Replace bogus resize-window-to-clear-screen logic.
> 
> This removes (ab)uses of SetConsoleScreenBufferSize
> which sometimes cause cmd.exe to badly misbehave.
> It probably doesn't like the closely spaced timing of
> shrinking the window down to one line followed by a restore
> of the size. Instead we just output newlines to clear
> the window.

The problem is obvious when observing your test application (thanks a
lot for that), but I'm not convinced that just calling WriteConsole with
a single LF in a loop is the way to go.  The Windows console API has
more nifty ways to fix this problem.

Here's another suggestion.  I attached a patch which utilizes
ScrollConsoleScreenBuffer to save the buffer content when the console
window is at the end of the console buffer.  So we either move the
console window down the buffer if there's enough buffer left, or we move
the buffer up under the window if we're operating at the end of the
buffer.

Please give it a try and report back.  Especially if I screwed up :}

Btw, since you're apparently building your own Cygwin DLL I thought just
sending the patch for testing is sufficient.  If you prefer to get a
developer snapshot with this patch, I can do that, too.  Just say the word.


Thanks,
Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #1.2: 0001-Fix-buffer-scrolling-when-performing-a-clear-screen.patch --]
[-- Type: text/plain, Size: 2979 bytes --]

From f9dd1288180b5d88b2915337aa58462811de3168 Mon Sep 17 00:00:00 2001
From: Corinna Vinschen <corinna@vinschen.de>
Date: Wed, 27 Jul 2016 20:39:24 +0200
Subject: [PATCH] Fix buffer scrolling when performing a "clear screen"

Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
---
 winsup/cygwin/fhandler_console.cc | 54 +++++++++++++++++++++++++--------------
 1 file changed, 35 insertions(+), 19 deletions(-)

diff --git a/winsup/cygwin/fhandler_console.cc b/winsup/cygwin/fhandler_console.cc
index d4c077b..9c490c5 100644
--- a/winsup/cygwin/fhandler_console.cc
+++ b/winsup/cygwin/fhandler_console.cc
@@ -1216,30 +1216,46 @@ dev_console::scroll_window (HANDLE h, int x1, int y1, int x2, int y2)
     return false;
 
   SMALL_RECT sr;
-  int toscroll = 2 + dwEnd.Y - b.srWindow.Top;
-  int shrink = 1 + toscroll + b.srWindow.Bottom - b.dwSize.Y;
+  int toscroll = dwEnd.Y - b.srWindow.Top + 1;
   sr.Left = sr.Right = dwEnd.X = 0;
-  /* Can't increment dwEnd yet since we may not have space in
-     the buffer.  */
-  SetConsoleCursorPosition (h, dwEnd);
-  if (shrink > 0)
+
+  if (b.srWindow.Bottom + toscroll >= b.dwSize.Y)
     {
-      COORD c = b.dwSize;
-      c.Y = dwEnd.Y - shrink;
-      SetConsoleScreenBufferSize (h, c);
-      SetConsoleScreenBufferSize (h, b.dwSize);
-      dwEnd.Y = 0;
-      fillin (h);
-      toscroll = 2 + dwEnd.Y - b.srWindow.Top;
+      /* So we're at the end of the buffer and scrolling the console window
+	 would move us beyond the buffer.  What we do here is to scroll the
+	 console buffer upward by just as much so that the current last line
+	 becomes the last line just prior to the first window line.  That
+	 keeps the end of the console buffer intact, as desired.
+
+	 Since we're moving the console buffer under the console window in
+	 this case, we must not move the console window. */
+      SMALL_RECT br;
+      COORD dest;
+      CHAR_INFO fill;
+
+      br.Left = 0;
+      br.Top = dwEnd.Y - b.srWindow.Top + 1;
+      br.Right = b.dwSize.X - 1;
+      br.Bottom = b.dwSize.Y - 1;
+      dest.X = dest.Y = 0;
+      fill.Char.AsciiChar = ' ';
+      fill.Attributes = current_win32_attr;
+      ScrollConsoleScreenBuffer (h, &br, NULL, dest, &fill);
+      /* Fix dwEnd to reflect the new cursor line (minus 1 to take the
+	 increment a few lines later into account) */
+      dwEnd.Y = b.dwCursorPosition.Y - 1;
     }
-
-  sr.Top = sr.Bottom = toscroll;
-
-  SetConsoleWindowInfo (h, FALSE, &sr);
-
+  else
+    {
+      /* The reminder of the console buffer is big enough to simply move
+         the console window. */
+      sr.Top = sr.Bottom = toscroll;
+      SetConsoleWindowInfo (h, FALSE, &sr);
+    }
+  /* Eventually set cursor to new end position at the top of the window. */
   dwEnd.Y++;
   SetConsoleCursorPosition (h, dwEnd);
-
+  /* Fix up console buffer info. */
   fillin (h);
   return true;
 }
-- 
2.5.5


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [PATCH] Strange behavior of cmd.exe when hammered with clear screen operations from Cygwin program.

[Kaz Kylheku]

On 27.07.2016 07:35, Kaz Kylheku wrote:
> On 27.07.2016 07:31, Kaz Kylheku wrote:
>> The patch below seems to solves it for me:
> 
> Oops, no it doesn't. Stay tuned. :)

This is a bit more like it:

 From: Kaz Kylheku <kaz@kylheku.com>
Date: Wed, 27 Jul 2016 07:49:54 -0700
Subject: Replace bogus resize-window-to-clear-screen logic.

This removes (ab)uses of SetConsoleScreenBufferSize
which sometimes cause cmd.exe to badly misbehave.
It probably doesn't like the closely spaced timing of
shrinking the window down to one line followed by a restore
of the size. Instead we just output newlines to clear
the window.

* winsup/cygwin/fhandler.h (dev_console::scroll_window):
Member function declaration removed.
(dev_console::clear_should_scroll): New member
function declared.

* winsup/cygwin/fhandler_console.cc
(dev_console::scroll_window): Member function removed.
(dev_console::clear_should_scroll): New member function.
Performs only the test that was performed by scroll_window,
not the actual scrolling. The scrolling is now done in
the caller in the fhandler_console class.
(fhandler_console::clear_screen): Call con.clear_should_scroll
instead of con.scroll_window. If this returns false, act
as before. Otherwise, clear the screen by scrolling the
window. This is done not by making SetConsoleScreenBufferSize
calls to shrink and restore the window, but by earnestly
emitting a number of carriage returns equal to the vertical
screen size and then restoring the cursor position.

diff --git a/winsup/cygwin/fhandler.h b/winsup/cygwin/fhandler.h
index 21c71d7..4af584b 100644
--- a/winsup/cygwin/fhandler.h
+++ b/winsup/cygwin/fhandler.h
@@ -1361,7 +1361,7 @@ class dev_console
    int set_cl_x (cltype);
    int set_cl_y (cltype);
    bool fillin (HANDLE);
-  bool __reg3 scroll_window (HANDLE, int, int, int, int);
+  bool __reg3 clear_should_scroll (HANDLE, int, int, int, int);
    void __reg3 scroll_buffer (HANDLE, int, int, int, int, int, int);
    void __reg3 clear_screen (HANDLE, int, int, int, int);
    void __reg3 save_restore (HANDLE, char);
diff --git a/winsup/cygwin/fhandler_console.cc 
b/winsup/cygwin/fhandler_console.cc
index 5fe4480..380aeec 100644
--- a/winsup/cygwin/fhandler_console.cc
+++ b/winsup/cygwin/fhandler_console.cc
@@ -1209,38 +1209,11 @@ dev_console::set_cl_y (cltype y)
  }

  bool
-dev_console::scroll_window (HANDLE h, int x1, int y1, int x2, int y2)
+dev_console::clear_should_scroll (HANDLE h, int x1, int y1, int x2, int 
y2)
  {
    if (save_buf || x1 != 0 || x2 != dwWinSize.X - 1 || y1 != 
b.srWindow.Top
        || y2 != b.srWindow.Bottom || b.dwSize.Y <= dwWinSize.Y)
      return false;
-
-  SMALL_RECT sr;
-  int toscroll = 2 + dwEnd.Y - b.srWindow.Top;
-  int shrink = 1 + toscroll + b.srWindow.Bottom - b.dwSize.Y;
-  sr.Left = sr.Right = dwEnd.X = 0;
-  /* Can't increment dwEnd yet since we may not have space in
-     the buffer.  */
-  SetConsoleCursorPosition (h, dwEnd);
-  if (shrink > 0)
-    {
-      COORD c = b.dwSize;
-      c.Y = dwEnd.Y - shrink;
-      SetConsoleScreenBufferSize (h, c);
-      SetConsoleScreenBufferSize (h, b.dwSize);
-      dwEnd.Y = 0;
-      fillin (h);
-      toscroll = 2 + dwEnd.Y - b.srWindow.Top;
-    }
-
-  sr.Top = sr.Bottom = toscroll;
-
-  SetConsoleWindowInfo (h, FALSE, &sr);
-
-  dwEnd.Y++;
-  SetConsoleCursorPosition (h, dwEnd);
-
-  fillin (h);
    return true;
  }

@@ -1261,8 +1234,27 @@ fhandler_console::clear_screen (cltype xc1, 
cltype yc1, cltype xc2, cltype yc2)

    /* Detect special case - scroll the screen if we have a buffer in 
order to
       preserve the buffer. */
-  if (!con.scroll_window (h, x1, y1, x2, y2))
-    con.clear_screen (h, x1, y1, x2, y2);
+  if (!con.clear_should_scroll (h, x1, y1, x2, y2))
+    {
+      con.clear_screen (h, x1, y1, x2, y2);
+      return;
+    }
+
+  int xpos, ypos;
+  DWORD done;
+  cursor_get(&xpos, &ypos);
+  cursor_set(false, 0, con.b.dwSize.Y - 1);
+
+  for (int i = 0; i < con.dwWinSize.Y; i += done)
+    {
+      if (!WriteConsoleW (h, L"\n", 1, &done, 0))
+        {
+	  __seterrno ();
+	  break;
+	}
+    }
+
+  cursor_set(false, xpos, ypos);
  }

  void __reg3


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: [PATCH] Strange behavior of cmd.exe when hammered with clear screen operations from Cygwin program.

[Kaz Kylheku]

On 27.07.2016 07:31, Kaz Kylheku wrote:
> The patch below seems to solves it for me:

Oops, no it doesn't. Stay tuned. :)


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

[PATCH] Strange behavior of cmd.exe when hammered with clear screen operations from Cygwin program.

[Kaz Kylheku]

The patch below seems to solves it for me:

Cygwin should not be faking a scroll of the console by shrinking it
and then restoring its size. Though clever, this is evidently
unreliable; cmd.exe misbehaves if the console handle
is hit with rapid SetConsoleScreenBufferSize calls.

------------

 From: Kaz Kylheku <kaz@kylheku.com>
Date: Wed, 27 Jul 2016 07:12:54 -0700
Subject: Replace bogus resize-window-to-clear-screen logic.

This removes (ab)uses of SetConsoleScreenBufferSize
which sometimes cause cmd.exe to badly misbehave.
It probably doesn't like the closely spaced timing of
shrinking the window down to one line followed by a restore
of the size. Instead we just output newlines to clear
the window.

* winsup/cygwin/fhandler.h (dev_console::scroll_window):
Member function declaration removed.
(dev_console::clear_should_scroll): New member
function declared.

* winsup/cygwin/fhandler_console.cc
(dev_console::scroll_window): Member function removed.
(dev_console::clear_should_scroll): New member function.
Performs only the test that was performed by scroll_window,
not the actual scrolling. The scrolling is now done in
the caller in the fhandler_console class.
(fhandler_console::clear_screen): Call con.clear_should_scroll
instead of con.scroll_window. If this returns false, act
as before. Otherwise, clear the screen by scrolling the
window. This is done not by making SetConsoleScreenBufferSize
calls to shrink and restore the window, but by earnestly
emitting a number of carriage returns equal to the vertical
screen size and then restoring the cursor position.

diff --git a/winsup/cygwin/fhandler.h b/winsup/cygwin/fhandler.h
index 21c71d7..4af584b 100644
--- a/winsup/cygwin/fhandler.h
+++ b/winsup/cygwin/fhandler.h
@@ -1361,7 +1361,7 @@ class dev_console
    int set_cl_x (cltype);
    int set_cl_y (cltype);
    bool fillin (HANDLE);
-  bool __reg3 scroll_window (HANDLE, int, int, int, int);
+  bool __reg3 clear_should_scroll (HANDLE, int, int, int, int);
    void __reg3 scroll_buffer (HANDLE, int, int, int, int, int, int);
    void __reg3 clear_screen (HANDLE, int, int, int, int);
    void __reg3 save_restore (HANDLE, char);
diff --git a/winsup/cygwin/fhandler_console.cc 
b/winsup/cygwin/fhandler_console.cc
index 5fe4480..f4806e0 100644
--- a/winsup/cygwin/fhandler_console.cc
+++ b/winsup/cygwin/fhandler_console.cc
@@ -1209,38 +1209,11 @@ dev_console::set_cl_y (cltype y)
  }

  bool
-dev_console::scroll_window (HANDLE h, int x1, int y1, int x2, int y2)
+dev_console::clear_should_scroll (HANDLE h, int x1, int y1, int x2, int 
y2)
  {
    if (save_buf || x1 != 0 || x2 != dwWinSize.X - 1 || y1 != 
b.srWindow.Top
        || y2 != b.srWindow.Bottom || b.dwSize.Y <= dwWinSize.Y)
      return false;
-
-  SMALL_RECT sr;
-  int toscroll = 2 + dwEnd.Y - b.srWindow.Top;
-  int shrink = 1 + toscroll + b.srWindow.Bottom - b.dwSize.Y;
-  sr.Left = sr.Right = dwEnd.X = 0;
-  /* Can't increment dwEnd yet since we may not have space in
-     the buffer.  */
-  SetConsoleCursorPosition (h, dwEnd);
-  if (shrink > 0)
-    {
-      COORD c = b.dwSize;
-      c.Y = dwEnd.Y - shrink;
-      SetConsoleScreenBufferSize (h, c);
-      SetConsoleScreenBufferSize (h, b.dwSize);
-      dwEnd.Y = 0;
-      fillin (h);
-      toscroll = 2 + dwEnd.Y - b.srWindow.Top;
-    }
-
-  sr.Top = sr.Bottom = toscroll;
-
-  SetConsoleWindowInfo (h, FALSE, &sr);
-
-  dwEnd.Y++;
-  SetConsoleCursorPosition (h, dwEnd);
-
-  fillin (h);
    return true;
  }

@@ -1261,8 +1234,28 @@ fhandler_console::clear_screen (cltype xc1, 
cltype yc1, cltype xc2, cltype yc2)

    /* Detect special case - scroll the screen if we have a buffer in 
order to
       preserve the buffer. */
-  if (!con.scroll_window (h, x1, y1, x2, y2))
-    con.clear_screen (h, x1, y1, x2, y2);
+  if (!con.clear_should_scroll (h, x1, y1, x2, y2))
+    {
+      con.clear_screen (h, x1, y1, x2, y2);
+      return;
+    }
+
+  int xpos, ypos;
+  cursor_get(&xpos, &ypos);
+
+  for (int i = 0; i < con.dwWinSize.Y; i++)
+    {
+      WCHAR buf[] = { L'\n' };
+      DWORD done;
+
+      if (!WriteConsoleW (get_output_handle (), buf, 1, &done, 0))
+        {
+	  __seterrno ();
+	  break;
+	}
+    }
+
+  cursor_set(false, xpos, ypos);
  }

  void __reg3


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple