Console output hangs in the middle of a line

Console output hangs in the middle of a line

[Orgad Shaneh]

Hi,

I'm using cygwin runtime 3.3.4-2. CYGWIN env variable is empty.
Running in Windows Terminal.

When I run git clean -dfx on a Node.JS project, which has huge
node_modules directories, that produces a lot of output in high rate,
the console seems to freeze for very long in the middle of a line, and
continues after a while. If I get it correctly, the writes are done in
4K chunks, freezing for a few seconds after each chunk.

The output looks more or less like this:

Removing common/auth.js
Removing common/auth.js.map
...
Removing common/bar.js
Removing common/bar.js.map
Remo<freeze for about 20s after 4K bytes>
ving common/foo.js  (continued on the same line)
Removing common/foo.js.map
...
etc.

No output was lost.

Is this a known issue?

- Orgad

Re: Console output hangs in the middle of a line

[Takashi Yano]

On Wed, 16 Feb 2022 11:49:36 +0200
Orgad Shaneh wrote:
> I'm using cygwin runtime 3.3.4-2. CYGWIN env variable is empty.
> Running in Windows Terminal.
> 
> When I run git clean -dfx on a Node.JS project, which has huge
> node_modules directories, that produces a lot of output in high rate,
> the console seems to freeze for very long in the middle of a line, and
> continues after a while. If I get it correctly, the writes are done in
> 4K chunks, freezing for a few seconds after each chunk.
> 
> The output looks more or less like this:
> 
> Removing common/auth.js
> Removing common/auth.js.map
> ...
> Removing common/bar.js
> Removing common/bar.js.map
> Remo<freeze for about 20s after 4K bytes>
> ving common/foo.js  (continued on the same line)
> Removing common/foo.js.map
> ...
> etc.
> 
> No output was lost.
> 
> Is this a known issue?

Could you please provide reproducible steps for the issue?

-- 
Takashi Yano <takashi.yano@nifty.ne.jp>

Re: Console output hangs in the middle of a line

[Takashi Yano]

On Wed, 16 Feb 2022 18:59:05 +0900
Takashi Yano wrote:
> On Wed, 16 Feb 2022 11:49:36 +0200
> Orgad Shaneh wrote:
> > I'm using cygwin runtime 3.3.4-2. CYGWIN env variable is empty.
> > Running in Windows Terminal.
> > 
> > When I run git clean -dfx on a Node.JS project, which has huge
> > node_modules directories, that produces a lot of output in high rate,
> > the console seems to freeze for very long in the middle of a line, and
> > continues after a while. If I get it correctly, the writes are done in
> > 4K chunks, freezing for a few seconds after each chunk.
> > 
> > The output looks more or less like this:
> > 
> > Removing common/auth.js
> > Removing common/auth.js.map
> > ...
> > Removing common/bar.js
> > Removing common/bar.js.map
> > Remo<freeze for about 20s after 4K bytes>
> > ving common/foo.js  (continued on the same line)
> > Removing common/foo.js.map
> > ...
> > etc.
> > 
> > No output was lost.
> > 
> > Is this a known issue?
> 
> Could you please provide reproducible steps for the issue?

And, also please check if this issue occurs in command prompt?
(I mean not in Windows Terminal, but cmd.exe.)

-- 
Takashi Yano <takashi.yano@nifty.ne.jp>

Re: Console output hangs in the middle of a line

[Orgad Shaneh]

Hi and thanks for replying.

On Wed, 16 Feb 2022 12:01:07 +0200
Takashi Yano wrote:
> On Wed, 16 Feb 2022 18:59:05 +0900
> Takashi Yano wrote:
> > On Wed, 16 Feb 2022 11:49:36 +0200
> > Orgad Shaneh wrote:
> > > I'm using cygwin runtime 3.3.4-2. CYGWIN env variable is empty.
> > > Running in Windows Terminal.
> > >
> > > When I run git clean -dfx on a Node.JS project, which has huge
> > > node_modules directories, that produces a lot of output in high rate,
> > > the console seems to freeze for very long in the middle of a line, and
> > > continues after a while. If I get it correctly, the writes are done in
> > > 4K chunks, freezing for a few seconds after each chunk.
> > >
> > > The output looks more or less like this:
> > >
> > > Removing common/auth.js
> > > Removing common/auth.js.map
> > > ...
> > > Removing common/bar.js
> > > Removing common/bar.js.map
> > > Remo<freeze for about 20s after 4K bytes>
> > > ving common/foo.js  (continued on the same line)
> > > Removing common/foo.js.map
> > > ...
> > > etc.
> > >
> > > No output was lost.
> > >
> > > Is this a known issue?
> >
> > Could you please provide reproducible steps for the issue?

mkdir -p chunked
cd chunked
git init
mkdir -p abc/def/ghi/jkl def
cd abc/def/ghi/jkl
touch foo
git add foo
seq 1 1000 | xargs touch
cd ../../../../def
seq 1 10000 | xargs touch
cd ..
git clean -dfx

Notice that the long operation of cleaning def (that only writes a
single line) is not printed immediately. The output here looks like
this:
Removing abc/def/ghi/jkl/1
Removing abc/def/ghi/jkl/10
Removing abc/def/ghi/jkl/100
Removing abc/def/ghi/jkl/1000
Removing abc/def/ghi/jkl/101
Removing abc/def/ghi/jkl/102
...
Removing abc/def/ghi/jkl/991
Removing ab<hangs>
c/def/ghi/jkl/992
Removing abc/def/ghi/jkl/993
Removing abc/def/ghi/jkl/994
Removing abc/def/ghi/jkl/995
Removing abc/def/ghi/jkl/996
Removing abc/def/ghi/jkl/997
Removing abc/def/ghi/jkl/998
Removing abc/def/ghi/jkl/999
Removing def/

> And, also please check if this issue occurs in command prompt?
> (I mean not in Windows Terminal, but cmd.exe.)

It does.

Please include me when replying. I'm not registered to the mailing list.

- Orgad

Re: Console output hangs in the middle of a line

[Takashi Yano]

On Sun, 20 Feb 2022 13:43:33 +0200
Orgad Shaneh <orgads@gmail.com> wrote:
> mkdir -p chunked
> cd chunked
> git init
> mkdir -p abc/def/ghi/jkl def
> cd abc/def/ghi/jkl
> touch foo
> git add foo
> seq 1 1000 | xargs touch
> cd ../../../../def
> seq 1 10000 | xargs touch
> cd ..
> git clean -dfx

Thanks for the test case. However, I cannot reproduce
your problem.

$ uname -a
CYGWIN_NT-10.0 Express5800-S70 3.3.4(0.341/5/3) 2022-01-31 19:35 x86_64 Cygwin
$ git --version
git version 2.35.1

I have made a shell script such as:

#!/bin/sh
mkdir -p chunked
cd chunked
git init
mkdir -p abc/def/ghi/jkl def
cd abc/def/ghi/jkl
touch foo
git add foo
seq 1 1000 | xargs touch
cd ../../../../def
seq 1 10000 | xargs touch
cd ..
time git clean -dfx

and resut is as follows.

Removing abc/def/ghi/jkl/1
Removing abc/def/ghi/jkl/10
Removing abc/def/ghi/jkl/100
Removing abc/def/ghi/jkl/1000
Removing abc/def/ghi/jkl/101
Removing abc/def/ghi/jkl/102
Removing abc/def/ghi/jkl/103
Removing abc/def/ghi/jkl/104
Removing abc/def/ghi/jkl/105
Removing abc/def/ghi/jkl/106
Removing abc/def/ghi/jkl/107
Removing abc/def/ghi/jkl/108
[...]
Removing abc/def/ghi/jkl/995
Removing abc/def/ghi/jkl/996
Removing abc/def/ghi/jkl/997
Removing abc/def/ghi/jkl/998
Removing abc/def/ghi/jkl/999
Removing def/

real    0m3.307s
user    0m0.296s
sys     0m2.983s

How long does the test case hang?

In my environment, from the line
Removing abc/def/ghi/jkl/1
to the line
Removing abc/def/ghi/jkl/999
takes less than 1 second, and the line
Removing def/
takes the rest of the time.


-- 
Takashi Yano <takashi.yano@nifty.ne.jp>

Re: Console output hangs in the middle of a line

[Takashi Yano]

On Sun, 20 Feb 2022 22:37:51 +0900
Takashi Yano wrote:
> $ uname -a
> CYGWIN_NT-10.0 Express5800-S70 3.3.4(0.341/5/3) 2022-01-31 19:35 x86_64 Cygwin
> $ git --version
> git version 2.35.1

Windoes Terminal version is: 1.11.3471.0

-- 
Takashi Yano <takashi.yano@nifty.ne.jp>

Re: Console output hangs in the middle of a line

[Takashi Yano]

On Sun, 20 Feb 2022 22:37:51 +0900
Takashi Yano wrote:
> real    0m3.307s
> user    0m0.296s
> sys     0m2.983s

Changing the line:
seq 1 1000 | xargs touch
to
seq 1 10000 | xargs touch
results in:

real    0m7.844s
user    0m0.921s
sys     0m6.500s

No hang occurs as well.

-- 
Takashi Yano <takashi.yano@nifty.ne.jp>

Re: Console output hangs in the middle of a line

[Orgad Shaneh]

Hi,

On Sun, Feb 20, 2022 at 3:38 PM Takashi Yano <takashi.yano@nifty.ne.jp> wrote:
>
> On Sun, 20 Feb 2022 13:43:33 +0200
> Orgad Shaneh <orgads@gmail.com> wrote:
> > mkdir -p chunked
> > cd chunked
> > git init
> > mkdir -p abc/def/ghi/jkl def
> > cd abc/def/ghi/jkl
> > touch foo
> > git add foo
> > seq 1 1000 | xargs touch
> > cd ../../../../def
> > seq 1 10000 | xargs touch
> > cd ..
> > git clean -dfx
>
> Thanks for the test case. However, I cannot reproduce
> your problem.
>
> $ uname -a
> CYGWIN_NT-10.0 Express5800-S70 3.3.4(0.341/5/3) 2022-01-31 19:35 x86_64 Cygwin
> $ git --version
> git version 2.35.1
>
> I have made a shell script such as:
>
> #!/bin/sh
> mkdir -p chunked
> cd chunked
> git init
> mkdir -p abc/def/ghi/jkl def
> cd abc/def/ghi/jkl
> touch foo
> git add foo
> seq 1 1000 | xargs touch
> cd ../../../../def
> seq 1 10000 | xargs touch
> cd ..
> time git clean -dfx
>
> and resut is as follows.
>
> Removing abc/def/ghi/jkl/1
> Removing abc/def/ghi/jkl/10
> Removing abc/def/ghi/jkl/100
> Removing abc/def/ghi/jkl/1000
> Removing abc/def/ghi/jkl/101
> Removing abc/def/ghi/jkl/102
> Removing abc/def/ghi/jkl/103
> Removing abc/def/ghi/jkl/104
> Removing abc/def/ghi/jkl/105
> Removing abc/def/ghi/jkl/106
> Removing abc/def/ghi/jkl/107
> Removing abc/def/ghi/jkl/108
> [...]
> Removing abc/def/ghi/jkl/995
> Removing abc/def/ghi/jkl/996
> Removing abc/def/ghi/jkl/997
> Removing abc/def/ghi/jkl/998
> Removing abc/def/ghi/jkl/999
> Removing def/
>
> real    0m3.307s
> user    0m0.296s
> sys     0m2.983s
>
> How long does the test case hang?
>
> In my environment, from the line
> Removing abc/def/ghi/jkl/1
> to the line
> Removing abc/def/ghi/jkl/999
> takes less than 1 second, and the line
> Removing def/
> takes the rest of the time.
>
>
> --
> Takashi Yano <takashi.yano@nifty.ne.jp>

Turns out I was using Git for Windows and not the cygwin release.

With cygwin git it works as expected.

I'll report to git-for-windows then.

Thanks,
- Orgad

Re: Console output hangs in the middle of a line

[Takashi Yano]

On Sun, 20 Feb 2022 15:51:11 +0200
Orgad Shaneh wrote:
> Turns out I was using Git for Windows and not the cygwin release.

I have tested using Git-for-windows (git version 2.34.1.vfs.0.0).

With
seq 1 1000 | xargs touch

real    0m1.812s
user    0m0.015s
sys     0m0.031s

With
seq 1 10000 | xargs touch

real    0m3.666s
user    0m0.000s
sys     0m0.000s

It seems that it may hang for about 1 sec at the line
Removing abc/def/ghi/jkl/991
Removing ab<hangs 1sec>
before
Removing def/
line.

With
seq 1 10000 | xargs touch
the short hang is here:
Removing abc/def/ghi/jkl/9878
Removing abc/def/ghi/jkl<hangs 1sec>

If the script

#!/bin/sh
mkdir -p chunked
cd chunked
/cygdrive/c/Program\ Files/Git/mingw64/bin/git init
mkdir -p abc/def/ghi/jkl def
cd abc/def/ghi/jkl
touch foo
/cygdrive/c/Program\ Files/Git/mingw64/bin/git add foo
seq 1 10000 | xargs touch
cd ../../../../def
seq 1 100000 | xargs touch
cd ..
time /cygdrive/c/Program\ Files/Git/mingw64/bin/git clean -dfx

is used, the hang here:
Removing abc/def/ghi/jkl/9878
Removing abc/def/ghi/jkl<hangs 15sec>
is more than 10sec.

It seems that removing def/ takes the time,
and the output stops before displying 
Removing abc/def/ghi/jkl/9879
[...]
Removing abc/def/ghi/jkl/9999

I guess the output is just buffered in somewhere
which is not flushed.

-- 
Takashi Yano <takashi.yano@nifty.ne.jp>

Re: Console output hangs in the middle of a line

[Takashi Yano]

On Sun, 20 Feb 2022 23:22:07 +0900
Takashi Yano wrote:
> If the script
> 
> #!/bin/sh
> mkdir -p chunked
> cd chunked
> /cygdrive/c/Program\ Files/Git/mingw64/bin/git init
> mkdir -p abc/def/ghi/jkl def
> cd abc/def/ghi/jkl
> touch foo
> /cygdrive/c/Program\ Files/Git/mingw64/bin/git add foo
> seq 1 10000 | xargs touch
> cd ../../../../def
> seq 1 100000 | xargs touch
> cd ..
> time /cygdrive/c/Program\ Files/Git/mingw64/bin/git clean -dfx
> 
> is used, the hang here:
> Removing abc/def/ghi/jkl/9878
> Removing abc/def/ghi/jkl<hangs 15sec>
> is more than 10sec.
> 
> It seems that removing def/ takes the time,
> and the output stops before displying 
> Removing abc/def/ghi/jkl/9879
> [...]
> Removing abc/def/ghi/jkl/9999
> 
> I guess the output is just buffered in somewhere
> which is not flushed.

The same happens in command prompt which started from 'Git CMD'
icon. In this case, not only cygwin but also msys2 is not used.
Therefore, this is the problem of git executable itself in
Git-for-windows.

-- 
Takashi Yano <takashi.yano@nifty.ne.jp>

Re: Console output hangs in the middle of a line

[Orgad Shaneh]

On Sun, Feb 20, 2022 at 4:34 PM Takashi Yano <takashi.yano@nifty.ne.jp> wrote:
>
> On Sun, 20 Feb 2022 23:22:07 +0900
> Takashi Yano wrote:
> > If the script
> >
> > #!/bin/sh
> > mkdir -p chunked
> > cd chunked
> > /cygdrive/c/Program\ Files/Git/mingw64/bin/git init
> > mkdir -p abc/def/ghi/jkl def
> > cd abc/def/ghi/jkl
> > touch foo
> > /cygdrive/c/Program\ Files/Git/mingw64/bin/git add foo
> > seq 1 10000 | xargs touch
> > cd ../../../../def
> > seq 1 100000 | xargs touch
> > cd ..
> > time /cygdrive/c/Program\ Files/Git/mingw64/bin/git clean -dfx
> >
> > is used, the hang here:
> > Removing abc/def/ghi/jkl/9878
> > Removing abc/def/ghi/jkl<hangs 15sec>
> > is more than 10sec.
> >
> > It seems that removing def/ takes the time,
> > and the output stops before displying
> > Removing abc/def/ghi/jkl/9879
> > [...]
> > Removing abc/def/ghi/jkl/9999
> >
> > I guess the output is just buffered in somewhere
> > which is not flushed.
>
> The same happens in command prompt which started from 'Git CMD'
> icon. In this case, not only cygwin but also msys2 is not used.
> Therefore, this is the problem of git executable itself in
> Git-for-windows.
>
> --
> Takashi Yano <takashi.yano@nifty.ne.jp>

Thank you very much.

- Orgad