Cygwin a bit slow

Cygwin a bit slow

[J M]

[-- Attachment #1: Type: text/plain, Size: 442 bytes --]

Hi,

I'm seeing that Cygwin is a bit slow, directly and after comparing to
simple ubuntu virtual machines by example.

Specifically:

- Copy and paste texts in vim, I see clearly the slow in paste.
- Using sed and/or grep that count approx. between 6x and 8x respect to
virtual machine simple ubuntu.
- In general multiple bash commands are slower.

Can you analyze this?

I'm use the last updated Windows 11 and a fresh Cygwin.

Cesar Jorge

Re: Cygwin a bit slow

[J M]

[-- Attachment #1: Type: text/plain, Size: 659 bytes --]

I added, sed and grex x60 to x80, no software running and no antivirus.

Regards

El vie., 5 abr. 2024 17:18, J M <cesarjorgemartinez@gmail.com> escribió:

> Hi,
>
> I'm seeing that Cygwin is a bit slow, directly and after comparing to
> simple ubuntu virtual machines by example.
>
> Specifically:
>
> - Copy and paste texts in vim, I see clearly the slow in paste.
> - Using sed and/or grep that count approx. between 6x and 8x respect to
> virtual machine simple ubuntu.
> - In general multiple bash commands are slower.
>
> Can you analyze this?
>
> I'm use the last updated Windows 11 and a fresh Cygwin.
>
> Cesar Jorge
>
>
>

Re: Cygwin a bit slow

[Brian Inglis]

On 2024-04-05 09:21, J M via Cygwin wrote:
> I added, sed and grex x60 to x80, no software running and no antivirus.
> El vie., 5 abr. 2024 17:18, J M escribió:
>> I'm seeing that Cygwin is a bit slow, directly and after comparing to
>> simple ubuntu virtual machines by example.
>> Specifically:
>> - Copy and paste texts in vim, I see clearly the slow in paste.
>> - Using sed and/or grep that count approx. between 6x and 8x respect to
>> virtual machine simple ubuntu.
>> - In general multiple bash commands are slower.
>> Can you analyze this?
>> I'm use the last updated Windows 11 and a fresh Cygwin.

Any chance Cygwin may be running under Hyper-V due to possibly unrequested 
security changes, such as enabling Device Security/Core Isolation/Memory 
Integrity, because of enabling logging/reporting back to MS, or checking 
signature on each execution of each executable, or other "security" features (by 
default bypassed by buggy/unsafe MS products, which are the ones that need it)?

-- 
Take care. Thanks, Brian Inglis              Calgary, Alberta, Canada

La perfection est atteinte                   Perfection is achieved
non pas lorsqu'il n'y a plus rien à ajouter  not when there is no more to add
mais lorsqu'il n'y a plus rien à retirer     but when there is no more to cut
                                 -- Antoine de Saint-Exupéry

Re: Cygwin a bit slow

[Lee]

On Fri, Apr 5, 2024 at 11:18 AM J M  wrote:
>
> Hi,
>
> I'm seeing that Cygwin is a bit slow, directly and after comparing to
> simple ubuntu virtual machines by example.
>
> Specifically:
>
> - Copy and paste texts in vim, I see clearly the slow in paste.

I don't know about the rest, but paste being slow is an old problem - eg:
Subject: speeding up a paste operation
To:      cygwin@cygwin.com
Date:    Fri, Aug 24, 2018 at 7:30 PM

at least on my machine, there's a clear
winner for pasting in an absurdly large amount of text:

$ time d2u < /dev/clipboard > hosts-3.txt

real    0m11.372s
user    0m3.749s
sys     0m6.984s

$ time cat /dev/clipboard | tr -d '\r' > hosts-2.txt

real    0m4.405s
user    0m0.124s
sys     0m3.577s

$ time getclip -u > hosts.txt

real    0m0.734s
user    0m0.031s
sys     0m0.031s

Regards,
Lee

Re: Cygwin a bit slow

[Adam Dinwoodie]

On Fri, 5 Apr 2024 at 16:19, J M via Cygwin wrote:
>
> Hi,
>
> I'm seeing that Cygwin is a bit slow, directly and after comparing to
> simple ubuntu virtual machines by example.
>
> Specifically:
>
> - Copy and paste texts in vim, I see clearly the slow in paste.
> - Using sed and/or grep that count approx. between 6x and 8x respect to
> virtual machine simple ubuntu.
> - In general multiple bash commands are slower.
>
> Can you analyze this?
>
> I'm use the last updated Windows 11 and a fresh Cygwin.

This is expected. Cygwin runs as a compatibility layer between Windows
and the POSIX applications, and that compatibility layer has
significant performance overheads. Running in a virtual machine –
including WSL – has far fewer of those overheads, at the expense of
requiring a complete separate operating system, all the virtualisation
infrastructure, and poorer access to the Windows OS.

There is clearly a trade-off here, and for a lot of folk who would
have used Cygwin in the past, WSL is going to be a better choice:
those disadvantages are much less relevant than they were five or ten
years ago. Obviously, if you have ideas for how to improve Cygwin
performance, the project is always looking for volunteers; there's
more information at https://cygwin.com/contrib.html.

HTH

Adam

Re: Cygwin a bit slow

[J M]

[-- Attachment #1: Type: text/plain, Size: 2527 bytes --]

Hi Adam,

It would be nice to contribute, although my knowledge is not high, I could
review problems and try things.

There are two things that are difficult for me. One is to search the Cygwin
Archives, you could have a way to search for them directly on the Cygwin
website. And another is to have powerful tools to look for problems and a
FAQ where they can explain their use in detail (windows and linux tools).

I think it's not all problems in Cygwin, many times they are conflicts with
other software (antivirus is one).


Specifically for this problem, I have investigated the problem and can be
related to pipes and antivirus.

Specifically
while true
do
echo ABC | grep AAA
done

It makes the cpu of that antivirus go up.

I have seen that the pipes make use of \??\pipe routes which I have seen at
https://cygwin.com/cygwin-ug-net/using.html

I have it configured to exclude c:\cygwin64 and perhaps those paths of
those pipes are having effects in that other software.

By now this...

Regards


El lun., 8 abr. 2024 21:48, Adam Dinwoodie <adam@dinwoodie.org> escribió:

> On Fri, 5 Apr 2024 at 16:19, J M via Cygwin wrote:
> >
> > Hi,
> >
> > I'm seeing that Cygwin is a bit slow, directly and after comparing to
> > simple ubuntu virtual machines by example.
> >
> > Specifically:
> >
> > - Copy and paste texts in vim, I see clearly the slow in paste.
> > - Using sed and/or grep that count approx. between 6x and 8x respect to
> > virtual machine simple ubuntu.
> > - In general multiple bash commands are slower.
> >
> > Can you analyze this?
> >
> > I'm use the last updated Windows 11 and a fresh Cygwin.
>
> This is expected. Cygwin runs as a compatibility layer between Windows
> and the POSIX applications, and that compatibility layer has
> significant performance overheads. Running in a virtual machine –
> including WSL – has far fewer of those overheads, at the expense of
> requiring a complete separate operating system, all the virtualisation
> infrastructure, and poorer access to the Windows OS.
>
> There is clearly a trade-off here, and for a lot of folk who would
> have used Cygwin in the past, WSL is going to be a better choice:
> those disadvantages are much less relevant than they were five or ten
> years ago. Obviously, if you have ideas for how to improve Cygwin
> performance, the project is always looking for volunteers; there's
> more information at https://cygwin.com/contrib.html.
>
> HTH
>
> Adam
>

Re: Cygwin a bit slow

[Christian Franke]

J M via Cygwin wrote:
> ...
>
> Specifically for this problem, I have investigated the problem and can be
> related to pipes and antivirus.
>
> Specifically
> while true
> do
> echo ABC | grep AAA
> done
>
> It makes the cpu of that antivirus go up.

This is as expected because malware scanners hook into Win32 API's 
CreateProcess*() calls which are also used by the fork()/exec() 
emulation of Cygwin. Each run of 'grep' above uses at least two 
CreateProcess*() calls.

This quick test shows how many 'date' commands could be run per second:

$ while :; do date +%s; done | uniq -c
...
      65 1712742865 <== Windows Defender off
      66 1712742866
      66 1712742867
      64 1712742868
      61 1712742869
      51 1712742870 <== Windows Defender turned on
      51 1712742871
      49 1712742872
      45 1712742873
      53 1712742874
      54 1712742875
...

The above could even slow down to 1-2 per second with certain malware 
scanners if expensive heuristics (which may also generate a lot of false 
positives, BTW) is enabled. So one problem is the lousy performance of 
CreateProcess*() calls. This is not Cygwin-specific but affects typical 
Cygwin use cases like shell scripts.

Using bash builtins in the above example speeds it up to ~21000/second 
on the same very old box:

$ while :; do printf '%(%s)T\n'; done | uniq -c

-- 
Regards,
Christian

Re: Cygwin a bit slow

[Sam Edge]

On 10/04/2024 11:34, Christian Franke via Cygwin wrote:
> J M via Cygwin wrote:
>> ...
>>
>> Specifically for this problem, I have investigated the problem and can be
>> related to pipes and antivirus.
>>
>> Specifically
>> while true
>> do
>> echo ABC | grep AAA
>> done
>>
>> It makes the cpu of that antivirus go up.
>
> This is as expected because malware scanners hook into Win32 API's
> CreateProcess*() calls which are also used by the fork()/exec()
> emulation of Cygwin. Each run of 'grep' above uses at least two
> CreateProcess*() calls.

This is very true and depends greatly on the AV being used. I find Trend
is particularly bad, even if you exclude all the Cygwin directories and
directories of files being accessed. Somehow, the way the hooks are
implemented stalls process creation and file open in ways that Windows
Defender does not. This is particularly noticeable when using
Cygwin-based build tools - build times generally increase at least
10-fold after installing Trend.

On one job, I wasted a lot of time and client's money collecting logs
for Trend to analyse to no avail. I think the product is basically very
badly written. The fact that it creates dozens of processes with
hundreds of threads just to do AV scanning does not fill me with confidence!

Wherever possible, I remove third-party AV from Windows machines and
install group policy to enforce Windows Defender and malware detection
in the browser and/or a proxy & the email server instead. Saves a lot of
CPU cycles. :-)


--
Sam Edge

Re: Cygwin a bit slow

[J M]

[-- Attachment #1: Type: text/plain, Size: 2815 bytes --]

Hi,

For me not use AV or disable parts is not an option...

Then, if AV is inspecting the CreateProcess, these processes can be known
the path of these process?

Ex, I launch grep. One AV process can discern the path of these process, or
it is impossible to find out if the executable is inside of c:\cygwin64
directory and discard and/or not catch the event, and then inform to the AV
enterprises howto to do these tasks?

I did the following tests with Avast AV:

With all shields stopped or all shields up, same result, one more time that
other:

Launch multiple while true with echo and grep by example and sleep to
results.

In all cases, cpu very high and memory progressively up and up until
windows crash memory exhausted.

The AVs not known howto discern this or it is impossible discern this?

Regards

El jue., 11 abr. 2024 1:17, Sam Edge via Cygwin <cygwin@cygwin.com>
escribió:

> On 10/04/2024 11:34, Christian Franke via Cygwin wrote:
> > J M via Cygwin wrote:
> >> ...
> >>
> >> Specifically for this problem, I have investigated the problem and can
> be
> >> related to pipes and antivirus.
> >>
> >> Specifically
> >> while true
> >> do
> >> echo ABC | grep AAA
> >> done
> >>
> >> It makes the cpu of that antivirus go up.
> >
> > This is as expected because malware scanners hook into Win32 API's
> > CreateProcess*() calls which are also used by the fork()/exec()
> > emulation of Cygwin. Each run of 'grep' above uses at least two
> > CreateProcess*() calls.
>
> This is very true and depends greatly on the AV being used. I find Trend
> is particularly bad, even if you exclude all the Cygwin directories and
> directories of files being accessed. Somehow, the way the hooks are
> implemented stalls process creation and file open in ways that Windows
> Defender does not. This is particularly noticeable when using
> Cygwin-based build tools - build times generally increase at least
> 10-fold after installing Trend.
>
> On one job, I wasted a lot of time and client's money collecting logs
> for Trend to analyse to no avail. I think the product is basically very
> badly written. The fact that it creates dozens of processes with
> hundreds of threads just to do AV scanning does not fill me with
> confidence!
>
> Wherever possible, I remove third-party AV from Windows machines and
> install group policy to enforce Windows Defender and malware detection
> in the browser and/or a proxy & the email server instead. Saves a lot of
> CPU cycles. :-)
>
>
> --
> Sam Edge
>
>
> --
> Problem reports:      https://cygwin.com/problems.html
> FAQ:                  https://cygwin.com/faq/
> Documentation:        https://cygwin.com/docs.html
> Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple
>

Re: Cygwin a bit slow

[Sam Edge]

On 12/04/2024 19:01, J M via Cygwin wrote:
> Hi,
>
> For me not use AV or disable parts is not an option...
>
> Then, if AV is inspecting the CreateProcess, these processes can be known
> the path of these process?

[Please bottom post in Cygwin mailing lists. TIA]

I'm not suggesting you don't use a real-time AV scanner. I'm suggesting
that Windows Defender is the best one to avoid slowing your Windows PC
down in general and especially with Cygwin. (FYI In independent tests,
its detection rate is very much as good as Trend, AVM, McAfee etc. as
well, so you save money & maintenance time without losing protection to
boot.)

As I said, with Trend and other badly-written AV services, simply
excluding an executable or directory does not prevent the slow down. In
some cases even 'disabling' the AV using its UI does not either,
probably because this doesn't actually remove the hooks.

Again, I recommend removing all third-party AV and using Defender (and
Smart Screen, Safe Browsing etc. in your browser) if you have control
over your PC. If you're locked down, convince your IT manager about the
cost (purchase & your time waiting for the PC) savings of doing so.
(Good luck with the last - IT managers are often petty empire builders
who don't like taking advice even when it's good & saves money. I know -
I have been one in past times!)


--
Sam Edge