possible grep V3.7 issue

possible grep V3.7 issue

[Gordon Grimes]

Hi,

I wanted to let you know about a possible issue I found in grep v3.7.  I run Cygwin and recently downgraded ‘grep’ to v3.6 from v3.7 to address a performance issue (shown below).
I just thought you’d want to know  (V3.6 is working well, so I’m in a good place 😊 )
Enjoy,
-g-


% time echo hi | grep h
hi

real        0m7.442s
user       0m0.030s
sys          0m6.576s

% /usr/bin/grep --version
grep (GNU grep) 3.7
Packaged by Cygwin (3.7-2)
Copyright (C) 2021 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later https://gnu.org/licenses/gpl.html.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by Mike Haertel and others; see
https://git.sv.gnu.org/cgit/grep.git/tree/AUTHORS.

After the downgrade…

% time echo hi | grep h
hi

real        0m0.292s
user       0m0.015s
sys          0m0.124s


% grep --version
grep (GNU grep) 3.6
Packaged by Cygwin (3.6-1)
Copyright (C) 2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later https://gnu.org/licenses/gpl.html.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by Mike Haertel and others; see
https://git.sv.gnu.org/cgit/grep.git/tree/AUTHORS.


System Details…

OS Name:                   Microsoft Windows 10 Enterprise
OS Version:                10.0.19042 N/A Build 19042
OS Manufacturer:           Microsoft Corporation
OS Configuration:          Standalone Workstation
OS Build Type:             Multiprocessor Free
Registered Owner:          Windows User
Registered Organization:   TestCo
Product ID:                xxxxxx
Original Install Date:     1/11/2022, 7:40:45 AM
System Boot Time:          4/29/2022, 8:40:28 AM
System Manufacturer:       LENOVO
System Model:              20ST004EUS
System Type:               x64-based PC
Processor(s):              1 Processor(s) Installed.
                           [01]: Intel64 Family 6 Model 165 Stepping 2 GenuineIntel ~2592 Mhz
BIOS Version:              LENOVO N30ET45W (1.28 ), 2/24/2022
Windows Directory:         C:\Windows
System Directory:          C:\Windows\system32
Boot Device:               \Device\HarddiskVolume2
System Locale:             en-us;English (United States)
Input Locale:              en-us;English (United States)
Time Zone:                 (UTC-05:00) Eastern Time (US & Canada)
Total Physical Memory:     32,519 MB
Available Physical Memory: 6,687 MB
Virtual Memory: Max Size:  57,190 MB
Virtual Memory: Available: 11,275 MB
Virtual Memory: In Use:    45,915 MB
Page File Location(s):     C:\pagefile.sys
Domain:                    WORKGROUP
Logon Server:              xxxxx
Hotfix(s):                 9 Hotfix(s) Installed.
                           [01]: KB5012117
                           [02]: KB4562830
                           [03]: KB4570334
                           [04]: KB5005716
                           [05]: KB5012599
                           [06]: KB5007273
                           [07]: KB5011352
                           [08]: KB5011651
                           [09]: KB5005260
Network Card(s):           5 NIC(s) Installed.
                           [01]: Intel(R) Wi-Fi 6 AX201 160MHz
                                 Connection Name: Wi-Fi
                                 Status:          Media disconnected
                           [02]: Intel(R) Ethernet Connection (11) I219-V
                                 Connection Name: Ethernet
                                 Status:          Media disconnected
                           [03]: Bluetooth Device (Personal Area Network)
                                 Connection Name: Bluetooth Network Connection
                                 Status:          Media disconnected
                           [04]: Juniper Networks Virtual Adapter
                                 Connection Name: Local Area Connection* 12
                                 DHCP Enabled:    No
                                 IP address(es)
                                 [01]: xxxxx
                                 [02]: xxxxx
                           [05]: DisplayLink Network Adapter NCM
                                 Connection Name: Ethernet 2
                                 DHCP Enabled:    Yes
                                 DHCP Server:     192.168.1.1
                                 IP address(es)
                                 [01]: xxxxx
                                 [02]: xxxxx
Hyper-V Requirements:      VM Monitor Mode Extensions: Yes
                           Virtualization Enabled In Firmware: Yes
                           Second Level Address Translation: Yes
                           Data Execution Prevention Available: Yes

Re: possible grep V3.7 issue

[Eliot Moss]

On 6/6/2022 8:26 AM, Gordon Grimes via Cygwin wrote:
> Hi,
> 
> I wanted to let you know about a possible issue I found in grep v3.7.  I run Cygwin and recently downgraded ‘grep’ to v3.6 from v3.7 to address a performance issue (shown below).
> I just thought you’d want to know  (V3.6 is working well, so I’m in a good place 😊 )
> Enjoy,
> -g-
> 
> 
> % time echo hi | grep h
> hi
> 
> real        0m7.442s
> user       0m0.030s
> sys          0m6.576s
> 
> % /usr/bin/grep --version
> grep (GNU grep) 3.7
> Packaged by Cygwin (3.7-2)
> Copyright (C) 2021 Free Software Foundation, Inc.
> License GPLv3+: GNU GPL version 3 or later https://gnu.org/licenses/gpl.html.
> This is free software: you are free to change and redistribute it.
> There is NO WARRANTY, to the extent permitted by law.
> 
> Written by Mike Haertel and others; see
> https://git.sv.gnu.org/cgit/grep.git/tree/AUTHORS.
> 
> After the downgrade…
> 
> % time echo hi | grep h
> hi
> 
> real        0m0.292s
> user       0m0.015s
> sys          0m0.124s

I'm running 3.7-2 and get timing similar to what you give
for 3.6:

real    0m0.167s
user    0m0.015s
sys     0m0.061s

Not sure what to say unless, if I downgraded, it would be
an order of magnitude faster ...

I wonder if maybe the new version triggers some anti-virus
activity or something like that.

Eliot Moss

RE: [EXT] Re: possible grep V3.7 issue

[Gordon Grimes]

Thanks Elliot.

Yeah, I suspected this wasn't 100% reproducible or else it'd have been reported/addressed by now.  As you say, perhaps there's some idiosyncrasy with my system/setup.  If there's a wave of reporting a similar issue then perhaps it's worth a look (e.g. perhaps a recent MS update triggered this).  Anyway, unless/until that happens, I'll simply stick with V3.6. 

Thanks for taking a look.
-g-

-----Original Message-----
From: Eliot Moss <moss@cs.umass.edu> 
Sent: Monday, June 6, 2022 8:34 AM
To: Gordon Grimes <Gordon.Grimes@nortonlifelock.com>; cygwin@cygwin.com
Subject: [EXT] Re: possible grep V3.7 issue

On 6/6/2022 8:26 AM, Gordon Grimes via Cygwin wrote:
> Hi,
> 
> I wanted to let you know about a possible issue I found in grep v3.7.  I run Cygwin and recently downgraded ‘grep’ to v3.6 from v3.7 to address a performance issue (shown below).
> I just thought you’d want to know  (V3.6 is working well, so I’m in a 
> good place 😊 ) Enjoy,
> -g-
> 
> 
> % time echo hi | grep h
> hi
> 
> real        0m7.442s
> user       0m0.030s
> sys          0m6.576s
> 
> % /usr/bin/grep --version
> grep (GNU grep) 3.7
> Packaged by Cygwin (3.7-2)
> Copyright (C) 2021 Free Software Foundation, Inc.
> License GPLv3+: GNU GPL version 3 or later https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgnu.org%2Flicenses%2Fgpl.html&amp;data=05%7C01%7CGordon.Grimes%40nortonlifelock.com%7C640f66e158104f7486ea08da47b8e328%7C94986b1d466f4fc0ab4b5c725603deab%7C0%7C0%7C637901156666044572%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&amp;sdata=FujGNPbnS6TMO8I1RT%2FuczWHhImX6KnEUnLWC5EYYTE%3D&amp;reserved=0.
> This is free software: you are free to change and redistribute it.
> There is NO WARRANTY, to the extent permitted by law.
> 
> Written by Mike Haertel and others; see 
> https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit.sv.gnu.org%2Fcgit%2Fgrep.git%2Ftree%2FAUTHORS&amp;data=05%7C01%7CGordon.Grimes%40nortonlifelock.com%7C640f66e158104f7486ea08da47b8e328%7C94986b1d466f4fc0ab4b5c725603deab%7C0%7C0%7C637901156666044572%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&amp;sdata=yXCYAppkPpbDwkD1h2BPA%2B5PPE6QYLRvmVHxvUwxP34%3D&amp;reserved=0.
> 
> After the downgrade…
> 
> % time echo hi | grep h
> hi
> 
> real        0m0.292s
> user       0m0.015s
> sys          0m0.124s

I'm running 3.7-2 and get timing similar to what you give for 3.6:

real    0m0.167s
user    0m0.015s
sys     0m0.061s

Not sure what to say unless, if I downgraded, it would be an order of magnitude faster ...

I wonder if maybe the new version triggers some anti-virus activity or something like that.

Eliot Moss

Re: [EXT] Re: possible grep V3.7 issue

[Brian Inglis]

On 2022-06-06 06:41, Gordon Grimes via Cygwin wrote:
> On Monday, June 6, 2022 8:34 AM, Eliot Moss wrote:
>> On 6/6/2022 8:26 AM, Gordon Grimes via Cygwin wrote:
>>> I wanted to let you know about a possible issue I found in grep 
>>> v3.7. I run Cygwin and recently downgraded ‘grep’ to v3.6 from 
>>> v3.7 to address a performance issue (shown below). I just thought
>>> you’d want to know  (V3.6 is working well, so I’m in a good place 😊)
>>> % time echo hi | grep h
>>> hi
>>> real    0m7.442s
>>> user    0m0.030s
>>> sys     0m6.576s
>>> % /usr/bin/grep --version
>>> grep (GNU grep) 3.7
>>> Packaged by Cygwin (3.7-2)
>>> Copyright (C) 2021 Free Software Foundation, Inc.
>>> License GPLv3+: GNU GPL version 3 or later https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgnu.org%2Flicenses%2Fgpl.html&data=05%7C01%7CGordon.Grimes%40nortonlifelock.com%7C640f66e158104f7486ea08da47b8e328%7C94986b1d466f4fc0ab4b5c725603deab%7C0%7C0%7C637901156666044572%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=FujGNPbnS6TMO8I1RT%2FuczWHhImX6KnEUnLWC5EYYTE%3D&reserved=0.
>>> This is free software: you are free to change and redistribute it.
>>> There is NO WARRANTY, to the extent permitted by law.
>>> Written by Mike Haertel and others; see
>>> https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit.sv.gnu.org%2Fcgit%2Fgrep.git%2Ftree%2FAUTHORS&data=05%7C01%7CGordon.Grimes%40nortonlifelock.com%7C640f66e158104f7486ea08da47b8e328%7C94986b1d466f4fc0ab4b5c725603deab%7C0%7C0%7C637901156666044572%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=yXCYAppkPpbDwkD1h2BPA%2B5PPE6QYLRvmVHxvUwxP34%3D&reserved=0.
>>> After the downgrade…
>>> % time echo hi | grep h
>>> hi
>>> real    0m0.292s
>>> user    0m0.015s
>>> sys     0m0.124s

>> I'm running 3.7-2 and get timing similar to what you give for 3.6:
>> real    0m0.167s
>> user    0m0.015s
>> sys     0m0.061s
>> Not sure what to say unless, if I downgraded, it would be an order
>> of magnitude faster ... I wonder if maybe the new version triggers
>> some anti-virus activity or something like that.
> I suspected this wasn't 100% reproducible or else it'd have been > 
> reported/addressed by now. As you say, perhaps there's some 
> idiosyncrasy with my system/setup. If there's a wave of reporting a 
> similar issue then perhaps it's worth a look (e.g. perhaps a recent
> MS update triggered this). Anyway, unless/until that happens, I'll
> simply stick with V3.6.
Not seeing any significant differences, over the timing granularity on 
Windows, between the installed or either of the built versions:

$ grep --version
grep (GNU grep) 3.7
Packaged by Cygwin (3.7-2)
Copyright (C) 2021 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later 
<https://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Written by Mike Haertel and others; see
<https://git.sv.gnu.org/cgit/grep.git/tree/AUTHORS>.
$ time echo hi
hi
real    0m0.000s
user    0m0.000s
sys     0m0.000s
$ time echo hi | grep hi
hi
real    0m0.226s
user    0m0.000s
sys     0m0.139s
$ time grep hi <<< hi
hi
real    0m0.214s
user    0m0.046s
sys     0m0.061s
$ time grep-3.6-1.x86_64/inst/usr/bin/grep hi <<< hi
hi
real    0m0.203s
user    0m0.015s
sys     0m0.094s
$ time echo hi | grep-3.6-1.x86_64/inst/usr/bin/grep hi
hi
real    0m0.198s
user    0m0.030s
sys     0m0.046s
$ time grep-3.7-2.x86_64/inst/usr/bin/grep hi <<< hi
hi
real    0m0.214s
user    0m0.015s
sys     0m0.124s
$ time echo hi | grep-3.7-2.x86_64/inst/usr/bin/grep hi
hi
real    0m0.287s
user    0m0.030s
sys     0m0.123s

Please try running commands like the above to see if any other obvious 
anomalies appear in timing, and also:

$ cygcheck /bin/bash /bin/grep
C:/.../bin/bash.exe
   C:/.../bin/cygwin1.dll
     C:/WINDOWS/system32/KERNEL32.dll
       C:/WINDOWS/system32/ntdll.dll
       C:/WINDOWS/system32/KERNELBASE.dll
   C:/.../bin/cygiconv-2.dll
   C:/.../bin/cygintl-8.dll
   C:/.../bin/cygreadline7.dll
     C:/.../bin/cygncursesw-10.dll

C:/.../bin/grep.exe
   C:/.../bin/cygwin1.dll
     C:/WINDOWS/system32/KERNEL32.dll
       C:/WINDOWS/system32/ntdll.dll
       C:/WINDOWS/system32/KERNELBASE.dll
   C:/.../bin/cygintl-8.dll
     C:/.../bin/cygiconv-2.dll
   C:/.../bin/cygpcre-1.dll

to see if any other obvious anomalies appear such as paths other than 
Windows/System32 and your Cygwin /bin/, or DLLs other than base Windows 
or Cygwin locale, I/O, or search.

Try running grep under strace, and look in the first column of the 
output for large time deltas (the second column is cumulative thread or 
process time) e.g.:

$ strace -o grep.strace grep hi <<< hi
hi
$ awk '$1 > 1000' grep.strace
  5864    6350 [main] grep (4000) sigprocmask: 0 = sigprocmask (0, 0x0, 
0x180323570)
  2157   30776 [main] grep 520 handle_to_fn: current match 'C:' = 
'\Device\HarddiskVolume2'
11188   44322 [main] grep 520 build_fh_pc: created an archetype 
(0x180350A90) for /dev/cons0(3/0)
  1909   51867 [main] grep 520 open: 
open(/usr/share/locale/locale.alias, 0x0)
  2933   65187 [main] grep 520 read: read(3, 0x800085510, 65536) blocking
  3581   69642 [main] grep 520 open: 
open(/usr/share/locale/en_CA.ISO8859-1/LC_MESSAGES/grep.mo, 0x10000)
  1713   87197 [main] grep 520 open: 
open(/usr/share/locale/en_GB.ISO8859-1/LC_MESSAGES/grep.mo, 0x10000)
  1808   97122 [main] grep 520 open: open(/proc/self/maps, 0x0)
  7609  105898 [main] grep 520 mmap64: addr 0x0, len 327680, prot 0x3, 
flags 0x22, fd -1, off 0x0
  4845  110951 [main] grep 520 munmap: munmap (addr 0x6FFFFFFB0000, len 
327680)
  1103  152703 [main] grep 520 fhandler_base::set_flags: flags 0x18000, 
supplied_bin 0x0
  6919  160062 [main] grep 520 mmap64: addr 0x0, len 327680, prot 0x3, 
flags 0x22, fd -1, off 0x0
  4692  165018 [main] grep 520 munmap: munmap (addr 0x6FFFFFFA0000, len 
327680)
  1103  204304 [main] grep 520 lseek64: 0 = lseek(3, 0, 0)
  4862  212220 [main] grep 520 fhandler_base::fstat_helper: 0 = fstat 
(\??\C:\$Extend\$Deleted\005C00000051F76035B2E0D4, 0xFFFFC870) 
st_size=0, st_mode=040600, 
st_ino=25895697862752096st_atim=629E2C2B.913C4C4 
st_ctim=629E2C2B.913C4C4 st_mtim=629E2C2B.913C4C4 
st_birthtim=629E2C2B.913C4C4
28876  244184 [main] grep 520 fhandler_console::free_console: freed 
console, res 1

and if you see any, look around those lines for traces of alien code 
injection.

Some alien Norton product seems to have been injected above, so that is 
a likely culprit, until someone complains about its effect on 
performance, and gets Cygwin bash, grep 3.7, etc. added to its bypass 
list; see the FAQ:

	https://cygwin.com/faq/faq.html#faq.using.bloda.

-- 
Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada

This email may be disturbing to some readers as it contains
too much technical detail. Reader discretion is advised.
[Data in binary units and prefixes, physical quantities in SI.]