From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp-out-no.shaw.ca (smtp-out-no.shaw.ca [64.59.134.12]) by sourceware.org (Postfix) with ESMTPS id 8BE8B3858002 for ; Mon, 22 Feb 2021 20:30:03 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 8BE8B3858002 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=SystematicSw.ab.ca Authentication-Results: sourceware.org; spf=none smtp.mailfrom=brian.inglis@systematicsw.ab.ca Received: from [192.168.1.104] ([68.147.0.90]) by shaw.ca with ESMTP id EHqDlv3bweHr9EHqElzXQ4; Mon, 22 Feb 2021 13:30:02 -0700 X-Authority-Analysis: v=2.4 cv=Yq/K+6UX c=1 sm=1 tr=0 ts=6034144a a=T+ovY1NZ+FAi/xYICV7Bgg==:117 a=T+ovY1NZ+FAi/xYICV7Bgg==:17 a=IkcTkHD0fZMA:10 a=QweF5vw4ChSBs4rJw5EA:9 a=QEXdDO2ut3YA:10 Reply-To: cygwin@cygwin.com Subject: Re: CRITICAL ls MEMORY LEAK To: cygwin@cygwin.com References: <003401d70864$cd3b3400$67b19c00$@gmail.com> From: Brian Inglis Organization: Systematic Software Message-ID: <306dd40d-666d-4a27-0a2c-dc03053d2f8c@SystematicSw.ab.ca> Date: Mon, 22 Feb 2021 13:30:01 -0700 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.7.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-CA Content-Transfer-Encoding: 7bit X-CMAE-Envelope: MS4xfNfNaLWw93QGME0tBFfJgbj/ykOEenXt4cjLIKRkEsTcsSQ7K60C9ujUyFg18wtAyiFcv9vuutsMe+z+/Q7m3rMDr/KDh8tR4OCYlfr8gKYlaKYff97T OzADBfy+Rx9Y23H1s1uKueiyuQtnbUqVJFgenap+ClHiz+7PhLcR3m6rkDDeiC5YetX2ZzM+X0uC7/RyW7fZksC1wId7c6DaMwE= X-Spam-Status: No, score=0.6 required=5.0 tests=BAYES_00, KAM_DMARC_STATUS, KAM_LAZY_DOMAIN_SECURITY, NICE_REPLY_A, RCVD_IN_BARRACUDACENTRAL, RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, SPF_HELO_NONE, SPF_NONE, TXREP autolearn=no autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: cygwin@cygwin.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: General Cygwin discussions and problem reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Feb 2021 20:30:04 -0000 On 2021-02-22 13:12, Doug Henderson wrote: > On Sun, 21 Feb 2021 at 08:21, Satalink wrote: >> I deal with a lot of very large files on a regular basis. I've noticed that >> when I delve into these directories using in mintty and issue the command ls >> -l (or ls -color=auto), a very large junk of memory is consumed. The >> memory leak seems to be proportionate to the number and size of files within >> the containing folder. > This is likely due to your virus scanner. If those files contain > non-executable content, it is probably safe to disable virus scans for > those files. > > Something that ls does is triggering the scan. That scan causes the > virus scanner to read the entire file. You should see extraordinary > GPU and disk activity for some time after the ls completes. There > might be processes or at least threads for each file being scanned. > > Hopefully you will be able to identify a common folder in the path to > those files where it is safe to disable scanning for that folder and > all folders and files within that folder. I've often wondered if the heavy activity is due to Windows' defaults to writing files with F+RX perms which triggers executable virus scans? You can't have separate directory and file default perms and unlike Unix, Windows appears dumb about applying the X bit to files, probably because that would render downloaded executables non-X, and there is no easy way for users to change that, whereas Unix requires tools and users explicitly grant X perms. -- Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada This email may be disturbing to some readers as it contains too much technical detail. Reader discretion is advised. [Data in binary units and prefixes, physical quantities in SI.]