From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 113661 invoked by alias); 5 Dec 2015 10:51:00 -0000 Mailing-List: contact cygwin-help@cygwin.com; run by ezmlm Precedence: bulk List-Id: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: cygwin-owner@cygwin.com Mail-Followup-To: cygwin@cygwin.com Received: (qmail 113649 invoked by uid 89); 5 Dec 2015 10:50:59 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-0.0 required=5.0 tests=AWL,BAYES_50,KAM_LAZY_DOMAIN_SECURITY autolearn=no version=3.3.2 X-HELO: m0.truegem.net Received: from m0.truegem.net (HELO m0.truegem.net) (69.55.228.47) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with (AES256-SHA encrypted) ESMTPS; Sat, 05 Dec 2015 10:50:57 +0000 Received: (from daemon@localhost) by m0.truegem.net (8.12.11/8.12.11) id tB5AotxI062160 for ; Sat, 5 Dec 2015 02:50:55 -0800 (PST) (envelope-from mark@maxrnd.com) Received: from 76-217-5-154.lightspeed.irvnca.sbcglobal.net(76.217.5.154), claiming to be "[192.168.1.100]" via SMTP by m0.truegem.net, id smtpdHnztw0; Sat Dec 5 02:50:51 2015 Subject: Re: Cygwin multithreading performance To: cygwin@cygwin.com References: <564E3017.90205@maxrnd.com> <5650379B.4030405@maxrnd.com> <20151121105301.GE2755@calimero.vinschen.de> <5652C402.7040006@maxrnd.com> <24780-1448274431-7444@sneakemail.com> <5653B52B.5000804@maxrnd.com> <20151126093427.GJ2755@calimero.vinschen.de> <5656DDEF.9070603@maxrnd.com> From: Mark Geisert Message-ID: <5662C199.7040906@maxrnd.com> Date: Sat, 05 Dec 2015 10:51:00 -0000 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:42.0) Gecko/20100101 Firefox/42.0 SeaMonkey/2.39 MIME-Version: 1.0 In-Reply-To: <5656DDEF.9070603@maxrnd.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-SW-Source: 2015-12/txt/msg00052.txt.bz2 Mark Geisert wrote: > Corinna Vinschen wrote: >> On Nov 23 16:54, Mark Geisert wrote: >>> John Hein wrote: >>>> Mark Geisert wrote at 23:45 -0800 on Nov 22, 2015: >>>> > Corinna Vinschen wrote: >>>> > > On Nov 21 01:21, Mark Geisert wrote: >>>> > [...] so I wonder if there's >>>> > >> some unintentional serialization going on somewhere, but I >>>> don't know yet >>>> > >> how I could verify that theory. >>>> > > >>>> > > If I'm allowed to make an educated guess, the big serializer >>>> in Cygwin >>>> > > are probably the calls to malloc, calloc, realloc, free. We >>>> desperately >>>> > > need a new malloc implementation better suited to >>>> multi-threading. > [...] >>>> >>>> Someone recently mentioned on this list they were working on porting >>>> jemalloc. That would be a good choice. >>> >>> Indeed; thanks for the reminder. Somehow I hadn't followed that thread. >> >> Indeed^2. Did you look into the locking any further to see if there's >> more than one culprit? I guess we've a rather long way to a "lock-less >> kernel"... [...] > But that is just groundwork to identifying which locks are suffering the > most contention. To identify them at source level I think I'll also > need to record the caller's RIP when they are being locked. In the OP's very good testcase the most heavily contended locks, by far, are those internal to git's builtin/pack-objects.c. I plan to show actual stats after some more cleanup, but I did notice something in that git source file that might explain the difference between Cygwin and MinGW when running this testcase... #ifndef NO_PTHREADS static pthread_mutex_t read_mutex; #define read_lock() pthread_mutex_lock(&read_mutex) #define read_unlock() pthread_mutex_unlock(&read_mutex) static pthread_mutex_t cache_mutex; #define cache_lock() pthread_mutex_lock(&cache_mutex) #define cache_unlock() pthread_mutex_unlock(&cache_mutex) static pthread_mutex_t progress_mutex; #define progress_lock() pthread_mutex_lock(&progress_mutex) #define progress_unlock() pthread_mutex_unlock(&progress_mutex) #else #define read_lock() (void)0 #define read_unlock() (void)0 #define cache_lock() (void)0 #define cache_unlock() (void)0 #define progress_lock() (void)0 #define progress_unlock() (void)0 #endif Is it possible the MinGW version of git is compiled with NO_PTHREADS #defined? If so, it would mean there's no locking being done at all and would explain the faster execution and near 100% CPU utilization when running under MinGW. ..mark -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple