From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 856 invoked by alias); 6 Dec 2015 20:56:31 -0000 Mailing-List: contact cygwin-help@cygwin.com; run by ezmlm Precedence: bulk List-Id: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: cygwin-owner@cygwin.com Mail-Followup-To: cygwin@cygwin.com Received: (qmail 844 invoked by uid 89); 6 Dec 2015 20:56:30 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=0.2 required=5.0 tests=AWL,BAYES_50,FREEMAIL_ENVFROM_END_DIGIT,FREEMAIL_FROM,RCVD_IN_DNSWL_LOW,SPF_PASS autolearn=ham version=3.3.2 X-HELO: mail-wm0-f53.google.com Received: from mail-wm0-f53.google.com (HELO mail-wm0-f53.google.com) (74.125.82.53) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with (AES128-GCM-SHA256 encrypted) ESMTPS; Sun, 06 Dec 2015 20:56:29 +0000 Received: by wmww144 with SMTP id w144so118344785wmw.1 for ; Sun, 06 Dec 2015 12:56:26 -0800 (PST) MIME-Version: 1.0 X-Received: by 10.194.79.72 with SMTP id h8mr32262415wjx.136.1449435386153; Sun, 06 Dec 2015 12:56:26 -0800 (PST) Received: by 10.27.87.195 with HTTP; Sun, 6 Dec 2015 12:56:26 -0800 (PST) In-Reply-To: <5663EB9A.40002@maxrnd.com> References: <564E3017.90205@maxrnd.com> <5650379B.4030405@maxrnd.com> <20151121105301.GE2755@calimero.vinschen.de> <5652C402.7040006@maxrnd.com> <24780-1448274431-7444@sneakemail.com> <5653B52B.5000804@maxrnd.com> <20151126093427.GJ2755@calimero.vinschen.de> <5656DDEF.9070603@maxrnd.com> <5662C199.7040906@maxrnd.com> <566367C8.5020703@maxrnd.com> <5663EB9A.40002@maxrnd.com> Date: Sun, 06 Dec 2015 20:56:00 -0000 Message-ID: Subject: Re: Cygwin multithreading performance From: Kacper Michajlow To: cygwin@cygwin.com Content-Type: text/plain; charset=UTF-8 X-IsSubscribed: yes X-SW-Source: 2015-12/txt/msg00079.txt.bz2 2015-12-06 9:02 GMT+01:00 Mark Geisert : > Kacper Michajlow wrote: >> >> 2015-12-05 23:40 GMT+01:00 Mark Geisert : >>> >>> It looks like we're going to have to compare actual pthread_mutex_lock() >>> implementations. Inspecting source is nice but I don't want to be >>> chasing a >>> mirage so I really hope there's a pthread_mutex_lock() function inside >>> the >>> MinGW git you are running. gdb could easily answer that question. Could >>> you please do an 'info func pthread_mutex_lock' after starting MinGW git >>> under MinGW gdb with a breakpoint at main() (so libraries are loaded). > > [...] >> >> Hmm, thinking about it mingw doesn't have pthread implementation or >> any wrapper for it. If someone needs pthread they would probably go >> for pthreads-w32 implementation. >> >> I started to wonder because I don't recall git would need pthreads to >> compile on Windows. And indeed they have a wrapper for Windows API... >> https://github.com/git/git/blob/master/compat/win32/pthread.h >> https://github.com/git/git/blob/master/compat/win32/pthread.c > > > OK, so git has its own pthread_mutex_lock/unlock ops which map to very > light-weight critical section operations. > >> Though it is not really a matter that "native" git build is fast and >> all, but that Cygwin's one really struggles if it comes to MT workload. > > > In the worst cases I see using your testcase, about half the time the > busiest locks are processed within 1 usec but there's a spectrum of longer > latencies for the other half of the time. I don't know (yet) if that can be > improved in Cygwin's more general implementation but at least the matter has > now been brought to our attention :). , Yes, I can imagine, git's objects are very small so threading overhead is very noticeable. >> And this not only issue with git unfortunately. Download speeds are >> also limited on Cygwin. I know POSIX compatibility layers comes with a >> price but I would love to see improvements in those areas. >> Cygwin: >> Receiving objects: 100% (230458/230458), 78.41 MiB | 1.53 MiB/s, done. >> "native" git: >> Receiving objects: 100% (230458/230458), 78.41 MiB | 18.54 MiB/s, done. > > > You're asserting this additional testcase has the same cause. What is > telling you that? And FTR what is the git command you are issuing? I can > then do the lock latency analysis on this new testcase if warranted. No, sorry, I mixed different things. It is just that I'm ruining both git build lately and I wanted to share another issue before I forget about it. This was git clone command for some random repository from github. There is a lot factors at hand here but the fact is with cygwin speed is capped on 1.5MB/s and this is reproducible. This is probably also related to the fact that git operates on large amount small object. But this time it is single thread workload. I tried strace this, but frankly I am not sure what to look for. All in all I just want to bring those issues to your attention. Whether it is fixable or not is another story. But we will not know unless someone with required knowledge analyze it. -Kacper -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple