From: Kacper Michajlow <kasper93@gmail.com>
To: cygwin@cygwin.com
Subject: Re: Cygwin multithreading performance
Date: Sat, 05 Dec 2015 13:07:00 -0000 [thread overview]
Message-ID: <CABPLAST5EnifrAQ2xKZmohKhyxQHh=K3x3DeCL+BTdHN8nN98w@mail.gmail.com> (raw)
In-Reply-To: <5662C199.7040906@maxrnd.com>
2015-12-05 11:51 GMT+01:00 Mark Geisert <mark@maxrnd.com>:
> Mark Geisert wrote:
>>
>> Corinna Vinschen wrote:
>>>
>>> On Nov 23 16:54, Mark Geisert wrote:
>>>>
>>>> John Hein wrote:
>>>>>
>>>>> Mark Geisert wrote at 23:45 -0800 on Nov 22, 2015:
>>>>> > Corinna Vinschen wrote:
>>>>> > > On Nov 21 01:21, Mark Geisert wrote:
>>>>> > [...] so I wonder if there's
>>>>> > >> some unintentional serialization going on somewhere, but I
>>>>> don't know yet
>>>>> > >> how I could verify that theory.
>>>>> > >
>>>>> > > If I'm allowed to make an educated guess, the big serializer
>>>>> in Cygwin
>>>>> > > are probably the calls to malloc, calloc, realloc, free. We
>>>>> desperately
>>>>> > > need a new malloc implementation better suited to
>>>>> multi-threading.
>>
>> [...]
>>>>>
>>>>>
>>>>> Someone recently mentioned on this list they were working on porting
>>>>> jemalloc. That would be a good choice.
>>>>
>>>>
>>>> Indeed; thanks for the reminder. Somehow I hadn't followed that thread.
>>>
>>>
>>> Indeed^2. Did you look into the locking any further to see if there's
>>> more than one culprit? I guess we've a rather long way to a "lock-less
>>> kernel"...
>
> [...]
>>
>> But that is just groundwork to identifying which locks are suffering the
>> most contention. To identify them at source level I think I'll also
>> need to record the caller's RIP when they are being locked.
>
>
> In the OP's very good testcase the most heavily contended locks, by far, are
> those internal to git's builtin/pack-objects.c. I plan to show actual stats
> after some more cleanup, but I did notice something in that git source file
> that might explain the difference between Cygwin and MinGW when running this
> testcase...
>
> #ifndef NO_PTHREADS
>
> static pthread_mutex_t read_mutex;
> #define read_lock() pthread_mutex_lock(&read_mutex)
> #define read_unlock() pthread_mutex_unlock(&read_mutex)
>
> static pthread_mutex_t cache_mutex;
> #define cache_lock() pthread_mutex_lock(&cache_mutex)
> #define cache_unlock() pthread_mutex_unlock(&cache_mutex)
>
> static pthread_mutex_t progress_mutex;
> #define progress_lock() pthread_mutex_lock(&progress_mutex)
> #define progress_unlock() pthread_mutex_unlock(&progress_mutex)
>
> #else
>
> #define read_lock() (void)0
> #define read_unlock() (void)0
> #define cache_lock() (void)0
> #define cache_unlock() (void)0
> #define progress_lock() (void)0
> #define progress_unlock() (void)0
>
> #endif
>
> Is it possible the MinGW version of git is compiled with NO_PTHREADS
> #defined? If so, it would mean there's no locking being done at all and
> would explain the faster execution and near 100% CPU utilization when
> running under MinGW.
Nah, there is no threading enabled when there is no pthreads. How
would that work? :D See thread-utils.h
#ifndef NO_PTHREADS
#include <pthread.h>
extern int online_cpus(void);
extern int init_recursive_mutex(pthread_mutex_t*);
#else
#define online_cpus() 1
#endif
Looks like there is indeed a bug in git code when passing "--threads"
explicitly to "git pack-objects", because they show warning about
threads being unsupported, but doesn't overwrite delta_search_threads
value. I will go to git's ML about it. This is completely not related
to our issue.
--
Problem reports: http://cygwin.com/problems.html
FAQ: http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
next prev parent reply other threads:[~2015-12-05 13:07 UTC|newest]
Thread overview: 21+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-11-14 0:24 Kacper Michajlow
2015-11-19 20:24 ` Mark Geisert
2015-11-20 14:25 ` Kacper Michajlow
2015-11-21 9:21 ` Mark Geisert
2015-11-21 10:53 ` Corinna Vinschen
2015-11-23 7:45 ` Mark Geisert
2015-11-23 10:27 ` John Hein
2015-11-24 1:05 ` Mark Geisert
2015-11-26 9:49 ` Corinna Vinschen
2015-11-26 10:49 ` Mark Geisert
2015-12-05 10:51 ` Mark Geisert
2015-12-05 13:07 ` Kacper Michajlow [this message]
2015-12-05 13:59 ` Kacper Michajlow
2015-12-05 22:40 ` Mark Geisert
2015-12-06 2:35 ` Kacper Michajlow
2015-12-06 8:02 ` Mark Geisert
2015-12-06 20:56 ` Kacper Michajlow
2015-12-08 10:51 ` Mark Geisert
2015-12-08 15:34 ` Corinna Vinschen
2015-12-08 17:02 ` Corinna Vinschen
2015-12-18 15:06 ` Achim Gratz
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CABPLAST5EnifrAQ2xKZmohKhyxQHh=K3x3DeCL+BTdHN8nN98w@mail.gmail.com' \
--to=kasper93@gmail.com \
--cc=cygwin@cygwin.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).