From: Adhemerval Zanella <adhemerval.zanella@linaro.org>
To: libc-alpha@sourceware.org
Subject: [PATCH 2/3] Mutex: Only read while spinning
Date: Thu, 05 Apr 2018 20:55:00 -0000 [thread overview]
Message-ID: <d45dd71a-0ab5-47b1-c1ca-1e4fe39a58d1@linaro.org> (raw)
In-Reply-To: <1522394093-9835-2-git-send-email-kemi.wang@intel.com>
On 30/03/2018 04:14, Kemi Wang wrote:
> The pthread adaptive spin mutex spins on the lock for a while before going
> to a sleep. While the lock is contended and we need to wait, going straight
> back to LLL_MUTEX_TRYLOCK(cmpxchg) is not a good idea on many targets as
> that will force expensive memory synchronization among processors and
> penalize other running threads. For example, it constantly floods the
> system with "read for ownership" requests, which are much more expensive to
> process than a single read. Thus, we only use MO read until we observe the
> lock to not be acquired anymore, as suggusted by Andi Kleen.
>
> Test machine:
> 2-sockets Skylake paltform, 112 cores with 62G RAM
>
> Test case: Contended pthread adaptive spin mutex with global update
> each thread of the workload does:
> a) Lock the mutex (adaptive spin type)
> b) Globle variable increment
> c) Unlock the mutex
> in a loop until timeout, and the main thread reports the total iteration
> number of all the threads in one second.
>
> This test case is as same as Will-it-scale.pthread_mutex3 except mutex type is
> modified to PTHREAD_MUTEX_ADAPTIVE_NP.
> github: https://github.com/antonblanchard/will-it-scale.git
>
> nr_threads base head(SPIN_COUNT=10) head(SPIN_COUNT=1000)
> 1 51644585 51307573(-0.7%) 51323778(-0.6%)
> 2 7914789 10011301(+26.5%) 9867343(+24.7%)
> 7 1687620 4224135(+150.3%) 3430504(+103.3%)
> 14 1026555 3784957(+268.7%) 1843458(+79.6%)
> 28 962001 2886885(+200.1%) 681965(-29.1%)
> 56 883770 2740755(+210.1%) 364879(-58.7%)
> 112 1150589 2707089(+135.3%) 415261(-63.9%)
In pthread_mutex3 it is basically more updates in a global variable synchronized
with a mutex, so if I am reading correct the benchmark, a higher value means
less contention. I also assume you use the 'threads' value in this table.
I checked on a 64 cores aarch64 machine to see what kind of improvement, if
any; one would get with this change:
nr_threads base head(SPIN_COUNT=10) head(SPIN_COUNT=1000)
1 27566206 28778254 (4.211680) 28778467 (4.212389)
2 8498813 7777589 (-9.273105) 7806043 (-8.874791)
7 5019434 2869629 (-74.915782) 3307812 (-51.744839)
14 4379155 2906255 (-50.680343) 2825041 (-55.012087)
28 4397464 3261094 (-34.846282) 3259486 (-34.912805)
56 4020956 3898386 (-3.144122) 4038505 (0.434542)
So I think this change should be platform-specific.
>
> Suggested-by: Andi Kleen <andi.kleen@intel.com>
> Signed-off-by: Kemi Wang <kemi.wang@intel.com>
> ---
> nptl/pthread_mutex_lock.c | 23 +++++++++++++++--------
> 1 file changed, 15 insertions(+), 8 deletions(-)
>
> diff --git a/nptl/pthread_mutex_lock.c b/nptl/pthread_mutex_lock.c
> index 1519c14..c3aca93 100644
> --- a/nptl/pthread_mutex_lock.c
> +++ b/nptl/pthread_mutex_lock.c
> @@ -26,6 +26,7 @@
> #include <atomic.h>
> #include <lowlevellock.h>
> #include <stap-probe.h>
> +#include <mutex-conf.h>
>
> #ifndef lll_lock_elision
> #define lll_lock_elision(lock, try_lock, private) ({ \
> @@ -124,16 +125,22 @@ __pthread_mutex_lock (pthread_mutex_t *mutex)
> if (LLL_MUTEX_TRYLOCK (mutex) != 0)
> {
> int cnt = 0;
> - int max_cnt = MIN (MAX_ADAPTIVE_COUNT,
> - mutex->__data.__spins * 2 + 10);
> + int max_cnt = MIN (__mutex_aconf.spin_count,
> + mutex->__data.__spins * 2 + 100);
> do
> {
> - if (cnt++ >= max_cnt)
> - {
> - LLL_MUTEX_LOCK (mutex);
> - break;
> - }
> - atomic_spin_nop ();
> + if (cnt >= max_cnt)
> + {
> + LLL_MUTEX_LOCK (mutex);
> + break;
> + }
> + /* MO read while spinning */
> + do
> + {
> + atomic_spin_nop ();
> + }
> + while (atomic_load_relaxed (&mutex->__data.__lock) != 0 &&
> + ++cnt < max_cnt);
> }
> while (LLL_MUTEX_TRYLOCK (mutex) != 0);
>
>
next prev parent reply other threads:[~2018-04-05 20:55 UTC|newest]
Thread overview: 14+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-03-30 7:17 [PATCH 1/3] Tunables: Add tunables of spin count for adaptive spin mutex Kemi Wang
2018-03-30 7:17 ` [PATCH 2/3] Mutex: Only read while spinning Kemi Wang
2018-04-05 20:55 ` Adhemerval Zanella [this message]
2018-04-08 8:30 ` kemi
2018-04-09 20:52 ` Adhemerval Zanella
2018-04-10 1:49 ` kemi
2018-04-11 13:28 ` Adhemerval Zanella
2018-03-30 7:17 ` [PATCH 3/3] Mutex: Avoid useless spinning Kemi Wang
2018-04-05 20:59 ` Adhemerval Zanella
2018-04-08 8:33 ` kemi
2018-04-02 15:19 ` [PATCH 1/3] Tunables: Add tunables of spin count for adaptive spin mutex Adhemerval Zanella
2018-04-04 10:27 ` kemi
2018-04-04 17:17 ` Adhemerval Zanella
2018-04-05 1:11 ` Carlos O'Donell
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=d45dd71a-0ab5-47b1-c1ca-1e4fe39a58d1@linaro.org \
--to=adhemerval.zanella@linaro.org \
--cc=libc-alpha@sourceware.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).