From: Adhemerval Zanella <adhemerval.zanella@linaro.org>
To: Florian Weimer <fweimer@redhat.com>, libc-alpha@sourceware.org
Subject: Re: [PATCH 6/6] nptl: Move elision implementations into libc
Date: Mon, 22 Feb 2021 13:53:25 -0300 [thread overview]
Message-ID: <6d274cde-bfed-49fb-5cda-cf438e18ad93@linaro.org> (raw)
In-Reply-To: <2bfbf479cfc97fac04d1e07c6934afefafbf6b56.1613582255.git.fweimer@redhat.com>
On 17/02/2021 14:22, Florian Weimer via Libc-alpha wrote:
> The elision interfaces are closely aligned between the targets that
> implement them, so declare them in the generic <lowlevellock.h>
> file.
>
> Empty .c stubs are provided, so that fewer makefile updates
> under sysdeps are needed. Also simplify initialization via
> __libc_early_init.
>
> The symbols __lll_clocklock_elision, __lll_lock_elision,
> __lll_trylock_elision, __lll_unlock_elision, __pthread_force_elision
> move into libc. For the time being, non-hidden references are used
> from libpthread to access them, but once that part of libpthread
> is moved into libc, hidden symbols will be used again. (Hidden
> references seem desirable to reduce the likelihood of transactions
> aborts.)
LGTM, some comments below.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
> ---
> elf/libc_early_init.c | 6 ++
> nptl/Makefile | 5 +
> nptl/Versions | 5 +
> .../elision-conf.c | 9 +-
> .../elision-lock.c | 9 +-
> .../elision-timed.c | 9 +-
> .../elision-trylock.c | 9 +-
> nptl/elision-unlock.c | 17 +++
> nptl/lowlevellock.c | 5 +
> nptl/pthread_mutex_lock.c | 13 ---
> nptl/pthread_mutex_timedlock.c | 13 ---
> nptl/pthread_mutex_trylock.c | 8 --
> nptl/pthread_mutex_unlock.c | 4 -
> sysdeps/nptl/lowlevellock.h | 102 ++++++++++++++++++
> .../elision-conf.h} | 17 ++-
> sysdeps/unix/sysv/linux/powerpc/Makefile | 2 -
> .../unix/sysv/linux/powerpc/elision-conf.c | 26 +----
> .../unix/sysv/linux/powerpc/elision-conf.h | 8 +-
> .../unix/sysv/linux/powerpc/elision-lock.c | 7 +-
> .../unix/sysv/linux/powerpc/elision-trylock.c | 1 +
> .../unix/sysv/linux/powerpc/elision-unlock.c | 1 +
> .../unix/sysv/linux/powerpc/force-elision.h | 62 -----------
> .../unix/sysv/linux/powerpc/lowlevellock.h | 49 ---------
> .../linux/powerpc/pthread_mutex_cond_lock.c | 22 ----
> sysdeps/unix/sysv/linux/s390/Makefile | 3 -
> sysdeps/unix/sysv/linux/s390/elision-conf.c | 27 +----
> sysdeps/unix/sysv/linux/s390/elision-conf.h | 8 +-
> sysdeps/unix/sysv/linux/s390/elision-lock.c | 7 +-
> .../unix/sysv/linux/s390/elision-trylock.c | 1 +
> sysdeps/unix/sysv/linux/s390/elision-unlock.c | 1 +
> sysdeps/unix/sysv/linux/s390/force-elision.h | 62 -----------
> sysdeps/unix/sysv/linux/s390/lowlevellock.h | 49 ---------
> .../sysv/linux/s390/pthread_mutex_cond_lock.c | 22 ----
> .../sysv/linux/s390/pthread_mutex_timedlock.c | 22 ----
> .../sysv/linux/s390/pthread_mutex_trylock.c | 22 ----
> sysdeps/unix/sysv/linux/x86/Makefile | 2 -
> sysdeps/unix/sysv/linux/x86/elision-conf.c | 25 +----
> sysdeps/unix/sysv/linux/x86/elision-conf.h | 8 +-
> sysdeps/unix/sysv/linux/x86/elision-lock.c | 7 +-
> sysdeps/unix/sysv/linux/x86/elision-trylock.c | 1 +
> sysdeps/unix/sysv/linux/x86/elision-unlock.c | 1 +
> sysdeps/unix/sysv/linux/x86/force-elision.h | 62 -----------
> sysdeps/unix/sysv/linux/x86/lowlevellock.h | 24 -----
> .../sysv/linux/x86/pthread_mutex_cond_lock.c | 22 ----
> .../sysv/linux/x86/pthread_mutex_timedlock.c | 22 ----
> .../sysv/linux/x86/pthread_mutex_trylock.c | 22 ----
> 46 files changed, 184 insertions(+), 645 deletions(-)
> rename sysdeps/unix/sysv/linux/s390/pthread_mutex_lock.c => nptl/elision-conf.c (79%)
> rename sysdeps/unix/sysv/linux/powerpc/pthread_mutex_lock.c => nptl/elision-lock.c (79%)
> rename sysdeps/unix/sysv/linux/powerpc/pthread_mutex_trylock.c => nptl/elision-timed.c (78%)
> rename sysdeps/unix/sysv/linux/x86/pthread_mutex_lock.c => nptl/elision-trylock.c (79%)
> create mode 100644 nptl/elision-unlock.c
> rename sysdeps/{unix/sysv/linux/powerpc/pthread_mutex_timedlock.c => pthread/elision-conf.h} (64%)
> delete mode 100644 sysdeps/unix/sysv/linux/powerpc/force-elision.h
> delete mode 100644 sysdeps/unix/sysv/linux/powerpc/lowlevellock.h
> delete mode 100644 sysdeps/unix/sysv/linux/powerpc/pthread_mutex_cond_lock.c
> delete mode 100644 sysdeps/unix/sysv/linux/s390/force-elision.h
> delete mode 100644 sysdeps/unix/sysv/linux/s390/lowlevellock.h
> delete mode 100644 sysdeps/unix/sysv/linux/s390/pthread_mutex_cond_lock.c
> delete mode 100644 sysdeps/unix/sysv/linux/s390/pthread_mutex_timedlock.c
> delete mode 100644 sysdeps/unix/sysv/linux/s390/pthread_mutex_trylock.c
> delete mode 100644 sysdeps/unix/sysv/linux/x86/force-elision.h
> delete mode 100644 sysdeps/unix/sysv/linux/x86/pthread_mutex_cond_lock.c
> delete mode 100644 sysdeps/unix/sysv/linux/x86/pthread_mutex_timedlock.c
> delete mode 100644 sysdeps/unix/sysv/linux/x86/pthread_mutex_trylock.c
>
> diff --git a/elf/libc_early_init.c b/elf/libc_early_init.c
> index 4755554f12..ce23393d90 100644
> --- a/elf/libc_early_init.c
> +++ b/elf/libc_early_init.c
> @@ -17,8 +17,10 @@
> <https://www.gnu.org/licenses/>. */
>
> #include <ctype.h>
> +#include <elision-conf.h>
> #include <libc-early-init.h>
> #include <libc-internal.h>
> +#include <lowlevellock.h>
> #include <sys/single_threaded.h>
>
> #ifdef SHARED
> @@ -37,4 +39,8 @@ __libc_early_init (_Bool initial)
> #ifdef SHARED
> __libc_initial = initial;
> #endif
> +
> +#if HAVE_ELISION
> + __lll_elision_init ();
> +#endif
> }
Ok.
> diff --git a/nptl/Makefile b/nptl/Makefile
> index 4dd9c6a6a8..5f85dd7854 100644
> --- a/nptl/Makefile
> +++ b/nptl/Makefile
> @@ -30,6 +30,11 @@ extra-libs-others := $(extra-libs)
>
> routines = \
> alloca_cutoff \
> + elision-conf \
> + elision-lock \
> + elision-timed \
> + elision-trylock \
> + elision-unlock \
> forward \
> futex-internal \
> libc-cancellation \
Ok.
> diff --git a/nptl/Versions b/nptl/Versions
> index 6cca579a0a..186befebfd 100644
> --- a/nptl/Versions
> +++ b/nptl/Versions
> @@ -79,8 +79,12 @@ libc {
> __libc_current_sigrtmin_private;
> __libc_dl_error_tsd;
> __libc_pthread_init;
> + __lll_clocklock_elision;
> + __lll_lock_elision;
> __lll_lock_wait;
> __lll_lock_wait_private;
> + __lll_trylock_elision;
> + __lll_unlock_elision;
> __pthread_attr_copy;
> __pthread_attr_destroy;
> __pthread_attr_init;
> @@ -88,6 +92,7 @@ libc {
> __pthread_attr_setsigmask_internal;
> __pthread_cond_destroy; # Used by the C11 threads.
> __pthread_cond_init; # Used by the C11 threads.
> + __pthread_force_elision;
> __pthread_getattr_default_np;
> }
> }
Ok.
> diff --git a/sysdeps/unix/sysv/linux/s390/pthread_mutex_lock.c b/nptl/elision-conf.c
> similarity index 79%
> rename from sysdeps/unix/sysv/linux/s390/pthread_mutex_lock.c
> rename to nptl/elision-conf.c
Ok.
> index b193b5b2b0..d71c30a01b 100644
> --- a/sysdeps/unix/sysv/linux/s390/pthread_mutex_lock.c
> +++ b/nptl/elision-conf.c
> @@ -1,5 +1,5 @@
> -/* Elided version of pthread_mutex_lock.
> - Copyright (C) 2014-2021 Free Software Foundation, Inc.
> +/* elision-conf.c: Lock elision tunable parameters. Stub version.
> + Copyright (C) 2021 Free Software Foundation, Inc.
> This file is part of the GNU C Library.
>
> The GNU C Library is free software; you can redistribute it and/or
> @@ -15,8 +15,3 @@
> You should have received a copy of the GNU Lesser General Public
> License along with the GNU C Library; if not, see
> <https://www.gnu.org/licenses/>. */
> -
> -#include <elision-conf.h>
> -#include <force-elision.h>
> -
> -#include <nptl/pthread_mutex_lock.c>
Ok.
> diff --git a/sysdeps/unix/sysv/linux/powerpc/pthread_mutex_lock.c b/nptl/elision-lock.c
> similarity index 79%
> rename from sysdeps/unix/sysv/linux/powerpc/pthread_mutex_lock.c
> rename to nptl/elision-lock.c
> index 110a6bf47b..a8235cbbea 100644
> --- a/sysdeps/unix/sysv/linux/powerpc/pthread_mutex_lock.c
> +++ b/nptl/elision-lock.c
> @@ -1,5 +1,5 @@
> -/* Elided version of pthread_mutex_lock.
> - Copyright (C) 2015-2021 Free Software Foundation, Inc.
> +/* elision-lock.c: Lock elision locking. Stub version.
> + Copyright (C) 2021 Free Software Foundation, Inc.
> This file is part of the GNU C Library.
>
> The GNU C Library is free software; you can redistribute it and/or
> @@ -15,8 +15,3 @@
> You should have received a copy of the GNU Lesser General Public
> License along with the GNU C Library; if not, see
> <https://www.gnu.org/licenses/>. */
> -
> -#include <elision-conf.h>
> -#include <force-elision.h>
> -
> -#include <nptl/pthread_mutex_lock.c>
Ok.
> diff --git a/sysdeps/unix/sysv/linux/powerpc/pthread_mutex_trylock.c b/nptl/elision-timed.c
> similarity index 78%
> rename from sysdeps/unix/sysv/linux/powerpc/pthread_mutex_trylock.c
> rename to nptl/elision-timed.c
> index ee9fe1d186..4723197363 100644
> --- a/sysdeps/unix/sysv/linux/powerpc/pthread_mutex_trylock.c
> +++ b/nptl/elision-timed.c
> @@ -1,5 +1,5 @@
> -/* Elided version of pthread_mutex_trylock.
> - Copyright (C) 2015-2021 Free Software Foundation, Inc.
> +/* elision-lock.c: Lock elision timed locking. Stub version.
> + Copyright (C) 2021 Free Software Foundation, Inc.
> This file is part of the GNU C Library.
>
> The GNU C Library is free software; you can redistribute it and/or
> @@ -15,8 +15,3 @@
> You should have received a copy of the GNU Lesser General Public
> License along with the GNU C Library; if not, see
> <https://www.gnu.org/licenses/>. */
> -
> -#include <elision-conf.h>
> -#include <force-elision.h>
> -
> -#include <nptl/pthread_mutex_trylock.c>
Ok.
> diff --git a/sysdeps/unix/sysv/linux/x86/pthread_mutex_lock.c b/nptl/elision-trylock.c
> similarity index 79%
> rename from sysdeps/unix/sysv/linux/x86/pthread_mutex_lock.c
> rename to nptl/elision-trylock.c
> index 6e5f1f9964..286a7417fa 100644
> --- a/sysdeps/unix/sysv/linux/x86/pthread_mutex_lock.c
> +++ b/nptl/elision-trylock.c
> @@ -1,5 +1,5 @@
> -/* Elided version of pthread_mutex_lock.
> - Copyright (C) 2011-2021 Free Software Foundation, Inc.
> +/* elision-lock.c: Lock elision locking attempts. Stub version.
> + Copyright (C) 2021 Free Software Foundation, Inc.
> This file is part of the GNU C Library.
>
> The GNU C Library is free software; you can redistribute it and/or
> @@ -15,8 +15,3 @@
> You should have received a copy of the GNU Lesser General Public
> License along with the GNU C Library; if not, see
> <https://www.gnu.org/licenses/>. */
> -
> -#include <elision-conf.h>
> -#include "force-elision.h"
> -
> -#include "nptl/pthread_mutex_lock.c"
Ok.
> diff --git a/nptl/elision-unlock.c b/nptl/elision-unlock.c
> new file mode 100644
> index 0000000000..552a90ade2
> --- /dev/null
> +++ b/nptl/elision-unlock.c
> @@ -0,0 +1,17 @@
> +/* elision-lock.c: Lock elision unlocking support. Stub version.
> + Copyright (C) 2021 Free Software Foundation, Inc.
> + This file is part of the GNU C Library.
> +
> + The GNU C Library is free software; you can redistribute it and/or
> + modify it under the terms of the GNU Lesser General Public
> + License as published by the Free Software Foundation; either
> + version 2.1 of the License, or (at your option) any later version.
> +
> + The GNU C Library is distributed in the hope that it will be useful,
> + but WITHOUT ANY WARRANTY; without even the implied warranty of
> + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> + Lesser General Public License for more details.
> +
> + You should have received a copy of the GNU Lesser General Public
> + License along with the GNU C Library; if not, see
> + <https://www.gnu.org/licenses/>. */
Ok.
> diff --git a/nptl/lowlevellock.c b/nptl/lowlevellock.c
> index 7976c6d8f5..0c0dd0ee00 100644
> --- a/nptl/lowlevellock.c
> +++ b/nptl/lowlevellock.c
> @@ -51,3 +51,8 @@ __lll_lock_wait (int *futex, int private)
> }
> }
> libc_hidden_def (__lll_lock_wait)
> +
> +#if HAVE_ELISION
> +int __pthread_force_elision __attribute__ ((nocommon));
> +libc_hidden_data_def (__pthread_force_elision)
> +#endif
Ok, attribute nocommon now that it exported from libc.
> diff --git a/nptl/pthread_mutex_lock.c b/nptl/pthread_mutex_lock.c
> index 319658adc4..0e24ae09d7 100644
> --- a/nptl/pthread_mutex_lock.c
> +++ b/nptl/pthread_mutex_lock.c
> @@ -27,15 +27,6 @@
> #include <futex-internal.h>
> #include <stap-probe.h>
>
> -#ifndef lll_lock_elision
> -#define lll_lock_elision(lock, try_lock, private) ({ \
> - lll_lock (lock, private); 0; })
> -#endif
> -
> -#ifndef lll_trylock_elision
> -#define lll_trylock_elision(a,t) lll_trylock(a)
> -#endif
> -
> /* Some of the following definitions differ when pthread_mutex_cond_lock.c
> includes this file. */
> #ifndef LLL_MUTEX_LOCK
Ok.
> @@ -52,10 +43,6 @@
> PTHREAD_MUTEX_PSHARED (mutex))
> #endif
>
> -#ifndef FORCE_ELISION
> -#define FORCE_ELISION(m, s)
> -#endif
> -
> static int __pthread_mutex_lock_full (pthread_mutex_t *mutex)
> __attribute_noinline__;
>
Ok.
> diff --git a/nptl/pthread_mutex_timedlock.c b/nptl/pthread_mutex_timedlock.c
> index 8428ebca0f..5e52a4d856 100644
> --- a/nptl/pthread_mutex_timedlock.c
> +++ b/nptl/pthread_mutex_timedlock.c
> @@ -29,19 +29,6 @@
>
> #include <stap-probe.h>
>
> -#ifndef lll_clocklock_elision
> -#define lll_clocklock_elision(futex, adapt_count, clockid, abstime, private) \
> - __futex_clocklock64 (&(futex), clockid, abstime, private)
> -#endif
> -
> -#ifndef lll_trylock_elision
> -#define lll_trylock_elision(a,t) lll_trylock(a)
> -#endif
> -
> -#ifndef FORCE_ELISION
> -#define FORCE_ELISION(m, s)
> -#endif
> -
> int
> __pthread_mutex_clocklock_common (pthread_mutex_t *mutex,
> clockid_t clockid,
Ok.
> diff --git a/nptl/pthread_mutex_trylock.c b/nptl/pthread_mutex_trylock.c
> index 519727580d..e52dddafbd 100644
> --- a/nptl/pthread_mutex_trylock.c
> +++ b/nptl/pthread_mutex_trylock.c
> @@ -23,14 +23,6 @@
> #include <lowlevellock.h>
> #include <futex-internal.h>
>
> -#ifndef lll_trylock_elision
> -#define lll_trylock_elision(a,t) lll_trylock(a)
> -#endif
> -
> -#ifndef FORCE_ELISION
> -#define FORCE_ELISION(m, s)
> -#endif
> -
> int
> __pthread_mutex_trylock (pthread_mutex_t *mutex)
> {
Ok.
> diff --git a/nptl/pthread_mutex_unlock.c b/nptl/pthread_mutex_unlock.c
> index e29bb7fa53..3b5ccdacf9 100644
> --- a/nptl/pthread_mutex_unlock.c
> +++ b/nptl/pthread_mutex_unlock.c
> @@ -24,10 +24,6 @@
> #include <stap-probe.h>
> #include <futex-internal.h>
>
> -#ifndef lll_unlock_elision
> -#define lll_unlock_elision(a,b,c) ({ lll_unlock (a,c); 0; })
> -#endif
> -
> static int
> __pthread_mutex_unlock_full (pthread_mutex_t *mutex, int decr)
> __attribute_noinline__;
Ok.
> diff --git a/sysdeps/nptl/lowlevellock.h b/sysdeps/nptl/lowlevellock.h
> index 176ba96251..c4b32756f9 100644
> --- a/sysdeps/nptl/lowlevellock.h
> +++ b/sysdeps/nptl/lowlevellock.h
> @@ -20,6 +20,7 @@
> #define _LOWLEVELLOCK_H 1
>
> #include <atomic.h>
> +#include <elision-conf.h>
> #include <lowlevellock-futex.h>
> #include <time.h>
>
> @@ -160,4 +161,105 @@ libc_hidden_proto (__lll_lock_wait)
> #define LLL_LOCK_INITIALIZER (0)
> #define LLL_LOCK_INITIALIZER_LOCKED (1)
>
> +/* Elision support. */
> +
> +/* Force elision for all new locks. This is used to decide whether
> + existing DEFAULT locks should be automatically upgraded to elision
> + in pthread_mutex_lock. Disabled for suid programs. Only used when
> + elision is available. */
> +extern int __pthread_force_elision;
> +libc_hidden_proto (__pthread_force_elision)
> +
> +extern void __lll_elision_init (void) attribute_hidden;
> +extern int __lll_clocklock_elision (int *futex, short *adapt_count,
> + clockid_t clockid,
> + const struct __timespec64 *timeout,
> + int private);
> +libc_hidden_proto (__lll_clocklock_elision)
> +
> +extern int __lll_lock_elision (int *futex, short *adapt_count, int private);
> +libc_hidden_proto (__lll_lock_elision)
> +
> +#if ELISION_UNLOCK_NEEDS_ADAPT_COUNT
> +extern int __lll_unlock_elision (int *lock, short *adapt_count, int private);
> +#else
> +extern int __lll_unlock_elision (int *lock, int private);
> +#endif
> +libc_hidden_proto (__lll_unlock_elision)
> +
> +extern int __lll_trylock_elision (int *lock, short *adapt_count);
> +libc_hidden_proto (__lll_trylock_elision)
Ok.
> +
> +#if HAVE_ELISION
> +# define lll_clocklock_elision(futex, adapt_count, clockid, timeout, private) \
> + __lll_clocklock_elision (&(futex), &(adapt_count), clockid, timeout, private)
> +# define lll_lock_elision(futex, adapt_count, private) \
> + __lll_lock_elision (&(futex), &(adapt_count), private)
> +# define lll_trylock_elision(futex, adapt_count) \
> + __lll_trylock_elision (&(futex), &(adapt_count))
> +# if ELISION_UNLOCK_NEEDS_ADAPT_COUNT
> +# define lll_unlock_elision(futex, adapt_count, private) \
> + __lll_unlock_elision (&(futex), &(adapt_count), private)
> +#else
> +# define lll_unlock_elision(futex, adapt_count, private) \
> + __lll_unlock_elision (&(futex), private)
> +#endif
> +
> +/* Automatically enable elision for existing user lock kinds. */
> +# define FORCE_ELISION(m, s) \
> + if (__pthread_force_elision) \
> + { \
> + /* See concurrency notes regarding __kind in \
> + struct __pthread_mutex_s in \
> + sysdeps/nptl/bits/thread-shared-types.h. \
> + \
> + There are the following cases for the kind of a mutex \
> + (The mask PTHREAD_MUTEX_ELISION_FLAGS_NP covers the flags \
> + PTHREAD_MUTEX_ELISION_NP and PTHREAD_MUTEX_NO_ELISION_NP where \
> + only one of both flags can be set): \
> + - both flags are not set: \
> + This is the first lock operation for this mutex. Enable \
> + elision as it is not enabled so far. \
> + Note: It can happen that multiple threads are calling e.g. \
> + pthread_mutex_lock at the same time as the first lock \
> + operation for this mutex. Then elision is enabled for this \
> + mutex by multiple threads. Storing with relaxed MO is enough \
> + as all threads will store the same new value for the kind of \
> + the mutex. But we have to ensure that we always use the \
> + elision path regardless if this thread has enabled elision or \
> + another one. \
> + \
> + - PTHREAD_MUTEX_ELISION_NP flag is set: \
> + Elision was already enabled for this mutex by a previous lock \
> + operation. See case above. Just use the elision path. \
> + \
> + - PTHREAD_MUTEX_NO_ELISION_NP flag is set: \
> + Elision was explicitly disabled by pthread_mutexattr_settype. \
> + Do not use the elision path. \
> + Note: The flag PTHREAD_MUTEX_NO_ELISION_NP will never be \
> + changed after mutex initialization. */ \
> + int mutex_kind = atomic_load_relaxed (&((m)->__data.__kind)); \
> + if ((mutex_kind & PTHREAD_MUTEX_ELISION_FLAGS_NP) == 0) \
> + { \
> + mutex_kind |= PTHREAD_MUTEX_ELISION_NP; \
> + atomic_store_relaxed (&((m)->__data.__kind), mutex_kind); \
> + } \
> + if ((mutex_kind & PTHREAD_MUTEX_ELISION_NP) != 0) \
> + { \
> + s; \
> + } \
> + }
Maybe since you are refactoring it, to use a static inline instead of macro
here?
> +
> +#else /* !HAVE_ELISION */
> +
> +#define lll_clocklock_elision(futex, adapt_count, clockid, abstime, private) \
> + __futex_clocklock64 (&(futex), clockid, abstime, private)
> +# define lll_lock_elision(lock, try_lock, private) \
> + ({ lll_lock (lock, private); 0; })
> +# define lll_trylock_elision(a,t) lll_trylock(a)
> +# define lll_unlock_elision(a,b,c) ({ lll_unlock (a,c); 0; })
> +# define FORCE_ELISION(m, s)
> +
> +#endif /* !HAVE_ELISION */
> +
> #endif /* lowlevellock.h */
Ok.
> diff --git a/sysdeps/unix/sysv/linux/powerpc/pthread_mutex_timedlock.c b/sysdeps/pthread/elision-conf.h
> similarity index 64%
> rename from sysdeps/unix/sysv/linux/powerpc/pthread_mutex_timedlock.c
> rename to sysdeps/pthread/elision-conf.h
> index 6f8b06d459..6635a7c56f 100644
> --- a/sysdeps/unix/sysv/linux/powerpc/pthread_mutex_timedlock.c
> +++ b/sysdeps/pthread/elision-conf.h
> @@ -1,5 +1,5 @@
> -/* Elided version of pthread_mutex_timedlock.
> - Copyright (C) 2015-2021 Free Software Foundation, Inc.
> +/* elision-conf.h: Lock elision configuration. Stub version.
> + Copyright (C) 2021 Free Software Foundation, Inc.
> This file is part of the GNU C Library.
>
> The GNU C Library is free software; you can redistribute it and/or
> @@ -16,7 +16,14 @@
> License along with the GNU C Library; if not, see
> <https://www.gnu.org/licenses/>. */
>
> -#include <elision-conf.h>
> -#include <force-elision.h>
> +#ifndef _ELISION_CONF_H
> +#define _ELISION_CONF_H 1
>
> -#include <nptl/pthread_mutex_timedlock.c>
> +/* No elision support by default. */
> +#define HAVE_ELISION 0
> +
> +/* Whether __lll_unlock_elision expects a pointer argument to the
> + adaptive counter. Here, an unused arbitrary value. */
> +#define ELISION_UNLOCK_NEEDS_ADAPT_COUNT 0
> +
> +#endif
Maybe prepend the define with a string related to code it relates to,
like 'ELISION_CONF_'? This links the define more directly to the
implementation, since HAVE_* is usually defined by config.h.
> diff --git a/sysdeps/unix/sysv/linux/powerpc/Makefile b/sysdeps/unix/sysv/linux/powerpc/Makefile
> index cc2f804d86..a093cda68b 100644
> --- a/sysdeps/unix/sysv/linux/powerpc/Makefile
> +++ b/sysdeps/unix/sysv/linux/powerpc/Makefile
> @@ -32,7 +32,5 @@ endif
>
> ifeq ($(subdir),nptl)
> libpthread-routines += sysdep
> -libpthread-sysdep_routines += elision-lock elision-unlock elision-timed \
> - elision-trylock
> libpthread-shared-only-routines += sysdep
> endif
Ok.
> diff --git a/sysdeps/unix/sysv/linux/powerpc/elision-conf.c b/sysdeps/unix/sysv/linux/powerpc/elision-conf.c
> index 003bc82343..1ecc2a7b8f 100644
> --- a/sysdeps/unix/sysv/linux/powerpc/elision-conf.c
> +++ b/sysdeps/unix/sysv/linux/powerpc/elision-conf.c
> @@ -51,12 +51,6 @@ struct elision_config __elision_aconf =
> .skip_trylock_internal_abort = 3,
> };
>
> -/* Force elision for all new locks. This is used to decide whether existing
> - DEFAULT locks should be automatically use elision in pthread_mutex_lock().
> - Disabled for suid programs. Only used when elision is available. */
> -
> -int __pthread_force_elision attribute_hidden = 0;
> -
> #if HAVE_TUNABLES
> static inline void
> __always_inline
> @@ -104,10 +98,8 @@ TUNABLE_CALLBACK_FNDECL (skip_trylock_internal_abort, int32_t);
>
> /* Initialize elision. */
>
> -static void
> -elision_init (int argc __attribute__ ((unused)),
> - char **argv __attribute__ ((unused)),
> - char **environ)
> +void
> +__lll_elision_init (void)
> {
> #if HAVE_TUNABLES
> /* Elision depends on tunables and must be explicitly turned on by setting
> @@ -150,17 +142,3 @@ elision_init (int argc __attribute__ ((unused)),
> if (!__pthread_force_elision)
> __elision_aconf.try_tbegin = 0; /* Disable elision on rwlocks. */
> }
> -
> -#ifdef SHARED
> -# define INIT_SECTION ".init_array"
> -# define MAYBE_CONST
> -#else
> -# define INIT_SECTION ".preinit_array"
> -# define MAYBE_CONST const
> -#endif
> -
> -void (*MAYBE_CONST __pthread_init_array []) (int, char **, char **)
> - __attribute__ ((section (INIT_SECTION), aligned (sizeof (void *)))) =
> -{
> - &elision_init
> -};
Ok.
> diff --git a/sysdeps/unix/sysv/linux/powerpc/elision-conf.h b/sysdeps/unix/sysv/linux/powerpc/elision-conf.h
> index 8c444d8695..1a1bd644ea 100644
> --- a/sysdeps/unix/sysv/linux/powerpc/elision-conf.h
> +++ b/sysdeps/unix/sysv/linux/powerpc/elision-conf.h
> @@ -22,6 +22,9 @@
> #include <pthread.h>
> #include <time.h>
>
> +#define HAVE_ELISION 1
> +#define ELISION_UNLOCK_NEEDS_ADAPT_COUNT 1
> +
> /* Should make sure there is no false sharing on this. */
> struct elision_config
> {
> @@ -34,9 +37,4 @@ struct elision_config
>
> extern struct elision_config __elision_aconf attribute_hidden;
>
> -extern int __pthread_force_elision attribute_hidden;
> -
> -/* Tell the test suite to test elision for this architecture. */
> -#define HAVE_ELISION 1
> -
> #endif
Ok.
> diff --git a/sysdeps/unix/sysv/linux/powerpc/elision-lock.c b/sysdeps/unix/sysv/linux/powerpc/elision-lock.c
> index 767d439f88..f5fb46df25 100644
> --- a/sysdeps/unix/sysv/linux/powerpc/elision-lock.c
> +++ b/sysdeps/unix/sysv/linux/powerpc/elision-lock.c
> @@ -23,12 +23,6 @@
> #include <elision-conf.h>
> #include "htm.h"
>
> -#if !defined(LLL_LOCK) && !defined(EXTRAARG)
> -/* Make sure the configuration code is always linked in for static
> - libraries. */
> -#include "elision-conf.c"
> -#endif
> -
> #ifndef EXTRAARG
> # define EXTRAARG
> #endif
> @@ -84,3 +78,4 @@ __lll_lock_elision (int *lock, short *adapt_count, EXTRAARG int pshared)
> use_lock:
> return LLL_LOCK ((*lock), pshared);
> }
> +libc_hidden_def (__lll_lock_elision)
Ok.
> diff --git a/sysdeps/unix/sysv/linux/powerpc/elision-trylock.c b/sysdeps/unix/sysv/linux/powerpc/elision-trylock.c
> index ab92f4d72f..5ebbced427 100644
> --- a/sysdeps/unix/sysv/linux/powerpc/elision-trylock.c
> +++ b/sysdeps/unix/sysv/linux/powerpc/elision-trylock.c
> @@ -67,3 +67,4 @@ __lll_trylock_elision (int *futex, short *adapt_count)
> use_lock:
> return lll_trylock (*futex);
> }
> +libc_hidden_def (__lll_trylock_elision)
Ok.
> diff --git a/sysdeps/unix/sysv/linux/powerpc/elision-unlock.c b/sysdeps/unix/sysv/linux/powerpc/elision-unlock.c
> index 35044af40b..5aa87521c5 100644
> --- a/sysdeps/unix/sysv/linux/powerpc/elision-unlock.c
> +++ b/sysdeps/unix/sysv/linux/powerpc/elision-unlock.c
> @@ -41,3 +41,4 @@ __lll_unlock_elision (int *lock, short *adapt_count, int pshared)
> }
> return 0;
> }
> +libc_hidden_def (__lll_unlock_elision)
Ok.
> diff --git a/sysdeps/unix/sysv/linux/powerpc/force-elision.h b/sysdeps/unix/sysv/linux/powerpc/force-elision.h
> deleted file mode 100644
> index d1fa611c52..0000000000
> --- a/sysdeps/unix/sysv/linux/powerpc/force-elision.h
> +++ /dev/null
> @@ -1,62 +0,0 @@
> -/* force-elision.h: Automatic enabling of elision for mutexes
> - Copyright (C) 2015-2021 Free Software Foundation, Inc.
> - This file is part of the GNU C Library.
> -
> - The GNU C Library is free software; you can redistribute it and/or
> - modify it under the terms of the GNU Lesser General Public
> - License as published by the Free Software Foundation; either
> - version 2.1 of the License, or (at your option) any later version.
> -
> - The GNU C Library is distributed in the hope that it will be useful,
> - but WITHOUT ANY WARRANTY; without even the implied warranty of
> - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> - Lesser General Public License for more details.
> -
> - You should have received a copy of the GNU Lesser General Public
> - License along with the GNU C Library; if not, see
> - <https://www.gnu.org/licenses/>. */
> -
> -/* Automatically enable elision for existing user lock kinds. */
> -#define FORCE_ELISION(m, s) \
> - if (__pthread_force_elision) \
> - { \
> - /* See concurrency notes regarding __kind in \
> - struct __pthread_mutex_s in \
> - sysdeps/nptl/bits/thread-shared-types.h. \
> - \
> - There are the following cases for the kind of a mutex \
> - (The mask PTHREAD_MUTEX_ELISION_FLAGS_NP covers the flags \
> - PTHREAD_MUTEX_ELISION_NP and PTHREAD_MUTEX_NO_ELISION_NP where \
> - only one of both flags can be set): \
> - - both flags are not set: \
> - This is the first lock operation for this mutex. Enable \
> - elision as it is not enabled so far. \
> - Note: It can happen that multiple threads are calling e.g. \
> - pthread_mutex_lock at the same time as the first lock \
> - operation for this mutex. Then elision is enabled for this \
> - mutex by multiple threads. Storing with relaxed MO is enough \
> - as all threads will store the same new value for the kind of \
> - the mutex. But we have to ensure that we always use the \
> - elision path regardless if this thread has enabled elision or \
> - another one. \
> - \
> - - PTHREAD_MUTEX_ELISION_NP flag is set: \
> - Elision was already enabled for this mutex by a previous lock \
> - operation. See case above. Just use the elision path. \
> - \
> - - PTHREAD_MUTEX_NO_ELISION_NP flag is set: \
> - Elision was explicitly disabled by pthread_mutexattr_settype. \
> - Do not use the elision path. \
> - Note: The flag PTHREAD_MUTEX_NO_ELISION_NP will never be \
> - changed after mutex initialization. */ \
> - int mutex_kind = atomic_load_relaxed (&((m)->__data.__kind)); \
> - if ((mutex_kind & PTHREAD_MUTEX_ELISION_FLAGS_NP) == 0) \
> - { \
> - mutex_kind |= PTHREAD_MUTEX_ELISION_NP; \
> - atomic_store_relaxed (&((m)->__data.__kind), mutex_kind); \
> - } \
> - if ((mutex_kind & PTHREAD_MUTEX_ELISION_NP) != 0) \
> - { \
> - s; \
> - } \
> - }
Ok.
> diff --git a/sysdeps/unix/sysv/linux/powerpc/lowlevellock.h b/sysdeps/unix/sysv/linux/powerpc/lowlevellock.h
> deleted file mode 100644
> index 65962020c6..0000000000
> --- a/sysdeps/unix/sysv/linux/powerpc/lowlevellock.h
> +++ /dev/null
> @@ -1,49 +0,0 @@
> -/* PowerPC specific lock definitions.
> - Copyright (C) 2015-2021 Free Software Foundation, Inc.
> - This file is part of the GNU C Library.
> -
> - The GNU C Library is free software; you can redistribute it and/or
> - modify it under the terms of the GNU Lesser General Public
> - License as published by the Free Software Foundation; either
> - version 2.1 of the License, or (at your option) any later version.
> -
> - The GNU C Library is distributed in the hope that it will be useful,
> - but WITHOUT ANY WARRANTY; without even the implied warranty of
> - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> - Lesser General Public License for more details.
> -
> - You should have received a copy of the GNU Lesser General Public
> - License along with the GNU C Library; if not, see
> - <https://www.gnu.org/licenses/>. */
> -
> -#ifndef _POWERPC_LOWLEVELLOCK_H
> -#define _POWERPC_LOWLEVELLOCK_H 1
> -
> -#include <sysdeps/nptl/lowlevellock.h>
> -
> -/* Transactional lock elision definitions. */
> -extern int __lll_clocklock_elision
> - (int *futex, short *adapt_count,
> - clockid_t clockid, const struct __timespec64 *timeout, int private)
> - attribute_hidden;
> -
> -#define lll_clocklock_elision(futex, adapt_count, clockid, timeout, private) \
> - __lll_clocklock_elision (&(futex), &(adapt_count), clockid, timeout, private)
> -
> -extern int __lll_lock_elision (int *futex, short *adapt_count, int private)
> - attribute_hidden;
> -
> -extern int __lll_unlock_elision (int *lock, short *adapt_count, int private)
> - attribute_hidden;
> -
> -extern int __lll_trylock_elision(int *lock, short *adapt_count)
> - attribute_hidden;
> -
> -#define lll_lock_elision(futex, adapt_count, private) \
> - __lll_lock_elision (&(futex), &(adapt_count), private)
> -#define lll_unlock_elision(futex, adapt_count, private) \
> - __lll_unlock_elision (&(futex), &(adapt_count), private)
> -#define lll_trylock_elision(futex, adapt_count) \
> - __lll_trylock_elision (&(futex), &(adapt_count))
> -
> -#endif
Ok.
> diff --git a/sysdeps/unix/sysv/linux/powerpc/pthread_mutex_cond_lock.c b/sysdeps/unix/sysv/linux/powerpc/pthread_mutex_cond_lock.c
> deleted file mode 100644
> index f33bac7ca1..0000000000
> --- a/sysdeps/unix/sysv/linux/powerpc/pthread_mutex_cond_lock.c
> +++ /dev/null
> @@ -1,22 +0,0 @@
> -/* Copyright (C) 2015-2021 Free Software Foundation, Inc.
> - This file is part of the GNU C Library.
> -
> - The GNU C Library is free software; you can redistribute it and/or
> - modify it under the terms of the GNU Lesser General Public
> - License as published by the Free Software Foundation; either
> - version 2.1 of the License, or (at your option) any later version.
> -
> - The GNU C Library is distributed in the hope that it will be useful,
> - but WITHOUT ANY WARRANTY; without even the implied warranty of
> - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> - Lesser General Public License for more details.
> -
> - You should have received a copy of the GNU Lesser General Public
> - License along with the GNU C Library; if not, see
> - <https://www.gnu.org/licenses/>. */
> -
> -/* The cond lock is not actually elided yet, but we still need to handle
> - already elided locks. */
> -#include <elision-conf.h>
> -
> -#include <nptl/pthread_mutex_cond_lock.c>
Ok.
> diff --git a/sysdeps/unix/sysv/linux/s390/Makefile b/sysdeps/unix/sysv/linux/s390/Makefile
> index d9db1b5422..3de9579e23 100644
> --- a/sysdeps/unix/sysv/linux/s390/Makefile
> +++ b/sysdeps/unix/sysv/linux/s390/Makefile
> @@ -12,9 +12,6 @@ gen-as-const-headers += ucontext_i.sym
> endif
>
> ifeq ($(subdir),nptl)
> -libpthread-sysdep_routines += elision-lock elision-unlock elision-timed \
> - elision-trylock
> -
> elision-CFLAGS = -mhtm -msoft-float
> CFLAGS-elision-lock.c = $(elision-CFLAGS)
> CFLAGS-elision-timed.c = $(elision-CFLAGS)
Ok.
> diff --git a/sysdeps/unix/sysv/linux/s390/elision-conf.c b/sysdeps/unix/sysv/linux/s390/elision-conf.c
> index c05c046e56..a7a90e73eb 100644
> --- a/sysdeps/unix/sysv/linux/s390/elision-conf.c
> +++ b/sysdeps/unix/sysv/linux/s390/elision-conf.c
> @@ -51,13 +51,6 @@ struct elision_config __elision_aconf =
> .skip_trylock_internal_abort = 3,
> };
>
> -/* Force elision for all new locks. This is used to decide whether existing
> - DEFAULT locks should be automatically upgraded to elision in
> - pthread_mutex_lock(). Disabled for suid programs. Only used when elision
> - is available. */
> -
> -int __pthread_force_elision attribute_hidden = 0;
> -
> #if HAVE_TUNABLES
> static inline void
> __always_inline
> @@ -104,10 +97,8 @@ TUNABLE_CALLBACK_FNDECL (skip_trylock_internal_abort, int32_t);
>
> /* Initialize elison. */
>
> -static void
> -elision_init (int argc __attribute__ ((unused)),
> - char **argv __attribute__ ((unused)),
> - char **environ)
> +void
> +__lll_elision_init (void)
> {
> #if HAVE_TUNABLES
> /* Elision depends on tunables and must be explicitly turned on by setting
> @@ -130,17 +121,3 @@ elision_init (int argc __attribute__ ((unused)),
> if (!__pthread_force_elision)
> __elision_aconf.try_tbegin = 0; /* Disable elision on rwlocks. */
> }
> -
> -#ifdef SHARED
> -# define INIT_SECTION ".init_array"
> -# define MAYBE_CONST
> -#else
> -# define INIT_SECTION ".preinit_array"
> -# define MAYBE_CONST const
> -#endif
> -
> -void (*MAYBE_CONST __pthread_init_array []) (int, char **, char **)
> - __attribute__ ((section (INIT_SECTION), aligned (sizeof (void *)))) =
> -{
> - &elision_init
> -};
Ok.
> diff --git a/sysdeps/unix/sysv/linux/s390/elision-conf.h b/sysdeps/unix/sysv/linux/s390/elision-conf.h
> index 846c5ff297..3789817c56 100644
> --- a/sysdeps/unix/sysv/linux/s390/elision-conf.h
> +++ b/sysdeps/unix/sysv/linux/s390/elision-conf.h
> @@ -21,6 +21,9 @@
> #include <pthread.h>
> #include <time.h>
>
> +#define HAVE_ELISION 1
> +#define ELISION_UNLOCK_NEEDS_ADAPT_COUNT 1
> +
> /* Should make sure there is no false sharing on this. */
>
> struct elision_config
> @@ -34,9 +37,4 @@ struct elision_config
>
> extern struct elision_config __elision_aconf attribute_hidden;
>
> -extern int __pthread_force_elision attribute_hidden;
> -
> -/* Tell the test suite to test elision for this architecture. */
> -#define HAVE_ELISION 1
> -
> #endif
Ok.
> diff --git a/sysdeps/unix/sysv/linux/s390/elision-lock.c b/sysdeps/unix/sysv/linux/s390/elision-lock.c
> index 26474c6398..d5dd6023a0 100644
> --- a/sysdeps/unix/sysv/linux/s390/elision-lock.c
> +++ b/sysdeps/unix/sysv/linux/s390/elision-lock.c
> @@ -23,12 +23,6 @@
> #include <elision-conf.h>
> #include <stdint.h>
>
> -#if !defined(LLL_LOCK) && !defined(EXTRAARG)
> -/* Make sure the configuration code is always linked in for static
> - libraries. */
> -#include "elision-conf.c"
> -#endif
> -
> #ifndef EXTRAARG
> #define EXTRAARG
> #endif
> @@ -123,3 +117,4 @@ __lll_lock_elision (int *futex, short *adapt_count, EXTRAARG int private)
> succeed. */
> return LLL_LOCK ((*futex), private);
> }
> +libc_hidden_def (__lll_lock_elision)
Ok.
> diff --git a/sysdeps/unix/sysv/linux/s390/elision-trylock.c b/sysdeps/unix/sysv/linux/s390/elision-trylock.c
> index 484c3cc913..2470955c89 100644
> --- a/sysdeps/unix/sysv/linux/s390/elision-trylock.c
> +++ b/sysdeps/unix/sysv/linux/s390/elision-trylock.c
> @@ -95,3 +95,4 @@ __lll_trylock_elision (int *futex, short *adapt_count)
> succeed. */
> return lll_trylock (*futex);
> }
> +libc_hidden_def (__lll_trylock_elision)
Ok.
> diff --git a/sysdeps/unix/sysv/linux/s390/elision-unlock.c b/sysdeps/unix/sysv/linux/s390/elision-unlock.c
> index 5ac6758df2..87a5bd7cbf 100644
> --- a/sysdeps/unix/sysv/linux/s390/elision-unlock.c
> +++ b/sysdeps/unix/sysv/linux/s390/elision-unlock.c
> @@ -59,3 +59,4 @@ __lll_unlock_elision(int *futex, short *adapt_count, int private)
> }
> return 0;
> }
> +libc_hidden_def (__lll_unlock_elision)
Ok.
> diff --git a/sysdeps/unix/sysv/linux/s390/force-elision.h b/sysdeps/unix/sysv/linux/s390/force-elision.h
> deleted file mode 100644
> index 154740d37f..0000000000
> --- a/sysdeps/unix/sysv/linux/s390/force-elision.h
> +++ /dev/null
> @@ -1,62 +0,0 @@
> -/* Automatic enabling of elision for mutexes
> - Copyright (C) 2014-2021 Free Software Foundation, Inc.
> - This file is part of the GNU C Library.
> -
> - The GNU C Library is free software; you can redistribute it and/or
> - modify it under the terms of the GNU Lesser General Public
> - License as published by the Free Software Foundation; either
> - version 2.1 of the License, or (at your option) any later version.
> -
> - The GNU C Library is distributed in the hope that it will be useful,
> - but WITHOUT ANY WARRANTY; without even the implied warranty of
> - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> - Lesser General Public License for more details.
> -
> - You should have received a copy of the GNU Lesser General Public
> - License along with the GNU C Library; if not, see
> - <https://www.gnu.org/licenses/>. */
> -
> -/* Automatically enable elision for existing user lock kinds. */
> -#define FORCE_ELISION(m, s) \
> - if (__pthread_force_elision) \
> - { \
> - /* See concurrency notes regarding __kind in \
> - struct __pthread_mutex_s in \
> - sysdeps/nptl/bits/thread-shared-types.h. \
> - \
> - There are the following cases for the kind of a mutex \
> - (The mask PTHREAD_MUTEX_ELISION_FLAGS_NP covers the flags \
> - PTHREAD_MUTEX_ELISION_NP and PTHREAD_MUTEX_NO_ELISION_NP where \
> - only one of both flags can be set): \
> - - both flags are not set: \
> - This is the first lock operation for this mutex. Enable \
> - elision as it is not enabled so far. \
> - Note: It can happen that multiple threads are calling e.g. \
> - pthread_mutex_lock at the same time as the first lock \
> - operation for this mutex. Then elision is enabled for this \
> - mutex by multiple threads. Storing with relaxed MO is enough \
> - as all threads will store the same new value for the kind of \
> - the mutex. But we have to ensure that we always use the \
> - elision path regardless if this thread has enabled elision or \
> - another one. \
> - \
> - - PTHREAD_MUTEX_ELISION_NP flag is set: \
> - Elision was already enabled for this mutex by a previous lock \
> - operation. See case above. Just use the elision path. \
> - \
> - - PTHREAD_MUTEX_NO_ELISION_NP flag is set: \
> - Elision was explicitly disabled by pthread_mutexattr_settype. \
> - Do not use the elision path. \
> - Note: The flag PTHREAD_MUTEX_NO_ELISION_NP will never be \
> - changed after mutex initialization. */ \
> - int mutex_kind = atomic_load_relaxed (&((m)->__data.__kind)); \
> - if ((mutex_kind & PTHREAD_MUTEX_ELISION_FLAGS_NP) == 0) \
> - { \
> - mutex_kind |= PTHREAD_MUTEX_ELISION_NP; \
> - atomic_store_relaxed (&((m)->__data.__kind), mutex_kind); \
> - } \
> - if ((mutex_kind & PTHREAD_MUTEX_ELISION_NP) != 0) \
> - { \
> - s; \
> - } \
> - }
Ok.
> diff --git a/sysdeps/unix/sysv/linux/s390/lowlevellock.h b/sysdeps/unix/sysv/linux/s390/lowlevellock.h
> deleted file mode 100644
> index 98d78f5270..0000000000
> --- a/sysdeps/unix/sysv/linux/s390/lowlevellock.h
> +++ /dev/null
> @@ -1,49 +0,0 @@
> -/* Copyright (C) 2003-2021 Free Software Foundation, Inc.
> - This file is part of the GNU C Library.
> - Contributed by Martin Schwidefsky <schwidefsky@de.ibm.com>, 2003.
> -
> - The GNU C Library is free software; you can redistribute it and/or
> - modify it under the terms of the GNU Lesser General Public
> - License as published by the Free Software Foundation; either
> - version 2.1 of the License, or (at your option) any later version.
> -
> - The GNU C Library is distributed in the hope that it will be useful,
> - but WITHOUT ANY WARRANTY; without even the implied warranty of
> - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> - Lesser General Public License for more details.
> -
> - You should have received a copy of the GNU Lesser General Public
> - License along with the GNU C Library; if not, see
> - <https://www.gnu.org/licenses/>. */
> -
> -#ifndef _S390_LOWLEVELLOCK_H
> -#define _S390_LOWLEVELLOCK_H 1
> -
> -#include <sysdeps/nptl/lowlevellock.h>
> -
> -/* Transactional lock elision definitions. */
> -extern int __lll_clocklock_elision
> - (int *futex, short *adapt_count,
> - clockid_t clockid, const struct __timespec64 *timeout, int private)
> - attribute_hidden;
> -
> -# define lll_clocklock_elision(futex, adapt_count, clockid, timeout, private) \
> - __lll_clocklock_elision (&(futex), &(adapt_count), clockid, timeout, private)
> -
> -extern int __lll_lock_elision (int *futex, short *adapt_count, int private)
> - attribute_hidden;
> -
> -extern int __lll_unlock_elision(int *futex, short *adapt_count, int private)
> - attribute_hidden;
> -
> -extern int __lll_trylock_elision(int *futex, short *adapt_count)
> - attribute_hidden;
> -
> -# define lll_lock_elision(futex, adapt_count, private) \
> - __lll_lock_elision (&(futex), &(adapt_count), private)
> -# define lll_unlock_elision(futex, adapt_count, private) \
> - __lll_unlock_elision (&(futex), &(adapt_count), private)
> -# define lll_trylock_elision(futex, adapt_count) \
> - __lll_trylock_elision(&(futex), &(adapt_count))
> -
> -#endif /* lowlevellock.h */
Ok.
> diff --git a/sysdeps/unix/sysv/linux/s390/pthread_mutex_cond_lock.c b/sysdeps/unix/sysv/linux/s390/pthread_mutex_cond_lock.c
> deleted file mode 100644
> index 8398ff6425..0000000000
> --- a/sysdeps/unix/sysv/linux/s390/pthread_mutex_cond_lock.c
> +++ /dev/null
> @@ -1,22 +0,0 @@
> -/* Copyright (C) 2014-2021 Free Software Foundation, Inc.
> - This file is part of the GNU C Library.
> -
> - The GNU C Library is free software; you can redistribute it and/or
> - modify it under the terms of the GNU Lesser General Public
> - License as published by the Free Software Foundation; either
> - version 2.1 of the License, or (at your option) any later version.
> -
> - The GNU C Library is distributed in the hope that it will be useful,
> - but WITHOUT ANY WARRANTY; without even the implied warranty of
> - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> - Lesser General Public License for more details.
> -
> - You should have received a copy of the GNU Lesser General Public
> - License along with the GNU C Library; if not, see
> - <https://www.gnu.org/licenses/>. */
> -
> -/* The cond lock is not actually elided yet, but we still need to handle
> - already elided locks. */
> -#include <elision-conf.h>
> -
> -#include <nptl/pthread_mutex_cond_lock.c>
Ok.
> diff --git a/sysdeps/unix/sysv/linux/s390/pthread_mutex_timedlock.c b/sysdeps/unix/sysv/linux/s390/pthread_mutex_timedlock.c
> deleted file mode 100644
> index 8538f157a0..0000000000
> --- a/sysdeps/unix/sysv/linux/s390/pthread_mutex_timedlock.c
> +++ /dev/null
> @@ -1,22 +0,0 @@
> -/* Elided version of pthread_mutex_timedlock.
> - Copyright (C) 2014-2021 Free Software Foundation, Inc.
> - This file is part of the GNU C Library.
> -
> - The GNU C Library is free software; you can redistribute it and/or
> - modify it under the terms of the GNU Lesser General Public
> - License as published by the Free Software Foundation; either
> - version 2.1 of the License, or (at your option) any later version.
> -
> - The GNU C Library is distributed in the hope that it will be useful,
> - but WITHOUT ANY WARRANTY; without even the implied warranty of
> - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> - Lesser General Public License for more details.
> -
> - You should have received a copy of the GNU Lesser General Public
> - License along with the GNU C Library; if not, see
> - <https://www.gnu.org/licenses/>. */
> -
> -#include <elision-conf.h>
> -#include <force-elision.h>
> -
> -#include <nptl/pthread_mutex_timedlock.c>
Ok.
> diff --git a/sysdeps/unix/sysv/linux/s390/pthread_mutex_trylock.c b/sysdeps/unix/sysv/linux/s390/pthread_mutex_trylock.c
> deleted file mode 100644
> index c4de1363cb..0000000000
> --- a/sysdeps/unix/sysv/linux/s390/pthread_mutex_trylock.c
> +++ /dev/null
> @@ -1,22 +0,0 @@
> -/* Elided version of pthread_mutex_trylock.
> - Copyright (C) 2014-2021 Free Software Foundation, Inc.
> - This file is part of the GNU C Library.
> -
> - The GNU C Library is free software; you can redistribute it and/or
> - modify it under the terms of the GNU Lesser General Public
> - License as published by the Free Software Foundation; either
> - version 2.1 of the License, or (at your option) any later version.
> -
> - The GNU C Library is distributed in the hope that it will be useful,
> - but WITHOUT ANY WARRANTY; without even the implied warranty of
> - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> - Lesser General Public License for more details.
> -
> - You should have received a copy of the GNU Lesser General Public
> - License along with the GNU C Library; if not, see
> - <https://www.gnu.org/licenses/>. */
> -
> -#include <elision-conf.h>
> -#include <force-elision.h>
> -
> -#include <nptl/pthread_mutex_trylock.c>
Ok.
> diff --git a/sysdeps/unix/sysv/linux/x86/Makefile b/sysdeps/unix/sysv/linux/x86/Makefile
> index 6bfd6bec49..9dfdd689a9 100644
> --- a/sysdeps/unix/sysv/linux/x86/Makefile
> +++ b/sysdeps/unix/sysv/linux/x86/Makefile
> @@ -12,8 +12,6 @@ sysdep_headers += sys/elf.h sys/perm.h sys/reg.h sys/vm86.h sys/debugreg.h sys/i
> endif
>
> ifeq ($(subdir),nptl)
> -libpthread-sysdep_routines += elision-lock elision-unlock elision-timed \
> - elision-trylock
> CFLAGS-elision-lock.c += -mrtm
> CFLAGS-elision-unlock.c += -mrtm
> CFLAGS-elision-timed.c += -mrtm
Ok.
> diff --git a/sysdeps/unix/sysv/linux/x86/elision-conf.c b/sysdeps/unix/sysv/linux/x86/elision-conf.c
> index e8ffe022f0..a5a4043d15 100644
> --- a/sysdeps/unix/sysv/linux/x86/elision-conf.c
> +++ b/sysdeps/unix/sysv/linux/x86/elision-conf.c
> @@ -48,13 +48,6 @@ struct elision_config __elision_aconf =
> .skip_trylock_internal_abort = 3,
> };
>
> -/* Force elision for all new locks. This is used to decide whether existing
> - DEFAULT locks should be automatically upgraded to elision in
> - pthread_mutex_lock(). Disabled for suid programs. Only used when elision
> - is available. */
> -
> -int __pthread_force_elision attribute_hidden = 0;
> -
> #if HAVE_TUNABLES
> static __always_inline void
> do_set_elision_enable (int32_t elision_enable)
Ok.
> @@ -98,10 +91,8 @@ TUNABLE_CALLBACK_FNDECL (skip_trylock_internal_abort, int32_t);
>
> /* Initialize elision. */
>
> -static void
> -elision_init (int argc __attribute__ ((unused)),
> - char **argv __attribute__ ((unused)),
> - char **environ)
> +void
> +__lll_elision_init (void)
> {
> #if HAVE_TUNABLES
> /* Elision depends on tunables and must be explicitly turned on by setting
> @@ -122,15 +113,3 @@ elision_init (int argc __attribute__ ((unused)),
> if (!__pthread_force_elision)
> __elision_aconf.retry_try_xbegin = 0; /* Disable elision on rwlocks. */
> }
> -
> -#ifdef SHARED
> -# define INIT_SECTION ".init_array"
> -#else
> -# define INIT_SECTION ".preinit_array"
> -#endif
> -
> -void (*const __pthread_init_array []) (int, char **, char **)
> - __attribute__ ((section (INIT_SECTION), aligned (sizeof (void *)))) =
> -{
> - &elision_init
> -};
Ok.
> diff --git a/sysdeps/unix/sysv/linux/x86/elision-conf.h b/sysdeps/unix/sysv/linux/x86/elision-conf.h
> index bc5e73bd10..78f06ae602 100644
> --- a/sysdeps/unix/sysv/linux/x86/elision-conf.h
> +++ b/sysdeps/unix/sysv/linux/x86/elision-conf.h
> @@ -21,6 +21,9 @@
> #include <pthread.h>
> #include <time.h>
>
> +#define HAVE_ELISION 1
> +#define ELISION_UNLOCK_NEEDS_ADAPT_COUNT 0
> +
> /* Should make sure there is no false sharing on this. */
>
> struct elision_config
> @@ -33,9 +36,4 @@ struct elision_config
>
> extern struct elision_config __elision_aconf attribute_hidden;
>
> -extern int __pthread_force_elision attribute_hidden;
> -
> -/* Tell the test suite to test elision for this architecture. */
> -#define HAVE_ELISION 1
> -
> #endif
Ok.
> diff --git a/sysdeps/unix/sysv/linux/x86/elision-lock.c b/sysdeps/unix/sysv/linux/x86/elision-lock.c
> index 16d4f6e4f1..ea5f373c67 100644
> --- a/sysdeps/unix/sysv/linux/x86/elision-lock.c
> +++ b/sysdeps/unix/sysv/linux/x86/elision-lock.c
> @@ -22,12 +22,6 @@
> #include "hle.h"
> #include <elision-conf.h>
>
> -#if !defined(LLL_LOCK) && !defined(EXTRAARG)
> -/* Make sure the configuration code is always linked in for static
> - libraries. */
> -#include "elision-conf.c"
> -#endif
> -
> #ifndef EXTRAARG
> #define EXTRAARG
> #endif
> @@ -105,3 +99,4 @@ __lll_lock_elision (int *futex, short *adapt_count, EXTRAARG int private)
> /* Use a normal lock here. */
> return LLL_LOCK ((*futex), private);
> }
> +libc_hidden_def (__lll_lock_elision)
Ok.
> diff --git a/sysdeps/unix/sysv/linux/x86/elision-trylock.c b/sysdeps/unix/sysv/linux/x86/elision-trylock.c
> index ef9aa23819..5e689f577f 100644
> --- a/sysdeps/unix/sysv/linux/x86/elision-trylock.c
> +++ b/sysdeps/unix/sysv/linux/x86/elision-trylock.c
> @@ -73,3 +73,4 @@ __lll_trylock_elision (int *futex, short *adapt_count)
>
> return lll_trylock (*futex);
> }
> +libc_hidden_def (__lll_trylock_elision)
Ok.
> diff --git a/sysdeps/unix/sysv/linux/x86/elision-unlock.c b/sysdeps/unix/sysv/linux/x86/elision-unlock.c
> index edd8b1fd08..eb877bc18c 100644
> --- a/sysdeps/unix/sysv/linux/x86/elision-unlock.c
> +++ b/sysdeps/unix/sysv/linux/x86/elision-unlock.c
> @@ -31,3 +31,4 @@ __lll_unlock_elision(int *lock, int private)
> lll_unlock ((*lock), private);
> return 0;
> }
> +libc_hidden_def (__lll_unlock_elision)
Ok.
> diff --git a/sysdeps/unix/sysv/linux/x86/force-elision.h b/sysdeps/unix/sysv/linux/x86/force-elision.h
> deleted file mode 100644
> index 3c0a0e0ac5..0000000000
> --- a/sysdeps/unix/sysv/linux/x86/force-elision.h
> +++ /dev/null
> @@ -1,62 +0,0 @@
> -/* force-elision.h: Automatic enabling of elision for mutexes
> - Copyright (C) 2013-2021 Free Software Foundation, Inc.
> - This file is part of the GNU C Library.
> -
> - The GNU C Library is free software; you can redistribute it and/or
> - modify it under the terms of the GNU Lesser General Public
> - License as published by the Free Software Foundation; either
> - version 2.1 of the License, or (at your option) any later version.
> -
> - The GNU C Library is distributed in the hope that it will be useful,
> - but WITHOUT ANY WARRANTY; without even the implied warranty of
> - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> - Lesser General Public License for more details.
> -
> - You should have received a copy of the GNU Lesser General Public
> - License along with the GNU C Library; if not, see
> - <https://www.gnu.org/licenses/>. */
> -
> -/* Automatically enable elision for existing user lock kinds. */
> -#define FORCE_ELISION(m, s) \
> - if (__pthread_force_elision) \
> - { \
> - /* See concurrency notes regarding __kind in \
> - struct __pthread_mutex_s in \
> - sysdeps/nptl/bits/thread-shared-types.h. \
> - \
> - There are the following cases for the kind of a mutex \
> - (The mask PTHREAD_MUTEX_ELISION_FLAGS_NP covers the flags \
> - PTHREAD_MUTEX_ELISION_NP and PTHREAD_MUTEX_NO_ELISION_NP where \
> - only one of both flags can be set): \
> - - both flags are not set: \
> - This is the first lock operation for this mutex. Enable \
> - elision as it is not enabled so far. \
> - Note: It can happen that multiple threads are calling e.g. \
> - pthread_mutex_lock at the same time as the first lock \
> - operation for this mutex. Then elision is enabled for this \
> - mutex by multiple threads. Storing with relaxed MO is enough \
> - as all threads will store the same new value for the kind of \
> - the mutex. But we have to ensure that we always use the \
> - elision path regardless if this thread has enabled elision or \
> - another one. \
> - \
> - - PTHREAD_MUTEX_ELISION_NP flag is set: \
> - Elision was already enabled for this mutex by a previous lock \
> - operation. See case above. Just use the elision path. \
> - \
> - - PTHREAD_MUTEX_NO_ELISION_NP flag is set: \
> - Elision was explicitly disabled by pthread_mutexattr_settype. \
> - Do not use the elision path. \
> - Note: The flag PTHREAD_MUTEX_NO_ELISION_NP will never be \
> - changed after mutex initialization. */ \
> - int mutex_kind = atomic_load_relaxed (&((m)->__data.__kind)); \
> - if ((mutex_kind & PTHREAD_MUTEX_ELISION_FLAGS_NP) == 0) \
> - { \
> - mutex_kind |= PTHREAD_MUTEX_ELISION_NP; \
> - atomic_store_relaxed (&((m)->__data.__kind), mutex_kind); \
> - } \
> - if ((mutex_kind & PTHREAD_MUTEX_ELISION_NP) != 0) \
> - { \
> - s; \
> - } \
> - }
Ok.
> diff --git a/sysdeps/unix/sysv/linux/x86/lowlevellock.h b/sysdeps/unix/sysv/linux/x86/lowlevellock.h
> index 435a190ec9..7205bcadfd 100644
> --- a/sysdeps/unix/sysv/linux/x86/lowlevellock.h
> +++ b/sysdeps/unix/sysv/linux/x86/lowlevellock.h
> @@ -82,30 +82,6 @@ __lll_cas_lock (int *futex)
> __lll_unlock (&(lock), private); \
> }))
>
> -extern int __lll_clocklock_elision (int *futex, short *adapt_count,
> - clockid_t clockid,
> - const struct __timespec64 *timeout,
> - int private) attribute_hidden;
> -
> -#define lll_clocklock_elision(futex, adapt_count, clockid, timeout, private) \
> - __lll_clocklock_elision (&(futex), &(adapt_count), clockid, timeout, private)
> -
> -extern int __lll_lock_elision (int *futex, short *adapt_count, int private)
> - attribute_hidden;
> -
> -extern int __lll_unlock_elision (int *lock, int private)
> - attribute_hidden;
> -
> -extern int __lll_trylock_elision (int *lock, short *adapt_count)
> - attribute_hidden;
> -
> -#define lll_lock_elision(futex, adapt_count, private) \
> - __lll_lock_elision (&(futex), &(adapt_count), private)
> -#define lll_unlock_elision(futex, adapt_count, private) \
> - __lll_unlock_elision (&(futex), private)
> -#define lll_trylock_elision(futex, adapt_count) \
> - __lll_trylock_elision (&(futex), &(adapt_count))
> -
> #endif /* !__ASSEMBLER__ */
>
> #endif /* lowlevellock.h */
Ok.
> diff --git a/sysdeps/unix/sysv/linux/x86/pthread_mutex_cond_lock.c b/sysdeps/unix/sysv/linux/x86/pthread_mutex_cond_lock.c
> deleted file mode 100644
> index 7604f8706f..0000000000
> --- a/sysdeps/unix/sysv/linux/x86/pthread_mutex_cond_lock.c
> +++ /dev/null
> @@ -1,22 +0,0 @@
> -/* Copyright (C) 2013-2021 Free Software Foundation, Inc.
> - This file is part of the GNU C Library.
> -
> - The GNU C Library is free software; you can redistribute it and/or
> - modify it under the terms of the GNU Lesser General Public
> - License as published by the Free Software Foundation; either
> - version 2.1 of the License, or (at your option) any later version.
> -
> - The GNU C Library is distributed in the hope that it will be useful,
> - but WITHOUT ANY WARRANTY; without even the implied warranty of
> - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> - Lesser General Public License for more details.
> -
> - You should have received a copy of the GNU Lesser General Public
> - License along with the GNU C Library; if not, see
> - <https://www.gnu.org/licenses/>. */
> -
> -/* The cond lock is not actually elided yet, but we still need to handle
> - already elided locks. */
> -#include <elision-conf.h>
> -
> -#include <nptl/pthread_mutex_cond_lock.c>
Ok.
> diff --git a/sysdeps/unix/sysv/linux/x86/pthread_mutex_timedlock.c b/sysdeps/unix/sysv/linux/x86/pthread_mutex_timedlock.c
> deleted file mode 100644
> index 3fbfe02690..0000000000
> --- a/sysdeps/unix/sysv/linux/x86/pthread_mutex_timedlock.c
> +++ /dev/null
> @@ -1,22 +0,0 @@
> -/* Elided version of pthread_mutex_timedlock.
> - Copyright (C) 2011-2021 Free Software Foundation, Inc.
> - This file is part of the GNU C Library.
> -
> - The GNU C Library is free software; you can redistribute it and/or
> - modify it under the terms of the GNU Lesser General Public
> - License as published by the Free Software Foundation; either
> - version 2.1 of the License, or (at your option) any later version.
> -
> - The GNU C Library is distributed in the hope that it will be useful,
> - but WITHOUT ANY WARRANTY; without even the implied warranty of
> - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> - Lesser General Public License for more details.
> -
> - You should have received a copy of the GNU Lesser General Public
> - License along with the GNU C Library; if not, see
> - <https://www.gnu.org/licenses/>. */
> -
> -#include <elision-conf.h>
> -#include "force-elision.h"
> -
> -#include "nptl/pthread_mutex_timedlock.c"
Ok.
> diff --git a/sysdeps/unix/sysv/linux/x86/pthread_mutex_trylock.c b/sysdeps/unix/sysv/linux/x86/pthread_mutex_trylock.c
> deleted file mode 100644
> index c7abb5ad4f..0000000000
> --- a/sysdeps/unix/sysv/linux/x86/pthread_mutex_trylock.c
> +++ /dev/null
> @@ -1,22 +0,0 @@
> -/* Elided version of pthread_mutex_trylock.
> - Copyright (C) 2011-2021 Free Software Foundation, Inc.
> - This file is part of the GNU C Library.
> -
> - The GNU C Library is free software; you can redistribute it and/or
> - modify it under the terms of the GNU Lesser General Public
> - License as published by the Free Software Foundation; either
> - version 2.1 of the License, or (at your option) any later version.
> -
> - The GNU C Library is distributed in the hope that it will be useful,
> - but WITHOUT ANY WARRANTY; without even the implied warranty of
> - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> - Lesser General Public License for more details.
> -
> - You should have received a copy of the GNU Lesser General Public
> - License along with the GNU C Library; if not, see
> - <https://www.gnu.org/licenses/>. */
> -
> -#include <elision-conf.h>
> -#include "force-elision.h"
> -
> -#include "nptl/pthread_mutex_trylock.c"
>
Ok.
next prev parent reply other threads:[~2021-02-22 16:53 UTC|newest]
Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-02-17 17:21 [PATCH 0/6] More libpthread removal preparations Florian Weimer
2021-02-17 17:21 ` [PATCH 1/6] nptl: Split libpthread-routines into one routine per line Florian Weimer
2021-02-22 14:23 ` Adhemerval Zanella
2021-02-17 17:21 ` [PATCH 2/6] nptl: Reformat Versions Florian Weimer
2021-02-22 16:31 ` Adhemerval Zanella
2021-02-17 17:21 ` [PATCH 3/6] nptl: Move futex-internal.c into main nptl directory Florian Weimer
2021-02-22 16:34 ` Adhemerval Zanella
2021-02-17 17:21 ` [PATCH 4/6] nptl: Move lowlevellock into libc Florian Weimer
2021-02-22 16:38 ` Adhemerval Zanella
2021-02-22 16:46 ` Florian Weimer
2021-02-17 17:21 ` [PATCH 5/6] nptl: Move futex-internal " Florian Weimer
2021-02-22 16:39 ` Adhemerval Zanella
2021-02-17 17:22 ` [PATCH 6/6] nptl: Move elision implementations " Florian Weimer
2021-02-22 16:53 ` Adhemerval Zanella [this message]
2021-02-22 17:02 ` Florian Weimer
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=6d274cde-bfed-49fb-5cda-cf438e18ad93@linaro.org \
--to=adhemerval.zanella@linaro.org \
--cc=fweimer@redhat.com \
--cc=libc-alpha@sourceware.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).