* [PATCH v2 0/4] Libatomic: Cleanup ifunc selector and aliasing
@ 2024-06-11 12:42 Victor Do Nascimento
2024-06-11 12:42 ` [PATCH v2 1/4] Libatomic: AArch64: Convert all lse128 assembly to .insn directives Victor Do Nascimento
` (4 more replies)
0 siblings, 5 replies; 6+ messages in thread
From: Victor Do Nascimento @ 2024-06-11 12:42 UTC (permalink / raw)
To: gcc-patches; +Cc: richard.sandiford, Richard.Earnshaw, Victor Do Nascimento
Changes in V2:
As explained in patch v2 1/4, it has become clear that the current
approach of querying assembler support for newer architectural
extensions at compile time is undesirable both from a maintainability
as well as a consistency standpoint - Different compiled versions of
Libatomic may have different features depending on the machine on
which they were built.
These issues make for difficult testing as the explosion in number of
`#ifdef' guards makes maintenance error-prone and the dependence on
binutils version means that, as well as deploying changes for testing
in a variety of target configurations, testing must also involve
compiling the library on an increasing number of host configurations,
meaning that the chance of bugs going undetected increases (as was
proved in the pre-commit CI which, due to the use of an older version
of Binutils, picked up on a runtime-error that had hitherto gone
unnoticed).
We therefore do away with the use of all assembly instructions
dependent on Binutils 2.42, choosing to replace them with `.inst's
instead. This eliminates the latent bug picked up by CI and will
ensure consistent builds of Libatomic across all versions of Binutils.
---
The recent introduction of the optional LSE128 and RCPC3 architectural
extensions to AArch64 has further led to the increased flexibility of
atomic support in the architecture, with many extensions providing
support for distinct atomic operations, each with different potential
applications in mind.
This has led to maintenance difficulties in Libatomic, in particular
regarding the way the ifunc selector is generated via a series of
macro expansions at compile-time.
Until now, irrespective of the atomic operation in question, all atomic
functions for a particular operand size were expected to have the same
number of ifunc alternatives, meaning that a one-size-fits-all
approach could reasonably be taken for the selector.
This meant that if, hypothetically, for a particular architecture and
operand size one particular atomic operation was to have 3 different
implementations associated with different extensions, libatomic would
likewise be required to present three ifunc alternatives for all other
atomic functions.
The consequence in the design choice was the unnecessary use of
function aliasing and the unwieldy code which resulted from this.
This patch series attempts to remediate this issue by making the
preprocessor macros defining the number of ifunc alternatives and
their respective selection functions dependent on the file importing
the ifunc selector-generating framework.
all files are given `LAT_<FILENAME>' macros, defined at the beginning
and undef'd at the end of the file. It is these macros that are
subsequently used to fine-tune the behaviors of `libatomic_i.h' and
`host-config.h'.
In particular, the definition of the `IFUNC_NCOND(N)' and
`IFUNC_COND_<n>' macros in host-config.h can now be guarded behind
these new file-specific macros, which ultimately control what the
`GEN_SELECTOR(X)' macro in `libatomic_i.h' expands to. As both of
these headers are imported once per file implementing some atomic
operation, fine-tuned control is now possible.
Regtested with both `--enable-gnu-indirect-function' and
`--disable-gnu-indirect-function' configurations on armv9.4-a target
with LRCPC3 and LSE128 support and without.
Victor Do Nascimento (4):
Libatomic: AArch64: Convert all lse128 assembly to .insn directives
Libatomic: Define per-file identifier macros
Libatomic: Make ifunc selector behavior contingent on importing file
Libatomic: Clean up AArch64 `atomic_16.S' implementation file
libatomic/acinclude.m4 | 18 -
libatomic/auto-config.h.in | 3 -
libatomic/cas_n.c | 2 +
libatomic/config/linux/aarch64/atomic_16.S | 511 +++++++++----------
libatomic/config/linux/aarch64/host-config.h | 35 +-
libatomic/configure | 43 --
libatomic/configure.ac | 3 -
libatomic/exch_n.c | 2 +
libatomic/fadd_n.c | 2 +
libatomic/fand_n.c | 2 +
libatomic/fence.c | 2 +
libatomic/fenv.c | 2 +
libatomic/fior_n.c | 2 +
libatomic/flag.c | 2 +
libatomic/fnand_n.c | 2 +
libatomic/fop_n.c | 2 +
libatomic/fsub_n.c | 2 +
libatomic/fxor_n.c | 2 +
libatomic/gcas.c | 2 +
libatomic/gexch.c | 2 +
libatomic/glfree.c | 2 +
libatomic/gload.c | 2 +
libatomic/gstore.c | 2 +
libatomic/load_n.c | 2 +
libatomic/store_n.c | 2 +
libatomic/tas_n.c | 2 +
26 files changed, 303 insertions(+), 350 deletions(-)
--
2.34.1
^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH v2 1/4] Libatomic: AArch64: Convert all lse128 assembly to .insn directives
2024-06-11 12:42 [PATCH v2 0/4] Libatomic: Cleanup ifunc selector and aliasing Victor Do Nascimento
@ 2024-06-11 12:42 ` Victor Do Nascimento
2024-06-11 12:42 ` [PATCH v2 2/4] Libatomic: Define per-file identifier macros Victor Do Nascimento
` (3 subsequent siblings)
4 siblings, 0 replies; 6+ messages in thread
From: Victor Do Nascimento @ 2024-06-11 12:42 UTC (permalink / raw)
To: gcc-patches; +Cc: richard.sandiford, Richard.Earnshaw, Victor Do Nascimento
Given the lack of support for the LSE128 instructions in all but the
the most up-to-date version of Binutils (2.42), having the build-time
test for assembler support for these instructions often leads to the
building of Libatomic without support for LSE128-dependent atomic
function implementations. This ultimately leads to different people
having different versions of Libatomic on their machines, depending on
which assembler was available at compilation time.
Furthermore, the conditional inclusion of these atomic function
implementations predicated on assembler support leads to a series of
`#if HAVE_FEAT_LSE128' guards scattered throughout the codebase and
the need for a series of aliases when the feature flag evaluates
to false. The preprocessor macro guards, together with the
conditional aliasing leads to code that is cumbersome to understand
and maintain.
Both of the issues highlighted above will only get worse with the
coming support for LRCPC3 atomics which under the current scheme will
also require build-time checks.
Consequently, a better option for both consistency across builds and
code cleanness is to make recourse to the `.inst' directive. By
replacing all novel assembly instructions for their hexadecimal
representation within `.inst's, we ensure that the Libatomic code is
both considerably cleaner and all machines build the same binary,
irrespective of binutils version available at compile time.
This patch therefore removes all configure checks for LSE128-support
in the assembler and all the guards and aliases that were associated
with `HAVE_FEAT_LSE128'
libatomic/ChangeLog:
* acinclude.m4 (LIBAT_TEST_FEAT_AARCH64_LSE128): Delete.
* auto-config.h.in (HAVE_FEAT_LSE128): Likewise
* config/linux/aarch64/atomic_16.S: Replace all LSE128
instructions with equivalent `.inst' directives.
(HAVE_FEAT_LSE128): Remove all references.
* configure: Regenerate.
* configure.ac: Remove call to LIBAT_TEST_FEAT_AARCH64_LSE128.
---
libatomic/acinclude.m4 | 18 -----
libatomic/auto-config.h.in | 3 -
libatomic/config/linux/aarch64/atomic_16.S | 76 +++++++++-------------
libatomic/configure | 43 ------------
libatomic/configure.ac | 3 -
5 files changed, 32 insertions(+), 111 deletions(-)
diff --git a/libatomic/acinclude.m4 b/libatomic/acinclude.m4
index 6d2e0b1c355..f35ab5b60a5 100644
--- a/libatomic/acinclude.m4
+++ b/libatomic/acinclude.m4
@@ -83,24 +83,6 @@ AC_DEFUN([LIBAT_TEST_ATOMIC_BUILTIN],[
])
])
-dnl
-dnl Test if the host assembler supports armv9.4-a LSE128 isns.
-dnl
-AC_DEFUN([LIBAT_TEST_FEAT_AARCH64_LSE128],[
- AC_CACHE_CHECK([for armv9.4-a LSE128 insn support],
- [libat_cv_have_feat_lse128],[
- AC_LANG_CONFTEST([AC_LANG_PROGRAM([],[asm(".arch armv9-a+lse128")])])
- if AC_TRY_EVAL(ac_compile); then
- eval libat_cv_have_feat_lse128=yes
- else
- eval libat_cv_have_feat_lse128=no
- fi
- rm -f conftest*
- ])
- LIBAT_DEFINE_YESNO([HAVE_FEAT_LSE128], [$libat_cv_have_feat_lse128],
- [Have LSE128 support for 16 byte integers.])
-])
-
dnl
dnl Test if we have __atomic_load and __atomic_store for mode $1, size $2
dnl
diff --git a/libatomic/auto-config.h.in b/libatomic/auto-config.h.in
index 7c78933b07d..ab3424a759e 100644
--- a/libatomic/auto-config.h.in
+++ b/libatomic/auto-config.h.in
@@ -105,9 +105,6 @@
/* Define to 1 if you have the <dlfcn.h> header file. */
#undef HAVE_DLFCN_H
-/* Have LSE128 support for 16 byte integers. */
-#undef HAVE_FEAT_LSE128
-
/* Define to 1 if you have the <fenv.h> header file. */
#undef HAVE_FENV_H
diff --git a/libatomic/config/linux/aarch64/atomic_16.S b/libatomic/config/linux/aarch64/atomic_16.S
index b63e97ac5a2..d6e71ba6e16 100644
--- a/libatomic/config/linux/aarch64/atomic_16.S
+++ b/libatomic/config/linux/aarch64/atomic_16.S
@@ -40,18 +40,9 @@
#include "auto-config.h"
-#if !HAVE_IFUNC
-# undef HAVE_FEAT_LSE128
-# define HAVE_FEAT_LSE128 0
-#endif
-
#define HAVE_FEAT_LSE2 HAVE_IFUNC
-#if HAVE_FEAT_LSE128
- .arch armv9-a+lse128
-#else
.arch armv8-a+lse
-#endif
#define LSE128(NAME) libat_##NAME##_i1
#define LSE2(NAME) libat_##NAME##_i2
@@ -226,7 +217,6 @@ ENTRY (exchange_16)
END (exchange_16)
-#if HAVE_FEAT_LSE128
ENTRY_FEAT (exchange_16, LSE128)
mov tmp0, x0
mov res0, in0
@@ -234,21 +224,23 @@ ENTRY_FEAT (exchange_16, LSE128)
cbnz w4, 1f
/* RELAXED. */
- swpp res0, res1, [tmp0]
+ /* swpp res0, res1, [tmp0] */
+ .inst 0x192180c0
ret
1:
cmp w4, ACQUIRE
b.hi 2f
/* ACQUIRE/CONSUME. */
- swppa res0, res1, [tmp0]
+ /* swppa res0, res1, [tmp0] */
+ .inst 0x19a180c0
ret
/* RELEASE/ACQ_REL/SEQ_CST. */
-2: swppal res0, res1, [tmp0]
+2: /* swppal res0, res1, [tmp0] */
+ .inst 0x19e180c0
ret
END_FEAT (exchange_16, LSE128)
-#endif
ENTRY (compare_exchange_16)
@@ -446,7 +438,6 @@ ENTRY (fetch_or_16)
END (fetch_or_16)
-#if HAVE_FEAT_LSE128
ENTRY_FEAT (fetch_or_16, LSE128)
mov tmp0, x0
mov res0, in0
@@ -454,21 +445,23 @@ ENTRY_FEAT (fetch_or_16, LSE128)
cbnz w4, 1f
/* RELAXED. */
- ldsetp res0, res1, [tmp0]
+ /* ldsetp res0, res1, [tmp0] */
+ .inst 0x192130c0
ret
1:
cmp w4, ACQUIRE
b.hi 2f
/* ACQUIRE/CONSUME. */
- ldsetpa res0, res1, [tmp0]
+ /* ldsetpa res0, res1, [tmp0] */
+ .inst 0x19a130c0
ret
/* RELEASE/ACQ_REL/SEQ_CST. */
-2: ldsetpal res0, res1, [tmp0]
+2: /* ldsetpal res0, res1, [tmp0] */
+ .inst 0x19e130c0
ret
END_FEAT (fetch_or_16, LSE128)
-#endif
ENTRY (or_fetch_16)
@@ -493,14 +486,14 @@ ENTRY (or_fetch_16)
END (or_fetch_16)
-#if HAVE_FEAT_LSE128
ENTRY_FEAT (or_fetch_16, LSE128)
cbnz w4, 1f
mov tmp0, in0
mov tmp1, in1
/* RELAXED. */
- ldsetp in0, in1, [x0]
+ /* ldsetp in0, in1, [x0] */
+ .inst 0x19233002
orr res0, in0, tmp0
orr res1, in1, tmp1
ret
@@ -509,18 +502,19 @@ ENTRY_FEAT (or_fetch_16, LSE128)
b.hi 2f
/* ACQUIRE/CONSUME. */
- ldsetpa in0, in1, [x0]
+ /* ldsetpa in0, in1, [x0] */
+ .inst 0x19a33002
orr res0, in0, tmp0
orr res1, in1, tmp1
ret
/* RELEASE/ACQ_REL/SEQ_CST. */
-2: ldsetpal in0, in1, [x0]
+2: /* ldsetpal in0, in1, [x0] */
+ .inst 0x19e33002
orr res0, in0, tmp0
orr res1, in1, tmp1
ret
END_FEAT (or_fetch_16, LSE128)
-#endif
ENTRY (fetch_and_16)
@@ -545,7 +539,6 @@ ENTRY (fetch_and_16)
END (fetch_and_16)
-#if HAVE_FEAT_LSE128
ENTRY_FEAT (fetch_and_16, LSE128)
mov tmp0, x0
mvn res0, in0
@@ -553,7 +546,8 @@ ENTRY_FEAT (fetch_and_16, LSE128)
cbnz w4, 1f
/* RELAXED. */
- ldclrp res0, res1, [tmp0]
+ /* ldclrp res0, res1, [tmp0] */
+ .inst 0x192110c0
ret
1:
@@ -561,14 +555,15 @@ ENTRY_FEAT (fetch_and_16, LSE128)
b.hi 2f
/* ACQUIRE/CONSUME. */
- ldclrpa res0, res1, [tmp0]
+ /* ldclrpa res0, res1, [tmp0] */
+ .inst 0x19a110c0
ret
/* RELEASE/ACQ_REL/SEQ_CST. */
-2: ldclrpal res0, res1, [tmp0]
+2: /* ldclrpal res0, res1, [tmp0] */
+ .inst 0x19e110c0
ret
END_FEAT (fetch_and_16, LSE128)
-#endif
ENTRY (and_fetch_16)
@@ -593,14 +588,14 @@ ENTRY (and_fetch_16)
END (and_fetch_16)
-#if HAVE_FEAT_LSE128
ENTRY_FEAT (and_fetch_16, LSE128)
mvn tmp0, in0
mvn tmp0, in1
cbnz w4, 1f
/* RELAXED. */
- ldclrp tmp0, tmp1, [x0]
+ /* ldclrp tmp0, tmp1, [x0] */
+ .inst 0x19271006
and res0, tmp0, in0
and res1, tmp1, in1
ret
@@ -610,18 +605,19 @@ ENTRY_FEAT (and_fetch_16, LSE128)
b.hi 2f
/* ACQUIRE/CONSUME. */
- ldclrpa tmp0, tmp1, [x0]
+ /* ldclrpa tmp0, tmp1, [x0] */
+ .inst 0x19a71006
and res0, tmp0, in0
and res1, tmp1, in1
ret
/* RELEASE/ACQ_REL/SEQ_CST. */
-2: ldclrpal tmp0, tmp1, [x5]
+2: /* ldclrpal tmp0, tmp1, [x5] */
+ .inst 0x19e710a6
and res0, tmp0, in0
and res1, tmp1, in1
ret
END_FEAT (and_fetch_16, LSE128)
-#endif
ENTRY (fetch_xor_16)
@@ -729,16 +725,9 @@ ENTRY (test_and_set_16)
END (test_and_set_16)
-/* Alias entry points which are the same in LSE2 and LSE128. */
-
#if HAVE_IFUNC
-# if !HAVE_FEAT_LSE128
-ALIAS (exchange_16, LSE128, LSE2)
-ALIAS (fetch_or_16, LSE128, LSE2)
-ALIAS (fetch_and_16, LSE128, LSE2)
-ALIAS (or_fetch_16, LSE128, LSE2)
-ALIAS (and_fetch_16, LSE128, LSE2)
-# endif
+
+/* Alias entry points which are the same in LSE2 and LSE128. */
ALIAS (load_16, LSE128, LSE2)
ALIAS (store_16, LSE128, LSE2)
ALIAS (compare_exchange_16, LSE128, LSE2)
@@ -753,7 +742,6 @@ ALIAS (nand_fetch_16, LSE128, LSE2)
ALIAS (test_and_set_16, LSE128, LSE2)
/* Alias entry points which are the same in baseline and LSE2. */
-
ALIAS (exchange_16, LSE2, CORE)
ALIAS (fetch_add_16, LSE2, CORE)
ALIAS (add_fetch_16, LSE2, CORE)
diff --git a/libatomic/configure b/libatomic/configure
index 32cb3ecac26..d579bab96f8 100755
--- a/libatomic/configure
+++ b/libatomic/configure
@@ -14697,49 +14697,6 @@ _ACEOF
-# Check for target-specific assembly-level support for atomic operations.
-
- { $as_echo "$as_me:${as_lineno-$LINENO}: checking for armv9.4-a LSE128 insn support" >&5
-$as_echo_n "checking for armv9.4-a LSE128 insn support... " >&6; }
-if ${libat_cv_have_feat_lse128+:} false; then :
- $as_echo_n "(cached) " >&6
-else
-
- cat confdefs.h - <<_ACEOF >conftest.$ac_ext
-/* end confdefs.h. */
-
-int
-main ()
-{
-asm(".arch armv9-a+lse128")
- ;
- return 0;
-}
-_ACEOF
- if { { eval echo "\"\$as_me\":${as_lineno-$LINENO}: \"$ac_compile\""; } >&5
- (eval $ac_compile) 2>&5
- ac_status=$?
- $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5
- test $ac_status = 0; }; then
- eval libat_cv_have_feat_lse128=yes
- else
- eval libat_cv_have_feat_lse128=no
- fi
- rm -f conftest*
-
-fi
-{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $libat_cv_have_feat_lse128" >&5
-$as_echo "$libat_cv_have_feat_lse128" >&6; }
-
- yesno=`echo $libat_cv_have_feat_lse128 | tr 'yesno' '1 0 '`
-
-cat >>confdefs.h <<_ACEOF
-#define HAVE_FEAT_LSE128 $yesno
-_ACEOF
-
-
-
-
{ $as_echo "$as_me:${as_lineno-$LINENO}: checking whether byte ordering is bigendian" >&5
$as_echo_n "checking whether byte ordering is bigendian... " >&6; }
if ${ac_cv_c_bigendian+:} false; then :
diff --git a/libatomic/configure.ac b/libatomic/configure.ac
index 85824fa7614..32a2cdb13ae 100644
--- a/libatomic/configure.ac
+++ b/libatomic/configure.ac
@@ -206,9 +206,6 @@ LIBAT_FORALL_MODES([LIBAT_HAVE_ATOMIC_CAS])
LIBAT_FORALL_MODES([LIBAT_HAVE_ATOMIC_FETCH_ADD])
LIBAT_FORALL_MODES([LIBAT_HAVE_ATOMIC_FETCH_OP])
-# Check for target-specific assembly-level support for atomic operations.
-LIBAT_TEST_FEAT_AARCH64_LSE128()
-
AC_C_BIGENDIAN
# I don't like the default behaviour of WORDS_BIGENDIAN undefined for LE.
AH_BOTTOM(
--
2.34.1
^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH v2 2/4] Libatomic: Define per-file identifier macros
2024-06-11 12:42 [PATCH v2 0/4] Libatomic: Cleanup ifunc selector and aliasing Victor Do Nascimento
2024-06-11 12:42 ` [PATCH v2 1/4] Libatomic: AArch64: Convert all lse128 assembly to .insn directives Victor Do Nascimento
@ 2024-06-11 12:42 ` Victor Do Nascimento
2024-06-11 12:42 ` [PATCH v2 3/4] Libatomic: Make ifunc selector behavior contingent on importing file Victor Do Nascimento
` (2 subsequent siblings)
4 siblings, 0 replies; 6+ messages in thread
From: Victor Do Nascimento @ 2024-06-11 12:42 UTC (permalink / raw)
To: gcc-patches; +Cc: richard.sandiford, Richard.Earnshaw, Victor Do Nascimento
In order to facilitate the fine-tuning of how `libatomic_i.h' and
`host-config.h' headers are used by different atomic functions, we
define distinct identifier macros for each file which, in implementing
atomic operations, imports these headers.
The idea is that different parts of these headers could then be
conditionally defined depending on the macros set by the file that
`#include'd them.
Given how it is possible that some file names are generic enough that
using them as-is for macro names (e.g. flag.c -> FLAG) may potentially
lead to name clashes with other macros, all file names first have LAT_
prepended to them such that, for example, flag.c is assigned the
LAT_FLAG macro.
Libatomic/ChangeLog:
* cas_n.c (LAT_CAS_N): New.
* exch_n.c (LAT_EXCH_N): Likewise.
* fadd_n.c (LAT_FADD_N): Likewise.
* fand_n.c (LAT_FAND_N): Likewise.
* fence.c (LAT_FENCE): Likewise.
* fenv.c (LAT_FENV): Likewise.
* fior_n.c (LAT_FIOR_N): Likewise.
* flag.c (LAT_FLAG): Likewise.
* fnand_n.c (LAT_FNAND_N): Likewise.
* fop_n.c (LAT_FOP_N): Likewise
* fsub_n.c (LAT_FSUB_N): Likewise.
* fxor_n.c (LAT_FXOR_N): Likewise.
* gcas.c (LAT_GCAS): Likewise.
* gexch.c (LAT_GEXCH): Likewise.
* glfree.c (LAT_GLFREE): Likewise.
* gload.c (LAT_GLOAD): Likewise.
* gstore.c (LAT_GSTORE): Likewise.
* load_n.c (LAT_LOAD_N): Likewise.
* store_n.c (LAT_STORE_N): Likewise.
* tas_n.c (LAT_TAS_N): Likewise.
---
libatomic/cas_n.c | 2 ++
libatomic/exch_n.c | 2 ++
libatomic/fadd_n.c | 2 ++
libatomic/fand_n.c | 2 ++
libatomic/fence.c | 2 ++
libatomic/fenv.c | 2 ++
libatomic/fior_n.c | 2 ++
libatomic/flag.c | 2 ++
libatomic/fnand_n.c | 2 ++
libatomic/fop_n.c | 2 ++
libatomic/fsub_n.c | 2 ++
libatomic/fxor_n.c | 2 ++
libatomic/gcas.c | 2 ++
libatomic/gexch.c | 2 ++
libatomic/glfree.c | 2 ++
libatomic/gload.c | 2 ++
libatomic/gstore.c | 2 ++
libatomic/load_n.c | 2 ++
libatomic/store_n.c | 2 ++
libatomic/tas_n.c | 2 ++
20 files changed, 40 insertions(+)
diff --git a/libatomic/cas_n.c b/libatomic/cas_n.c
index a080b990371..2a6357e48db 100644
--- a/libatomic/cas_n.c
+++ b/libatomic/cas_n.c
@@ -22,6 +22,7 @@
see the files COPYING3 and COPYING.RUNTIME respectively. If not, see
<http://www.gnu.org/licenses/>. */
+#define LAT_CAS_N
#include "libatomic_i.h"
@@ -122,3 +123,4 @@ SIZE(libat_compare_exchange) (UTYPE *mptr, UTYPE *eptr, UTYPE newval,
#endif
EXPORT_ALIAS (SIZE(compare_exchange));
+#undef LAT_CAS_N
diff --git a/libatomic/exch_n.c b/libatomic/exch_n.c
index e5ff80769b9..184d3de1009 100644
--- a/libatomic/exch_n.c
+++ b/libatomic/exch_n.c
@@ -22,6 +22,7 @@
see the files COPYING3 and COPYING.RUNTIME respectively. If not, see
<http://www.gnu.org/licenses/>. */
+#define LAT_EXCH_N
#include "libatomic_i.h"
@@ -126,3 +127,4 @@ SIZE(libat_exchange) (UTYPE *mptr, UTYPE newval, int smodel UNUSED)
#endif
EXPORT_ALIAS (SIZE(exchange));
+#undef LAT_EXCH_N
diff --git a/libatomic/fadd_n.c b/libatomic/fadd_n.c
index bc15b8bc0e6..32b75cec654 100644
--- a/libatomic/fadd_n.c
+++ b/libatomic/fadd_n.c
@@ -22,6 +22,7 @@
see the files COPYING3 and COPYING.RUNTIME respectively. If not, see
<http://www.gnu.org/licenses/>. */
+#define LAT_FADD_N
#include <libatomic_i.h>
#define NAME add
@@ -43,3 +44,4 @@
#endif
#include "fop_n.c"
+#undef LAT_FADD_N
diff --git a/libatomic/fand_n.c b/libatomic/fand_n.c
index ffe9ed8700f..9eab55bcd72 100644
--- a/libatomic/fand_n.c
+++ b/libatomic/fand_n.c
@@ -1,3 +1,5 @@
+#define LAT_FAND_N
#define NAME and
#define OP(X,Y) ((X) & (Y))
#include "fop_n.c"
+#undef LAT_FAND_N
diff --git a/libatomic/fence.c b/libatomic/fence.c
index a9b1e280c5a..4022194a57a 100644
--- a/libatomic/fence.c
+++ b/libatomic/fence.c
@@ -21,6 +21,7 @@
see the files COPYING3 and COPYING.RUNTIME respectively. If not, see
<http://www.gnu.org/licenses/>. */
+#define LAT_FENCE
#include "libatomic_i.h"
#include <stdatomic.h>
@@ -43,3 +44,4 @@ void
{
atomic_signal_fence (order);
}
+#undef LAT_FENCE
diff --git a/libatomic/fenv.c b/libatomic/fenv.c
index 41f187c1f85..dccad356a31 100644
--- a/libatomic/fenv.c
+++ b/libatomic/fenv.c
@@ -21,6 +21,7 @@
see the files COPYING3 and COPYING.RUNTIME respectively. If not, see
<http://www.gnu.org/licenses/>. */
+#define LAT_FENV
#include "libatomic_i.h"
#ifdef HAVE_FENV_H
@@ -70,3 +71,4 @@ __atomic_feraiseexcept (int excepts __attribute__ ((unused)))
}
#endif
}
+#undef LAT_FENV
diff --git a/libatomic/fior_n.c b/libatomic/fior_n.c
index 55d0d66b469..2b58d4805d6 100644
--- a/libatomic/fior_n.c
+++ b/libatomic/fior_n.c
@@ -1,3 +1,5 @@
+#define LAT_FIOR_N
#define NAME or
#define OP(X,Y) ((X) | (Y))
#include "fop_n.c"
+#undef LAT_FIOR_N
diff --git a/libatomic/flag.c b/libatomic/flag.c
index e4a5a27819a..8afd80c9130 100644
--- a/libatomic/flag.c
+++ b/libatomic/flag.c
@@ -21,6 +21,7 @@
see the files COPYING3 and COPYING.RUNTIME respectively. If not, see
<http://www.gnu.org/licenses/>. */
+#define LAT_FLAG
#include "libatomic_i.h"
#include <stdatomic.h>
@@ -62,3 +63,4 @@ void
{
return atomic_flag_clear_explicit (object, order);
}
+#undef LAT_FLAG
diff --git a/libatomic/fnand_n.c b/libatomic/fnand_n.c
index a3c98c70494..84a02709cbb 100644
--- a/libatomic/fnand_n.c
+++ b/libatomic/fnand_n.c
@@ -1,3 +1,5 @@
+#define LAT_FNAND_N
#define NAME nand
#define OP(X,Y) ~((X) & (Y))
#include "fop_n.c"
+#undef LAT_FNAND_N
diff --git a/libatomic/fop_n.c b/libatomic/fop_n.c
index f5eb07e859f..fefff3a57a4 100644
--- a/libatomic/fop_n.c
+++ b/libatomic/fop_n.c
@@ -22,6 +22,7 @@
see the files COPYING3 and COPYING.RUNTIME respectively. If not, see
<http://www.gnu.org/licenses/>. */
+#define LAT_FOP_N
#include <libatomic_i.h>
@@ -198,3 +199,4 @@ SIZE(C3(libat_,NAME,_fetch)) (UTYPE *mptr, UTYPE opval, int smodel UNUSED)
EXPORT_ALIAS (SIZE(C2(fetch_,NAME)));
EXPORT_ALIAS (SIZE(C2(NAME,_fetch)));
+#undef LAT_FOP_N
diff --git a/libatomic/fsub_n.c b/libatomic/fsub_n.c
index e9f8d7d25e1..49b375a543f 100644
--- a/libatomic/fsub_n.c
+++ b/libatomic/fsub_n.c
@@ -1,3 +1,5 @@
+#define LAT_FSUB_N
#define NAME sub
#define OP(X,Y) ((X) - (Y))
#include "fop_n.c"
+#undef LAT_FSUB_N
diff --git a/libatomic/fxor_n.c b/libatomic/fxor_n.c
index 0f2d9624127..d9a91bc3b23 100644
--- a/libatomic/fxor_n.c
+++ b/libatomic/fxor_n.c
@@ -1,3 +1,5 @@
+#define LAT_FXOR_N
#define NAME xor
#define OP(X,Y) ((X) ^ (Y))
#include "fop_n.c"
+#undef LAT_FXOR_N
diff --git a/libatomic/gcas.c b/libatomic/gcas.c
index 21d11305f1e..af4a5f5c5ee 100644
--- a/libatomic/gcas.c
+++ b/libatomic/gcas.c
@@ -22,6 +22,7 @@
see the files COPYING3 and COPYING.RUNTIME respectively. If not, see
<http://www.gnu.org/licenses/>. */
+#define LAT_GCAS
#include "libatomic_i.h"
@@ -118,3 +119,4 @@ libat_compare_exchange (size_t n, void *mptr, void *eptr, void *dptr,
}
EXPORT_ALIAS (compare_exchange);
+#undef LAT_GCAS
diff --git a/libatomic/gexch.c b/libatomic/gexch.c
index 6233759a2e8..afb054c0ef2 100644
--- a/libatomic/gexch.c
+++ b/libatomic/gexch.c
@@ -22,6 +22,7 @@
see the files COPYING3 and COPYING.RUNTIME respectively. If not, see
<http://www.gnu.org/licenses/>. */
+#define LAT_GEXCH
#include "libatomic_i.h"
@@ -142,3 +143,4 @@ libat_exchange (size_t n, void *mptr, void *vptr, void *rptr, int smodel)
}
EXPORT_ALIAS (exchange);
+#undef LAT_GEXCH
diff --git a/libatomic/glfree.c b/libatomic/glfree.c
index 58a45126194..1051ceb81cd 100644
--- a/libatomic/glfree.c
+++ b/libatomic/glfree.c
@@ -22,6 +22,7 @@
see the files COPYING3 and COPYING.RUNTIME respectively. If not, see
<http://www.gnu.org/licenses/>. */
+#define LAT_GLFREE
#include "libatomic_i.h"
/* Accesses with a power-of-two size are not lock-free if we don't have an
@@ -80,3 +81,4 @@ libat_is_lock_free (size_t n, void *ptr)
}
EXPORT_ALIAS (is_lock_free);
+#undef LAT_GLFREE
diff --git a/libatomic/gload.c b/libatomic/gload.c
index 4b3198cc5ae..9b499672161 100644
--- a/libatomic/gload.c
+++ b/libatomic/gload.c
@@ -22,6 +22,7 @@
see the files COPYING3 and COPYING.RUNTIME respectively. If not, see
<http://www.gnu.org/licenses/>. */
+#define LAT_GLOAD
#include "libatomic_i.h"
@@ -98,3 +99,4 @@ libat_load (size_t n, void *mptr, void *rptr, int smodel)
}
EXPORT_ALIAS (load);
+#undef LAT_GLOAD
diff --git a/libatomic/gstore.c b/libatomic/gstore.c
index 505a7b9b2df..b2636059bd8 100644
--- a/libatomic/gstore.c
+++ b/libatomic/gstore.c
@@ -22,6 +22,7 @@
see the files COPYING3 and COPYING.RUNTIME respectively. If not, see
<http://www.gnu.org/licenses/>. */
+#define LAT_GSTORE
#include "libatomic_i.h"
@@ -106,3 +107,4 @@ libat_store (size_t n, void *mptr, void *vptr, int smodel)
}
EXPORT_ALIAS (store);
+#undef LAT_GSTORE
diff --git a/libatomic/load_n.c b/libatomic/load_n.c
index 7513f191833..657c8e23ed2 100644
--- a/libatomic/load_n.c
+++ b/libatomic/load_n.c
@@ -22,6 +22,7 @@
see the files COPYING3 and COPYING.RUNTIME respectively. If not, see
<http://www.gnu.org/licenses/>. */
+#define LAT_LOAD_N
#include "libatomic_i.h"
@@ -113,3 +114,4 @@ SIZE(libat_load) (UTYPE *mptr, int smodel)
#endif
EXPORT_ALIAS (SIZE(load));
+#undef LAT_LOAD_N
diff --git a/libatomic/store_n.c b/libatomic/store_n.c
index d8ab5e69a50..079e22d75ba 100644
--- a/libatomic/store_n.c
+++ b/libatomic/store_n.c
@@ -22,6 +22,7 @@
see the files COPYING3 and COPYING.RUNTIME respectively. If not, see
<http://www.gnu.org/licenses/>. */
+#define LAT_STORE_N
#include "libatomic_i.h"
@@ -110,3 +111,4 @@ SIZE(libat_store) (UTYPE *mptr, UTYPE newval, int smodel)
#endif
EXPORT_ALIAS (SIZE(store));
+#undef LAT_STORE_N
diff --git a/libatomic/tas_n.c b/libatomic/tas_n.c
index 4a01cd2a5c8..9321b3a4e02 100644
--- a/libatomic/tas_n.c
+++ b/libatomic/tas_n.c
@@ -22,6 +22,7 @@
see the files COPYING3 and COPYING.RUNTIME respectively. If not, see
<http://www.gnu.org/licenses/>. */
+#define LAT_TAS_N
#include "libatomic_i.h"
@@ -113,3 +114,4 @@ SIZE(libat_test_and_set) (UTYPE *mptr, int smodel UNUSED)
#endif
EXPORT_ALIAS (SIZE(test_and_set));
+#undef LAT_TAS_N
--
2.34.1
^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH v2 3/4] Libatomic: Make ifunc selector behavior contingent on importing file
2024-06-11 12:42 [PATCH v2 0/4] Libatomic: Cleanup ifunc selector and aliasing Victor Do Nascimento
2024-06-11 12:42 ` [PATCH v2 1/4] Libatomic: AArch64: Convert all lse128 assembly to .insn directives Victor Do Nascimento
2024-06-11 12:42 ` [PATCH v2 2/4] Libatomic: Define per-file identifier macros Victor Do Nascimento
@ 2024-06-11 12:42 ` Victor Do Nascimento
2024-06-11 12:42 ` [PATCH v2 4/4] Libatomic: Clean up AArch64 `atomic_16.S' implementation file Victor Do Nascimento
2024-06-12 8:50 ` [PATCH v2 0/4] Libatomic: Cleanup ifunc selector and aliasing Richard Sandiford
4 siblings, 0 replies; 6+ messages in thread
From: Victor Do Nascimento @ 2024-06-11 12:42 UTC (permalink / raw)
To: gcc-patches; +Cc: richard.sandiford, Richard.Earnshaw, Victor Do Nascimento
By querying previously-defined file-identifier macros, `host-config.h'
is able to get information about its environment and, based on this
information, select more appropriate function-specific ifunc
selectors. This reduces the number of unnecessary feature tests that
need to be carried out in order to find the best atomic implementation
for a function at run-time.
An immediate benefit of this is that we can further fine-tune the
architectural requirements for each atomic function without risk of
incurring the maintenance and runtime-performance penalties of having
to maintain an ifunc selector with a huge number of alternatives, most
of which are irrelevant for any particular function. Consequently,
for AArch64 targets, we relax the architectural requirements of
`compare_exchange_16', which now requires only LSE as opposed to the
newer LSE2.
The new flexibility provided by this approach also means that certain
functions can now be called directly, doing away with ifunc selectors
altogether when only a single implementation is available for it on a
given target. As per the macro expansion framework laid out in
`libatomic_i.h', such functions should have their names prefixed with
`__atomic_' as opposed to `libat_'. This is the same prefix applied
to function names when Libatomic is configured with
`--disable-gnu-indirect-function'.
To achieve this, these functions unconditionally apply the aliasing
rule that at present is conditionally applied only when libatomic is
built without ifunc support, which ensures that the default
`libat_##NAME' is accessible via the equivalent `__atomic_##NAME' too.
This is ensured by using the new `ENTRY_ALIASED' macro.
Finally, this means we are able to do away with a whole set of
function aliases that were needed until now, thus considerably
cleaning up the implementation.
libatomic/ChangeLog:
* config/linux/aarch64/atomic_16.S: Remove unnecessary
aliasing.
(LSE): New.
(ENTRY_ALIASED): Likewise.
* config/linux/aarch64/host-config.h (LSE_ATOP): New.
(LSE2_ATOP): Likewise.
(LSE128_ATOP): Likewise.
(IFUNC_COND_1): Make its definition conditional on above 3
macros.
(IFUNC_NCOND): Likewise.
---
libatomic/config/linux/aarch64/atomic_16.S | 64 ++++++--------------
libatomic/config/linux/aarch64/host-config.h | 35 ++++++++---
2 files changed, 45 insertions(+), 54 deletions(-)
diff --git a/libatomic/config/linux/aarch64/atomic_16.S b/libatomic/config/linux/aarch64/atomic_16.S
index d6e71ba6e16..11a296dacc3 100644
--- a/libatomic/config/linux/aarch64/atomic_16.S
+++ b/libatomic/config/linux/aarch64/atomic_16.S
@@ -45,17 +45,20 @@
.arch armv8-a+lse
#define LSE128(NAME) libat_##NAME##_i1
-#define LSE2(NAME) libat_##NAME##_i2
+#define LSE(NAME) libat_##NAME##_i1
+#define LSE2(NAME) libat_##NAME##_i1
#define CORE(NAME) libat_##NAME
#define ATOMIC(NAME) __atomic_##NAME
+/* Emit __atomic_* entrypoints if no ifuncs. */
+#define ENTRY_ALIASED(NAME) ENTRY2 (CORE (NAME), ALIAS (NAME, ATOMIC, CORE))
+
#if HAVE_IFUNC
# define ENTRY(NAME) ENTRY2 (CORE (NAME), )
# define ENTRY_FEAT(NAME, FEAT) ENTRY2 (FEAT (NAME), )
# define END_FEAT(NAME, FEAT) END2 (FEAT (NAME))
#else
-/* Emit __atomic_* entrypoints if no ifuncs. */
-# define ENTRY(NAME) ENTRY2 (CORE (NAME), ALIAS (NAME, ATOMIC, CORE))
+# define ENTRY(NAME) ENTRY_ALIASED (NAME)
#endif
#define END(NAME) END2 (CORE (NAME))
@@ -291,7 +294,7 @@ END (compare_exchange_16)
#if HAVE_FEAT_LSE2
-ENTRY_FEAT (compare_exchange_16, LSE2)
+ENTRY_FEAT (compare_exchange_16, LSE)
ldp exp0, exp1, [x1]
mov tmp0, exp0
mov tmp1, exp1
@@ -324,11 +327,11 @@ ENTRY_FEAT (compare_exchange_16, LSE2)
/* ACQ_REL/SEQ_CST. */
4: caspal exp0, exp1, in0, in1, [x0]
b 0b
-END_FEAT (compare_exchange_16, LSE2)
+END_FEAT (compare_exchange_16, LSE)
#endif
-ENTRY (fetch_add_16)
+ENTRY_ALIASED (fetch_add_16)
mov x5, x0
cbnz w4, 2f
@@ -350,7 +353,7 @@ ENTRY (fetch_add_16)
END (fetch_add_16)
-ENTRY (add_fetch_16)
+ENTRY_ALIASED (add_fetch_16)
mov x5, x0
cbnz w4, 2f
@@ -372,7 +375,7 @@ ENTRY (add_fetch_16)
END (add_fetch_16)
-ENTRY (fetch_sub_16)
+ENTRY_ALIASED (fetch_sub_16)
mov x5, x0
cbnz w4, 2f
@@ -394,7 +397,7 @@ ENTRY (fetch_sub_16)
END (fetch_sub_16)
-ENTRY (sub_fetch_16)
+ENTRY_ALIASED (sub_fetch_16)
mov x5, x0
cbnz w4, 2f
@@ -620,7 +623,7 @@ ENTRY_FEAT (and_fetch_16, LSE128)
END_FEAT (and_fetch_16, LSE128)
-ENTRY (fetch_xor_16)
+ENTRY_ALIASED (fetch_xor_16)
mov x5, x0
cbnz w4, 2f
@@ -642,7 +645,7 @@ ENTRY (fetch_xor_16)
END (fetch_xor_16)
-ENTRY (xor_fetch_16)
+ENTRY_ALIASED (xor_fetch_16)
mov x5, x0
cbnz w4, 2f
@@ -664,7 +667,7 @@ ENTRY (xor_fetch_16)
END (xor_fetch_16)
-ENTRY (fetch_nand_16)
+ENTRY_ALIASED (fetch_nand_16)
mov x5, x0
mvn in0, in0
mvn in1, in1
@@ -688,7 +691,7 @@ ENTRY (fetch_nand_16)
END (fetch_nand_16)
-ENTRY (nand_fetch_16)
+ENTRY_ALIASED (nand_fetch_16)
mov x5, x0
mvn in0, in0
mvn in1, in1
@@ -714,7 +717,7 @@ END (nand_fetch_16)
/* __atomic_test_and_set is always inlined, so this entry is unused and
only required for completeness. */
-ENTRY (test_and_set_16)
+ENTRY_ALIASED (test_and_set_16)
/* RELAXED/ACQUIRE/CONSUME/RELEASE/ACQ_REL/SEQ_CST. */
mov x5, x0
@@ -725,39 +728,6 @@ ENTRY (test_and_set_16)
END (test_and_set_16)
-#if HAVE_IFUNC
-
-/* Alias entry points which are the same in LSE2 and LSE128. */
-ALIAS (load_16, LSE128, LSE2)
-ALIAS (store_16, LSE128, LSE2)
-ALIAS (compare_exchange_16, LSE128, LSE2)
-ALIAS (fetch_add_16, LSE128, LSE2)
-ALIAS (add_fetch_16, LSE128, LSE2)
-ALIAS (fetch_sub_16, LSE128, LSE2)
-ALIAS (sub_fetch_16, LSE128, LSE2)
-ALIAS (fetch_xor_16, LSE128, LSE2)
-ALIAS (xor_fetch_16, LSE128, LSE2)
-ALIAS (fetch_nand_16, LSE128, LSE2)
-ALIAS (nand_fetch_16, LSE128, LSE2)
-ALIAS (test_and_set_16, LSE128, LSE2)
-
-/* Alias entry points which are the same in baseline and LSE2. */
-ALIAS (exchange_16, LSE2, CORE)
-ALIAS (fetch_add_16, LSE2, CORE)
-ALIAS (add_fetch_16, LSE2, CORE)
-ALIAS (fetch_sub_16, LSE2, CORE)
-ALIAS (sub_fetch_16, LSE2, CORE)
-ALIAS (fetch_or_16, LSE2, CORE)
-ALIAS (or_fetch_16, LSE2, CORE)
-ALIAS (fetch_and_16, LSE2, CORE)
-ALIAS (and_fetch_16, LSE2, CORE)
-ALIAS (fetch_xor_16, LSE2, CORE)
-ALIAS (xor_fetch_16, LSE2, CORE)
-ALIAS (fetch_nand_16, LSE2, CORE)
-ALIAS (nand_fetch_16, LSE2, CORE)
-ALIAS (test_and_set_16, LSE2, CORE)
-#endif
-
/* GNU_PROPERTY_AARCH64_* macros from elf.h for use in asm code. */
#define FEATURE_1_AND 0xc0000000
#define FEATURE_1_BTI 1
diff --git a/libatomic/config/linux/aarch64/host-config.h b/libatomic/config/linux/aarch64/host-config.h
index e1a699948f4..d05e9eb628f 100644
--- a/libatomic/config/linux/aarch64/host-config.h
+++ b/libatomic/config/linux/aarch64/host-config.h
@@ -48,15 +48,36 @@ typedef struct __ifunc_arg_t {
# define _IFUNC_ARG_HWCAP (1ULL << 62)
#endif
-#if N == 16
-# define IFUNC_COND_1 (has_lse128 (hwcap, features))
-# define IFUNC_COND_2 (has_lse2 (hwcap, features))
-# define IFUNC_NCOND(N) 2
-#else
-# define IFUNC_COND_1 (hwcap & HWCAP_ATOMICS)
-# define IFUNC_NCOND(N) 1
+/* From the file which imported `host-config.h' we can ascertain which
+ architectural extension provides relevant atomic support. From this,
+ we can proceed to tweak the ifunc selector behavior. */
+#if defined (LAT_CAS_N)
+# define LSE_ATOP
+#elif defined (LAT_LOAD_N) || defined (LAT_STORE_N)
+# define LSE2_ATOP
+#elif defined (LAT_EXCH_N) || defined (LAT_FIOR_N) || defined (LAT_FAND_N)
+# define LSE128_ATOP
#endif
+# if N == 16
+# if defined (LSE_ATOP)
+# define IFUNC_NCOND(N) 1
+# define IFUNC_COND_1 (hwcap & HWCAP_ATOMICS)
+# elif defined (LSE2_ATOP)
+# define IFUNC_NCOND(N) 1
+# define IFUNC_COND_1 (has_lse2 (hwcap, features))
+# elif defined (LSE128_ATOP)
+# define IFUNC_NCOND(N) 1
+# define IFUNC_COND_1 (has_lse128 (hwcap, features))
+# else
+# define IFUNC_NCOND(N) 0
+# define IFUNC_ALT 1
+# endif
+# else
+# define IFUNC_COND_1 (hwcap & HWCAP_ATOMICS)
+# define IFUNC_NCOND(N) 1
+# endif
+
#define MIDR_IMPLEMENTOR(midr) (((midr) >> 24) & 255)
#define MIDR_PARTNUM(midr) (((midr) >> 4) & 0xfff)
--
2.34.1
^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH v2 4/4] Libatomic: Clean up AArch64 `atomic_16.S' implementation file
2024-06-11 12:42 [PATCH v2 0/4] Libatomic: Cleanup ifunc selector and aliasing Victor Do Nascimento
` (2 preceding siblings ...)
2024-06-11 12:42 ` [PATCH v2 3/4] Libatomic: Make ifunc selector behavior contingent on importing file Victor Do Nascimento
@ 2024-06-11 12:42 ` Victor Do Nascimento
2024-06-12 8:50 ` [PATCH v2 0/4] Libatomic: Cleanup ifunc selector and aliasing Richard Sandiford
4 siblings, 0 replies; 6+ messages in thread
From: Victor Do Nascimento @ 2024-06-11 12:42 UTC (permalink / raw)
To: gcc-patches; +Cc: richard.sandiford, Richard.Earnshaw, Victor Do Nascimento
At present, `atomic_16.S' groups different implementations of the
same functions together in the file. Therefore, as an example,
the LSE2 implementation of `load_16' follows on immediately from its
core implementation, as does the `store_16' LSE2 implementation.
Such architectural extension-dependent implementations are dependent
on ifunc support, such that they are guarded by the relevant
preprocessor macro, i.e. `#if HAVE_IFUNC'.
Having to apply these guards on a per-function basis adds unnecessary
clutter to the file and makes its maintenance more error-prone.
We therefore reorganize the layout of the file in such a way that all
core implementations needing no `#ifdef's are placed first, followed
by all ifunc-dependent implementations, which can all be guarded by a
single `#if HAVE_IFUNC', greatly reducing the overall number of
required `#ifdef' macros.
libatomic/ChangeLog:
* config/linux/aarch64/atomic_16.S: Reorganize functions in
file.
(HAVE_FEAT_LSE2): Delete.
---
libatomic/config/linux/aarch64/atomic_16.S | 445 +++++++++++----------
1 file changed, 223 insertions(+), 222 deletions(-)
diff --git a/libatomic/config/linux/aarch64/atomic_16.S b/libatomic/config/linux/aarch64/atomic_16.S
index 11a296dacc3..c44c31c6418 100644
--- a/libatomic/config/linux/aarch64/atomic_16.S
+++ b/libatomic/config/linux/aarch64/atomic_16.S
@@ -40,8 +40,6 @@
#include "auto-config.h"
-#define HAVE_FEAT_LSE2 HAVE_IFUNC
-
.arch armv8-a+lse
#define LSE128(NAME) libat_##NAME##_i1
@@ -116,6 +114,9 @@ NAME: \
#define SEQ_CST 5
+/* Core implementations: Not dependent on the presence of further architectural
+ extensions. */
+
ENTRY (load_16)
mov x5, x0
cbnz w1, 2f
@@ -134,31 +135,6 @@ ENTRY (load_16)
END (load_16)
-#if HAVE_FEAT_LSE2
-ENTRY_FEAT (load_16, LSE2)
- cbnz w1, 1f
-
- /* RELAXED. */
- ldp res0, res1, [x0]
- ret
-1:
- cmp w1, SEQ_CST
- b.eq 2f
-
- /* ACQUIRE/CONSUME (Load-AcquirePC semantics). */
- ldp res0, res1, [x0]
- dmb ishld
- ret
-
- /* SEQ_CST. */
-2: ldar tmp0, [x0] /* Block reordering with Store-Release instr. */
- ldp res0, res1, [x0]
- dmb ishld
- ret
-END_FEAT (load_16, LSE2)
-#endif
-
-
ENTRY (store_16)
cbnz w4, 2f
@@ -176,23 +152,6 @@ ENTRY (store_16)
END (store_16)
-#if HAVE_FEAT_LSE2
-ENTRY_FEAT (store_16, LSE2)
- cbnz w4, 1f
-
- /* RELAXED. */
- stp in0, in1, [x0]
- ret
-
- /* RELEASE/SEQ_CST. */
-1: ldxp xzr, tmp0, [x0]
- stlxp w4, in0, in1, [x0]
- cbnz w4, 1b
- ret
-END_FEAT (store_16, LSE2)
-#endif
-
-
ENTRY (exchange_16)
mov x5, x0
cbnz w4, 2f
@@ -220,32 +179,6 @@ ENTRY (exchange_16)
END (exchange_16)
-ENTRY_FEAT (exchange_16, LSE128)
- mov tmp0, x0
- mov res0, in0
- mov res1, in1
- cbnz w4, 1f
-
- /* RELAXED. */
- /* swpp res0, res1, [tmp0] */
- .inst 0x192180c0
- ret
-1:
- cmp w4, ACQUIRE
- b.hi 2f
-
- /* ACQUIRE/CONSUME. */
- /* swppa res0, res1, [tmp0] */
- .inst 0x19a180c0
- ret
-
- /* RELEASE/ACQ_REL/SEQ_CST. */
-2: /* swppal res0, res1, [tmp0] */
- .inst 0x19e180c0
- ret
-END_FEAT (exchange_16, LSE128)
-
-
ENTRY (compare_exchange_16)
ldp exp0, exp1, [x1]
cbz w4, 3f
@@ -293,42 +226,6 @@ ENTRY (compare_exchange_16)
END (compare_exchange_16)
-#if HAVE_FEAT_LSE2
-ENTRY_FEAT (compare_exchange_16, LSE)
- ldp exp0, exp1, [x1]
- mov tmp0, exp0
- mov tmp1, exp1
- cbz w4, 2f
- cmp w4, RELEASE
- b.hs 3f
-
- /* ACQUIRE/CONSUME. */
- caspa exp0, exp1, in0, in1, [x0]
-0:
- cmp exp0, tmp0
- ccmp exp1, tmp1, 0, eq
- bne 1f
- mov x0, 1
- ret
-1:
- stp exp0, exp1, [x1]
- mov x0, 0
- ret
-
- /* RELAXED. */
-2: casp exp0, exp1, in0, in1, [x0]
- b 0b
-
- /* RELEASE. */
-3: b.hi 4f
- caspl exp0, exp1, in0, in1, [x0]
- b 0b
-
- /* ACQ_REL/SEQ_CST. */
-4: caspal exp0, exp1, in0, in1, [x0]
- b 0b
-END_FEAT (compare_exchange_16, LSE)
-#endif
ENTRY_ALIASED (fetch_add_16)
@@ -441,32 +338,6 @@ ENTRY (fetch_or_16)
END (fetch_or_16)
-ENTRY_FEAT (fetch_or_16, LSE128)
- mov tmp0, x0
- mov res0, in0
- mov res1, in1
- cbnz w4, 1f
-
- /* RELAXED. */
- /* ldsetp res0, res1, [tmp0] */
- .inst 0x192130c0
- ret
-1:
- cmp w4, ACQUIRE
- b.hi 2f
-
- /* ACQUIRE/CONSUME. */
- /* ldsetpa res0, res1, [tmp0] */
- .inst 0x19a130c0
- ret
-
- /* RELEASE/ACQ_REL/SEQ_CST. */
-2: /* ldsetpal res0, res1, [tmp0] */
- .inst 0x19e130c0
- ret
-END_FEAT (fetch_or_16, LSE128)
-
-
ENTRY (or_fetch_16)
mov x5, x0
cbnz w4, 2f
@@ -489,37 +360,6 @@ ENTRY (or_fetch_16)
END (or_fetch_16)
-ENTRY_FEAT (or_fetch_16, LSE128)
- cbnz w4, 1f
- mov tmp0, in0
- mov tmp1, in1
-
- /* RELAXED. */
- /* ldsetp in0, in1, [x0] */
- .inst 0x19233002
- orr res0, in0, tmp0
- orr res1, in1, tmp1
- ret
-1:
- cmp w4, ACQUIRE
- b.hi 2f
-
- /* ACQUIRE/CONSUME. */
- /* ldsetpa in0, in1, [x0] */
- .inst 0x19a33002
- orr res0, in0, tmp0
- orr res1, in1, tmp1
- ret
-
- /* RELEASE/ACQ_REL/SEQ_CST. */
-2: /* ldsetpal in0, in1, [x0] */
- .inst 0x19e33002
- orr res0, in0, tmp0
- orr res1, in1, tmp1
- ret
-END_FEAT (or_fetch_16, LSE128)
-
-
ENTRY (fetch_and_16)
mov x5, x0
cbnz w4, 2f
@@ -542,33 +382,6 @@ ENTRY (fetch_and_16)
END (fetch_and_16)
-ENTRY_FEAT (fetch_and_16, LSE128)
- mov tmp0, x0
- mvn res0, in0
- mvn res1, in1
- cbnz w4, 1f
-
- /* RELAXED. */
- /* ldclrp res0, res1, [tmp0] */
- .inst 0x192110c0
- ret
-
-1:
- cmp w4, ACQUIRE
- b.hi 2f
-
- /* ACQUIRE/CONSUME. */
- /* ldclrpa res0, res1, [tmp0] */
- .inst 0x19a110c0
- ret
-
- /* RELEASE/ACQ_REL/SEQ_CST. */
-2: /* ldclrpal res0, res1, [tmp0] */
- .inst 0x19e110c0
- ret
-END_FEAT (fetch_and_16, LSE128)
-
-
ENTRY (and_fetch_16)
mov x5, x0
cbnz w4, 2f
@@ -591,38 +404,6 @@ ENTRY (and_fetch_16)
END (and_fetch_16)
-ENTRY_FEAT (and_fetch_16, LSE128)
- mvn tmp0, in0
- mvn tmp0, in1
- cbnz w4, 1f
-
- /* RELAXED. */
- /* ldclrp tmp0, tmp1, [x0] */
- .inst 0x19271006
- and res0, tmp0, in0
- and res1, tmp1, in1
- ret
-
-1:
- cmp w4, ACQUIRE
- b.hi 2f
-
- /* ACQUIRE/CONSUME. */
- /* ldclrpa tmp0, tmp1, [x0] */
- .inst 0x19a71006
- and res0, tmp0, in0
- and res1, tmp1, in1
- ret
-
- /* RELEASE/ACQ_REL/SEQ_CST. */
-2: /* ldclrpal tmp0, tmp1, [x5] */
- .inst 0x19e710a6
- and res0, tmp0, in0
- and res1, tmp1, in1
- ret
-END_FEAT (and_fetch_16, LSE128)
-
-
ENTRY_ALIASED (fetch_xor_16)
mov x5, x0
cbnz w4, 2f
@@ -728,6 +509,226 @@ ENTRY_ALIASED (test_and_set_16)
END (test_and_set_16)
+#if HAVE_IFUNC
+/* ifunc implementations: Carries run-time dependence on the presence of further
+ architectural extensions. */
+
+ENTRY_FEAT (exchange_16, LSE128)
+ mov tmp0, x0
+ mov res0, in0
+ mov res1, in1
+ cbnz w4, 1f
+
+ /* RELAXED. */
+ /* swpp res0, res1, [tmp0] */
+ .inst 0x192180c0
+ ret
+1:
+ cmp w4, ACQUIRE
+ b.hi 2f
+
+ /* ACQUIRE/CONSUME. */
+ /* swppa res0, res1, [tmp0] */
+ .inst 0x19a180c0
+ ret
+
+ /* RELEASE/ACQ_REL/SEQ_CST. */
+2: /* swppal res0, res1, [tmp0] */
+ .inst 0x19e180c0
+ ret
+END_FEAT (exchange_16, LSE128)
+
+
+ENTRY_FEAT (fetch_or_16, LSE128)
+ mov tmp0, x0
+ mov res0, in0
+ mov res1, in1
+ cbnz w4, 1f
+
+ /* RELAXED. */
+ /* ldsetp res0, res1, [tmp0] */
+ .inst 0x192130c0
+ ret
+1:
+ cmp w4, ACQUIRE
+ b.hi 2f
+
+ /* ACQUIRE/CONSUME. */
+ /* ldsetpa res0, res1, [tmp0] */
+ .inst 0x19a130c0
+ ret
+
+ /* RELEASE/ACQ_REL/SEQ_CST. */
+2: /* ldsetpal res0, res1, [tmp0] */
+ .inst 0x19e130c0
+ ret
+END_FEAT (fetch_or_16, LSE128)
+
+
+ENTRY_FEAT (or_fetch_16, LSE128)
+ cbnz w4, 1f
+ mov tmp0, in0
+ mov tmp1, in1
+
+ /* RELAXED. */
+ /* ldsetp in0, in1, [x0] */
+ .inst 0x19233002
+ orr res0, in0, tmp0
+ orr res1, in1, tmp1
+ ret
+1:
+ cmp w4, ACQUIRE
+ b.hi 2f
+
+ /* ACQUIRE/CONSUME. */
+ /* ldsetpa in0, in1, [x0] */
+ .inst 0x19a33002
+ orr res0, in0, tmp0
+ orr res1, in1, tmp1
+ ret
+
+ /* RELEASE/ACQ_REL/SEQ_CST. */
+2: /* ldsetpal in0, in1, [x0] */
+ .inst 0x19e33002
+ orr res0, in0, tmp0
+ orr res1, in1, tmp1
+ ret
+END_FEAT (or_fetch_16, LSE128)
+
+
+ENTRY_FEAT (fetch_and_16, LSE128)
+ mov tmp0, x0
+ mvn res0, in0
+ mvn res1, in1
+ cbnz w4, 1f
+
+ /* RELAXED. */
+ /* ldclrp res0, res1, [tmp0] */
+ .inst 0x192110c0
+ ret
+
+1:
+ cmp w4, ACQUIRE
+ b.hi 2f
+
+ /* ACQUIRE/CONSUME. */
+ /* ldclrpa res0, res1, [tmp0] */
+ .inst 0x19a110c0
+ ret
+
+ /* RELEASE/ACQ_REL/SEQ_CST. */
+2: /* ldclrpal res0, res1, [tmp0] */
+ .inst 0x19e110c0
+ ret
+END_FEAT (fetch_and_16, LSE128)
+
+
+ENTRY_FEAT (and_fetch_16, LSE128)
+ mvn tmp0, in0
+ mvn tmp0, in1
+ cbnz w4, 1f
+
+ /* RELAXED. */
+ /* ldclrp tmp0, tmp1, [x0] */
+ .inst 0x19271006
+ and res0, tmp0, in0
+ and res1, tmp1, in1
+ ret
+
+1:
+ cmp w4, ACQUIRE
+ b.hi 2f
+
+ /* ACQUIRE/CONSUME. */
+ /* ldclrpa tmp0, tmp1, [x0] */
+ .inst 0x19a71006
+ and res0, tmp0, in0
+ and res1, tmp1, in1
+ ret
+
+ /* RELEASE/ACQ_REL/SEQ_CST. */
+2: /* ldclrpal tmp0, tmp1, [x5] */
+ .inst 0x19e710a6
+ and res0, tmp0, in0
+ and res1, tmp1, in1
+ ret
+END_FEAT (and_fetch_16, LSE128)
+
+
+ENTRY_FEAT (load_16, LSE2)
+ cbnz w1, 1f
+
+ /* RELAXED. */
+ ldp res0, res1, [x0]
+ ret
+1:
+ cmp w1, SEQ_CST
+ b.eq 2f
+
+ /* ACQUIRE/CONSUME (Load-AcquirePC semantics). */
+ ldp res0, res1, [x0]
+ dmb ishld
+ ret
+
+ /* SEQ_CST. */
+2: ldar tmp0, [x0] /* Block reordering with Store-Release instr. */
+ ldp res0, res1, [x0]
+ dmb ishld
+ ret
+END_FEAT (load_16, LSE2)
+
+
+ENTRY_FEAT (store_16, LSE2)
+ cbnz w4, 1f
+
+ /* RELAXED. */
+ stp in0, in1, [x0]
+ ret
+
+ /* RELEASE/SEQ_CST. */
+1: ldxp xzr, tmp0, [x0]
+ stlxp w4, in0, in1, [x0]
+ cbnz w4, 1b
+ ret
+END_FEAT (store_16, LSE2)
+
+
+ENTRY_FEAT (compare_exchange_16, LSE)
+ ldp exp0, exp1, [x1]
+ mov tmp0, exp0
+ mov tmp1, exp1
+ cbz w4, 2f
+ cmp w4, RELEASE
+ b.hs 3f
+
+ /* ACQUIRE/CONSUME. */
+ caspa exp0, exp1, in0, in1, [x0]
+0:
+ cmp exp0, tmp0
+ ccmp exp1, tmp1, 0, eq
+ bne 1f
+ mov x0, 1
+ ret
+1:
+ stp exp0, exp1, [x1]
+ mov x0, 0
+ ret
+
+ /* RELAXED. */
+2: casp exp0, exp1, in0, in1, [x0]
+ b 0b
+
+ /* RELEASE. */
+3: b.hi 4f
+ caspl exp0, exp1, in0, in1, [x0]
+ b 0b
+
+ /* ACQ_REL/SEQ_CST. */
+4: caspal exp0, exp1, in0, in1, [x0]
+ b 0b
+END_FEAT (compare_exchange_16, LSE)
+#endif
+
/* GNU_PROPERTY_AARCH64_* macros from elf.h for use in asm code. */
#define FEATURE_1_AND 0xc0000000
#define FEATURE_1_BTI 1
--
2.34.1
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH v2 0/4] Libatomic: Cleanup ifunc selector and aliasing
2024-06-11 12:42 [PATCH v2 0/4] Libatomic: Cleanup ifunc selector and aliasing Victor Do Nascimento
` (3 preceding siblings ...)
2024-06-11 12:42 ` [PATCH v2 4/4] Libatomic: Clean up AArch64 `atomic_16.S' implementation file Victor Do Nascimento
@ 2024-06-12 8:50 ` Richard Sandiford
4 siblings, 0 replies; 6+ messages in thread
From: Richard Sandiford @ 2024-06-12 8:50 UTC (permalink / raw)
To: Victor Do Nascimento; +Cc: gcc-patches, Richard.Earnshaw
Victor Do Nascimento <victor.donascimento@arm.com> writes:
> Changes in V2:
>
> As explained in patch v2 1/4, it has become clear that the current
> approach of querying assembler support for newer architectural
> extensions at compile time is undesirable both from a maintainability
> as well as a consistency standpoint - Different compiled versions of
> Libatomic may have different features depending on the machine on
> which they were built.
>
> These issues make for difficult testing as the explosion in number of
> `#ifdef' guards makes maintenance error-prone and the dependence on
> binutils version means that, as well as deploying changes for testing
> in a variety of target configurations, testing must also involve
> compiling the library on an increasing number of host configurations,
> meaning that the chance of bugs going undetected increases (as was
> proved in the pre-commit CI which, due to the use of an older version
> of Binutils, picked up on a runtime-error that had hitherto gone
> unnoticed).
>
> We therefore do away with the use of all assembly instructions
> dependent on Binutils 2.42, choosing to replace them with `.inst's
> instead. This eliminates the latent bug picked up by CI and will
> ensure consistent builds of Libatomic across all versions of Binutils.
Nice! Thanks for doing this. It seems much cleaner and more flexible
than the current approach.
Thanks also for the clear organisation of the series.
OK for trunk. (For the record, I didn't hand-check the encodings of the
.insts ...)
Richard
> ---
>
> The recent introduction of the optional LSE128 and RCPC3 architectural
> extensions to AArch64 has further led to the increased flexibility of
> atomic support in the architecture, with many extensions providing
> support for distinct atomic operations, each with different potential
> applications in mind.
>
> This has led to maintenance difficulties in Libatomic, in particular
> regarding the way the ifunc selector is generated via a series of
> macro expansions at compile-time.
>
> Until now, irrespective of the atomic operation in question, all atomic
> functions for a particular operand size were expected to have the same
> number of ifunc alternatives, meaning that a one-size-fits-all
> approach could reasonably be taken for the selector.
>
> This meant that if, hypothetically, for a particular architecture and
> operand size one particular atomic operation was to have 3 different
> implementations associated with different extensions, libatomic would
> likewise be required to present three ifunc alternatives for all other
> atomic functions.
>
> The consequence in the design choice was the unnecessary use of
> function aliasing and the unwieldy code which resulted from this.
>
> This patch series attempts to remediate this issue by making the
> preprocessor macros defining the number of ifunc alternatives and
> their respective selection functions dependent on the file importing
> the ifunc selector-generating framework.
>
> all files are given `LAT_<FILENAME>' macros, defined at the beginning
> and undef'd at the end of the file. It is these macros that are
> subsequently used to fine-tune the behaviors of `libatomic_i.h' and
> `host-config.h'.
>
> In particular, the definition of the `IFUNC_NCOND(N)' and
> `IFUNC_COND_<n>' macros in host-config.h can now be guarded behind
> these new file-specific macros, which ultimately control what the
> `GEN_SELECTOR(X)' macro in `libatomic_i.h' expands to. As both of
> these headers are imported once per file implementing some atomic
> operation, fine-tuned control is now possible.
>
> Regtested with both `--enable-gnu-indirect-function' and
> `--disable-gnu-indirect-function' configurations on armv9.4-a target
> with LRCPC3 and LSE128 support and without.
>
> Victor Do Nascimento (4):
> Libatomic: AArch64: Convert all lse128 assembly to .insn directives
> Libatomic: Define per-file identifier macros
> Libatomic: Make ifunc selector behavior contingent on importing file
> Libatomic: Clean up AArch64 `atomic_16.S' implementation file
>
> libatomic/acinclude.m4 | 18 -
> libatomic/auto-config.h.in | 3 -
> libatomic/cas_n.c | 2 +
> libatomic/config/linux/aarch64/atomic_16.S | 511 +++++++++----------
> libatomic/config/linux/aarch64/host-config.h | 35 +-
> libatomic/configure | 43 --
> libatomic/configure.ac | 3 -
> libatomic/exch_n.c | 2 +
> libatomic/fadd_n.c | 2 +
> libatomic/fand_n.c | 2 +
> libatomic/fence.c | 2 +
> libatomic/fenv.c | 2 +
> libatomic/fior_n.c | 2 +
> libatomic/flag.c | 2 +
> libatomic/fnand_n.c | 2 +
> libatomic/fop_n.c | 2 +
> libatomic/fsub_n.c | 2 +
> libatomic/fxor_n.c | 2 +
> libatomic/gcas.c | 2 +
> libatomic/gexch.c | 2 +
> libatomic/glfree.c | 2 +
> libatomic/gload.c | 2 +
> libatomic/gstore.c | 2 +
> libatomic/load_n.c | 2 +
> libatomic/store_n.c | 2 +
> libatomic/tas_n.c | 2 +
> 26 files changed, 303 insertions(+), 350 deletions(-)
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2024-06-12 8:50 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2024-06-11 12:42 [PATCH v2 0/4] Libatomic: Cleanup ifunc selector and aliasing Victor Do Nascimento
2024-06-11 12:42 ` [PATCH v2 1/4] Libatomic: AArch64: Convert all lse128 assembly to .insn directives Victor Do Nascimento
2024-06-11 12:42 ` [PATCH v2 2/4] Libatomic: Define per-file identifier macros Victor Do Nascimento
2024-06-11 12:42 ` [PATCH v2 3/4] Libatomic: Make ifunc selector behavior contingent on importing file Victor Do Nascimento
2024-06-11 12:42 ` [PATCH v2 4/4] Libatomic: Clean up AArch64 `atomic_16.S' implementation file Victor Do Nascimento
2024-06-12 8:50 ` [PATCH v2 0/4] Libatomic: Cleanup ifunc selector and aliasing Richard Sandiford
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).