[Aarch64][1/3] Use atomic compare-and-swap instructions when available.

[Aarch64][1/3] Use atomic compare-and-swap instructions when available.

[Matthew Wahab]

[-- Attachment #1: Type: text/plain, Size: 872 bytes --]

ARMv8.1 adds instructions for atomic compare-and-swap with optional
memory ordering specifiers. This patch series adds the instructions to
GCC, making them available with -march=armv8.1-a or with
-march=armv8-a+lse, and using them to implement the __sync and __atomic
builtins.

This patch adds an internal TARGET_LSE macro, to check target support
for the atomic instructions.

Subsequent patches
- add and use atomic compare-and-swap instructions;
- add tests for the compare-and-swap;

Tested the series for aarch64-none-linux-gnu with native bootstrap and make
check and for aarch64-none-elf with cross-compiled check-gcc. Also tested
aarch64-none-elf with cross-compiled check-gcc on an emulator that supports
ARMv8.1.

Ok for trunk?
Matthew

2015-08-12  Matthew Wahab  <matthew.wahab@arm.com>

	* config/aarch64/aarch64.h (AARCH64_ISA_LSE): New.
	(TARGET_LSE): New.

[-- Attachment #2: 0001-Add-LSE-flag-testing-macro.patch --]
[-- Type: text/x-patch, Size: 1434 bytes --]

From 50a71e7572de7eb47c7913ae3d0984cd61f8d425 Mon Sep 17 00:00:00 2001
From: Matthew Wahab <matthew.wahab@arm.com>
Date: Mon, 20 Jul 2015 12:42:52 +0100
Subject: [PATCH 1/3] Add LSE flag testing macro.

Change-Id: I4bfeb5ec617f13026fb7dcd28a9b5c7efa4c1719
---
 gcc/config/aarch64/aarch64.h | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/gcc/config/aarch64/aarch64.h b/gcc/config/aarch64/aarch64.h
index 721927f..d6c7a74 100644
--- a/gcc/config/aarch64/aarch64.h
+++ b/gcc/config/aarch64/aarch64.h
@@ -156,6 +156,7 @@ extern unsigned aarch64_architecture_version;
 #define AARCH64_ISA_CRYPTO         (aarch64_isa_flags & AARCH64_FL_CRYPTO)
 #define AARCH64_ISA_FP             (aarch64_isa_flags & AARCH64_FL_FP)
 #define AARCH64_ISA_SIMD           (aarch64_isa_flags & AARCH64_FL_SIMD)
+#define AARCH64_ISA_LSE		   (aarch64_isa_flags & AARCH64_FL_LSE)
 
 /* Crypto is an optional extension to AdvSIMD.  */
 #define TARGET_CRYPTO (TARGET_SIMD && AARCH64_ISA_CRYPTO)
@@ -163,6 +164,9 @@ extern unsigned aarch64_architecture_version;
 /* CRC instructions that can be enabled through +crc arch extension.  */
 #define TARGET_CRC32 (AARCH64_ISA_CRC)
 
+/* Atomic instructions that can be enabled through the +lse extension.  */
+#define TARGET_LSE (AARCH64_ISA_LSE)
+
 /* Make sure this is always defined so we don't have to check for ifdefs
    but rather use normal ifs.  */
 #ifndef TARGET_FIX_ERR_A53_835769_DEFAULT
-- 
1.9.1

[Aarch64][2/3] Use the atomic compare-and-swap instructions when available.

[Matthew Wahab]

[-- Attachment #1: Type: text/plain, Size: 1737 bytes --]

ARMv8.1 adds instructions for atomic compare-and-swap with optional memory
ordering specifiers. This patch adds the compare-and-swap instructions and
changes the atomic_compare_and_swap patterns to use them

The changes to the atomic_compare_and_swap pattern makes the new
instructions available as an alternative for the existing expander, to
be used when the target supports them. The existing expander is reused
so that it can generate code needed to meet the requirements of the
atomic_compare_and_swap name.

Using the atomic CAS instructions, the code generated for a call to
__atomic_compare_exchange (ptr, expected, desired, weak, smodel, fmodel)
becomes:

   mov r, r1
   cas<smo><sz> r, r2, [r0]
   cmp r, r1
   cset r0, eq
   cbnz r0, L
   strb r, [r1]
L:
   ret

where
   r0 = ptr, r1 = *expected, r2 = *desired,
   r is some temporary.
   mo is one of {'', 'a', 'l', 'al'}, depending on smodel
   sz is one of {'', 'b', 'h'} depending on the data size.

Tested the series for aarch64-none-linux-gnu with native bootstrap and make
check and for aarch64-none-elf with cross-compiled check-gcc. Also tested
aarch64-none-elf with cross-compiled check-gcc on an emulator that supports
ARMv8.1.

Ok for trunk?
Matthew

2015-08-12  Matthew Wahab  <matthew.wahab@arm.com>

	* config/aarch64/aarch64-protos.h
	(aarch64_gen_atomic_cas): Declare.
	* config/aarch64/aarch64.c (aarch64_expand_compare_and_swap):
	Choose appropriate instruction pattern for the target.
	(aarch64_gen_atomic_cas): New.
	* config/aarch64/atomics.md (UNSPECV_ATOMIC_CAS): New.
	(atomic_compare_and_swap<mode>_1): Rename to
	aarch64_compare_and_swap<mode>.  Fix some indentation.
	(aarch64_compare_and_swap<mode>_lse): New.
	(aarch64_atomic_cas<mode>): New.


[-- Attachment #2: 0002-Add-and-use-atomic-CAS-instruction.patch --]
[-- Type: text/x-patch, Size: 10417 bytes --]

From a84d8f8202a30fca18ed79e617d5ba3422eb021f Mon Sep 17 00:00:00 2001
From: Matthew Wahab <matthew.wahab@arm.com>
Date: Mon, 10 Aug 2015 16:59:18 +0100
Subject: [PATCH 2/3] Add and use atomic CAS instruction.

Change-Id: Ie40f345d414fc9dc6c8cbac0eb9457547aa9ec2d
---
 gcc/config/aarch64/aarch64-protos.h |   1 +
 gcc/config/aarch64/aarch64.c        |  66 ++++++++++++++++++--
 gcc/config/aarch64/atomics.md       | 117 +++++++++++++++++++++++++++++++++---
 3 files changed, 172 insertions(+), 12 deletions(-)

diff --git a/gcc/config/aarch64/aarch64-protos.h b/gcc/config/aarch64/aarch64-protos.h
index 32b5d09..0b09d49 100644
--- a/gcc/config/aarch64/aarch64-protos.h
+++ b/gcc/config/aarch64/aarch64-protos.h
@@ -362,6 +362,7 @@ rtx aarch64_load_tp (rtx);
 
 void aarch64_expand_compare_and_swap (rtx op[]);
 void aarch64_split_compare_and_swap (rtx op[]);
+void aarch64_gen_atomic_cas (rtx, rtx, rtx, rtx, rtx);
 void aarch64_split_atomic_op (enum rtx_code, rtx, rtx, rtx, rtx, rtx, rtx);
 
 bool aarch64_gen_adjusted_ldpstp (rtx *, bool, enum machine_mode, RTX_CODE);
diff --git a/gcc/config/aarch64/aarch64.c b/gcc/config/aarch64/aarch64.c
index 6da7245..259e049 100644
--- a/gcc/config/aarch64/aarch64.c
+++ b/gcc/config/aarch64/aarch64.c
@@ -10749,7 +10749,23 @@ aarch64_expand_compare_and_swap (rtx operands[])
 {
   rtx bval, rval, mem, oldval, newval, is_weak, mod_s, mod_f, x;
   machine_mode mode, cmp_mode;
-  rtx (*gen) (rtx, rtx, rtx, rtx, rtx, rtx, rtx);
+  typedef rtx (*gen_cas_fn) (rtx, rtx, rtx, rtx, rtx, rtx, rtx);
+  int idx;
+  gen_cas_fn gen;
+  const gen_cas_fn split_cas[] =
+  {
+    gen_aarch64_compare_and_swapqi,
+    gen_aarch64_compare_and_swaphi,
+    gen_aarch64_compare_and_swapsi,
+    gen_aarch64_compare_and_swapdi
+  };
+  const gen_cas_fn atomic_cas[] =
+  {
+    gen_aarch64_compare_and_swapqi_lse,
+    gen_aarch64_compare_and_swaphi_lse,
+    gen_aarch64_compare_and_swapsi_lse,
+    gen_aarch64_compare_and_swapdi_lse
+  };
 
   bval = operands[0];
   rval = operands[1];
@@ -10794,13 +10810,17 @@ aarch64_expand_compare_and_swap (rtx operands[])
 
   switch (mode)
     {
-    case QImode: gen = gen_atomic_compare_and_swapqi_1; break;
-    case HImode: gen = gen_atomic_compare_and_swaphi_1; break;
-    case SImode: gen = gen_atomic_compare_and_swapsi_1; break;
-    case DImode: gen = gen_atomic_compare_and_swapdi_1; break;
+    case QImode: idx = 0; break;
+    case HImode: idx = 1; break;
+    case SImode: idx = 2; break;
+    case DImode: idx = 3; break;
     default:
       gcc_unreachable ();
     }
+  if (TARGET_LSE)
+    gen = atomic_cas[idx];
+  else
+    gen = split_cas[idx];
 
   emit_insn (gen (rval, mem, oldval, newval, is_weak, mod_s, mod_f));
 
@@ -10829,6 +10849,42 @@ aarch64_emit_post_barrier (enum memmodel model)
     }
 }
 
+/* Emit an atomic compare-and-swap operation.  RVAL is the destination register
+   for the data in memory.  EXPECTED is the value expected to be in memory.
+   DESIRED is the value to store to memory.  MEM is the memory location.  MODEL
+   is the memory ordering to use.  */
+
+void
+aarch64_gen_atomic_cas (rtx rval, rtx mem,
+			rtx expected, rtx desired,
+			rtx model)
+{
+  rtx (*gen) (rtx, rtx, rtx, rtx);
+  machine_mode mode;
+
+  mode = GET_MODE (mem);
+
+  switch (mode)
+    {
+    case QImode: gen = gen_aarch64_atomic_casqi; break;
+    case HImode: gen = gen_aarch64_atomic_cashi; break;
+    case SImode: gen = gen_aarch64_atomic_cassi; break;
+    case DImode: gen = gen_aarch64_atomic_casdi; break;
+    default:
+      gcc_unreachable ();
+    }
+
+  /* Move the expected value into the CAS destination register.  */
+  emit_insn (gen_rtx_SET (rval, expected));
+
+  /* Emit the CAS.  */
+  emit_insn (gen (rval, mem, desired, model));
+
+  /* Compare the expected value with the value loaded by the CAS, to establish
+     whether the swap was made.  */
+  aarch64_gen_compare_reg (EQ, rval, expected);
+}
+
 /* Split a compare and swap pattern.  */
 
 void
diff --git a/gcc/config/aarch64/atomics.md b/gcc/config/aarch64/atomics.md
index 1a38ac0..7082f61 100644
--- a/gcc/config/aarch64/atomics.md
+++ b/gcc/config/aarch64/atomics.md
@@ -26,6 +26,7 @@
     UNSPECV_STL				; Represent an atomic store or store-release.
     UNSPECV_ATOMIC_CMPSW		; Represent an atomic compare swap.
     UNSPECV_ATOMIC_EXCHG		; Represent an atomic exchange.
+    UNSPECV_ATOMIC_CAS			; Represent an atomic CAS.
     UNSPECV_ATOMIC_OP			; Represent an atomic operation.
 ])
 
@@ -45,10 +46,10 @@
   }
 )
 
-(define_insn_and_split "atomic_compare_and_swap<mode>_1"
+(define_insn_and_split "aarch64_compare_and_swap<mode>"
   [(set (reg:CC CC_REGNUM)					;; bool out
     (unspec_volatile:CC [(const_int 0)] UNSPECV_ATOMIC_CMPSW))
-   (set (match_operand:SI 0 "register_operand" "=&r")		;; val out
+   (set (match_operand:SI 0 "register_operand" "=&r")	   ;; val out
     (zero_extend:SI
       (match_operand:SHORT 1 "aarch64_sync_memory_operand" "+Q"))) ;; memory
    (set (match_dup 1)
@@ -57,7 +58,7 @@
        (match_operand:SHORT 3 "register_operand" "r")	;; desired
        (match_operand:SI 4 "const_int_operand")		;; is_weak
        (match_operand:SI 5 "const_int_operand")		;; mod_s
-       (match_operand:SI 6 "const_int_operand")]		;; mod_f
+       (match_operand:SI 6 "const_int_operand")]	;; mod_f
       UNSPECV_ATOMIC_CMPSW))
    (clobber (match_scratch:SI 7 "=&r"))]
   ""
@@ -70,17 +71,17 @@
   }
 )
 
-(define_insn_and_split "atomic_compare_and_swap<mode>_1"
+(define_insn_and_split "aarch64_compare_and_swap<mode>"
   [(set (reg:CC CC_REGNUM)					;; bool out
     (unspec_volatile:CC [(const_int 0)] UNSPECV_ATOMIC_CMPSW))
    (set (match_operand:GPI 0 "register_operand" "=&r")		;; val out
-    (match_operand:GPI 1 "aarch64_sync_memory_operand" "+Q")) ;; memory
+    (match_operand:GPI 1 "aarch64_sync_memory_operand" "+Q"))   ;; memory
    (set (match_dup 1)
     (unspec_volatile:GPI
       [(match_operand:GPI 2 "aarch64_plus_operand" "rI")	;; expect
        (match_operand:GPI 3 "register_operand" "r")		;; desired
-       (match_operand:SI 4 "const_int_operand")		;; is_weak
-       (match_operand:SI 5 "const_int_operand")		;; mod_s
+       (match_operand:SI 4 "const_int_operand")			;; is_weak
+       (match_operand:SI 5 "const_int_operand")			;; mod_s
        (match_operand:SI 6 "const_int_operand")]		;; mod_f
       UNSPECV_ATOMIC_CMPSW))
    (clobber (match_scratch:SI 7 "=&r"))]
@@ -94,6 +95,57 @@
   }
 )
 
+(define_insn_and_split "aarch64_compare_and_swap<mode>_lse"
+  [(set (reg:CC CC_REGNUM)					;; bool out
+    (unspec_volatile:CC [(const_int 0)] UNSPECV_ATOMIC_CMPSW))
+   (set (match_operand:SI 0 "register_operand" "=&r")		;; val out
+    (zero_extend:SI
+      (match_operand:SHORT 1 "aarch64_sync_memory_operand" "+Q"))) ;; memory
+   (set (match_dup 1)
+    (unspec_volatile:SHORT
+      [(match_operand:SI 2 "aarch64_plus_operand" "rI")	;; expected
+       (match_operand:SHORT 3 "register_operand" "r")	;; desired
+       (match_operand:SI 4 "const_int_operand")		;; is_weak
+       (match_operand:SI 5 "const_int_operand")		;; mod_s
+       (match_operand:SI 6 "const_int_operand")]	;; mod_f
+      UNSPECV_ATOMIC_CMPSW))]
+  "TARGET_LSE"
+  "#"
+  "&& reload_completed"
+  [(const_int 0)]
+  {
+    aarch64_gen_atomic_cas (operands[0], operands[1],
+			    operands[2], operands[3],
+			    operands[5]);
+    DONE;
+  }
+)
+
+(define_insn_and_split "aarch64_compare_and_swap<mode>_lse"
+  [(set (reg:CC CC_REGNUM)					;; bool out
+    (unspec_volatile:CC [(const_int 0)] UNSPECV_ATOMIC_CMPSW))
+   (set (match_operand:GPI 0 "register_operand" "=&r")		;; val out
+    (match_operand:GPI 1 "aarch64_sync_memory_operand" "+Q"))   ;; memory
+   (set (match_dup 1)
+    (unspec_volatile:GPI
+      [(match_operand:GPI 2 "aarch64_plus_operand" "rI")	;; expect
+       (match_operand:GPI 3 "register_operand" "r")		;; desired
+       (match_operand:SI 4 "const_int_operand")			;; is_weak
+       (match_operand:SI 5 "const_int_operand")			;; mod_s
+       (match_operand:SI 6 "const_int_operand")]		;; mod_f
+      UNSPECV_ATOMIC_CMPSW))]
+  "TARGET_LSE "
+  "#"
+  "&& reload_completed"
+  [(const_int 0)]
+  {
+    aarch64_gen_atomic_cas (operands[0], operands[1],
+			    operands[2], operands[3],
+			    operands[5]);
+    DONE;
+  }
+)
+
 (define_insn_and_split "atomic_exchange<mode>"
   [(set (match_operand:ALLI 0 "register_operand" "=&r")		;; output
     (match_operand:ALLI 1 "aarch64_sync_memory_operand" "+Q")) ;; memory
@@ -370,3 +422,54 @@
       return "dmb\\tish";
   }
 )
+
+;; ARMv8.1 LSE instructions.
+
+;; Atomic compare-and-swap: HI and smaller modes.
+
+(define_insn "aarch64_atomic_cas<mode>"
+ [(set (match_operand:SI 0 "register_operand" "+&r")		  ;; out
+   (zero_extend:SI
+    (match_operand:SHORT 1 "aarch64_sync_memory_operand" "+Q")))  ;; memory.
+  (set (match_dup 1)
+   (unspec_volatile:SHORT
+    [(match_dup 0)
+     (match_operand:SHORT 2 "register_operand" "r")	;; value.
+     (match_operand:SI 3 "const_int_operand" "")]	;; model.
+    UNSPECV_ATOMIC_CAS))]
+ "TARGET_LSE && reload_completed"
+{
+  enum memmodel model = memmodel_from_int (INTVAL (operands[3]));
+  if (is_mm_relaxed (model))
+    return "cas<atomic_sfx>\t%<w>0, %<w>2, %1";
+  else if (is_mm_acquire (model) || is_mm_consume (model))
+    return "casa<atomic_sfx>\t%<w>0, %<w>2, %1";
+  else if (is_mm_release (model))
+    return "casl<atomic_sfx>\t%<w>0, %<w>2, %1";
+  else
+    return "casal<atomic_sfx>\t%<w>0, %<w>2, %1";
+})
+
+;; Atomic compare-and-swap: SI and larger modes.
+
+(define_insn "aarch64_atomic_cas<mode>"
+ [(set (match_operand:GPI 0 "register_operand" "+&r")	      ;; out
+   (match_operand:GPI 1 "aarch64_sync_memory_operand" "+Q"))  ;; memory.
+  (set (match_dup 1)
+   (unspec_volatile:GPI
+    [(match_dup 0)
+     (match_operand:GPI 2 "register_operand" "r")	;; value.
+     (match_operand:SI 3 "const_int_operand" "")]	;; model.
+    UNSPECV_ATOMIC_CAS))]
+  "TARGET_LSE && reload_completed"
+{
+    enum memmodel model = memmodel_from_int (INTVAL (operands[3]));
+    if (is_mm_relaxed (model))
+      return "cas<atomic_sfx>\t%<w>0, %<w>2, %1";
+    else if (is_mm_acquire (model) || is_mm_consume (model))
+      return "casa<atomic_sfx>\t%<w>0, %<w>2, %1";
+    else if (is_mm_release (model))
+      return "casl<atomic_sfx>\t%<w>0, %<w>2, %1";
+    else
+      return "casal<atomic_sfx>\t%<w>0, %<w>2, %1";
+})
-- 
1.9.1

[Aarch64][3/3] Add tests for atomic compare-and-swap instructions.

[Matthew Wahab]

[-- Attachment #1: Type: text/plain, Size: 722 bytes --]

ARMv8.1 adds instructions for atomic compare-and-swap with optional
memory ordering specifiers. This patch adds tests for the
compare-and-swap instructions as two files. The first is support code to
run the test with a range of types and memory models, the second is the
test for the CAS instruction.

Tested the series for aarch64-none-linux-gnu with native bootstrap and make
check and for aarch64-none-elf with cross-compiled check-gcc. Also tested
aarch64-none-elf with cross-compiled check-gcc on an emulator that supports
ARMv8.1.

Ok for trunk?
Matthew

gcc/testsuite
2015-08-12  Matthew Wahab  <matthew.wahab@arm.com>

	* gcc.target/aarch64/atomic-inst-cas.c: New.
	* gcc.target/aarch64/atomic-inst-ops.inc: New.


[-- Attachment #2: 0003-Add-tests-for-CAS-instruction.patch --]
[-- Type: text/x-patch, Size: 5548 bytes --]

From c72302f2a0bc4d95a0b933e54332d551295040bf Mon Sep 17 00:00:00 2001
From: Matthew Wahab <matthew.wahab@arm.com>
Date: Mon, 3 Aug 2015 18:10:37 +0100
Subject: [PATCH 3/3] Add tests for CAS instruction.

Change-Id: I42a46c2f81f1200a893620ba96323ce785873e8d
---
 gcc/testsuite/gcc.target/aarch64/atomic-inst-cas.c | 61 ++++++++++++++++++++++
 .../gcc.target/aarch64/atomic-inst-ops.inc         | 53 +++++++++++++++++++
 2 files changed, 114 insertions(+)
 create mode 100644 gcc/testsuite/gcc.target/aarch64/atomic-inst-cas.c
 create mode 100644 gcc/testsuite/gcc.target/aarch64/atomic-inst-ops.inc

diff --git a/gcc/testsuite/gcc.target/aarch64/atomic-inst-cas.c b/gcc/testsuite/gcc.target/aarch64/atomic-inst-cas.c
new file mode 100644
index 0000000..f6f2892
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/atomic-inst-cas.c
@@ -0,0 +1,61 @@
+/* { dg-do compile } */
+/* { dg-options "-O2 -march=armv8-a+lse" } */
+
+/* Test ARMv8.1-A CAS instruction.  */
+
+#include "atomic-inst-ops.inc"
+
+#define TEST TEST_TWO
+
+#define CAS_ATOMIC(FN, TY, MODEL1, MODEL2)				\
+  int FNNAME (FN, TY) (TY* val, TY* foo, TY* bar)			\
+  {									\
+    int model_s = MODEL1;						\
+    int model_f = MODEL2;						\
+    /* The success memory ordering must be at least as strong as	\
+       the failure memory ordering.  */					\
+    if (model_s < model_f)						\
+      return 0;								\
+    /* Ignore invalid memory orderings.  */				\
+    if (model_f == __ATOMIC_RELEASE || model_f == __ATOMIC_ACQ_REL)	\
+      return 0;								\
+    return __atomic_compare_exchange_n (val, foo, bar, 0, model_s, model_f); \
+  }
+
+#define CAS_ATOMIC_NORETURN(FN, TY, MODEL1, MODEL2)			\
+  void FNNAME (FN, TY) (TY* val, TY* foo, TY* bar)			\
+  {									\
+    int model_s = MODEL1;						\
+    int model_f = MODEL2;						\
+    /* The success memory ordering must be at least as strong as	\
+       the failure memory ordering.  */					\
+    if (model_s < model_f)						\
+      return;								\
+    /* Ignore invalid memory orderings.  */				\
+    if (model_f == __ATOMIC_RELEASE || model_f == __ATOMIC_ACQ_REL)	\
+      return;								\
+    __atomic_compare_exchange_n (val, foo, bar, 0, model_s, model_f);	\
+  }
+
+TEST (cas_atomic, CAS_ATOMIC)
+TEST (cas_atomic_noreturn, CAS_ATOMIC_NORETURN)
+
+
+/* { dg-final { scan-assembler-times "casb\t" 4} } */
+/* { dg-final { scan-assembler-times "casab\t" 20} } */
+/* { dg-final { scan-assembler-times "caslb\t" 4} } */
+/* { dg-final { scan-assembler-times "casalb\t" 36} } */
+
+/* { dg-final { scan-assembler-times "cash\t" 4} } */
+/* { dg-final { scan-assembler-times "casah\t" 20} } */
+/* { dg-final { scan-assembler-times "caslh\t" 4} } */
+/* { dg-final { scan-assembler-times "casalh\t" 36} } */
+
+/* { dg-final { scan-assembler-times "cas\t" 8} } */
+/* { dg-final { scan-assembler-times "casa\t" 40} } */
+/* { dg-final { scan-assembler-times "casl\t" 8} } */
+/* { dg-final { scan-assembler-times "casal\t" 72} } */
+
+/* { dg-final { scan-assembler-not "ldaxr\t" } } */
+/* { dg-final { scan-assembler-not "stlxr\t" } } */
+/* { dg-final { scan-assembler-not "dmb" } } */
diff --git a/gcc/testsuite/gcc.target/aarch64/atomic-inst-ops.inc b/gcc/testsuite/gcc.target/aarch64/atomic-inst-ops.inc
new file mode 100644
index 0000000..72c7e5c
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/atomic-inst-ops.inc
@@ -0,0 +1,53 @@
+/* Support code for atomic instruction tests.  */
+
+/* Define types names without spaces.  */
+typedef unsigned char uchar;
+typedef unsigned short ushort;
+typedef unsigned int uint;
+typedef long long longlong;
+typedef unsigned long long ulonglong;
+typedef __int128_t int128;
+typedef __uint128_t uint128;
+
+#define FNNAME(NAME,TY) NAME
+
+/* Expand one-model functions.  */
+#define TEST_M1(NAME, FN, TY, MODEL, DUMMY)	\
+  FN (test_##NAME##_##TY, TY, MODEL)
+
+/* Expand two-model functions.  */
+#define TEST_M2(NAME, FN, TY, MODEL1, MODEL2)	\
+  FN (test_##NAME##_##TY, TY, MODEL1, MODEL2)
+
+/* Typest to test.  */
+#define TEST_TY(NAME, FN, N, MODEL1, MODEL2)		\
+  TEST_M##N (NAME, FN, char, MODEL1, MODEL2)		\
+  TEST_M##N (NAME, FN, uchar, MODEL1, MODEL2)		\
+  TEST_M##N (NAME, FN, short, MODEL1, MODEL2)		\
+  TEST_M##N (NAME, FN, ushort, MODEL1, MODEL2)		\
+  TEST_M##N (NAME, FN, int, MODEL1, MODEL2)		\
+  TEST_M##N (NAME, FN, uint, MODEL1, MODEL2)		\
+  TEST_M##N (NAME, FN, longlong, MODEL1, MODEL2)	\
+  TEST_M##N (NAME, FN, ulonglong, MODEL1, MODEL2)	\
+  TEST_M##N (NAME, FN, int128, MODEL1, MODEL2)		\
+  TEST_M##N (NAME, FN, uint128, MODEL1, MODEL2)
+
+/* Cross-product of models to test.  */
+#define TEST_MODEL_M1(NAME, FN, N, M)			\
+  TEST_TY (NAME##_relaxed, FN, N, M, __ATOMIC_RELAXED)	\
+  TEST_TY (NAME##_consume, FN, N, M, __ATOMIC_CONSUME)	\
+  TEST_TY (NAME##_acquire, FN, N, M, __ATOMIC_ACQUIRE)	\
+  TEST_TY (NAME##_release, FN, N, M, __ATOMIC_RELEASE)	\
+  TEST_TY (NAME##_acq_rel, FN, N, M, __ATOMIC_ACQ_REL)	\
+  TEST_TY (NAME##_seq_cst, FN, N, M, __ATOMIC_SEQ_CST)	\
+
+#define TEST_MODEL_M2(NAME, FN)					\
+  TEST_MODEL_M1 (NAME##_relaxed, FN, 2, __ATOMIC_RELAXED)	\
+  TEST_MODEL_M1 (NAME##_consume, FN, 2, __ATOMIC_CONSUME)	\
+  TEST_MODEL_M1 (NAME##_acquire, FN, 2, __ATOMIC_ACQUIRE)	\
+  TEST_MODEL_M1 (NAME##_release, FN, 2, __ATOMIC_RELEASE)	\
+  TEST_MODEL_M1 (NAME##_acq_rel, FN, 2, __ATOMIC_ACQ_REL)	\
+  TEST_MODEL_M1 (NAME##_seq_cst, FN, 2, __ATOMIC_SEQ_CST)	\
+
+/* Expand functions for a cross-product of memory models and types.  */
+#define TEST_TWO(NAME, FN) TEST_MODEL_M2 (NAME, FN)
-- 
1.9.1

Re: [Aarch64][1/3] Use atomic compare-and-swap instructions when available.

[James Greenhalgh]

On Wed, Aug 12, 2015 at 11:12:29AM +0100, Matthew Wahab wrote:
> ARMv8.1 adds instructions for atomic compare-and-swap with optional
> memory ordering specifiers. This patch series adds the instructions to
> GCC, making them available with -march=armv8.1-a or with
> -march=armv8-a+lse, and using them to implement the __sync and __atomic
> builtins.
> 
> This patch adds an internal TARGET_LSE macro, to check target support
> for the atomic instructions.
> 
> Subsequent patches
> - add and use atomic compare-and-swap instructions;
> - add tests for the compare-and-swap;
> 
> Tested the series for aarch64-none-linux-gnu with native bootstrap and make
> check and for aarch64-none-elf with cross-compiled check-gcc. Also tested
> aarch64-none-elf with cross-compiled check-gcc on an emulator that supports
> ARMv8.1.
> 
> Ok for trunk?
> Matthew

OK.

Thanks,
James

> 
> 2015-08-12  Matthew Wahab  <matthew.wahab@arm.com>
> 
> 	* config/aarch64/aarch64.h (AARCH64_ISA_LSE): New.
> 	(TARGET_LSE): New.

Re: [Aarch64][3/3] Add tests for atomic compare-and-swap instructions.

[Andrew Pinski]

On Wed, Aug 12, 2015 at 6:16 PM, Matthew Wahab
<matthew.wahab@foss.arm.com> wrote:
> ARMv8.1 adds instructions for atomic compare-and-swap with optional
> memory ordering specifiers. This patch adds tests for the
> compare-and-swap instructions as two files. The first is support code to
> run the test with a range of types and memory models, the second is the
> test for the CAS instruction.
>
> Tested the series for aarch64-none-linux-gnu with native bootstrap and make
> check and for aarch64-none-elf with cross-compiled check-gcc. Also tested
> aarch64-none-elf with cross-compiled check-gcc on an emulator that supports
> ARMv8.1.
>
> Ok for trunk?


I noticed you did not change the other cas testcases so they don't set
lse.  This can cause those testcases to fail if a cpu defaults to
having LSE.
I am going to be modifying a cpu to default to LSE enabled soonish.

Thanks,
Andrew

> Matthew
>
> gcc/testsuite
> 2015-08-12  Matthew Wahab  <matthew.wahab@arm.com>
>
>         * gcc.target/aarch64/atomic-inst-cas.c: New.
>         * gcc.target/aarch64/atomic-inst-ops.inc: New.
>

Re: [Aarch64][3/3] Add tests for atomic compare-and-swap instructions.

[Matthew Wahab]

On 12/08/15 11:51, Andrew Pinski wrote:
> On Wed, Aug 12, 2015 at 6:16 PM, Matthew Wahab
> <matthew.wahab@foss.arm.com> wrote:
>> ARMv8.1 adds instructions for atomic compare-and-swap with optional
>> memory ordering specifiers. This patch adds tests for the
>> compare-and-swap instructions as two files. The first is support code to
>> run the test with a range of types and memory models, the second is the
>> test for the CAS instruction.
>
> I noticed you did not change the other cas testcases so they don't set
> lse.  This can cause those testcases to fail if a cpu defaults to
> having LSE.
> I am going to be modifying a cpu to default to LSE enabled soonish.
>

I'll see what I can do.
Matthew

Re: [Aarch64][2/3] Use the atomic compare-and-swap instructions when available.

[James Greenhalgh]

On Wed, Aug 12, 2015 at 11:15:25AM +0100, Matthew Wahab wrote:
> ARMv8.1 adds instructions for atomic compare-and-swap with optional memory
> ordering specifiers. This patch adds the compare-and-swap instructions and
> changes the atomic_compare_and_swap patterns to use them
> 
> The changes to the atomic_compare_and_swap pattern makes the new
> instructions available as an alternative for the existing expander, to
> be used when the target supports them. The existing expander is reused
> so that it can generate code needed to meet the requirements of the
> atomic_compare_and_swap name.
> 
> Using the atomic CAS instructions, the code generated for a call to
> __atomic_compare_exchange (ptr, expected, desired, weak, smodel, fmodel)
> becomes:
> 
>    mov r, r1
>    cas<smo><sz> r, r2, [r0]
>    cmp r, r1
>    cset r0, eq
>    cbnz r0, L
>    strb r, [r1]
> L:
>    ret
> 
> where
>    r0 = ptr, r1 = *expected, r2 = *desired,
>    r is some temporary.
>    mo is one of {'', 'a', 'l', 'al'}, depending on smodel
>    sz is one of {'', 'b', 'h'} depending on the data size.
> 
> Tested the series for aarch64-none-linux-gnu with native bootstrap and make
> check and for aarch64-none-elf with cross-compiled check-gcc. Also tested
> aarch64-none-elf with cross-compiled check-gcc on an emulator that supports
> ARMv8.1.
> 
> Ok for trunk?
> Matthew

OK, but please fix up the testcases Andrew mentioned for those configuring
their toolchains/test runs with a default -mcpu or -march which targets
these instructions. I would just hardwire them
(with dg-additional-options) to -march=armv8-a.

Thanks,
James

> 2015-08-12  Matthew Wahab  <matthew.wahab@arm.com>
> 
> 	* config/aarch64/aarch64-protos.h
> 	(aarch64_gen_atomic_cas): Declare.
> 	* config/aarch64/aarch64.c (aarch64_expand_compare_and_swap):
> 	Choose appropriate instruction pattern for the target.
> 	(aarch64_gen_atomic_cas): New.
> 	* config/aarch64/atomics.md (UNSPECV_ATOMIC_CAS): New.
> 	(atomic_compare_and_swap<mode>_1): Rename to
> 	aarch64_compare_and_swap<mode>.  Fix some indentation.
> 	(aarch64_compare_and_swap<mode>_lse): New.
> 	(aarch64_atomic_cas<mode>): New.
> 

Re: [Aarch64][3/3] Add tests for atomic compare-and-swap instructions.

[James Greenhalgh]

On Wed, Aug 12, 2015 at 11:16:41AM +0100, Matthew Wahab wrote:
> ARMv8.1 adds instructions for atomic compare-and-swap with optional
> memory ordering specifiers. This patch adds tests for the
> compare-and-swap instructions as two files. The first is support code to
> run the test with a range of types and memory models, the second is the
> test for the CAS instruction.
> 
> Tested the series for aarch64-none-linux-gnu with native bootstrap and make
> check and for aarch64-none-elf with cross-compiled check-gcc. Also tested
> aarch64-none-elf with cross-compiled check-gcc on an emulator that supports
> ARMv8.1.
> 
> Ok for trunk?
> Matthew

OK, but please write a follow-up to fix the pre-existing tests.

Thanks,
James

> 
> gcc/testsuite
> 2015-08-12  Matthew Wahab  <matthew.wahab@arm.com>
> 
> 	* gcc.target/aarch64/atomic-inst-cas.c: New.
> 	* gcc.target/aarch64/atomic-inst-ops.inc: New.
>