[PATCH v2 0/8] RISCV: Implement ISA Manual Table A.6 Mappings

[PATCH v2 0/8] RISCV: Implement ISA Manual Table A.6 Mappings

[Patrick O'Neill]

This patchset aims to bring the RISCV atomics implementation in line
with the recommended mapping present in table A.6 of the ISA manual.
  https://github.com/riscv/riscv-isa-manual/blob/c7cf84547b3aefacab5463add1734c1602b67a49/src/memory.tex#L1083-L1157

The current mapping in GCC is not internally consistent. Andrea Parri
pointed this out here along with a litmus test:
  https://inbox.sourceware.org/gcc-patches/Y1GbJuhcBFpPGJQ0@andrea/

As a result, we have an opportunity to jump straight to the A.6
implementation (meaning we will be compatible with LLVM's mappings which
are A.6). Ideally we move to the A.6 mappings before more binaries are
built/distributed and it becomes more painful to switch.

Patch 1 simplifies the memmodel to ignore MEMMODEL_SYNC_* cases (legacy
cases that aren't handled differently for RISC-V).
Patches 2-4 make the mappings strictly stronger.
Patches 5-8 weaken the mappings to be in line with table A.6 of the ISA
manual.

Christoph Muellner also submitted a similar patchset here:
  https://gcc.gnu.org/pipermail/gcc-patches/2022-May/595712.html
I used my previous patchset as a starting point since it was easier for 
me.

LLVM mapping notes:
* LLVM emits corresponding fences for atomic_signal_fence instructions.
  This seems to be an oversight since AFAIK atomic_signal_fence acts as
  a compiler directive. GCC does not emit any fences for
  atomic_signal_fence instructions.

Remaining work:
* After this patchset, GCC still emits a full fence rw,rw after a
  SEQ_CST load (fence r,rw is recommended by table A.6).
  This can be relaxed in a future patch as it is strictly stronger than
  A.6's recommendation - I wanted to get this series out to be
  considered for GCC 13.

Patchset v1:
  https://gcc.gnu.org/pipermail/gcc-patches/2022-April/592950.html

Changes for v2:
* Use memmodel_base rather than a custom simplify_memmodel function
  (Inspired by Christoph Muellner's patch 1/9)
* Move instruction styling change from [v1 5/7] to [v2 3/8] to reduce
  [v2 6/8]'s complexity
* Eliminated %K flag for atomic store introduced in v1 in favor of
  if/else
* Rebase/test

* PR target/89835: The RISC-V target uses amoswap.w for relaxed stores

Patrick O'Neill (8):
  RISCV: Eliminate SYNC memory models
  RISCV: Enforce Libatomic LR/SC SEQ_CST
  RISCV: Enforce atomic compare_exchange SEQ_CST
  RISCV: Add AMO release bits
  RISCV: Eliminate AMO op fences
  RISCV: Weaken compare_exchange LR/SC pairs
  RISCV: Weaken atomic stores
  RISCV: Weaken mem_thread_fence

 gcc/config/riscv/riscv-protos.h               |  5 ++
 gcc/config/riscv/riscv.cc                     | 70 ++++++++++++++-----
 gcc/config/riscv/sync.md                      | 62 +++++++++++-----
 .../gcc.target/riscv/amo-thread-fence-1.c     |  6 ++
 .../gcc.target/riscv/amo-thread-fence-2.c     |  6 ++
 .../gcc.target/riscv/amo-thread-fence-3.c     |  6 ++
 .../gcc.target/riscv/amo-thread-fence-4.c     |  6 ++
 .../gcc.target/riscv/amo-thread-fence-5.c     |  6 ++
 .../riscv/compare-exchange-atomics-model-1.c  | 12 ++++
 .../riscv/compare-exchange-atomics-model-2.c  | 12 ++++
 .../riscv/compare-exchange-atomics-model-3.c  | 12 ++++
 .../riscv/compare-exchange-atomics-model-4.c  | 12 ++++
 .../riscv/compare-exchange-atomics-model-5.c  | 12 ++++
 gcc/testsuite/gcc.target/riscv/pr89835.c      |  9 +++
 libgcc/config/riscv/atomic.c                  |  4 +-
 15 files changed, 206 insertions(+), 34 deletions(-)
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-thread-fence-1.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-thread-fence-2.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-thread-fence-3.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-thread-fence-4.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-thread-fence-5.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/compare-exchange-atomics-model-1.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/compare-exchange-atomics-model-2.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/compare-exchange-atomics-model-3.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/compare-exchange-atomics-model-4.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/compare-exchange-atomics-model-5.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/pr89835.c

-- 
2.25.1

[PATCH v2 1/8] RISCV: Eliminate SYNC memory models

[Patrick O'Neill]

Remove references to MEMMODEL_SYNC_* models by converting via
memmodel_base().

2023-04-05 Patrick O'Neill <patrick@rivosinc.com>

	* atomic.c: Remove MEMMODEL_SYNC_* cases and sanitize memmodel 
	input with memmodel_base

Signed-off-by: Patrick O'Neill <patrick@rivosinc.com>
---
 gcc/config/riscv/riscv.cc | 11 +++--------
 1 file changed, 3 insertions(+), 8 deletions(-)

diff --git a/gcc/config/riscv/riscv.cc b/gcc/config/riscv/riscv.cc
index 76eee4a55e9..8f5636c93ed 100644
--- a/gcc/config/riscv/riscv.cc
+++ b/gcc/config/riscv/riscv.cc
@@ -4288,14 +4288,11 @@ riscv_memmodel_needs_amo_acquire (enum memmodel model)
     {
       case MEMMODEL_ACQ_REL:
       case MEMMODEL_SEQ_CST:
-      case MEMMODEL_SYNC_SEQ_CST:
       case MEMMODEL_ACQUIRE:
       case MEMMODEL_CONSUME:
-      case MEMMODEL_SYNC_ACQUIRE:
 	return true;
 
       case MEMMODEL_RELEASE:
-      case MEMMODEL_SYNC_RELEASE:
       case MEMMODEL_RELAXED:
 	return false;
 
@@ -4314,14 +4311,11 @@ riscv_memmodel_needs_release_fence (enum memmodel model)
     {
       case MEMMODEL_ACQ_REL:
       case MEMMODEL_SEQ_CST:
-      case MEMMODEL_SYNC_SEQ_CST:
       case MEMMODEL_RELEASE:
-      case MEMMODEL_SYNC_RELEASE:
 	return true;
 
       case MEMMODEL_ACQUIRE:
       case MEMMODEL_CONSUME:
-      case MEMMODEL_SYNC_ACQUIRE:
       case MEMMODEL_RELAXED:
 	return false;
 
@@ -4360,6 +4354,7 @@ riscv_print_operand (FILE *file, rtx op, int letter)
     }
   machine_mode mode = GET_MODE (op);
   enum rtx_code code = GET_CODE (op);
+  const enum memmodel model = memmodel_base (INTVAL (op));
 
   switch (letter)
     {
@@ -4497,12 +4492,12 @@ riscv_print_operand (FILE *file, rtx op, int letter)
       break;
 
     case 'A':
-      if (riscv_memmodel_needs_amo_acquire ((enum memmodel) INTVAL (op)))
+      if (riscv_memmodel_needs_amo_acquire (model))
 	fputs (".aq", file);
       break;
 
     case 'F':
-      if (riscv_memmodel_needs_release_fence ((enum memmodel) INTVAL (op)))
+      if (riscv_memmodel_needs_release_fence (model))
 	fputs ("fence iorw,ow; ", file);
       break;
 
-- 
2.25.1

[PATCH v2 2/8] RISCV: Enforce Libatomic LR/SC SEQ_CST

[Patrick O'Neill]

Replace LR.aq/SC.rl pairs with the SEQ_CST LR.aqrl/SC.rl pairs
recommended by table A.6 of the ISA manual.

2023-04-05 Patrick O'Neill <patrick@rivosinc.com>

	* atomic.c: Change LR.aq/SC.rl pairs into sequentially
	consistent LR.aqrl/SC.rl pair.

Signed-off-by: Patrick O'Neill <patrick@rivosinc.com>
---
 libgcc/config/riscv/atomic.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/libgcc/config/riscv/atomic.c b/libgcc/config/riscv/atomic.c
index 69f53623509..5f895939b0b 100644
--- a/libgcc/config/riscv/atomic.c
+++ b/libgcc/config/riscv/atomic.c
@@ -39,7 +39,7 @@ see the files COPYING3 and COPYING.RUNTIME respectively.  If not, see
     unsigned old, tmp1, tmp2;						\
 									\
     asm volatile ("1:\n\t"						\
-		  "lr.w.aq %[old], %[mem]\n\t"				\
+		  "lr.w.aqrl %[old], %[mem]\n\t"			\
 		  #insn " %[tmp1], %[old], %[value]\n\t"		\
 		  invert						\
 		  "and %[tmp1], %[tmp1], %[mask]\n\t"			\
@@ -73,7 +73,7 @@ see the files COPYING3 and COPYING.RUNTIME respectively.  If not, see
     unsigned old, tmp1;							\
 									\
     asm volatile ("1:\n\t"						\
-		  "lr.w.aq %[old], %[mem]\n\t"				\
+		  "lr.w.aqrl %[old], %[mem]\n\t"			\
 		  "and %[tmp1], %[old], %[mask]\n\t"			\
 		  "bne %[tmp1], %[o], 1f\n\t"				\
 		  "and %[tmp1], %[old], %[not_mask]\n\t"		\
-- 
2.25.1

[PATCH v2 3/8] RISCV: Enforce atomic compare_exchange SEQ_CST

[Patrick O'Neill]

This patch enforces SEQ_CST for atomic compare_exchange ops.

Replace Fence/LR.aq/SC.aq pairs with strong SEQ_CST LR.aqrl/SC.rl pairs
recommended by table A.6 of the ISA manual.

2023-04-05 Patrick O'Neill <patrick@rivosinc.com>

	* sync.md: Change FENCE/LR.aq/SC.aq into sequentially
	consistent LR.aqrl/SC.rl pair.

Signed-off-by: Patrick O'Neill <patrick@rivosinc.com>
---
 gcc/config/riscv/sync.md | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)

diff --git a/gcc/config/riscv/sync.md b/gcc/config/riscv/sync.md
index c932ef87b9d..de42245981b 100644
--- a/gcc/config/riscv/sync.md
+++ b/gcc/config/riscv/sync.md
@@ -115,9 +115,16 @@
 	 UNSPEC_COMPARE_AND_SWAP))
    (clobber (match_scratch:GPR 6 "=&r"))]
   "TARGET_ATOMIC"
-  "%F5 1: lr.<amo>%A5 %0,%1; bne %0,%z2,1f; sc.<amo>%A4 %6,%z3,%1; bnez %6,1b; 1:"
+  {
+    return "1:\;"
+	   "lr.<amo>.aqrl\t%0,%1\;"
+	   "bne\t%0,%z2,1f\;"
+	   "sc.<amo>.rl\t%6,%z3,%1\;"
+	   "bnez\t%6,1b\;"
+	   "1:";
+  }
   [(set_attr "type" "atomic")
-   (set (attr "length") (const_int 20))])
+   (set (attr "length") (const_int 16))])
 
 (define_expand "atomic_compare_and_swap<mode>"
   [(match_operand:SI 0 "register_operand" "")   ;; bool output
-- 
2.25.1

[PATCH v2 4/8] RISCV: Add AMO release bits

[Patrick O'Neill]

This patch sets the relevant .rl bits on amo operations.

2023-04-05 Patrick O'Neill <patrick@rivosinc.com>

	* riscv.cc (riscv_print_operand): change behavior of %A to
	include release bits.

Signed-off-by: Patrick O'Neill <patrick@rivosinc.com>
---
 gcc/config/riscv/riscv.cc | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/gcc/config/riscv/riscv.cc b/gcc/config/riscv/riscv.cc
index 8f5636c93ed..8ffee494fbe 100644
--- a/gcc/config/riscv/riscv.cc
+++ b/gcc/config/riscv/riscv.cc
@@ -4492,8 +4492,13 @@ riscv_print_operand (FILE *file, rtx op, int letter)
       break;
 
     case 'A':
-      if (riscv_memmodel_needs_amo_acquire (model))
+      if (riscv_memmodel_needs_amo_acquire (model) &&
+	  riscv_memmodel_needs_release_fence (model))
+	fputs (".aqrl", file);
+      else if (riscv_memmodel_needs_amo_acquire (model))
 	fputs (".aq", file);
+      else if (riscv_memmodel_needs_release_fence (model))
+	fputs (".rl", file);
       break;
 
     case 'F':
-- 
2.25.1

[PATCH v2 5/8] RISCV: Eliminate AMO op fences

[Patrick O'Neill]

Atomic operations with the appropriate bits set already enfore release
semantics. Remove unnecessary release fences from atomic ops.

This change brings AMO ops in line with table A.6 of the ISA manual.

2023-04-05 Patrick O'Neill <patrick@rivosinc.com>

	* riscv.cc (riscv_memmodel_needs_amo_acquire): Change function
	name.
	* riscv.cc (riscv_print_operand): Remove unneeded %F case.
	* sync.md: Remove unneeded fences.

Signed-off-by: Patrick O'Neill <patrick@rivosinc.com>
---
 gcc/config/riscv/riscv.cc | 16 +++++-----------
 gcc/config/riscv/sync.md  | 16 ++++++++--------
 2 files changed, 13 insertions(+), 19 deletions(-)

diff --git a/gcc/config/riscv/riscv.cc b/gcc/config/riscv/riscv.cc
index 8ffee494fbe..6576e9ae524 100644
--- a/gcc/config/riscv/riscv.cc
+++ b/gcc/config/riscv/riscv.cc
@@ -4301,11 +4301,11 @@ riscv_memmodel_needs_amo_acquire (enum memmodel model)
     }
 }
 
-/* Return true if a FENCE should be emitted to before a memory access to
-   implement the release portion of memory model MODEL.  */
+/* Return true if the .RL suffix should be added to an AMO to implement the
+   release portion of memory model MODEL.  */
 
 static bool
-riscv_memmodel_needs_release_fence (enum memmodel model)
+riscv_memmodel_needs_amo_release (enum memmodel model)
 {
   switch (model)
     {
@@ -4331,7 +4331,6 @@ riscv_memmodel_needs_release_fence (enum memmodel model)
    'R'	Print the low-part relocation associated with OP.
    'C'	Print the integer branch condition for comparison OP.
    'A'	Print the atomic operation suffix for memory model OP.
-   'F'	Print a FENCE if the memory model requires a release.
    'z'	Print x0 if OP is zero, otherwise print OP normally.
    'i'	Print i if the operand is not a register.
    'S'	Print shift-index of single-bit mask OP.
@@ -4493,19 +4492,14 @@ riscv_print_operand (FILE *file, rtx op, int letter)
 
     case 'A':
       if (riscv_memmodel_needs_amo_acquire (model) &&
-	  riscv_memmodel_needs_release_fence (model))
+	  riscv_memmodel_needs_amo_release (model))
 	fputs (".aqrl", file);
       else if (riscv_memmodel_needs_amo_acquire (model))
 	fputs (".aq", file);
-      else if (riscv_memmodel_needs_release_fence (model))
+      else if (riscv_memmodel_needs_amo_release (model))
 	fputs (".rl", file);
       break;
 
-    case 'F':
-      if (riscv_memmodel_needs_release_fence (model))
-	fputs ("fence iorw,ow; ", file);
-      break;
-
     case 'i':
       if (code != REG)
         fputs ("i", file);
diff --git a/gcc/config/riscv/sync.md b/gcc/config/riscv/sync.md
index de42245981b..1aa9ac81cee 100644
--- a/gcc/config/riscv/sync.md
+++ b/gcc/config/riscv/sync.md
@@ -61,9 +61,9 @@
        (match_operand:SI 2 "const_int_operand")]      ;; model
       UNSPEC_ATOMIC_STORE))]
   "TARGET_ATOMIC"
-  "%F2amoswap.<amo>%A2 zero,%z1,%0"
+  "amoswap.<amo>%A2 zero,%z1,%0"
   [(set_attr "type" "atomic")
-   (set (attr "length") (const_int 8))])
+   (set (attr "length") (const_int 4))])
 
 (define_insn "atomic_<atomic_optab><mode>"
   [(set (match_operand:GPR 0 "memory_operand" "+A")
@@ -73,9 +73,9 @@
 	   (match_operand:SI 2 "const_int_operand")] ;; model
 	 UNSPEC_SYNC_OLD_OP))]
   "TARGET_ATOMIC"
-  "%F2amo<insn>.<amo>%A2 zero,%z1,%0"
+  "amo<insn>.<amo>%A2 zero,%z1,%0"
   [(set_attr "type" "atomic")
-   (set (attr "length") (const_int 8))])
+   (set (attr "length") (const_int 4))])
 
 (define_insn "atomic_fetch_<atomic_optab><mode>"
   [(set (match_operand:GPR 0 "register_operand" "=&r")
@@ -87,9 +87,9 @@
 	   (match_operand:SI 3 "const_int_operand")] ;; model
 	 UNSPEC_SYNC_OLD_OP))]
   "TARGET_ATOMIC"
-  "%F3amo<insn>.<amo>%A3 %0,%z2,%1"
+  "amo<insn>.<amo>%A3 %0,%z2,%1"
   [(set_attr "type" "atomic")
-   (set (attr "length") (const_int 8))])
+   (set (attr "length") (const_int 4))])
 
 (define_insn "atomic_exchange<mode>"
   [(set (match_operand:GPR 0 "register_operand" "=&r")
@@ -100,9 +100,9 @@
    (set (match_dup 1)
 	(match_operand:GPR 2 "register_operand" "0"))]
   "TARGET_ATOMIC"
-  "%F3amoswap.<amo>%A3 %0,%z2,%1"
+  "amoswap.<amo>%A3 %0,%z2,%1"
   [(set_attr "type" "atomic")
-   (set (attr "length") (const_int 8))])
+   (set (attr "length") (const_int 4))])
 
 (define_insn "atomic_cas_value_strong<mode>"
   [(set (match_operand:GPR 0 "register_operand" "=&r")
-- 
2.25.1

[PATCH v2 6/8] RISCV: Weaken compare_exchange LR/SC pairs

[Patrick O'Neill]

Introduce the %I and %J flags for setting the .aqrl bits on LR/SC pairs
as needed.

Atomic compare and exchange ops provide success and failure memory
models. C++17 and later place no restrictions on the relative strength
of each model, so ensure we cover both by using a model that enforces
the ordering of both given models.

This change brings compare_exchange LR/SC ops in line with table A.6 of the ISA
manual.

2023-04-05 Patrick O'Neill <patrick@rivosinc.com>
	
	* riscv.cc: Add function to get the union of two
	memmodels in sync.md.
	* riscv-protos.h: Likewise.
	* sync.md (atomic_cas_value_strong<mode>): Remove static
	.aqrl bits on SC op/.rl bits on LR op and replace with
	optimized %I, %J flags.
	* compare-exchange-atomics-model-1.c: New test.
	* compare-exchange-atomics-model-2.c: Likewise.
	* compare-exchange-atomics-model-3.c: Likewise.
	* compare-exchange-atomics-model-4.c: Likewise.
	* compare-exchange-atomics-model-5.c: Likewise.
	* compare-exchange-atomics-model-6.c: Likewise.

Signed-off-by: Patrick O'Neill <patrick@rivosinc.com>
---
 gcc/config/riscv/riscv-protos.h               |  3 ++
 gcc/config/riscv/riscv.cc                     | 44 +++++++++++++++++++
 gcc/config/riscv/sync.md                      |  9 +++-
 .../riscv/compare-exchange-atomics-model-1.c  | 12 +++++
 .../riscv/compare-exchange-atomics-model-2.c  | 12 +++++
 .../riscv/compare-exchange-atomics-model-3.c  | 12 +++++
 .../riscv/compare-exchange-atomics-model-4.c  | 12 +++++
 .../riscv/compare-exchange-atomics-model-5.c  | 12 +++++
 8 files changed, 114 insertions(+), 2 deletions(-)
 create mode 100644 gcc/testsuite/gcc.target/riscv/compare-exchange-atomics-model-1.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/compare-exchange-atomics-model-2.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/compare-exchange-atomics-model-3.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/compare-exchange-atomics-model-4.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/compare-exchange-atomics-model-5.c

diff --git a/gcc/config/riscv/riscv-protos.h b/gcc/config/riscv/riscv-protos.h
index 4611447ddde..b03edc3e8a5 100644
--- a/gcc/config/riscv/riscv-protos.h
+++ b/gcc/config/riscv/riscv-protos.h
@@ -22,6 +22,8 @@ along with GCC; see the file COPYING3.  If not see
 #ifndef GCC_RISCV_PROTOS_H
 #define GCC_RISCV_PROTOS_H
 
+#include "memmodel.h"
+
 /* Symbol types we understand.  The order of this list must match that of
    the unspec enum in riscv.md, subsequent to UNSPEC_ADDRESS_FIRST.  */
 enum riscv_symbol_type {
@@ -79,6 +81,7 @@ extern void riscv_reinit (void);
 extern poly_uint64 riscv_regmode_natural_size (machine_mode);
 extern bool riscv_v_ext_vector_mode_p (machine_mode);
 extern bool riscv_shamt_matches_mask_p (int, HOST_WIDE_INT);
+extern enum memmodel riscv_union_memmodels (enum memmodel, enum memmodel);
 
 /* Routines implemented in riscv-c.cc.  */
 void riscv_cpu_cpp_builtins (cpp_reader *);
diff --git a/gcc/config/riscv/riscv.cc b/gcc/config/riscv/riscv.cc
index 6576e9ae524..061d2cf42b4 100644
--- a/gcc/config/riscv/riscv.cc
+++ b/gcc/config/riscv/riscv.cc
@@ -4278,6 +4278,36 @@ riscv_print_operand_reloc (FILE *file, rtx op, bool hi_reloc)
   fputc (')', file);
 }
 
+/* Return the memory model that encapuslates both given models.  */
+
+enum memmodel
+riscv_union_memmodels (enum memmodel model1, enum memmodel model2)
+{
+  model1 = memmodel_base (model1);
+  model2 = memmodel_base (model2);
+
+  enum memmodel weaker = model1 <= model2 ? model1: model2;
+  enum memmodel stronger = model1 > model2 ? model1: model2;
+
+  switch (stronger)
+    {
+      case MEMMODEL_SEQ_CST:
+      case MEMMODEL_ACQ_REL:
+	return stronger;
+      case MEMMODEL_RELEASE:
+	if (weaker == MEMMODEL_ACQUIRE || weaker == MEMMODEL_CONSUME)
+	  return MEMMODEL_ACQ_REL;
+	else
+	  return stronger;
+      case MEMMODEL_ACQUIRE:
+      case MEMMODEL_CONSUME:
+      case MEMMODEL_RELAXED:
+	return stronger;
+      default:
+	gcc_unreachable ();
+    }
+}
+
 /* Return true if the .AQ suffix should be added to an AMO to implement the
    acquire portion of memory model MODEL.  */
 
@@ -4331,6 +4361,8 @@ riscv_memmodel_needs_amo_release (enum memmodel model)
    'R'	Print the low-part relocation associated with OP.
    'C'	Print the integer branch condition for comparison OP.
    'A'	Print the atomic operation suffix for memory model OP.
+   'I'	Print the LR suffix for memory model OP.
+   'J'	Print the SC suffix for memory model OP.
    'z'	Print x0 if OP is zero, otherwise print OP normally.
    'i'	Print i if the operand is not a register.
    'S'	Print shift-index of single-bit mask OP.
@@ -4500,6 +4532,18 @@ riscv_print_operand (FILE *file, rtx op, int letter)
 	fputs (".rl", file);
       break;
 
+    case 'I':
+      if (model == MEMMODEL_SEQ_CST)
+	fputs (".aqrl", file);
+      else if (riscv_memmodel_needs_amo_acquire (model))
+	fputs (".aq", file);
+      break;
+
+    case 'J':
+      if (riscv_memmodel_needs_amo_release (model))
+	fputs (".rl", file);
+      break;
+
     case 'i':
       if (code != REG)
         fputs ("i", file);
diff --git a/gcc/config/riscv/sync.md b/gcc/config/riscv/sync.md
index 1aa9ac81cee..b1a12545a19 100644
--- a/gcc/config/riscv/sync.md
+++ b/gcc/config/riscv/sync.md
@@ -116,10 +116,15 @@
    (clobber (match_scratch:GPR 6 "=&r"))]
   "TARGET_ATOMIC"
   {
+    enum memmodel model_success = (enum memmodel) INTVAL(operands[4]);
+    enum memmodel model_failure = (enum memmodel) INTVAL(operands[5]);
+    /* Find the union of the two memory models so we can satisfy both success
+       and failure memory models.  */
+    operands[5] = GEN_INT(riscv_union_memmodels(model_success, model_failure));
     return "1:\;"
-	   "lr.<amo>.aqrl\t%0,%1\;"
+	   "lr.<amo>%I5\t%0,%1\;"
 	   "bne\t%0,%z2,1f\;"
-	   "sc.<amo>.rl\t%6,%z3,%1\;"
+	   "sc.<amo>%J5\t%6,%z3,%1\;"
 	   "bnez\t%6,1b\;"
 	   "1:";
   }
diff --git a/gcc/testsuite/gcc.target/riscv/compare-exchange-atomics-model-1.c b/gcc/testsuite/gcc.target/riscv/compare-exchange-atomics-model-1.c
new file mode 100644
index 00000000000..a2c3fc7a1b6
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/compare-exchange-atomics-model-1.c
@@ -0,0 +1,12 @@
+/* { dg-do compile } */
+/* Verify that appropriate bits are placed per memory model.  */
+/* { dg-final { scan-assembler-not "lr.w.aq" } } */
+/* { dg-final { scan-assembler-not "lr.w.rl" } } */
+/* { dg-final { scan-assembler-not "sc.w.aq" } } */
+/* { dg-final { scan-assembler-not "sc.w.rl" } } */
+
+void
+foo (int bar, int baz, int qux)
+{
+  __atomic_compare_exchange_n(&bar, &baz, qux, 1, __ATOMIC_RELAXED, __ATOMIC_RELAXED);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/compare-exchange-atomics-model-2.c b/gcc/testsuite/gcc.target/riscv/compare-exchange-atomics-model-2.c
new file mode 100644
index 00000000000..d23d4db945f
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/compare-exchange-atomics-model-2.c
@@ -0,0 +1,12 @@
+/* { dg-do compile } */
+/* Verify that appropriate bits are placed per memory model.  */
+/* { dg-final { scan-assembler "lr.w.aq" } } */
+/* { dg-final { scan-assembler-not "lr.w.rl" } } */
+/* { dg-final { scan-assembler-not "sc.w.aq" } } */
+/* { dg-final { scan-assembler-not "sc.w.rl" } } */
+
+void
+foo (int bar, int baz, int qux)
+{
+  __atomic_compare_exchange_n(&bar, &baz, qux, 1, __ATOMIC_CONSUME, __ATOMIC_CONSUME);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/compare-exchange-atomics-model-3.c b/gcc/testsuite/gcc.target/riscv/compare-exchange-atomics-model-3.c
new file mode 100644
index 00000000000..7379825c6f7
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/compare-exchange-atomics-model-3.c
@@ -0,0 +1,12 @@
+/* { dg-do compile } */
+/* Verify that appropriate bits are placed per memory model.  */
+/* { dg-final { scan-assembler "lr.w.aq" } } */
+/* { dg-final { scan-assembler-not "lr.w.rl" } } */
+/* { dg-final { scan-assembler-not "sc.w.aq" } } */
+/* { dg-final { scan-assembler-not "sc.w.rl" } } */
+
+void
+foo (int bar, int baz, int qux)
+{
+  __atomic_compare_exchange_n(&bar, &baz, qux, 1, __ATOMIC_ACQUIRE, __ATOMIC_ACQUIRE);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/compare-exchange-atomics-model-4.c b/gcc/testsuite/gcc.target/riscv/compare-exchange-atomics-model-4.c
new file mode 100644
index 00000000000..80ab9889288
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/compare-exchange-atomics-model-4.c
@@ -0,0 +1,12 @@
+/* { dg-do compile } */
+/* Verify that appropriate bits are placed per memory model.  */
+/* { dg-final { scan-assembler "lr.w.aqrl" } } */
+/* { dg-final { scan-assembler "sc.w.rl" } } */
+/* { dg-final { scan-assembler-not "lr.w.rl" } } */
+/* { dg-final { scan-assembler-not "sc.w.aq" } } */
+
+void
+foo (int bar, int baz, int qux)
+{
+  __atomic_compare_exchange_n(&bar, &baz, qux, 1, __ATOMIC_SEQ_CST, __ATOMIC_SEQ_CST);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/compare-exchange-atomics-model-5.c b/gcc/testsuite/gcc.target/riscv/compare-exchange-atomics-model-5.c
new file mode 100644
index 00000000000..da905242317
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/compare-exchange-atomics-model-5.c
@@ -0,0 +1,12 @@
+/* { dg-do compile } */
+/* Verify that appropriate bits are placed per memory model.  */
+/* { dg-final { scan-assembler "lr.w.aq" } } */
+/* { dg-final { scan-assembler "sc.w.rl" } } */
+/* { dg-final { scan-assembler-not "lr.w.rl" } } */
+/* { dg-final { scan-assembler-not "sc.w.aq" } } */
+
+void
+foo (int bar, int baz, int qux)
+{
+  __atomic_compare_exchange_n(&bar, &baz, qux, 1, __ATOMIC_RELEASE, __ATOMIC_ACQUIRE);
+}
-- 
2.25.1

[PATCH v2 7/8] RISCV: Weaken atomic stores

[Patrick O'Neill]

This change brings atomic stores in line with table A.6 of the ISA
manual.

2023-04-05 Patrick O'Neill <patrick@rivosinc.com>

	PR target/89835
	* sync.md (atomic_store<mode>): Use simple store instruction in
	combination with a fence.
	* pr89835.c: New test.
	
Signed-off-by: Patrick O'Neill <patrick@rivosinc.com>
---
 gcc/config/riscv/sync.md                 | 17 ++++++++++++-----
 gcc/testsuite/gcc.target/riscv/pr89835.c |  9 +++++++++
 2 files changed, 21 insertions(+), 5 deletions(-)
 create mode 100644 gcc/testsuite/gcc.target/riscv/pr89835.c

diff --git a/gcc/config/riscv/sync.md b/gcc/config/riscv/sync.md
index b1a12545a19..cdd227721e1 100644
--- a/gcc/config/riscv/sync.md
+++ b/gcc/config/riscv/sync.md
@@ -53,17 +53,24 @@
 
 ;; Atomic memory operations.
 
-;; Implement atomic stores with amoswap.  Fall back to fences for atomic loads.
+;; Implement atomic stores with a leading fence.  Fall back to fences for atomic loads.
 (define_insn "atomic_store<mode>"
   [(set (match_operand:GPR 0 "memory_operand" "=A")
     (unspec_volatile:GPR
       [(match_operand:GPR 1 "reg_or_0_operand" "rJ")
        (match_operand:SI 2 "const_int_operand")]      ;; model
       UNSPEC_ATOMIC_STORE))]
-  "TARGET_ATOMIC"
-  "amoswap.<amo>%A2 zero,%z1,%0"
-  [(set_attr "type" "atomic")
-   (set (attr "length") (const_int 4))])
+  ""
+  {
+    enum memmodel model = (enum memmodel) INTVAL (operands[2]);
+    model = memmodel_base (model);
+    if (model == MEMMODEL_SEQ_CST || model == MEMMODEL_RELEASE)
+      return "fence\tr,rw\;"
+	     "s<amo>\t%z1,%0";
+    else
+      return "s<amo>\t%z1,%0\;";
+  }
+  [(set (attr "length") (const_int 8))])
 
 (define_insn "atomic_<atomic_optab><mode>"
   [(set (match_operand:GPR 0 "memory_operand" "+A")
diff --git a/gcc/testsuite/gcc.target/riscv/pr89835.c b/gcc/testsuite/gcc.target/riscv/pr89835.c
new file mode 100644
index 00000000000..ab190e11b60
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/pr89835.c
@@ -0,0 +1,9 @@
+/* { dg-do compile } */
+/* Verify that relaxed atomic stores use simple store instuctions.  */
+/* { dg-final { scan-assembler-not "amoswap" } } */
+
+void
+foo(int bar, int baz)
+{
+  __atomic_store_n(&bar, baz, __ATOMIC_RELAXED);
+}
-- 
2.25.1

[PATCH v2 8/8] RISCV: Weaken mem_thread_fence

[Patrick O'Neill]

This change brings atomic fences in line with table A.6 of the ISA
manual.

Relax mem_thread_fence according to the memmodel given.

2023-04-05 Patrick O'Neill <patrick@rivosinc.com>

	* riscv.cc: Expose helper functions to sync.md.
	* riscv-protos.h: Likewise.
	* sync.md (mem_thread_fence_1): Change fence depending on
	aquire/release requirements.
	* amo-thread-fence-1: New test.
	* amo-thread-fence-2: Likewise.
	* amo-thread-fence-3: Likewise.
	* amo-thread-fence-4: Likewise.
	* amo-thread-fence-5: Likewise.

Signed-off-by: Patrick O'Neill <patrick@rivosinc.com>
---
 gcc/config/riscv/riscv-protos.h                 |  2 ++
 gcc/config/riscv/riscv.cc                       |  4 ++--
 gcc/config/riscv/sync.md                        | 17 ++++++++++++++---
 .../gcc.target/riscv/amo-thread-fence-1.c       |  6 ++++++
 .../gcc.target/riscv/amo-thread-fence-2.c       |  6 ++++++
 .../gcc.target/riscv/amo-thread-fence-3.c       |  6 ++++++
 .../gcc.target/riscv/amo-thread-fence-4.c       |  6 ++++++
 .../gcc.target/riscv/amo-thread-fence-5.c       |  6 ++++++
 8 files changed, 48 insertions(+), 5 deletions(-)
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-thread-fence-1.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-thread-fence-2.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-thread-fence-3.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-thread-fence-4.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-thread-fence-5.c

diff --git a/gcc/config/riscv/riscv-protos.h b/gcc/config/riscv/riscv-protos.h
index b03edc3e8a5..233b8070047 100644
--- a/gcc/config/riscv/riscv-protos.h
+++ b/gcc/config/riscv/riscv-protos.h
@@ -81,6 +81,8 @@ extern void riscv_reinit (void);
 extern poly_uint64 riscv_regmode_natural_size (machine_mode);
 extern bool riscv_v_ext_vector_mode_p (machine_mode);
 extern bool riscv_shamt_matches_mask_p (int, HOST_WIDE_INT);
+extern bool riscv_memmodel_needs_amo_acquire (enum memmodel);
+extern bool riscv_memmodel_needs_amo_release (enum memmodel);
 extern enum memmodel riscv_union_memmodels (enum memmodel, enum memmodel);
 
 /* Routines implemented in riscv-c.cc.  */
diff --git a/gcc/config/riscv/riscv.cc b/gcc/config/riscv/riscv.cc
index 061d2cf42b4..27d876ac44c 100644
--- a/gcc/config/riscv/riscv.cc
+++ b/gcc/config/riscv/riscv.cc
@@ -4311,7 +4311,7 @@ riscv_union_memmodels (enum memmodel model1, enum memmodel model2)
 /* Return true if the .AQ suffix should be added to an AMO to implement the
    acquire portion of memory model MODEL.  */
 
-static bool
+bool
 riscv_memmodel_needs_amo_acquire (enum memmodel model)
 {
   switch (model)
@@ -4334,7 +4334,7 @@ riscv_memmodel_needs_amo_acquire (enum memmodel model)
 /* Return true if the .RL suffix should be added to an AMO to implement the
    release portion of memory model MODEL.  */
 
-static bool
+bool
 riscv_memmodel_needs_amo_release (enum memmodel model)
 {
   switch (model)
diff --git a/gcc/config/riscv/sync.md b/gcc/config/riscv/sync.md
index cdd227721e1..4204c956bd6 100644
--- a/gcc/config/riscv/sync.md
+++ b/gcc/config/riscv/sync.md
@@ -42,14 +42,25 @@
   DONE;
 })
 
-;; Until the RISC-V memory model (hence its mapping from C++) is finalized,
-;; conservatively emit a full FENCE.
 (define_insn "mem_thread_fence_1"
   [(set (match_operand:BLK 0 "" "")
 	(unspec:BLK [(match_dup 0)] UNSPEC_MEMORY_BARRIER))
    (match_operand:SI 1 "const_int_operand" "")] ;; model
   ""
-  "fence\tiorw,iorw")
+  {
+    enum memmodel model = (enum memmodel) INTVAL (operands[1]);
+    model = memmodel_base (model);
+    if (model == MEMMODEL_ACQ_REL)
+	return "fence.tso";
+    else if (riscv_memmodel_needs_amo_acquire (model) &&
+	     riscv_memmodel_needs_amo_release (model))
+	return "fence\trw,rw";
+    else if (riscv_memmodel_needs_amo_acquire (model))
+	return "fence\tr,rw";
+    else if (riscv_memmodel_needs_amo_release (model))
+	return "fence\trw,w";
+  }
+  [(set (attr "length") (const_int 4))])
 
 ;; Atomic memory operations.
 
diff --git a/gcc/testsuite/gcc.target/riscv/amo-thread-fence-1.c b/gcc/testsuite/gcc.target/riscv/amo-thread-fence-1.c
new file mode 100644
index 00000000000..833629bf2f2
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-thread-fence-1.c
@@ -0,0 +1,6 @@
+/* { dg-do compile } */
+/* { dg-final { scan-assembler-not "fence\t" } } */
+
+int main() {
+  __atomic_thread_fence(__ATOMIC_RELAXED);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-thread-fence-2.c b/gcc/testsuite/gcc.target/riscv/amo-thread-fence-2.c
new file mode 100644
index 00000000000..3395ee41dbb
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-thread-fence-2.c
@@ -0,0 +1,6 @@
+/* { dg-do compile } */
+/* { dg-final { scan-assembler "fence\tr,rw" } } */
+
+int main() {
+  __atomic_thread_fence(__ATOMIC_ACQUIRE);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-thread-fence-3.c b/gcc/testsuite/gcc.target/riscv/amo-thread-fence-3.c
new file mode 100644
index 00000000000..59cc4e5d394
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-thread-fence-3.c
@@ -0,0 +1,6 @@
+/* { dg-do compile } */
+/* { dg-final { scan-assembler "fence\trw,w" } } */
+
+int main() {
+  __atomic_thread_fence(__ATOMIC_RELEASE);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-thread-fence-4.c b/gcc/testsuite/gcc.target/riscv/amo-thread-fence-4.c
new file mode 100644
index 00000000000..2afed9a9e38
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-thread-fence-4.c
@@ -0,0 +1,6 @@
+/* { dg-do compile } */
+/* { dg-final { scan-assembler "fence.tso" } } */
+
+int main() {
+  __atomic_thread_fence(__ATOMIC_ACQ_REL);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-thread-fence-5.c b/gcc/testsuite/gcc.target/riscv/amo-thread-fence-5.c
new file mode 100644
index 00000000000..b8d56c0f066
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-thread-fence-5.c
@@ -0,0 +1,6 @@
+/* { dg-do compile } */
+/* { dg-final { scan-assembler "fence\trw,rw" } } */
+
+int main() {
+  __atomic_thread_fence(__ATOMIC_SEQ_CST);
+}
-- 
2.25.1

[PATCH v3 00/10] RISCV: Implement ISA Manual Table A.6 Mappings

[Patrick O'Neill]

This patchset aims to make the RISCV atomics implementation stronger
than the recommended mapping present in table A.6 of the ISA manual.
  https://github.com/riscv/riscv-isa-manual/blob/c7cf84547b3aefacab5463add1734c1602b67a49/src/memory.tex#L1083-L1157

The current mapping in GCC is not internally consistent. Andrea Parri
pointed this out here along with a litmus test:
  https://inbox.sourceware.org/gcc-patches/Y1GbJuhcBFpPGJQ0@andrea/

As a result, we have an opportunity to jump straight to the A.6
implementation (meaning we will be compatible with LLVM's mappings which
are A.6). In light of a proposal by Hans Boehm and to avoid an ABI break
in the future, the mapping implemented is strictly stronger than the one
in table A.6 in order to be compatible with Table A.7.
  https://lists.riscv.org/g/tech-unprivileged/topic/risc_v_memory_model_topics/92916241

If Hans' proposal is accepted, it makes sense to migrate to the mapping
recommended by table A.7. Since the stronger mapping in this patchset
(provided by Hans Boehm) appears to be compatible with both A.6 and A.7,
this transition should not result in an ABI break for GCC.

Patch 1 simplifies the memmodel to ignore MEMMODEL_SYNC_* cases (legacy
cases that aren't handled differently for RISC-V).
Patches 2-5 make the mappings strictly stronger.
Patches 5-9 weaken the mappings to be in line with table A.6 of the ISA
manual.
Patch 10 adds some basic conformance tests to ensure the implemented
mapping matches table A.6 with stronger SEQ_CST stores.

Christoph Muellner also submitted a similar patchset here:
  https://gcc.gnu.org/pipermail/gcc-patches/2022-May/595712.html
I used my previous patchset as a starting point since it was easier for 
me.

LLVM mapping notes:
* LLVM emits corresponding fences for atomic_signal_fence instructions.
  This seems to be an oversight since AFAIK atomic_signal_fence acts as
  a compiler directive. GCC does not emit any fences for
  atomic_signal_fence instructions.

Patchset v1:
  https://gcc.gnu.org/pipermail/gcc-patches/2022-April/592950.html

Patchset v2:
  https://gcc.gnu.org/pipermail/gcc-patches/2023-April/615264.html

Changes for v2:
* Use memmodel_base rather than a custom simplify_memmodel function
  (Inspired by Christoph Muellner's patch 1/9)
* Move instruction styling change from [v1 5/7] to [v2 3/8] to reduce
  [v2 6/8]'s complexity
* Eliminated %K flag for atomic store introduced in v1 in favor of
  if/else
* Rebase/test

Changes for v3:
* Use a trailing fence for atomic stores to be compatible with Table A.7
* Emit an optimized fence r,rw following a SEQ_CST load
* Consolidate tests in [PATCH v3 10/10]
* Add tests for basic A.6 conformance

Patrick O'Neill (10):
  RISCV: Eliminate SYNC memory models
  RISCV: Enforce Libatomic LR/SC SEQ_CST
  RISCV: Enforce atomic compare_exchange SEQ_CST
  RISCV: Add AMO release bits
  RISCV: Strengthen atomic stores
  RISCV: Eliminate AMO op fences
  RISCV: Weaken compare_exchange LR/SC pairs
  RISCV: Weaken mem_thread_fence
  RISCV: Weaken atomic loads
  RISCV: Table A.6 conformance tests

 gcc/config/riscv/riscv-protos.h               |  3 +
 gcc/config/riscv/riscv.cc                     | 66 +++++++++++---
 gcc/config/riscv/sync.md                      | 89 ++++++++++++++++---
 .../riscv/amo-table-a-6-amo-add-1.c           |  8 ++
 .../riscv/amo-table-a-6-amo-add-2.c           |  8 ++
 .../riscv/amo-table-a-6-amo-add-3.c           |  8 ++
 .../riscv/amo-table-a-6-amo-add-4.c           |  8 ++
 .../riscv/amo-table-a-6-amo-add-5.c           |  8 ++
 .../riscv/amo-table-a-6-compare-exchange-1.c  | 12 +++
 .../riscv/amo-table-a-6-compare-exchange-2.c  | 12 +++
 .../riscv/amo-table-a-6-compare-exchange-3.c  | 12 +++
 .../riscv/amo-table-a-6-compare-exchange-4.c  | 12 +++
 .../riscv/amo-table-a-6-compare-exchange-5.c  | 12 +++
 .../gcc.target/riscv/amo-table-a-6-fence-1.c  |  9 ++
 .../gcc.target/riscv/amo-table-a-6-fence-2.c  |  7 ++
 .../gcc.target/riscv/amo-table-a-6-fence-3.c  |  7 ++
 .../gcc.target/riscv/amo-table-a-6-fence-4.c  |  7 ++
 .../gcc.target/riscv/amo-table-a-6-fence-5.c  |  7 ++
 .../gcc.target/riscv/amo-table-a-6-load-1.c   |  9 ++
 .../gcc.target/riscv/amo-table-a-6-load-2.c   | 10 +++
 .../gcc.target/riscv/amo-table-a-6-load-3.c   | 10 +++
 .../gcc.target/riscv/amo-table-a-6-store-1.c  |  9 ++
 .../gcc.target/riscv/amo-table-a-6-store-2.c  | 10 +++
 .../riscv/amo-table-a-6-store-compat-3.c      | 10 +++
 gcc/testsuite/gcc.target/riscv/pr89835.c      |  9 ++
 libgcc/config/riscv/atomic.c                  |  4 +-
 26 files changed, 336 insertions(+), 30 deletions(-)
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-1.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-2.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-3.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-4.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-5.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-1.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-2.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-3.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-4.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-5.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-1.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-2.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-3.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-4.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-5.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-1.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-2.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-3.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-1.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-2.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-compat-3.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/pr89835.c

-- 
2.25.1

[PATCH v3 01/10] RISCV: Eliminate SYNC memory models

[Patrick O'Neill]

Remove references to MEMMODEL_SYNC_* models by converting via
memmodel_base().

2023-04-10 Patrick O'Neill <patrick@rivosinc.com>

	* riscv.cc: Remove MEMMODEL_SYNC_* cases and sanitize memmodel
	input with memmodel_base

Signed-off-by: Patrick O'Neill <patrick@rivosinc.com>
---
 gcc/config/riscv/riscv.cc | 11 +++--------
 1 file changed, 3 insertions(+), 8 deletions(-)

diff --git a/gcc/config/riscv/riscv.cc b/gcc/config/riscv/riscv.cc
index 76eee4a55e9..8f5636c93ed 100644
--- a/gcc/config/riscv/riscv.cc
+++ b/gcc/config/riscv/riscv.cc
@@ -4288,14 +4288,11 @@ riscv_memmodel_needs_amo_acquire (enum memmodel model)
     {
       case MEMMODEL_ACQ_REL:
       case MEMMODEL_SEQ_CST:
-      case MEMMODEL_SYNC_SEQ_CST:
       case MEMMODEL_ACQUIRE:
       case MEMMODEL_CONSUME:
-      case MEMMODEL_SYNC_ACQUIRE:
 	return true;
 
       case MEMMODEL_RELEASE:
-      case MEMMODEL_SYNC_RELEASE:
       case MEMMODEL_RELAXED:
 	return false;
 
@@ -4314,14 +4311,11 @@ riscv_memmodel_needs_release_fence (enum memmodel model)
     {
       case MEMMODEL_ACQ_REL:
       case MEMMODEL_SEQ_CST:
-      case MEMMODEL_SYNC_SEQ_CST:
       case MEMMODEL_RELEASE:
-      case MEMMODEL_SYNC_RELEASE:
 	return true;
 
       case MEMMODEL_ACQUIRE:
       case MEMMODEL_CONSUME:
-      case MEMMODEL_SYNC_ACQUIRE:
       case MEMMODEL_RELAXED:
 	return false;
 
@@ -4360,6 +4354,7 @@ riscv_print_operand (FILE *file, rtx op, int letter)
     }
   machine_mode mode = GET_MODE (op);
   enum rtx_code code = GET_CODE (op);
+  const enum memmodel model = memmodel_base (INTVAL (op));
 
   switch (letter)
     {
@@ -4497,12 +4492,12 @@ riscv_print_operand (FILE *file, rtx op, int letter)
       break;
 
     case 'A':
-      if (riscv_memmodel_needs_amo_acquire ((enum memmodel) INTVAL (op)))
+      if (riscv_memmodel_needs_amo_acquire (model))
 	fputs (".aq", file);
       break;
 
     case 'F':
-      if (riscv_memmodel_needs_release_fence ((enum memmodel) INTVAL (op)))
+      if (riscv_memmodel_needs_release_fence (model))
 	fputs ("fence iorw,ow; ", file);
       break;
 
-- 
2.25.1

[PATCH v3 02/10] RISCV: Enforce Libatomic LR/SC SEQ_CST

[Patrick O'Neill]

Replace LR.aq/SC.rl pairs with the SEQ_CST LR.aqrl/SC.rl pairs
recommended by table A.6 of the ISA manual.

2023-04-10 Patrick O'Neill <patrick@rivosinc.com>

	* atomic.c: Change LR.aq/SC.rl pairs into sequentially
	consistent LR.aqrl/SC.rl pair.

Signed-off-by: Patrick O'Neill <patrick@rivosinc.com>
---
 libgcc/config/riscv/atomic.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/libgcc/config/riscv/atomic.c b/libgcc/config/riscv/atomic.c
index 69f53623509..5f895939b0b 100644
--- a/libgcc/config/riscv/atomic.c
+++ b/libgcc/config/riscv/atomic.c
@@ -39,7 +39,7 @@ see the files COPYING3 and COPYING.RUNTIME respectively.  If not, see
     unsigned old, tmp1, tmp2;						\
 									\
     asm volatile ("1:\n\t"						\
-		  "lr.w.aq %[old], %[mem]\n\t"				\
+		  "lr.w.aqrl %[old], %[mem]\n\t"			\
 		  #insn " %[tmp1], %[old], %[value]\n\t"		\
 		  invert						\
 		  "and %[tmp1], %[tmp1], %[mask]\n\t"			\
@@ -73,7 +73,7 @@ see the files COPYING3 and COPYING.RUNTIME respectively.  If not, see
     unsigned old, tmp1;							\
 									\
     asm volatile ("1:\n\t"						\
-		  "lr.w.aq %[old], %[mem]\n\t"				\
+		  "lr.w.aqrl %[old], %[mem]\n\t"			\
 		  "and %[tmp1], %[old], %[mask]\n\t"			\
 		  "bne %[tmp1], %[o], 1f\n\t"				\
 		  "and %[tmp1], %[old], %[not_mask]\n\t"		\
-- 
2.25.1

[PATCH v3 03/10] RISCV: Enforce atomic compare_exchange SEQ_CST

[Patrick O'Neill]

This patch enforces SEQ_CST for atomic compare_exchange ops.

Replace Fence/LR.aq/SC.aq pairs with SEQ_CST LR.aqrl/SC.rl pairs
recommended by table A.6 of the ISA manual.

2023-04-10 Patrick O'Neill <patrick@rivosinc.com>

	* sync.md: Change FENCE/LR.aq/SC.aq into sequentially
	consistent LR.aqrl/SC.rl pair.

Signed-off-by: Patrick O'Neill <patrick@rivosinc.com>
---
 gcc/config/riscv/sync.md | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)

diff --git a/gcc/config/riscv/sync.md b/gcc/config/riscv/sync.md
index c932ef87b9d..de42245981b 100644
--- a/gcc/config/riscv/sync.md
+++ b/gcc/config/riscv/sync.md
@@ -115,9 +115,16 @@
 	 UNSPEC_COMPARE_AND_SWAP))
    (clobber (match_scratch:GPR 6 "=&r"))]
   "TARGET_ATOMIC"
-  "%F5 1: lr.<amo>%A5 %0,%1; bne %0,%z2,1f; sc.<amo>%A4 %6,%z3,%1; bnez %6,1b; 1:"
+  {
+    return "1:\;"
+	   "lr.<amo>.aqrl\t%0,%1\;"
+	   "bne\t%0,%z2,1f\;"
+	   "sc.<amo>.rl\t%6,%z3,%1\;"
+	   "bnez\t%6,1b\;"
+	   "1:";
+  }
   [(set_attr "type" "atomic")
-   (set (attr "length") (const_int 20))])
+   (set (attr "length") (const_int 16))])
 
 (define_expand "atomic_compare_and_swap<mode>"
   [(match_operand:SI 0 "register_operand" "")   ;; bool output
-- 
2.25.1

[PATCH v3 04/10] RISCV: Add AMO release bits

[Patrick O'Neill]

This patch sets the relevant .rl bits on amo operations.

2023-04-10 Patrick O'Neill <patrick@rivosinc.com>

	* riscv.cc (riscv_print_operand): change behavior of %A to
	include release bits.

Signed-off-by: Patrick O'Neill <patrick@rivosinc.com>
---
 gcc/config/riscv/riscv.cc | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/gcc/config/riscv/riscv.cc b/gcc/config/riscv/riscv.cc
index 8f5636c93ed..8ffee494fbe 100644
--- a/gcc/config/riscv/riscv.cc
+++ b/gcc/config/riscv/riscv.cc
@@ -4492,8 +4492,13 @@ riscv_print_operand (FILE *file, rtx op, int letter)
       break;
 
     case 'A':
-      if (riscv_memmodel_needs_amo_acquire (model))
+      if (riscv_memmodel_needs_amo_acquire (model) &&
+	  riscv_memmodel_needs_release_fence (model))
+	fputs (".aqrl", file);
+      else if (riscv_memmodel_needs_amo_acquire (model))
 	fputs (".aq", file);
+      else if (riscv_memmodel_needs_release_fence (model))
+	fputs (".rl", file);
       break;
 
     case 'F':
-- 
2.25.1

[PATCH v3 05/10] RISCV: Strengthen atomic stores

[Patrick O'Neill]

This change makes atomic stores strictly stronger than table A.6 of the
ISA manual. This mapping makes the overall patchset compatible with
table A.7 as well.

2023-04-10 Patrick O'Neill <patrick@rivosinc.com>

	PR target/89835
	* sync.md (atomic_store<mode>): Use simple store instruction in
	combination with a fence.
	* pr89835.c: New test.

Signed-off-by: Patrick O'Neill <patrick@rivosinc.com>
---
v3 Changelog:
* Use a trailing fence for atomic stores to be compatible with Table A.7
---
 gcc/config/riscv/sync.md                 | 20 +++++++++++++++++---
 gcc/testsuite/gcc.target/riscv/pr89835.c |  9 +++++++++
 2 files changed, 26 insertions(+), 3 deletions(-)
 create mode 100644 gcc/testsuite/gcc.target/riscv/pr89835.c

diff --git a/gcc/config/riscv/sync.md b/gcc/config/riscv/sync.md
index de42245981b..eef083b06e8 100644
--- a/gcc/config/riscv/sync.md
+++ b/gcc/config/riscv/sync.md
@@ -53,7 +53,8 @@
 
 ;; Atomic memory operations.
 
-;; Implement atomic stores with amoswap.  Fall back to fences for atomic loads.
+;; Implement atomic stores with conservative fences. Fall back to fences for atomic loads.
+;; This allows us to be compatible with the ISA manual Table A.6 and Table A.7.
 (define_insn "atomic_store<mode>"
   [(set (match_operand:GPR 0 "memory_operand" "=A")
     (unspec_volatile:GPR
@@ -61,9 +62,22 @@
        (match_operand:SI 2 "const_int_operand")]      ;; model
       UNSPEC_ATOMIC_STORE))]
   "TARGET_ATOMIC"
-  "%F2amoswap.<amo>%A2 zero,%z1,%0"
+  {
+    enum memmodel model = (enum memmodel) INTVAL (operands[2]);
+    model = memmodel_base (model);
+
+    if (model == MEMMODEL_SEQ_CST)
+      return "fence\trw,w\;"
+	     "s<amo>\t%z1,%0\;"
+	     "fence\trw,rw";
+    if (model == MEMMODEL_RELEASE)
+      return "fence\trw,w\;"
+	     "s<amo>\t%z1,%0";
+    else
+      return "s<amo>\t%z1,%0";
+  }
   [(set_attr "type" "atomic")
-   (set (attr "length") (const_int 8))])
+   (set (attr "length") (const_int 12))])
 
 (define_insn "atomic_<atomic_optab><mode>"
   [(set (match_operand:GPR 0 "memory_operand" "+A")
diff --git a/gcc/testsuite/gcc.target/riscv/pr89835.c b/gcc/testsuite/gcc.target/riscv/pr89835.c
new file mode 100644
index 00000000000..ab190e11b60
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/pr89835.c
@@ -0,0 +1,9 @@
+/* { dg-do compile } */
+/* Verify that relaxed atomic stores use simple store instuctions.  */
+/* { dg-final { scan-assembler-not "amoswap" } } */
+
+void
+foo(int bar, int baz)
+{
+  __atomic_store_n(&bar, baz, __ATOMIC_RELAXED);
+}
-- 
2.25.1

[PATCH v3 06/10] RISCV: Eliminate AMO op fences

[Patrick O'Neill]

Atomic operations with the appropriate bits set already enfore release
semantics. Remove unnecessary release fences from atomic ops.

This change brings AMO ops in line with table A.6 of the ISA manual.

2023-04-10 Patrick O'Neill <patrick@rivosinc.com>

	* riscv.cc (riscv_memmodel_needs_amo_acquire): Change function
	name.
	* riscv.cc (riscv_print_operand): Remove unneeded %F case.
	* sync.md: Remove unneeded fences.

Signed-off-by: Patrick O'Neill <patrick@rivosinc.com>
---
 gcc/config/riscv/riscv.cc | 16 +++++-----------
 gcc/config/riscv/sync.md  | 12 ++++++------
 2 files changed, 11 insertions(+), 17 deletions(-)

diff --git a/gcc/config/riscv/riscv.cc b/gcc/config/riscv/riscv.cc
index 8ffee494fbe..6576e9ae524 100644
--- a/gcc/config/riscv/riscv.cc
+++ b/gcc/config/riscv/riscv.cc
@@ -4301,11 +4301,11 @@ riscv_memmodel_needs_amo_acquire (enum memmodel model)
     }
 }
 
-/* Return true if a FENCE should be emitted to before a memory access to
-   implement the release portion of memory model MODEL.  */
+/* Return true if the .RL suffix should be added to an AMO to implement the
+   release portion of memory model MODEL.  */
 
 static bool
-riscv_memmodel_needs_release_fence (enum memmodel model)
+riscv_memmodel_needs_amo_release (enum memmodel model)
 {
   switch (model)
     {
@@ -4331,7 +4331,6 @@ riscv_memmodel_needs_release_fence (enum memmodel model)
    'R'	Print the low-part relocation associated with OP.
    'C'	Print the integer branch condition for comparison OP.
    'A'	Print the atomic operation suffix for memory model OP.
-   'F'	Print a FENCE if the memory model requires a release.
    'z'	Print x0 if OP is zero, otherwise print OP normally.
    'i'	Print i if the operand is not a register.
    'S'	Print shift-index of single-bit mask OP.
@@ -4493,19 +4492,14 @@ riscv_print_operand (FILE *file, rtx op, int letter)
 
     case 'A':
       if (riscv_memmodel_needs_amo_acquire (model) &&
-	  riscv_memmodel_needs_release_fence (model))
+	  riscv_memmodel_needs_amo_release (model))
 	fputs (".aqrl", file);
       else if (riscv_memmodel_needs_amo_acquire (model))
 	fputs (".aq", file);
-      else if (riscv_memmodel_needs_release_fence (model))
+      else if (riscv_memmodel_needs_amo_release (model))
 	fputs (".rl", file);
       break;
 
-    case 'F':
-      if (riscv_memmodel_needs_release_fence (model))
-	fputs ("fence iorw,ow; ", file);
-      break;
-
     case 'i':
       if (code != REG)
         fputs ("i", file);
diff --git a/gcc/config/riscv/sync.md b/gcc/config/riscv/sync.md
index eef083b06e8..fdfc56d64a1 100644
--- a/gcc/config/riscv/sync.md
+++ b/gcc/config/riscv/sync.md
@@ -87,9 +87,9 @@
 	   (match_operand:SI 2 "const_int_operand")] ;; model
 	 UNSPEC_SYNC_OLD_OP))]
   "TARGET_ATOMIC"
-  "%F2amo<insn>.<amo>%A2 zero,%z1,%0"
+  "amo<insn>.<amo>%A2\tzero,%z1,%0"
   [(set_attr "type" "atomic")
-   (set (attr "length") (const_int 8))])
+   (set (attr "length") (const_int 4))])
 
 (define_insn "atomic_fetch_<atomic_optab><mode>"
   [(set (match_operand:GPR 0 "register_operand" "=&r")
@@ -101,9 +101,9 @@
 	   (match_operand:SI 3 "const_int_operand")] ;; model
 	 UNSPEC_SYNC_OLD_OP))]
   "TARGET_ATOMIC"
-  "%F3amo<insn>.<amo>%A3 %0,%z2,%1"
+  "amo<insn>.<amo>%A3\t%0,%z2,%1"
   [(set_attr "type" "atomic")
-   (set (attr "length") (const_int 8))])
+   (set (attr "length") (const_int 4))])
 
 (define_insn "atomic_exchange<mode>"
   [(set (match_operand:GPR 0 "register_operand" "=&r")
@@ -114,9 +114,9 @@
    (set (match_dup 1)
 	(match_operand:GPR 2 "register_operand" "0"))]
   "TARGET_ATOMIC"
-  "%F3amoswap.<amo>%A3 %0,%z2,%1"
+  "amoswap.<amo>%A3\t%0,%z2,%1"
   [(set_attr "type" "atomic")
-   (set (attr "length") (const_int 8))])
+   (set (attr "length") (const_int 4))])
 
 (define_insn "atomic_cas_value_strong<mode>"
   [(set (match_operand:GPR 0 "register_operand" "=&r")
-- 
2.25.1

[PATCH v3 07/10] RISCV: Weaken compare_exchange LR/SC pairs

[Patrick O'Neill]

Introduce the %I and %J flags for setting the .aqrl bits on LR/SC pairs
as needed.

Atomic compare and exchange ops provide success and failure memory
models. C++17 and later place no restrictions on the relative strength
of each model, so ensure we cover both by using a model that enforces
the ordering of both given models.

This change brings compare_exchange LR/SC ops in line with table A.6 of the ISA
manual.

2023-04-10 Patrick O'Neill <patrick@rivosinc.com>

	* riscv.cc: Add function to get the union of two
	memmodels in sync.md.
	* riscv-protos.h: Likewise.
	* sync.md (atomic_cas_value_strong<mode>): Remove static
	.aqrl bits on SC op/.rl bits on LR op and replace with
	optimized %I, %J flags.

Signed-off-by: Patrick O'Neill <patrick@rivosinc.com>
---
v3 Changelog:
* Consolidate tests in [PATCH v3 10/10]
---
 gcc/config/riscv/riscv-protos.h |  3 +++
 gcc/config/riscv/riscv.cc       | 44 +++++++++++++++++++++++++++++++++
 gcc/config/riscv/sync.md        |  9 +++++--
 3 files changed, 54 insertions(+), 2 deletions(-)

diff --git a/gcc/config/riscv/riscv-protos.h b/gcc/config/riscv/riscv-protos.h
index 4611447ddde..b03edc3e8a5 100644
--- a/gcc/config/riscv/riscv-protos.h
+++ b/gcc/config/riscv/riscv-protos.h
@@ -22,6 +22,8 @@ along with GCC; see the file COPYING3.  If not see
 #ifndef GCC_RISCV_PROTOS_H
 #define GCC_RISCV_PROTOS_H
 
+#include "memmodel.h"
+
 /* Symbol types we understand.  The order of this list must match that of
    the unspec enum in riscv.md, subsequent to UNSPEC_ADDRESS_FIRST.  */
 enum riscv_symbol_type {
@@ -79,6 +81,7 @@ extern void riscv_reinit (void);
 extern poly_uint64 riscv_regmode_natural_size (machine_mode);
 extern bool riscv_v_ext_vector_mode_p (machine_mode);
 extern bool riscv_shamt_matches_mask_p (int, HOST_WIDE_INT);
+extern enum memmodel riscv_union_memmodels (enum memmodel, enum memmodel);
 
 /* Routines implemented in riscv-c.cc.  */
 void riscv_cpu_cpp_builtins (cpp_reader *);
diff --git a/gcc/config/riscv/riscv.cc b/gcc/config/riscv/riscv.cc
index 6576e9ae524..061d2cf42b4 100644
--- a/gcc/config/riscv/riscv.cc
+++ b/gcc/config/riscv/riscv.cc
@@ -4278,6 +4278,36 @@ riscv_print_operand_reloc (FILE *file, rtx op, bool hi_reloc)
   fputc (')', file);
 }
 
+/* Return the memory model that encapuslates both given models.  */
+
+enum memmodel
+riscv_union_memmodels (enum memmodel model1, enum memmodel model2)
+{
+  model1 = memmodel_base (model1);
+  model2 = memmodel_base (model2);
+
+  enum memmodel weaker = model1 <= model2 ? model1: model2;
+  enum memmodel stronger = model1 > model2 ? model1: model2;
+
+  switch (stronger)
+    {
+      case MEMMODEL_SEQ_CST:
+      case MEMMODEL_ACQ_REL:
+	return stronger;
+      case MEMMODEL_RELEASE:
+	if (weaker == MEMMODEL_ACQUIRE || weaker == MEMMODEL_CONSUME)
+	  return MEMMODEL_ACQ_REL;
+	else
+	  return stronger;
+      case MEMMODEL_ACQUIRE:
+      case MEMMODEL_CONSUME:
+      case MEMMODEL_RELAXED:
+	return stronger;
+      default:
+	gcc_unreachable ();
+    }
+}
+
 /* Return true if the .AQ suffix should be added to an AMO to implement the
    acquire portion of memory model MODEL.  */
 
@@ -4331,6 +4361,8 @@ riscv_memmodel_needs_amo_release (enum memmodel model)
    'R'	Print the low-part relocation associated with OP.
    'C'	Print the integer branch condition for comparison OP.
    'A'	Print the atomic operation suffix for memory model OP.
+   'I'	Print the LR suffix for memory model OP.
+   'J'	Print the SC suffix for memory model OP.
    'z'	Print x0 if OP is zero, otherwise print OP normally.
    'i'	Print i if the operand is not a register.
    'S'	Print shift-index of single-bit mask OP.
@@ -4500,6 +4532,18 @@ riscv_print_operand (FILE *file, rtx op, int letter)
 	fputs (".rl", file);
       break;
 
+    case 'I':
+      if (model == MEMMODEL_SEQ_CST)
+	fputs (".aqrl", file);
+      else if (riscv_memmodel_needs_amo_acquire (model))
+	fputs (".aq", file);
+      break;
+
+    case 'J':
+      if (riscv_memmodel_needs_amo_release (model))
+	fputs (".rl", file);
+      break;
+
     case 'i':
       if (code != REG)
         fputs ("i", file);
diff --git a/gcc/config/riscv/sync.md b/gcc/config/riscv/sync.md
index fdfc56d64a1..a31b8c4f28a 100644
--- a/gcc/config/riscv/sync.md
+++ b/gcc/config/riscv/sync.md
@@ -130,10 +130,15 @@
    (clobber (match_scratch:GPR 6 "=&r"))]
   "TARGET_ATOMIC"
   {
+    enum memmodel model_success = (enum memmodel) INTVAL(operands[4]);
+    enum memmodel model_failure = (enum memmodel) INTVAL(operands[5]);
+    /* Find the union of the two memory models so we can satisfy both success
+       and failure memory models.  */
+    operands[5] = GEN_INT(riscv_union_memmodels(model_success, model_failure));
     return "1:\;"
-	   "lr.<amo>.aqrl\t%0,%1\;"
+	   "lr.<amo>%I5\t%0,%1\;"
 	   "bne\t%0,%z2,1f\;"
-	   "sc.<amo>.rl\t%6,%z3,%1\;"
+	   "sc.<amo>%J5\t%6,%z3,%1\;"
 	   "bnez\t%6,1b\;"
 	   "1:";
   }
-- 
2.25.1

[PATCH v3 08/10] RISCV: Weaken mem_thread_fence

[Patrick O'Neill]

This change brings atomic fences in line with table A.6 of the ISA
manual.

Relax mem_thread_fence according to the memmodel given.

2023-04-10 Patrick O'Neill <patrick@rivosinc.com>

	* sync.md (mem_thread_fence_1): Change fence depending on the
	given memory model.

Signed-off-by: Patrick O'Neill <patrick@rivosinc.com>
---
v3 Changelog:
* Consolidate tests in [PATCH v3 10/10]
* Remove helper functions
---
 gcc/config/riscv/sync.md | 16 +++++++++++++---
 1 file changed, 13 insertions(+), 3 deletions(-)

diff --git a/gcc/config/riscv/sync.md b/gcc/config/riscv/sync.md
index a31b8c4f28a..e91fa29da51 100644
--- a/gcc/config/riscv/sync.md
+++ b/gcc/config/riscv/sync.md
@@ -42,14 +42,24 @@
   DONE;
 })
 
-;; Until the RISC-V memory model (hence its mapping from C++) is finalized,
-;; conservatively emit a full FENCE.
 (define_insn "mem_thread_fence_1"
   [(set (match_operand:BLK 0 "" "")
 	(unspec:BLK [(match_dup 0)] UNSPEC_MEMORY_BARRIER))
    (match_operand:SI 1 "const_int_operand" "")] ;; model
   ""
-  "fence\tiorw,iorw")
+  {
+    enum memmodel model = (enum memmodel) INTVAL (operands[1]);
+    model = memmodel_base (model);
+    if (model == MEMMODEL_SEQ_CST)
+	return "fence\trw,rw";
+    else if (model == MEMMODEL_ACQ_REL)
+	return "fence.tso";
+    else if (model == MEMMODEL_ACQUIRE)
+	return "fence\tr,rw";
+    else if (model == MEMMODEL_RELEASE)
+	return "fence\trw,w";
+  }
+  [(set (attr "length") (const_int 4))])
 
 ;; Atomic memory operations.
 
-- 
2.25.1

[PATCH v3 09/10] RISCV: Weaken atomic loads

[Patrick O'Neill]

This change brings atomic loads in line with table A.6 of the ISA
manual.

2023-04-10 Patrick O'Neill <patrick@rivosinc.com>

	* sync.md (atomic_load<mode>): Implement atomic load mapping.

Signed-off-by: Patrick O'Neill <patrick@rivosinc.com>
---
v3 Changelog:
* Add this patch
---
 gcc/config/riscv/sync.md | 27 ++++++++++++++++++++++++++-
 1 file changed, 26 insertions(+), 1 deletion(-)

diff --git a/gcc/config/riscv/sync.md b/gcc/config/riscv/sync.md
index e91fa29da51..9e3685f5b1c 100644
--- a/gcc/config/riscv/sync.md
+++ b/gcc/config/riscv/sync.md
@@ -23,6 +23,7 @@
   UNSPEC_COMPARE_AND_SWAP
   UNSPEC_SYNC_OLD_OP
   UNSPEC_SYNC_EXCHANGE
+  UNSPEC_ATOMIC_LOAD
   UNSPEC_ATOMIC_STORE
   UNSPEC_MEMORY_BARRIER
 ])
@@ -63,7 +64,31 @@
 
 ;; Atomic memory operations.
 
-;; Implement atomic stores with conservative fences. Fall back to fences for atomic loads.
+(define_insn "atomic_load<mode>"
+  [(set (match_operand:GPR 0 "register_operand" "=r")
+    (unspec_volatile:GPR
+      [(match_operand:GPR 1 "memory_operand" "A")
+       (match_operand:SI 2 "const_int_operand")]      ;; model
+      UNSPEC_ATOMIC_LOAD))]
+  "TARGET_ATOMIC"
+  {
+    enum memmodel model = (enum memmodel) INTVAL (operands[2]);
+    model = memmodel_base (model);
+
+    if (model == MEMMODEL_SEQ_CST)
+      return "fence\trw,rw\;"
+	     "l<amo>\t%0,%1\;"
+	     "fence\tr,rw";
+    if (model == MEMMODEL_ACQUIRE)
+      return "l<amo>\t%0,%1\;"
+	     "fence\tr,rw";
+    else
+      return "l<amo>\t%0,%1";
+  }
+  [(set_attr "type" "atomic")
+   (set (attr "length") (const_int 12))])
+
+;; Implement atomic stores with conservative fences.
 ;; This allows us to be compatible with the ISA manual Table A.6 and Table A.7.
 (define_insn "atomic_store<mode>"
   [(set (match_operand:GPR 0 "memory_operand" "=A")
-- 
2.25.1

[PATCH v3 10/10] RISCV: Table A.6 conformance tests

[Patrick O'Neill]

These tests cover basic cases to ensure the atomic mappings follow the
strengthened Table A.6 mappings that are compatible with Table A.7.

2023-04-10 Patrick O'Neill <patrick@rivosinc.com>

	* amo-table-a-6-amo-add-1.c: New test.
	* amo-table-a-6-amo-add-2.c: Likewise.
	* amo-table-a-6-amo-add-3.c: Likewise.
	* amo-table-a-6-amo-add-4.c: Likewise.
	* amo-table-a-6-amo-add-5.c: Likewise.
	* amo-table-a-6-compare-exchange-1.c: Likewise.
	* amo-table-a-6-compare-exchange-2.c: Likewise.
	* amo-table-a-6-compare-exchange-3.c: Likewise.
	* amo-table-a-6-compare-exchange-4.c: Likewise.
	* amo-table-a-6-compare-exchange-5.c: Likewise.
	* amo-table-a-6-fence-1.c: Likewise.
	* amo-table-a-6-fence-2.c: Likewise.
	* amo-table-a-6-fence-3.c: Likewise.
	* amo-table-a-6-fence-4.c: Likewise.
	* amo-table-a-6-fence-5.c: Likewise.
	* amo-table-a-6-load-1.c: Likewise.
	* amo-table-a-6-load-2.c: Likewise.
	* amo-table-a-6-load-3.c: Likewise.
	* amo-table-a-6-store-1.c: Likewise.
	* amo-table-a-6-store-2.c: Likewise.
	* amo-table-a-6-store-compat-3.c: Likewise.

Signed-off-by: Patrick O'Neill <patrick@rivosinc.com>
---
v3 Changelog:
* Consolidate existing tests in this patch
* Add new tests for store/load/amoadd
---
 .../gcc.target/riscv/amo-table-a-6-amo-add-1.c       |  8 ++++++++
 .../gcc.target/riscv/amo-table-a-6-amo-add-2.c       |  8 ++++++++
 .../gcc.target/riscv/amo-table-a-6-amo-add-3.c       |  8 ++++++++
 .../gcc.target/riscv/amo-table-a-6-amo-add-4.c       |  8 ++++++++
 .../gcc.target/riscv/amo-table-a-6-amo-add-5.c       |  8 ++++++++
 .../riscv/amo-table-a-6-compare-exchange-1.c         | 12 ++++++++++++
 .../riscv/amo-table-a-6-compare-exchange-2.c         | 12 ++++++++++++
 .../riscv/amo-table-a-6-compare-exchange-3.c         | 12 ++++++++++++
 .../riscv/amo-table-a-6-compare-exchange-4.c         | 12 ++++++++++++
 .../riscv/amo-table-a-6-compare-exchange-5.c         | 12 ++++++++++++
 .../gcc.target/riscv/amo-table-a-6-fence-1.c         |  9 +++++++++
 .../gcc.target/riscv/amo-table-a-6-fence-2.c         |  7 +++++++
 .../gcc.target/riscv/amo-table-a-6-fence-3.c         |  7 +++++++
 .../gcc.target/riscv/amo-table-a-6-fence-4.c         |  7 +++++++
 .../gcc.target/riscv/amo-table-a-6-fence-5.c         |  7 +++++++
 .../gcc.target/riscv/amo-table-a-6-load-1.c          |  9 +++++++++
 .../gcc.target/riscv/amo-table-a-6-load-2.c          | 10 ++++++++++
 .../gcc.target/riscv/amo-table-a-6-load-3.c          | 10 ++++++++++
 .../gcc.target/riscv/amo-table-a-6-store-1.c         |  9 +++++++++
 .../gcc.target/riscv/amo-table-a-6-store-2.c         | 10 ++++++++++
 .../gcc.target/riscv/amo-table-a-6-store-compat-3.c  | 10 ++++++++++
 21 files changed, 195 insertions(+)
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-1.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-2.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-3.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-4.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-5.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-1.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-2.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-3.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-4.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-5.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-1.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-2.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-3.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-4.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-5.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-1.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-2.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-3.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-1.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-2.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-compat-3.c

diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-1.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-1.c
new file mode 100644
index 00000000000..ae7e407befc
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-1.c
@@ -0,0 +1,8 @@
+/* { dg-do compile } */
+/* Verify that fence mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler "amoadd.w\t" } } */
+
+void
+foo (int* bar, int* baz) {
+  __atomic_add_fetch(bar, baz, __ATOMIC_RELAXED);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-2.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-2.c
new file mode 100644
index 00000000000..60d84f32481
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-2.c
@@ -0,0 +1,8 @@
+/* { dg-do compile } */
+/* Verify that fence mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler "amoadd.w.aq\t" } } */
+
+void
+foo (int* bar, int* baz) {
+  __atomic_add_fetch(bar, baz, __ATOMIC_ACQUIRE);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-3.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-3.c
new file mode 100644
index 00000000000..a97231e4e73
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-3.c
@@ -0,0 +1,8 @@
+/* { dg-do compile } */
+/* Verify that fence mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler "amoadd.w.rl\t" } } */
+
+void
+foo (int* bar, int* baz) {
+  __atomic_add_fetch(bar, baz, __ATOMIC_RELEASE);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-4.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-4.c
new file mode 100644
index 00000000000..3c843afdd5f
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-4.c
@@ -0,0 +1,8 @@
+/* { dg-do compile } */
+/* Verify that fence mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler "amoadd.w.aqrl\t" } } */
+
+void
+foo (int* bar, int* baz) {
+  __atomic_add_fetch(bar, baz, __ATOMIC_ACQ_REL);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-5.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-5.c
new file mode 100644
index 00000000000..3434229f5e0
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-5.c
@@ -0,0 +1,8 @@
+/* { dg-do compile } */
+/* Verify that fence mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler "amoadd.w.aqrl\t" } } */
+
+void
+foo (int* bar, int* baz) {
+  __atomic_add_fetch(bar, baz, __ATOMIC_SEQ_CST);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-1.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-1.c
new file mode 100644
index 00000000000..d96bc15e23b
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-1.c
@@ -0,0 +1,12 @@
+/* { dg-do compile } */
+/* Verify that compare exchange mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler-not "lr.w.aq" } } */
+/* { dg-final { scan-assembler-not "lr.w.rl" } } */
+/* { dg-final { scan-assembler-not "sc.w.aq" } } */
+/* { dg-final { scan-assembler-not "sc.w.rl" } } */
+
+void
+foo (int bar, int baz, int qux)
+{
+  __atomic_compare_exchange_n(&bar, &baz, qux, 1, __ATOMIC_RELAXED, __ATOMIC_RELAXED);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-2.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-2.c
new file mode 100644
index 00000000000..5d173641459
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-2.c
@@ -0,0 +1,12 @@
+/* { dg-do compile } */
+/* Verify that compare exchange mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler "lr.w.aq" } } */
+/* { dg-final { scan-assembler-not "lr.w.rl" } } */
+/* { dg-final { scan-assembler-not "sc.w.aq" } } */
+/* { dg-final { scan-assembler-not "sc.w.rl" } } */
+
+void
+foo (int bar, int baz, int qux)
+{
+  __atomic_compare_exchange_n(&bar, &baz, qux, 1, __ATOMIC_CONSUME, __ATOMIC_CONSUME);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-3.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-3.c
new file mode 100644
index 00000000000..36201027ab9
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-3.c
@@ -0,0 +1,12 @@
+/* { dg-do compile } */
+/* Verify that compare exchange mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler "lr.w.aq" } } */
+/* { dg-final { scan-assembler-not "lr.w.rl" } } */
+/* { dg-final { scan-assembler-not "sc.w.aq" } } */
+/* { dg-final { scan-assembler-not "sc.w.rl" } } */
+
+void
+foo (int bar, int baz, int qux)
+{
+  __atomic_compare_exchange_n(&bar, &baz, qux, 1, __ATOMIC_ACQUIRE, __ATOMIC_ACQUIRE);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-4.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-4.c
new file mode 100644
index 00000000000..3711bf7ddec
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-4.c
@@ -0,0 +1,12 @@
+/* { dg-do compile } */
+/* Verify that compare exchange mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler "lr.w.aqrl" } } */
+/* { dg-final { scan-assembler "sc.w.rl" } } */
+/* { dg-final { scan-assembler-not "lr.w.rl" } } */
+/* { dg-final { scan-assembler-not "sc.w.aq" } } */
+
+void
+foo (int bar, int baz, int qux)
+{
+  __atomic_compare_exchange_n(&bar, &baz, qux, 1, __ATOMIC_SEQ_CST, __ATOMIC_SEQ_CST);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-5.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-5.c
new file mode 100644
index 00000000000..fb0075a1a33
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-5.c
@@ -0,0 +1,12 @@
+/* { dg-do compile } */
+/* Verify that compare exchange mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler "lr.w.aq" } } */
+/* { dg-final { scan-assembler "sc.w.rl" } } */
+/* { dg-final { scan-assembler-not "lr.w.rl" } } */
+/* { dg-final { scan-assembler-not "sc.w.aq" } } */
+
+void
+foo (int bar, int baz, int qux)
+{
+  __atomic_compare_exchange_n(&bar, &baz, qux, 1, __ATOMIC_RELEASE, __ATOMIC_ACQUIRE);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-1.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-1.c
new file mode 100644
index 00000000000..827a0f6781b
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-1.c
@@ -0,0 +1,9 @@
+/* { dg-do compile } */
+/* Verify that fence mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler-not "fence\t" } } */
+/* { dg-final { scan-assembler-not "fence.tso" } } */
+
+
+int main() {
+  __atomic_thread_fence(__ATOMIC_RELAXED);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-2.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-2.c
new file mode 100644
index 00000000000..2d14a5b9681
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-2.c
@@ -0,0 +1,7 @@
+/* { dg-do compile } */
+/* Verify that fence mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler "fence\tr,rw" } } */
+
+int main() {
+  __atomic_thread_fence(__ATOMIC_ACQUIRE);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-3.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-3.c
new file mode 100644
index 00000000000..c215dc19c90
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-3.c
@@ -0,0 +1,7 @@
+/* { dg-do compile } */
+/* Verify that fence mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler "fence\trw,w" } } */
+
+int main() {
+  __atomic_thread_fence(__ATOMIC_RELEASE);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-4.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-4.c
new file mode 100644
index 00000000000..6dec57593f9
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-4.c
@@ -0,0 +1,7 @@
+/* { dg-do compile } */
+/* Verify that fence mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler "fence.tso" } } */
+
+int main() {
+  __atomic_thread_fence(__ATOMIC_ACQ_REL);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-5.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-5.c
new file mode 100644
index 00000000000..f05a33b1bcd
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-5.c
@@ -0,0 +1,7 @@
+/* { dg-do compile } */
+/* Verify that fence mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler "fence\trw,rw" } } */
+
+int main() {
+  __atomic_thread_fence(__ATOMIC_SEQ_CST);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-1.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-1.c
new file mode 100644
index 00000000000..8278198072e
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-1.c
@@ -0,0 +1,9 @@
+/* { dg-do compile } */
+/* Verify that load mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler-not "fence\t" } } */
+/* { dg-final { scan-assembler-not "fence.tso" } } */
+
+void
+foo (int* bar, int* baz) {
+  __atomic_load(bar, baz, __ATOMIC_RELAXED);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-2.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-2.c
new file mode 100644
index 00000000000..faef2ce25c4
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-2.c
@@ -0,0 +1,10 @@
+/* { dg-do compile } */
+/* Verify that fence mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler "fence\tr,rw" } } */
+/* { dg-final { scan-assembler-not "fence\trw,rw" } } */
+/* { dg-final { scan-assembler-not "fence.tso" } } */
+
+void
+foo (int* bar, int* baz) {
+  __atomic_load(bar, baz, __ATOMIC_ACQUIRE);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-3.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-3.c
new file mode 100644
index 00000000000..9210fd10a53
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-3.c
@@ -0,0 +1,10 @@
+/* { dg-do compile } */
+/* Verify that fence mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler "fence\tr,rw" } } */
+/* { dg-final { scan-assembler "fence\trw,rw" } } */
+/* { dg-final { scan-assembler-not "fence.tso" } } */
+
+void
+foo (int* bar, int* baz) {
+  __atomic_load(bar, baz, __ATOMIC_SEQ_CST);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-1.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-1.c
new file mode 100644
index 00000000000..e2fb71b17c3
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-1.c
@@ -0,0 +1,9 @@
+/* { dg-do compile } */
+/* Verify that load mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler-not "fence\t" } } */
+/* { dg-final { scan-assembler-not "fence.tso" } } */
+
+void
+foo (int* bar, int* baz) {
+  __atomic_store(bar, baz, __ATOMIC_RELAXED);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-2.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-2.c
new file mode 100644
index 00000000000..adaace33b30
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-2.c
@@ -0,0 +1,10 @@
+/* { dg-do compile } */
+/* Verify that fence mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler "fence\trw,w" } } */
+/* { dg-final { scan-assembler-not "fence\trw,rw" } } */
+/* { dg-final { scan-assembler-not "fence.tso" } } */
+
+void
+foo (int* bar, int* baz) {
+  __atomic_store(bar, baz, __ATOMIC_RELEASE);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-compat-3.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-compat-3.c
new file mode 100644
index 00000000000..1fff6ce2857
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-compat-3.c
@@ -0,0 +1,10 @@
+/* { dg-do compile } */
+/* Verify that store mapping are compatible with Table A.6 & A.7.  */
+/* { dg-final { scan-assembler "fence\trw,w" } } */
+/* { dg-final { scan-assembler "fence\trw,rw" } } */
+/* { dg-final { scan-assembler-not "fence.tso" } } */
+
+void
+foo (int* bar, int* baz) {
+  __atomic_store(bar, baz, __ATOMIC_SEQ_CST);
+}
-- 
2.25.1

[PATCH v4 00/10] RISCV: Implement ISA Manual Table A.6 Mappings

[Patrick O'Neill]

This patchset aims to make the RISCV atomics implementation stronger
than the recommended mapping present in table A.6 of the ISA manual.
  https://github.com/riscv/riscv-isa-manual/blob/c7cf84547b3aefacab5463add1734c1602b67a49/src/memory.tex#L1083-L1157

The current mapping in GCC is not internally consistent. Andrea Parri
pointed this out here along with a litmus test:
  https://inbox.sourceware.org/gcc-patches/Y1GbJuhcBFpPGJQ0@andrea/

As a result, we have an opportunity to jump straight to the A.6
implementation (meaning we will be compatible with LLVM's mappings which
are A.6). In light of a proposal by Hans Boehm and to avoid an ABI break
in the future, the mapping implemented is strictly stronger than the one
in table A.6 in order to be compatible with table A.7.
  https://lists.riscv.org/g/tech-unprivileged/topic/risc_v_memory_model_topics/92916241

This change incurs a performance penalty on SEQ_CST stores due to the
added trailing fence. Systems implementing table A.7 will have
significant performance gains relative to table A.6 and are expected to
be the standard memory model mapping in the RISCV ecosystem. Incurring
this added cost now will make it significantly easier for compiled
RISC-V binaries to transition to the A.7 memory model mapping.

If Hans' proposal is accepted, it makes sense to migrate to the mapping
recommended by table A.7. Since the stronger mapping in this patchset
(provided by Hans Boehm) appears to be compatible with both A.6 and A.7,
this transition should not result in an ABI break for GCC.

Patch 1 simplifies the memmodel to ignore MEMMODEL_SYNC_* cases (legacy
cases that aren't handled differently for RISC-V).
Patches 2-5 make the mappings strictly stronger.
Patches 5-9 weaken the mappings to be in line with table A.6 of the ISA
manual.
Patch 10 adds some basic conformance tests to ensure the implemented
mapping matches table A.6 with stronger SEQ_CST stores.

Christoph Muellner also submitted a similar patchset here:
  https://gcc.gnu.org/pipermail/gcc-patches/2022-May/595712.html
I used my previous patchset as a starting point since it was easier for 
me.

LLVM mapping notes:
* LLVM emits corresponding fences for atomic_signal_fence instructions.
  This seems to be an oversight since AFAIK atomic_signal_fence acts as
  a compiler directive. GCC does not emit any fences for
  atomic_signal_fence instructions.

Patchset v1:
  https://gcc.gnu.org/pipermail/gcc-patches/2022-April/592950.html

Patchset v2:
  https://gcc.gnu.org/pipermail/gcc-patches/2023-April/615264.html

Patchset v3:
  https://gcc.gnu.org/pipermail/gcc-patches/2023-April/615431.html

Changes for v2:
* Use memmodel_base rather than a custom simplify_memmodel function
  (Inspired by Christoph Muellner's patch 1/9)
* Move instruction styling change from [v1 5/7] to [v2 3/8] to reduce
  [v2 6/8]'s complexity
* Eliminated %K flag for atomic store introduced in v1 in favor of
  if/else
* Rebase/test

Changes for v3:
* Use a trailing fence for atomic stores to be compatible with table A.7
* Emit an optimized fence r,rw following a SEQ_CST load
* Consolidate tests in [PATCH v3 10/10]
* Add tests for basic A.6 conformance

Changes for v4:
* Update cover letter to cover more of the reasoning behind moving to a
  compatability mapping
* Improve conformance testcases patch assertions and add new
  compare-exchange testcases

Patrick O'Neill (10):
  RISCV: Eliminate SYNC memory models
  RISCV: Enforce Libatomic LR/SC SEQ_CST
  RISCV: Enforce atomic compare_exchange SEQ_CST
  RISCV: Add AMO release bits
  RISCV: Strengthen atomic stores
  RISCV: Eliminate AMO op fences
  RISCV: Weaken compare_exchange LR/SC pairs
  RISCV: Weaken mem_thread_fence
  RISCV: Weaken atomic loads
  RISCV: Table A.6 conformance tests

 gcc/config/riscv/riscv-protos.h               |  3 +
 gcc/config/riscv/riscv.cc                     | 66 +++++++++++---
 gcc/config/riscv/sync.md                      | 89 ++++++++++++++++---
 .../riscv/amo-table-a-6-amo-add-1.c           |  8 ++
 .../riscv/amo-table-a-6-amo-add-2.c           |  8 ++
 .../riscv/amo-table-a-6-amo-add-3.c           |  8 ++
 .../riscv/amo-table-a-6-amo-add-4.c           |  8 ++
 .../riscv/amo-table-a-6-amo-add-5.c           |  8 ++
 .../riscv/amo-table-a-6-compare-exchange-1.c  | 10 +++
 .../riscv/amo-table-a-6-compare-exchange-2.c  | 10 +++
 .../riscv/amo-table-a-6-compare-exchange-3.c  | 10 +++
 .../riscv/amo-table-a-6-compare-exchange-4.c  | 10 +++
 .../riscv/amo-table-a-6-compare-exchange-5.c  | 10 +++
 .../riscv/amo-table-a-6-compare-exchange-6.c  | 11 +++
 .../riscv/amo-table-a-6-compare-exchange-7.c  | 10 +++
 .../gcc.target/riscv/amo-table-a-6-fence-1.c  |  8 ++
 .../gcc.target/riscv/amo-table-a-6-fence-2.c  | 10 +++
 .../gcc.target/riscv/amo-table-a-6-fence-3.c  | 10 +++
 .../gcc.target/riscv/amo-table-a-6-fence-4.c  | 10 +++
 .../gcc.target/riscv/amo-table-a-6-fence-5.c  | 10 +++
 .../gcc.target/riscv/amo-table-a-6-load-1.c   |  9 ++
 .../gcc.target/riscv/amo-table-a-6-load-2.c   | 11 +++
 .../gcc.target/riscv/amo-table-a-6-load-3.c   | 11 +++
 .../gcc.target/riscv/amo-table-a-6-store-1.c  |  9 ++
 .../gcc.target/riscv/amo-table-a-6-store-2.c  | 11 +++
 .../riscv/amo-table-a-6-store-compat-3.c      | 11 +++
 gcc/testsuite/gcc.target/riscv/pr89835.c      |  9 ++
 libgcc/config/riscv/atomic.c                  |  4 +-
 28 files changed, 362 insertions(+), 30 deletions(-)
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-1.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-2.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-3.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-4.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-5.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-1.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-2.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-3.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-4.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-5.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-6.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-7.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-1.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-2.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-3.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-4.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-5.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-1.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-2.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-3.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-1.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-2.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-compat-3.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/pr89835.c

-- 
2.25.1

[PATCH v4 01/10] RISCV: Eliminate SYNC memory models

[Patrick O'Neill]

Remove references to MEMMODEL_SYNC_* models by converting via
memmodel_base().

2023-04-14 Patrick O'Neill <patrick@rivosinc.com>

	* riscv.cc: Remove MEMMODEL_SYNC_* cases and sanitize memmodel
	input with memmodel_base

Signed-off-by: Patrick O'Neill <patrick@rivosinc.com>
---
 gcc/config/riscv/riscv.cc | 11 +++--------
 1 file changed, 3 insertions(+), 8 deletions(-)

diff --git a/gcc/config/riscv/riscv.cc b/gcc/config/riscv/riscv.cc
index 5f542932d13..2226241217d 100644
--- a/gcc/config/riscv/riscv.cc
+++ b/gcc/config/riscv/riscv.cc
@@ -4288,14 +4288,11 @@ riscv_memmodel_needs_amo_acquire (enum memmodel model)
     {
       case MEMMODEL_ACQ_REL:
       case MEMMODEL_SEQ_CST:
-      case MEMMODEL_SYNC_SEQ_CST:
       case MEMMODEL_ACQUIRE:
       case MEMMODEL_CONSUME:
-      case MEMMODEL_SYNC_ACQUIRE:
 	return true;
 
       case MEMMODEL_RELEASE:
-      case MEMMODEL_SYNC_RELEASE:
       case MEMMODEL_RELAXED:
 	return false;
 
@@ -4314,14 +4311,11 @@ riscv_memmodel_needs_release_fence (enum memmodel model)
     {
       case MEMMODEL_ACQ_REL:
       case MEMMODEL_SEQ_CST:
-      case MEMMODEL_SYNC_SEQ_CST:
       case MEMMODEL_RELEASE:
-      case MEMMODEL_SYNC_RELEASE:
 	return true;
 
       case MEMMODEL_ACQUIRE:
       case MEMMODEL_CONSUME:
-      case MEMMODEL_SYNC_ACQUIRE:
       case MEMMODEL_RELAXED:
 	return false;
 
@@ -4360,6 +4354,7 @@ riscv_print_operand (FILE *file, rtx op, int letter)
     }
   machine_mode mode = GET_MODE (op);
   enum rtx_code code = GET_CODE (op);
+  const enum memmodel model = memmodel_base (INTVAL (op));
 
   switch (letter)
     {
@@ -4497,12 +4492,12 @@ riscv_print_operand (FILE *file, rtx op, int letter)
       break;
 
     case 'A':
-      if (riscv_memmodel_needs_amo_acquire ((enum memmodel) INTVAL (op)))
+      if (riscv_memmodel_needs_amo_acquire (model))
 	fputs (".aq", file);
       break;
 
     case 'F':
-      if (riscv_memmodel_needs_release_fence ((enum memmodel) INTVAL (op)))
+      if (riscv_memmodel_needs_release_fence (model))
 	fputs ("fence iorw,ow; ", file);
       break;
 
-- 
2.25.1

[PATCH v4 02/10] RISCV: Enforce Libatomic LR/SC SEQ_CST

[Patrick O'Neill]

Replace LR.aq/SC.rl pairs with the SEQ_CST LR.aqrl/SC.rl pairs
recommended by table A.6 of the ISA manual.

2023-04-14 Patrick O'Neill <patrick@rivosinc.com>

	* atomic.c: Change LR.aq/SC.rl pairs into sequentially
	consistent LR.aqrl/SC.rl pair.

Signed-off-by: Patrick O'Neill <patrick@rivosinc.com>
---
 libgcc/config/riscv/atomic.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/libgcc/config/riscv/atomic.c b/libgcc/config/riscv/atomic.c
index 69f53623509..5f895939b0b 100644
--- a/libgcc/config/riscv/atomic.c
+++ b/libgcc/config/riscv/atomic.c
@@ -39,7 +39,7 @@ see the files COPYING3 and COPYING.RUNTIME respectively.  If not, see
     unsigned old, tmp1, tmp2;						\
 									\
     asm volatile ("1:\n\t"						\
-		  "lr.w.aq %[old], %[mem]\n\t"				\
+		  "lr.w.aqrl %[old], %[mem]\n\t"			\
 		  #insn " %[tmp1], %[old], %[value]\n\t"		\
 		  invert						\
 		  "and %[tmp1], %[tmp1], %[mask]\n\t"			\
@@ -73,7 +73,7 @@ see the files COPYING3 and COPYING.RUNTIME respectively.  If not, see
     unsigned old, tmp1;							\
 									\
     asm volatile ("1:\n\t"						\
-		  "lr.w.aq %[old], %[mem]\n\t"				\
+		  "lr.w.aqrl %[old], %[mem]\n\t"			\
 		  "and %[tmp1], %[old], %[mask]\n\t"			\
 		  "bne %[tmp1], %[o], 1f\n\t"				\
 		  "and %[tmp1], %[old], %[not_mask]\n\t"		\
-- 
2.25.1

[PATCH v4 03/10] RISCV: Enforce atomic compare_exchange SEQ_CST

[Patrick O'Neill]

This patch enforces SEQ_CST for atomic compare_exchange ops.

Replace Fence/LR.aq/SC.aq pairs with SEQ_CST LR.aqrl/SC.rl pairs
recommended by table A.6 of the ISA manual.

2023-04-14 Patrick O'Neill <patrick@rivosinc.com>

	* sync.md: Change FENCE/LR.aq/SC.aq into sequentially
	consistent LR.aqrl/SC.rl pair.

Signed-off-by: Patrick O'Neill <patrick@rivosinc.com>
---
 gcc/config/riscv/sync.md | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)

diff --git a/gcc/config/riscv/sync.md b/gcc/config/riscv/sync.md
index c932ef87b9d..de42245981b 100644
--- a/gcc/config/riscv/sync.md
+++ b/gcc/config/riscv/sync.md
@@ -115,9 +115,16 @@
 	 UNSPEC_COMPARE_AND_SWAP))
    (clobber (match_scratch:GPR 6 "=&r"))]
   "TARGET_ATOMIC"
-  "%F5 1: lr.<amo>%A5 %0,%1; bne %0,%z2,1f; sc.<amo>%A4 %6,%z3,%1; bnez %6,1b; 1:"
+  {
+    return "1:\;"
+	   "lr.<amo>.aqrl\t%0,%1\;"
+	   "bne\t%0,%z2,1f\;"
+	   "sc.<amo>.rl\t%6,%z3,%1\;"
+	   "bnez\t%6,1b\;"
+	   "1:";
+  }
   [(set_attr "type" "atomic")
-   (set (attr "length") (const_int 20))])
+   (set (attr "length") (const_int 16))])
 
 (define_expand "atomic_compare_and_swap<mode>"
   [(match_operand:SI 0 "register_operand" "")   ;; bool output
-- 
2.25.1

[PATCH v4 04/10] RISCV: Add AMO release bits

[Patrick O'Neill]

This patch sets the relevant .rl bits on amo operations.

2023-04-14 Patrick O'Neill <patrick@rivosinc.com>

	* riscv.cc (riscv_print_operand): change behavior of %A to
	include release bits.

Signed-off-by: Patrick O'Neill <patrick@rivosinc.com>
---
 gcc/config/riscv/riscv.cc | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/gcc/config/riscv/riscv.cc b/gcc/config/riscv/riscv.cc
index 2226241217d..088b8d25773 100644
--- a/gcc/config/riscv/riscv.cc
+++ b/gcc/config/riscv/riscv.cc
@@ -4492,8 +4492,13 @@ riscv_print_operand (FILE *file, rtx op, int letter)
       break;
 
     case 'A':
-      if (riscv_memmodel_needs_amo_acquire (model))
+      if (riscv_memmodel_needs_amo_acquire (model) &&
+	  riscv_memmodel_needs_release_fence (model))
+	fputs (".aqrl", file);
+      else if (riscv_memmodel_needs_amo_acquire (model))
 	fputs (".aq", file);
+      else if (riscv_memmodel_needs_release_fence (model))
+	fputs (".rl", file);
       break;
 
     case 'F':
-- 
2.25.1

[PATCH v4 05/10] RISCV: Strengthen atomic stores

[Patrick O'Neill]

This change makes atomic stores strictly stronger than table A.6 of the
ISA manual. This mapping makes the overall patchset compatible with
table A.7 as well.

2023-04-14 Patrick O'Neill <patrick@rivosinc.com>

	PR target/89835
	* sync.md (atomic_store<mode>): Use simple store instruction in
	combination with a fence.
	* pr89835.c: New test.

Signed-off-by: Patrick O'Neill <patrick@rivosinc.com>
---
v3 Changelog:
* Use a trailing fence for atomic stores to be compatible with Table A.7
---
 gcc/config/riscv/sync.md                 | 20 +++++++++++++++++---
 gcc/testsuite/gcc.target/riscv/pr89835.c |  9 +++++++++
 2 files changed, 26 insertions(+), 3 deletions(-)
 create mode 100644 gcc/testsuite/gcc.target/riscv/pr89835.c

diff --git a/gcc/config/riscv/sync.md b/gcc/config/riscv/sync.md
index de42245981b..eef083b06e8 100644
--- a/gcc/config/riscv/sync.md
+++ b/gcc/config/riscv/sync.md
@@ -53,7 +53,8 @@
 
 ;; Atomic memory operations.
 
-;; Implement atomic stores with amoswap.  Fall back to fences for atomic loads.
+;; Implement atomic stores with conservative fences. Fall back to fences for atomic loads.
+;; This allows us to be compatible with the ISA manual Table A.6 and Table A.7.
 (define_insn "atomic_store<mode>"
   [(set (match_operand:GPR 0 "memory_operand" "=A")
     (unspec_volatile:GPR
@@ -61,9 +62,22 @@
        (match_operand:SI 2 "const_int_operand")]      ;; model
       UNSPEC_ATOMIC_STORE))]
   "TARGET_ATOMIC"
-  "%F2amoswap.<amo>%A2 zero,%z1,%0"
+  {
+    enum memmodel model = (enum memmodel) INTVAL (operands[2]);
+    model = memmodel_base (model);
+
+    if (model == MEMMODEL_SEQ_CST)
+      return "fence\trw,w\;"
+	     "s<amo>\t%z1,%0\;"
+	     "fence\trw,rw";
+    if (model == MEMMODEL_RELEASE)
+      return "fence\trw,w\;"
+	     "s<amo>\t%z1,%0";
+    else
+      return "s<amo>\t%z1,%0";
+  }
   [(set_attr "type" "atomic")
-   (set (attr "length") (const_int 8))])
+   (set (attr "length") (const_int 12))])
 
 (define_insn "atomic_<atomic_optab><mode>"
   [(set (match_operand:GPR 0 "memory_operand" "+A")
diff --git a/gcc/testsuite/gcc.target/riscv/pr89835.c b/gcc/testsuite/gcc.target/riscv/pr89835.c
new file mode 100644
index 00000000000..ab190e11b60
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/pr89835.c
@@ -0,0 +1,9 @@
+/* { dg-do compile } */
+/* Verify that relaxed atomic stores use simple store instuctions.  */
+/* { dg-final { scan-assembler-not "amoswap" } } */
+
+void
+foo(int bar, int baz)
+{
+  __atomic_store_n(&bar, baz, __ATOMIC_RELAXED);
+}
-- 
2.25.1

[PATCH v4 06/10] RISCV: Eliminate AMO op fences

[Patrick O'Neill]

Atomic operations with the appropriate bits set already enfore release
semantics. Remove unnecessary release fences from atomic ops.

This change brings AMO ops in line with table A.6 of the ISA manual.

2023-04-14 Patrick O'Neill <patrick@rivosinc.com>

	* riscv.cc (riscv_memmodel_needs_amo_acquire): Change function
	name.
	* riscv.cc (riscv_print_operand): Remove unneeded %F case.
	* sync.md: Remove unneeded fences.

Signed-off-by: Patrick O'Neill <patrick@rivosinc.com>
---
 gcc/config/riscv/riscv.cc | 16 +++++-----------
 gcc/config/riscv/sync.md  | 12 ++++++------
 2 files changed, 11 insertions(+), 17 deletions(-)

diff --git a/gcc/config/riscv/riscv.cc b/gcc/config/riscv/riscv.cc
index 088b8d25773..70031b83391 100644
--- a/gcc/config/riscv/riscv.cc
+++ b/gcc/config/riscv/riscv.cc
@@ -4301,11 +4301,11 @@ riscv_memmodel_needs_amo_acquire (enum memmodel model)
     }
 }
 
-/* Return true if a FENCE should be emitted to before a memory access to
-   implement the release portion of memory model MODEL.  */
+/* Return true if the .RL suffix should be added to an AMO to implement the
+   release portion of memory model MODEL.  */
 
 static bool
-riscv_memmodel_needs_release_fence (enum memmodel model)
+riscv_memmodel_needs_amo_release (enum memmodel model)
 {
   switch (model)
     {
@@ -4331,7 +4331,6 @@ riscv_memmodel_needs_release_fence (enum memmodel model)
    'R'	Print the low-part relocation associated with OP.
    'C'	Print the integer branch condition for comparison OP.
    'A'	Print the atomic operation suffix for memory model OP.
-   'F'	Print a FENCE if the memory model requires a release.
    'z'	Print x0 if OP is zero, otherwise print OP normally.
    'i'	Print i if the operand is not a register.
    'S'	Print shift-index of single-bit mask OP.
@@ -4493,19 +4492,14 @@ riscv_print_operand (FILE *file, rtx op, int letter)
 
     case 'A':
       if (riscv_memmodel_needs_amo_acquire (model) &&
-	  riscv_memmodel_needs_release_fence (model))
+	  riscv_memmodel_needs_amo_release (model))
 	fputs (".aqrl", file);
       else if (riscv_memmodel_needs_amo_acquire (model))
 	fputs (".aq", file);
-      else if (riscv_memmodel_needs_release_fence (model))
+      else if (riscv_memmodel_needs_amo_release (model))
 	fputs (".rl", file);
       break;
 
-    case 'F':
-      if (riscv_memmodel_needs_release_fence (model))
-	fputs ("fence iorw,ow; ", file);
-      break;
-
     case 'i':
       if (code != REG)
         fputs ("i", file);
diff --git a/gcc/config/riscv/sync.md b/gcc/config/riscv/sync.md
index eef083b06e8..fdfc56d64a1 100644
--- a/gcc/config/riscv/sync.md
+++ b/gcc/config/riscv/sync.md
@@ -87,9 +87,9 @@
 	   (match_operand:SI 2 "const_int_operand")] ;; model
 	 UNSPEC_SYNC_OLD_OP))]
   "TARGET_ATOMIC"
-  "%F2amo<insn>.<amo>%A2 zero,%z1,%0"
+  "amo<insn>.<amo>%A2\tzero,%z1,%0"
   [(set_attr "type" "atomic")
-   (set (attr "length") (const_int 8))])
+   (set (attr "length") (const_int 4))])
 
 (define_insn "atomic_fetch_<atomic_optab><mode>"
   [(set (match_operand:GPR 0 "register_operand" "=&r")
@@ -101,9 +101,9 @@
 	   (match_operand:SI 3 "const_int_operand")] ;; model
 	 UNSPEC_SYNC_OLD_OP))]
   "TARGET_ATOMIC"
-  "%F3amo<insn>.<amo>%A3 %0,%z2,%1"
+  "amo<insn>.<amo>%A3\t%0,%z2,%1"
   [(set_attr "type" "atomic")
-   (set (attr "length") (const_int 8))])
+   (set (attr "length") (const_int 4))])
 
 (define_insn "atomic_exchange<mode>"
   [(set (match_operand:GPR 0 "register_operand" "=&r")
@@ -114,9 +114,9 @@
    (set (match_dup 1)
 	(match_operand:GPR 2 "register_operand" "0"))]
   "TARGET_ATOMIC"
-  "%F3amoswap.<amo>%A3 %0,%z2,%1"
+  "amoswap.<amo>%A3\t%0,%z2,%1"
   [(set_attr "type" "atomic")
-   (set (attr "length") (const_int 8))])
+   (set (attr "length") (const_int 4))])
 
 (define_insn "atomic_cas_value_strong<mode>"
   [(set (match_operand:GPR 0 "register_operand" "=&r")
-- 
2.25.1

[PATCH v4 07/10] RISCV: Weaken compare_exchange LR/SC pairs

[Patrick O'Neill]

Introduce the %I and %J flags for setting the .aqrl bits on LR/SC pairs
as needed.

Atomic compare and exchange ops provide success and failure memory
models. C++17 and later place no restrictions on the relative strength
of each model, so ensure we cover both by using a model that enforces
the ordering of both given models.

This change brings compare_exchange LR/SC ops in line with table A.6 of the ISA
manual.

2023-04-14 Patrick O'Neill <patrick@rivosinc.com>

	* riscv.cc: Add function to get the union of two
	memmodels in sync.md.
	* riscv-protos.h: Likewise.
	* sync.md (atomic_cas_value_strong<mode>): Remove static
	.aqrl bits on SC op/.rl bits on LR op and replace with
	optimized %I, %J flags.

Signed-off-by: Patrick O'Neill <patrick@rivosinc.com>
---
v3 Changelog:
* Consolidate tests in [PATCH v3 10/10]
---
 gcc/config/riscv/riscv-protos.h |  3 +++
 gcc/config/riscv/riscv.cc       | 44 +++++++++++++++++++++++++++++++++
 gcc/config/riscv/sync.md        |  9 +++++--
 3 files changed, 54 insertions(+), 2 deletions(-)

diff --git a/gcc/config/riscv/riscv-protos.h b/gcc/config/riscv/riscv-protos.h
index 4611447ddde..b03edc3e8a5 100644
--- a/gcc/config/riscv/riscv-protos.h
+++ b/gcc/config/riscv/riscv-protos.h
@@ -22,6 +22,8 @@ along with GCC; see the file COPYING3.  If not see
 #ifndef GCC_RISCV_PROTOS_H
 #define GCC_RISCV_PROTOS_H
 
+#include "memmodel.h"
+
 /* Symbol types we understand.  The order of this list must match that of
    the unspec enum in riscv.md, subsequent to UNSPEC_ADDRESS_FIRST.  */
 enum riscv_symbol_type {
@@ -79,6 +81,7 @@ extern void riscv_reinit (void);
 extern poly_uint64 riscv_regmode_natural_size (machine_mode);
 extern bool riscv_v_ext_vector_mode_p (machine_mode);
 extern bool riscv_shamt_matches_mask_p (int, HOST_WIDE_INT);
+extern enum memmodel riscv_union_memmodels (enum memmodel, enum memmodel);
 
 /* Routines implemented in riscv-c.cc.  */
 void riscv_cpu_cpp_builtins (cpp_reader *);
diff --git a/gcc/config/riscv/riscv.cc b/gcc/config/riscv/riscv.cc
index 70031b83391..b3b89bf444b 100644
--- a/gcc/config/riscv/riscv.cc
+++ b/gcc/config/riscv/riscv.cc
@@ -4278,6 +4278,36 @@ riscv_print_operand_reloc (FILE *file, rtx op, bool hi_reloc)
   fputc (')', file);
 }
 
+/* Return the memory model that encapuslates both given models.  */
+
+enum memmodel
+riscv_union_memmodels (enum memmodel model1, enum memmodel model2)
+{
+  model1 = memmodel_base (model1);
+  model2 = memmodel_base (model2);
+
+  enum memmodel weaker = model1 <= model2 ? model1: model2;
+  enum memmodel stronger = model1 > model2 ? model1: model2;
+
+  switch (stronger)
+    {
+      case MEMMODEL_SEQ_CST:
+      case MEMMODEL_ACQ_REL:
+	return stronger;
+      case MEMMODEL_RELEASE:
+	if (weaker == MEMMODEL_ACQUIRE || weaker == MEMMODEL_CONSUME)
+	  return MEMMODEL_ACQ_REL;
+	else
+	  return stronger;
+      case MEMMODEL_ACQUIRE:
+      case MEMMODEL_CONSUME:
+      case MEMMODEL_RELAXED:
+	return stronger;
+      default:
+	gcc_unreachable ();
+    }
+}
+
 /* Return true if the .AQ suffix should be added to an AMO to implement the
    acquire portion of memory model MODEL.  */
 
@@ -4331,6 +4361,8 @@ riscv_memmodel_needs_amo_release (enum memmodel model)
    'R'	Print the low-part relocation associated with OP.
    'C'	Print the integer branch condition for comparison OP.
    'A'	Print the atomic operation suffix for memory model OP.
+   'I'	Print the LR suffix for memory model OP.
+   'J'	Print the SC suffix for memory model OP.
    'z'	Print x0 if OP is zero, otherwise print OP normally.
    'i'	Print i if the operand is not a register.
    'S'	Print shift-index of single-bit mask OP.
@@ -4500,6 +4532,18 @@ riscv_print_operand (FILE *file, rtx op, int letter)
 	fputs (".rl", file);
       break;
 
+    case 'I':
+      if (model == MEMMODEL_SEQ_CST)
+	fputs (".aqrl", file);
+      else if (riscv_memmodel_needs_amo_acquire (model))
+	fputs (".aq", file);
+      break;
+
+    case 'J':
+      if (riscv_memmodel_needs_amo_release (model))
+	fputs (".rl", file);
+      break;
+
     case 'i':
       if (code != REG)
         fputs ("i", file);
diff --git a/gcc/config/riscv/sync.md b/gcc/config/riscv/sync.md
index fdfc56d64a1..a31b8c4f28a 100644
--- a/gcc/config/riscv/sync.md
+++ b/gcc/config/riscv/sync.md
@@ -130,10 +130,15 @@
    (clobber (match_scratch:GPR 6 "=&r"))]
   "TARGET_ATOMIC"
   {
+    enum memmodel model_success = (enum memmodel) INTVAL(operands[4]);
+    enum memmodel model_failure = (enum memmodel) INTVAL(operands[5]);
+    /* Find the union of the two memory models so we can satisfy both success
+       and failure memory models.  */
+    operands[5] = GEN_INT(riscv_union_memmodels(model_success, model_failure));
     return "1:\;"
-	   "lr.<amo>.aqrl\t%0,%1\;"
+	   "lr.<amo>%I5\t%0,%1\;"
 	   "bne\t%0,%z2,1f\;"
-	   "sc.<amo>.rl\t%6,%z3,%1\;"
+	   "sc.<amo>%J5\t%6,%z3,%1\;"
 	   "bnez\t%6,1b\;"
 	   "1:";
   }
-- 
2.25.1

[PATCH v4 08/10] RISCV: Weaken mem_thread_fence

[Patrick O'Neill]

This change brings atomic fences in line with table A.6 of the ISA
manual.

Relax mem_thread_fence according to the memmodel given.

2023-04-14 Patrick O'Neill <patrick@rivosinc.com>

	* sync.md (mem_thread_fence_1): Change fence depending on the
	given memory model.

Signed-off-by: Patrick O'Neill <patrick@rivosinc.com>
---
v3 Changelog:
* Consolidate tests in [PATCH v3 10/10]
* Remove helper functions
---
 gcc/config/riscv/sync.md | 16 +++++++++++++---
 1 file changed, 13 insertions(+), 3 deletions(-)

diff --git a/gcc/config/riscv/sync.md b/gcc/config/riscv/sync.md
index a31b8c4f28a..e91fa29da51 100644
--- a/gcc/config/riscv/sync.md
+++ b/gcc/config/riscv/sync.md
@@ -42,14 +42,24 @@
   DONE;
 })
 
-;; Until the RISC-V memory model (hence its mapping from C++) is finalized,
-;; conservatively emit a full FENCE.
 (define_insn "mem_thread_fence_1"
   [(set (match_operand:BLK 0 "" "")
 	(unspec:BLK [(match_dup 0)] UNSPEC_MEMORY_BARRIER))
    (match_operand:SI 1 "const_int_operand" "")] ;; model
   ""
-  "fence\tiorw,iorw")
+  {
+    enum memmodel model = (enum memmodel) INTVAL (operands[1]);
+    model = memmodel_base (model);
+    if (model == MEMMODEL_SEQ_CST)
+	return "fence\trw,rw";
+    else if (model == MEMMODEL_ACQ_REL)
+	return "fence.tso";
+    else if (model == MEMMODEL_ACQUIRE)
+	return "fence\tr,rw";
+    else if (model == MEMMODEL_RELEASE)
+	return "fence\trw,w";
+  }
+  [(set (attr "length") (const_int 4))])
 
 ;; Atomic memory operations.
 
-- 
2.25.1

[PATCH v4 09/10] RISCV: Weaken atomic loads

[Patrick O'Neill]

This change brings atomic loads in line with table A.6 of the ISA
manual.

2023-04-14 Patrick O'Neill <patrick@rivosinc.com>

	* sync.md (atomic_load<mode>): Implement atomic load mapping.

Signed-off-by: Patrick O'Neill <patrick@rivosinc.com>
---
v3 Changelog:
* Add this patch
---
 gcc/config/riscv/sync.md | 27 ++++++++++++++++++++++++++-
 1 file changed, 26 insertions(+), 1 deletion(-)

diff --git a/gcc/config/riscv/sync.md b/gcc/config/riscv/sync.md
index e91fa29da51..9e3685f5b1c 100644
--- a/gcc/config/riscv/sync.md
+++ b/gcc/config/riscv/sync.md
@@ -23,6 +23,7 @@
   UNSPEC_COMPARE_AND_SWAP
   UNSPEC_SYNC_OLD_OP
   UNSPEC_SYNC_EXCHANGE
+  UNSPEC_ATOMIC_LOAD
   UNSPEC_ATOMIC_STORE
   UNSPEC_MEMORY_BARRIER
 ])
@@ -63,7 +64,31 @@
 
 ;; Atomic memory operations.
 
-;; Implement atomic stores with conservative fences. Fall back to fences for atomic loads.
+(define_insn "atomic_load<mode>"
+  [(set (match_operand:GPR 0 "register_operand" "=r")
+    (unspec_volatile:GPR
+      [(match_operand:GPR 1 "memory_operand" "A")
+       (match_operand:SI 2 "const_int_operand")]      ;; model
+      UNSPEC_ATOMIC_LOAD))]
+  "TARGET_ATOMIC"
+  {
+    enum memmodel model = (enum memmodel) INTVAL (operands[2]);
+    model = memmodel_base (model);
+
+    if (model == MEMMODEL_SEQ_CST)
+      return "fence\trw,rw\;"
+	     "l<amo>\t%0,%1\;"
+	     "fence\tr,rw";
+    if (model == MEMMODEL_ACQUIRE)
+      return "l<amo>\t%0,%1\;"
+	     "fence\tr,rw";
+    else
+      return "l<amo>\t%0,%1";
+  }
+  [(set_attr "type" "atomic")
+   (set (attr "length") (const_int 12))])
+
+;; Implement atomic stores with conservative fences.
 ;; This allows us to be compatible with the ISA manual Table A.6 and Table A.7.
 (define_insn "atomic_store<mode>"
   [(set (match_operand:GPR 0 "memory_operand" "=A")
-- 
2.25.1

[PATCH v4 10/10] RISCV: Table A.6 conformance tests

[Patrick O'Neill]

These tests cover basic cases to ensure the atomic mappings follow the
strengthened Table A.6 mappings that are compatible with Table A.7.

2023-04-14 Patrick O'Neill <patrick@rivosinc.com>

	* amo-table-a-6-amo-add-1.c: New test.
	* amo-table-a-6-amo-add-2.c: Likewise.
	* amo-table-a-6-amo-add-3.c: Likewise.
	* amo-table-a-6-amo-add-4.c: Likewise.
	* amo-table-a-6-amo-add-5.c: Likewise.
	* amo-table-a-6-compare-exchange-1.c: Likewise.
	* amo-table-a-6-compare-exchange-2.c: Likewise.
	* amo-table-a-6-compare-exchange-3.c: Likewise.
	* amo-table-a-6-compare-exchange-4.c: Likewise.
	* amo-table-a-6-compare-exchange-5.c: Likewise.
	* amo-table-a-6-compare-exchange-6.c: Likewise.
	* amo-table-a-6-compare-exchange-7.c: Likewise.
	* amo-table-a-6-fence-1.c: Likewise.
	* amo-table-a-6-fence-2.c: Likewise.
	* amo-table-a-6-fence-3.c: Likewise.
	* amo-table-a-6-fence-4.c: Likewise.
	* amo-table-a-6-fence-5.c: Likewise.
	* amo-table-a-6-load-1.c: Likewise.
	* amo-table-a-6-load-2.c: Likewise.
	* amo-table-a-6-load-3.c: Likewise.
	* amo-table-a-6-store-1.c: Likewise.
	* amo-table-a-6-store-2.c: Likewise.
	* amo-table-a-6-store-compat-3.c: Likewise.

Signed-off-by: Patrick O'Neill <patrick@rivosinc.com>
---
v3 Changelog:
* Consolidate existing tests in this patch
* Add new tests for store/load/amoadd
---
v4 Changelog:
* Add additional compare-exchange testcases
* Update assertions to use scan-assembler-times for fences
---
 .../gcc.target/riscv/amo-table-a-6-amo-add-1.c        |  8 ++++++++
 .../gcc.target/riscv/amo-table-a-6-amo-add-2.c        |  8 ++++++++
 .../gcc.target/riscv/amo-table-a-6-amo-add-3.c        |  8 ++++++++
 .../gcc.target/riscv/amo-table-a-6-amo-add-4.c        |  8 ++++++++
 .../gcc.target/riscv/amo-table-a-6-amo-add-5.c        |  8 ++++++++
 .../riscv/amo-table-a-6-compare-exchange-1.c          | 10 ++++++++++
 .../riscv/amo-table-a-6-compare-exchange-2.c          | 10 ++++++++++
 .../riscv/amo-table-a-6-compare-exchange-3.c          | 10 ++++++++++
 .../riscv/amo-table-a-6-compare-exchange-4.c          | 10 ++++++++++
 .../riscv/amo-table-a-6-compare-exchange-5.c          | 10 ++++++++++
 .../riscv/amo-table-a-6-compare-exchange-6.c          | 11 +++++++++++
 .../riscv/amo-table-a-6-compare-exchange-7.c          | 10 ++++++++++
 .../gcc.target/riscv/amo-table-a-6-fence-1.c          |  8 ++++++++
 .../gcc.target/riscv/amo-table-a-6-fence-2.c          | 10 ++++++++++
 .../gcc.target/riscv/amo-table-a-6-fence-3.c          | 10 ++++++++++
 .../gcc.target/riscv/amo-table-a-6-fence-4.c          | 10 ++++++++++
 .../gcc.target/riscv/amo-table-a-6-fence-5.c          | 10 ++++++++++
 gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-1.c |  9 +++++++++
 gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-2.c | 11 +++++++++++
 gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-3.c | 11 +++++++++++
 .../gcc.target/riscv/amo-table-a-6-store-1.c          |  9 +++++++++
 .../gcc.target/riscv/amo-table-a-6-store-2.c          | 11 +++++++++++
 .../gcc.target/riscv/amo-table-a-6-store-compat-3.c   | 11 +++++++++++
 23 files changed, 221 insertions(+)
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-1.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-2.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-3.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-4.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-5.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-1.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-2.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-3.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-4.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-5.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-6.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-7.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-1.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-2.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-3.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-4.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-5.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-1.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-2.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-3.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-1.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-2.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-compat-3.c

diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-1.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-1.c
new file mode 100644
index 00000000000..ae7e407befc
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-1.c
@@ -0,0 +1,8 @@
+/* { dg-do compile } */
+/* Verify that fence mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler "amoadd.w\t" } } */
+
+void
+foo (int* bar, int* baz) {
+  __atomic_add_fetch(bar, baz, __ATOMIC_RELAXED);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-2.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-2.c
new file mode 100644
index 00000000000..60d84f32481
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-2.c
@@ -0,0 +1,8 @@
+/* { dg-do compile } */
+/* Verify that fence mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler "amoadd.w.aq\t" } } */
+
+void
+foo (int* bar, int* baz) {
+  __atomic_add_fetch(bar, baz, __ATOMIC_ACQUIRE);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-3.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-3.c
new file mode 100644
index 00000000000..a97231e4e73
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-3.c
@@ -0,0 +1,8 @@
+/* { dg-do compile } */
+/* Verify that fence mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler "amoadd.w.rl\t" } } */
+
+void
+foo (int* bar, int* baz) {
+  __atomic_add_fetch(bar, baz, __ATOMIC_RELEASE);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-4.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-4.c
new file mode 100644
index 00000000000..3c843afdd5f
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-4.c
@@ -0,0 +1,8 @@
+/* { dg-do compile } */
+/* Verify that fence mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler "amoadd.w.aqrl\t" } } */
+
+void
+foo (int* bar, int* baz) {
+  __atomic_add_fetch(bar, baz, __ATOMIC_ACQ_REL);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-5.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-5.c
new file mode 100644
index 00000000000..3434229f5e0
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-5.c
@@ -0,0 +1,8 @@
+/* { dg-do compile } */
+/* Verify that fence mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler "amoadd.w.aqrl\t" } } */
+
+void
+foo (int* bar, int* baz) {
+  __atomic_add_fetch(bar, baz, __ATOMIC_SEQ_CST);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-1.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-1.c
new file mode 100644
index 00000000000..a9141cde48f
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-1.c
@@ -0,0 +1,10 @@
+/* { dg-do compile } */
+/* Verify that compare exchange mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler-times "lr.w\t" 1 } } */
+/* { dg-final { scan-assembler-times "sc.w\t" 1 } } */
+
+void
+foo (int bar, int baz, int qux)
+{
+  __atomic_compare_exchange_n(&bar, &baz, qux, 1, __ATOMIC_RELAXED, __ATOMIC_RELAXED);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-2.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-2.c
new file mode 100644
index 00000000000..b1ebb20e2f3
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-2.c
@@ -0,0 +1,10 @@
+/* { dg-do compile } */
+/* Verify that compare exchange mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler-times "lr.w.aq\t" 1 } } */
+/* { dg-final { scan-assembler-times "sc.w\t" 1 } } */
+
+void
+foo (int bar, int baz, int qux)
+{
+  __atomic_compare_exchange_n(&bar, &baz, qux, 1, __ATOMIC_CONSUME, __ATOMIC_CONSUME);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-3.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-3.c
new file mode 100644
index 00000000000..47d8d02f7e8
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-3.c
@@ -0,0 +1,10 @@
+/* { dg-do compile } */
+/* Verify that compare exchange mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler-times "lr.w.aq\t" 1 } } */
+/* { dg-final { scan-assembler-times "sc.w\t" 1 } } */
+
+void
+foo (int bar, int baz, int qux)
+{
+  __atomic_compare_exchange_n(&bar, &baz, qux, 1, __ATOMIC_ACQUIRE, __ATOMIC_ACQUIRE);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-4.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-4.c
new file mode 100644
index 00000000000..af6e1d69c75
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-4.c
@@ -0,0 +1,10 @@
+/* { dg-do compile } */
+/* Verify that compare exchange mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler-times "lr.w\t" 1 } } */
+/* { dg-final { scan-assembler-times "sc.w.rl\t" 1 } } */
+
+void
+foo (int bar, int baz, int qux)
+{
+  __atomic_compare_exchange_n(&bar, &baz, qux, 1, __ATOMIC_RELEASE, __ATOMIC_RELAXED);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-5.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-5.c
new file mode 100644
index 00000000000..ceb5660b6af
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-5.c
@@ -0,0 +1,10 @@
+/* { dg-do compile } */
+/* Verify that compare exchange mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler-times "lr.w.aqrl\t" 1 } } */
+/* { dg-final { scan-assembler-times "sc.w.rl\t" 1 } } */
+
+void
+foo (int bar, int baz, int qux)
+{
+  __atomic_compare_exchange_n(&bar, &baz, qux, 1, __ATOMIC_SEQ_CST, __ATOMIC_SEQ_CST);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-6.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-6.c
new file mode 100644
index 00000000000..7b012fb1288
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-6.c
@@ -0,0 +1,11 @@
+/* { dg-do compile } */
+/* Verify that compare exchange mappings match Table A.6's recommended mapping.  */
+/* Mixed mappings need to be unioned.  */
+/* { dg-final { scan-assembler-times "lr.w.aq\t" 1 } } */
+/* { dg-final { scan-assembler-times "sc.w.rl\t" 1 } } */
+
+void
+foo (int bar, int baz, int qux)
+{
+  __atomic_compare_exchange_n(&bar, &baz, qux, 1, __ATOMIC_RELEASE, __ATOMIC_ACQUIRE);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-7.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-7.c
new file mode 100644
index 00000000000..5adec6f7a19
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-7.c
@@ -0,0 +1,10 @@
+/* { dg-do compile } */
+/* Verify that compare exchange mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler-times "lr.w.aqrl\t" 1 } } */
+/* { dg-final { scan-assembler-times "sc.w.rl\t" 1 } } */
+
+void
+foo (int bar, int baz, int qux)
+{
+  __atomic_compare_exchange_n(&bar, &baz, qux, 1, __ATOMIC_SEQ_CST, __ATOMIC_RELAXED);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-1.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-1.c
new file mode 100644
index 00000000000..b8c28013ef3
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-1.c
@@ -0,0 +1,8 @@
+/* { dg-do compile } */
+/* Verify that fence mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler-not "fence\t" } } */
+/* { dg-final { scan-assembler-not "fence.tso" } } */
+
+int main() {
+  __atomic_thread_fence(__ATOMIC_RELAXED);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-2.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-2.c
new file mode 100644
index 00000000000..117f9036e39
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-2.c
@@ -0,0 +1,10 @@
+/* { dg-do compile } */
+/* Verify that fence mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler-times "fence\tr,rw" 1 } } */
+/* { dg-final { scan-assembler-not "fence\trw,w" } } */
+/* { dg-final { scan-assembler-not "fence.tso" } } */
+/* { dg-final { scan-assembler-not "fence\trw,rw" } } */
+
+int main() {
+  __atomic_thread_fence(__ATOMIC_ACQUIRE);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-3.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-3.c
new file mode 100644
index 00000000000..4b6dd7a9aa7
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-3.c
@@ -0,0 +1,10 @@
+/* { dg-do compile } */
+/* Verify that fence mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler-times "fence\trw,w" 1 } } */
+/* { dg-final { scan-assembler-not "fence\tr,rw" } } */
+/* { dg-final { scan-assembler-not "fence.tso" } } */
+/* { dg-final { scan-assembler-not "fence\trw,rw" } } */
+
+int main() {
+  __atomic_thread_fence(__ATOMIC_RELEASE);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-4.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-4.c
new file mode 100644
index 00000000000..d40d3bc37db
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-4.c
@@ -0,0 +1,10 @@
+/* { dg-do compile } */
+/* Verify that fence mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler-times "fence.tso" 1 } } */
+/* { dg-final { scan-assembler-not "fence\trw,w" } } */
+/* { dg-final { scan-assembler-not "fence\tr,rw" } } */
+/* { dg-final { scan-assembler-not "fence\trw,rw" } } */
+
+int main() {
+  __atomic_thread_fence(__ATOMIC_ACQ_REL);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-5.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-5.c
new file mode 100644
index 00000000000..71f76c27789
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-5.c
@@ -0,0 +1,10 @@
+/* { dg-do compile } */
+/* Verify that fence mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler-times "fence\trw,rw" 1 } } */
+/* { dg-final { scan-assembler-not "fence.tso" } } */
+/* { dg-final { scan-assembler-not "fence\trw,w" } } */
+/* { dg-final { scan-assembler-not "fence\tr,rw" } } */
+
+int main() {
+  __atomic_thread_fence(__ATOMIC_SEQ_CST);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-1.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-1.c
new file mode 100644
index 00000000000..8278198072e
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-1.c
@@ -0,0 +1,9 @@
+/* { dg-do compile } */
+/* Verify that load mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler-not "fence\t" } } */
+/* { dg-final { scan-assembler-not "fence.tso" } } */
+
+void
+foo (int* bar, int* baz) {
+  __atomic_load(bar, baz, __ATOMIC_RELAXED);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-2.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-2.c
new file mode 100644
index 00000000000..84b6cc542ac
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-2.c
@@ -0,0 +1,11 @@
+/* { dg-do compile } */
+/* Verify that load mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler-times "fence\tr,rw" 1 } } */
+/* { dg-final { scan-assembler-not "fence\trw,w" } } */
+/* { dg-final { scan-assembler-not "fence\trw,rw" } } */
+/* { dg-final { scan-assembler-not "fence.tso" } } */
+
+void
+foo (int* bar, int* baz) {
+  __atomic_load(bar, baz, __ATOMIC_ACQUIRE);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-3.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-3.c
new file mode 100644
index 00000000000..3f15041d117
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-3.c
@@ -0,0 +1,11 @@
+/* { dg-do compile } */
+/* Verify that load mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler-times "fence\tr,rw" 1 } } */
+/* { dg-final { scan-assembler-times "fence\trw,rw" 1 } } */
+/* { dg-final { scan-assembler-not "fence\trw,w" } } */
+/* { dg-final { scan-assembler-not "fence.tso" } } */
+
+void
+foo (int* bar, int* baz) {
+  __atomic_load(bar, baz, __ATOMIC_SEQ_CST);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-1.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-1.c
new file mode 100644
index 00000000000..c200bd1d11d
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-1.c
@@ -0,0 +1,9 @@
+/* { dg-do compile } */
+/* Verify that store mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler-not "fence\t" } } */
+/* { dg-final { scan-assembler-not "fence.tso" } } */
+
+void
+foo (int* bar, int* baz) {
+  __atomic_store(bar, baz, __ATOMIC_RELAXED);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-2.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-2.c
new file mode 100644
index 00000000000..1cf366b5986
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-2.c
@@ -0,0 +1,11 @@
+/* { dg-do compile } */
+/* Verify that store mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler-times "fence\trw,w" 1 } } */
+/* { dg-final { scan-assembler-not "fence\tr,rw" } } */
+/* { dg-final { scan-assembler-not "fence\trw,rw" } } */
+/* { dg-final { scan-assembler-not "fence.tso" } } */
+
+void
+foo (int* bar, int* baz) {
+  __atomic_store(bar, baz, __ATOMIC_RELEASE);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-compat-3.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-compat-3.c
new file mode 100644
index 00000000000..288e1493156
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-compat-3.c
@@ -0,0 +1,11 @@
+/* { dg-do compile } */
+/* Verify that store mapping are compatible with Table A.6 & A.7.  */
+/* { dg-final { scan-assembler-times "fence\trw,w" 1 } } */
+/* { dg-final { scan-assembler-times "fence\trw,rw" 1 } } */
+/* { dg-final { scan-assembler-not "fence\tr,rw" } } */
+/* { dg-final { scan-assembler-not "fence.tso" } } */
+
+void
+foo (int* bar, int* baz) {
+  __atomic_store(bar, baz, __ATOMIC_SEQ_CST);
+}
-- 
2.25.1

[PATCH v5 00/11] RISC-V: Implement ISA Manual Table A.6 Mappings

[Patrick O'Neill]

This patchset aims to make the RISCV atomics implementation stronger
than the recommended mapping present in table A.6 of the ISA manual.
https://github.com/riscv/riscv-isa-manual/blob/c7cf84547b3aefacab5463add1734c1602b67a49/src/memory.tex#L1083-L1157 

Context
---------
GCC defined RISC-V mappings [1] before the Memory Model task group
finalized their work and provided the ISA Manual Table A.6/A.7 mappings[2].

For at least a year now, we've known that the mappings were different,
but it wasn't clear if these unique mappings had correctness issues.

Andrea Parri found an issue with the GCC mappings, showing that
atomic_compare_exchange_weak_explicit(-,-,-,release,relaxed) mappings do
not enforce release ordering guarantees. (Meaning the GCC mappings have
a correctness issue).
  https://inbox.sourceware.org/gcc-patches/Y1GbJuhcBFpPGJQ0@andrea/ 

Why not A.6?
---------
We can update our mappings now, so the obvious choice would be to
implement Table A.6 (what LLVM implements/ISA manual recommends).

The reason why that isn't the best path forward for GCC is due to a
proposal by Hans Boehm to add L{d|w|b|h}.aq/rl and S{d|w|b|h}.aq/rl.

For context, there is discussion about fast-tracking the addition of
these instructions. The RISCV architectural review committee supports
adopting a "new and common atomics ABI for gcc and LLVM toochains ...
that assumes the addition of the preceding instructions”. That common
ABI is likely to be A.7.
  https://lists.riscv.org/g/tech-privileged/message/1284 

Transitioning from A.6 to A.7 will cause an ABI break. We can hedge
against that risk by emitting a conservative fence after SEQ_CST stores
to make the mapping compatible with both A.6 and A.7.

What does a mapping compatible with both A.6 & A.7 look like?
---------
It is exactly the same as Table A.6, but SEQ_CST stores have a trailing
fence rw,rw. It's strictly stronger than Table A.6.

Microbenchmark
---------
Hans Boehm helpfully wrote a microbenchmark [3] that uses ARM to give a
rough estimate for the performance benefits/penalties of the different
mappings. The microbenchmark is single threaded and almost-write-only.
This case seems unlikely but is useful for getting a rough idea of the
workload that would be impacted the most.

Testcases
-------
Control: A simple volatile store. This is most similar to a relaxed
store.
Release Store: This is most similar to Sw.rl (one of the instructions in
Hans' proposal).
Store with release fence: This is most similar to the mapping present in
Table A.6.
Store with two fences: This is most similar to the compatibility mapping
present in this patchset.

Machines
-------
Intel(R) Core(TM) i7-8650U (sanity check only): x86 TSO
Cortex A53 (Raspberry pi): ARM In order core
Cortex A55 (Pixel 6 Pro): ARM In order core
Cortex A76 (Pixel 6 Pro): ARM Out of order core
Cortex X1 (Pixel 6 Pro): ARM Out of order core

Microbenchmark Results [4]
--------
Units are nsecs per iteration.

Sanity check
Machine    	   CONTROL   REL_STORE   STORE_REL_FENCE   STORE_TWO_FENCE
-------    	   -------   ---------   ---------------   ---------------
Intel i7-8650U 1.34812   1.30038     1.2933            18.0474


Machine    	CONTROL   REL_STORE   STORE_REL_FENCE   STORE_TWO_FENCE
-------    	-------   ---------   ---------------   ---------------
Cortex A53 	7.15224   10.7282     7.15221           10.013
Cortex A55 	2.77965   8.89654     4.44787           7.78331
Cortex A76 	1.78021   1.86095     5.33088           8.88462
Cortex X1  	2.14252   2.14258     4.32982           7.05234

Reordered tests (using -r flag on microbenchmark)
Machine    	CONTROL   REL_STORE   STORE_REL_FENCE   STORE_TWO_FENCE
-------    	-------   ---------   ---------------   ---------------
Cortex A53 	7.15227   10.7282     7.16113           10.034
Cortex A55 	2.78024   8.89574     4.44844           7.78428
Cortex A76 	1.77686   1.81081     5.3301            8.88346
Cortex X1  	2.14254   2.14251     4.3273            7.05239

Benchmark Interpretation
--------
As expected, out of order machines are significantly faster with the
REL_STORE mappings. Unexpectedly, the in-order machines are
significantly slower with REL_STORE rather than REL_STORE_FENCE.

Most machines in the wild are expected to use Table A.7 once the
instructions are introduced. 
Incurring this added cost now will make it easier for compiled RISC-V
binaries to transition to the A.7 memory model mapping.

The performance benefits of moving to A.7 can be more clearly seen using
an almost-all-load microbenchmark (included on page 3 of Hans’
proposal). The code for that microbenchmark is attached below [5].
  https://lists.riscv.org/g/tech-unprivileged/attachment/382/0/load-acquire110422.pdf 
  https://lists.riscv.org/g/tech-unprivileged/topic/92916241 

Caveats
--------
This is a very synthetic microbenchmark that represents what is expected
to be a very unlikely workload. Nevertheless, it's helpful to see the
worst-case price we are paying for compatibility. 

“All times include an entire loop iteration, indirect dispatch and all.
The benchmark alternates tests, but does not lock CPU frequency. Since a
single core was in use, I expect this was running at basically full
speed. Any throttling affected everything more or less uniformly.”
- Hans Boehm

Patchset overview
--------
Patch 1 simplifies the memmodel to ignore MEMMODEL_SYNC_* cases (legacy
cases that aren't handled differently for RISC-V).
Patches 2-6 make the mappings strictly stronger.
Patches 7-9 weaken the mappings to be in line with table A.6 of the ISA
manual.
Patch 11 adds some basic conformance tests to ensure the implemented
mapping matches table A.6 with stronger SEQ_CST stores.

Conformance test cases notes
--------
The conformance tests in this patch are a good sanity check but do not
guarantee exactly following Table A.6. It checks that the right
instructions are emitted (ex. fence rw,r) but not the order of those
instructions.

LLVM mapping notes
--------
LLVM emits corresponding fences for atomic_signal_fence instructions.
This seems to be an oversight since AFAIK atomic_signal_fence acts as a
compiler directive. GCC does not emit any fences for atomic_signal_fence
instructions.

Future work
--------
There still remains some work to be done in this space after this
patchset fixes the correctness of the GCC mappings. 
* Look into explicitly handling subword loads/stores.
* Look into using AMOSWAP.rl for store words/doubles.
* L{b|h|w|d}.aq/rl & S{b|h|w|d}.aq/rl support once ratified.
* zTSO mappings.

Prior Patchsets
--------
Patchset v1:
  https://gcc.gnu.org/pipermail/gcc-patches/2022-April/592950.html 

Patchset v2:
  https://gcc.gnu.org/pipermail/gcc-patches/2023-April/615264.html 

Patchset v3:
  https://gcc.gnu.org/pipermail/gcc-patches/2023-April/615431.html 

Patchset v4:
  https://gcc.gnu.org/pipermail/gcc-patches/2023-April/615748.html 

Changelogs
--------
Changes for v2:
* Use memmodel_base rather than a custom simplify_memmodel function
  (Inspired by Christoph Muellner's patch 1/9)
* Move instruction styling change from [v1 5/7] to [v2 3/8] to reduce
  [v2 6/8]'s complexity
* Eliminated %K flag for atomic store introduced in v1 in favor of
  if/else
* Rebase/test

Changes for v3:
* Use a trailing fence for atomic stores to be compatible with table A.7
* Emit an optimized fence r,rw following a SEQ_CST load
* Consolidate tests in [PATCH v3 10/10]
* Add tests for basic A.6 conformance

Changes for v4:
* Update cover letter to cover more of the reasoning behind moving to a
  compatibility mapping
* Improve conformance testcases patch assertions and add new
  compare-exchange testcases

Changes for v5:
* Update cover letter to cover more context and reasoning behind moving
  to a compatibility mapping
* Rebase to include the subword-atomic cases introduced here:
  https://gcc.gnu.org/pipermail/gcc-patches/2023-April/616080.html
* Add basic amo-add subword atomic testcases
* Reformat changelogs
* Fix misc. whitespace issues

[1] GCC port with mappings merged 06 Feb 2017
  https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=09cae7507d9e88f2b05cf3a9404bf181e65ccbac

[2] A.6 mappings added to ISA manual 12 Dec 2017
https://github.com/riscv/riscv-isa-manual/commit/9da1a115bcc4fe327f35acceb851d4850d12e9fa 

[3] Hans Boehm almost-all-store Microbenchmark:
// Copyright 2023 Google LLC.
// SPDX-License-Identifier: Apache-2.0

#include <atomic>
#include <iostream>
#include <time.h>

static constexpr int INNER_ITERS = 10'000'000;
static constexpr int OUTER_ITERS = 20;
static constexpr int N_TESTS = 4;

volatile int the_volatile(17);
std::atomic<int> the_atomic(17);

void test1(int i) {
  the_volatile = i;
}

void test2(int i) {
  the_atomic.store(i, std::memory_order_release);
}

void test3(int i) {
  atomic_thread_fence(std::memory_order_release);
  the_atomic.store(i, std::memory_order_relaxed);
}

void test4(int i) {
  atomic_thread_fence(std::memory_order_release);
  the_atomic.store(i, std::memory_order_relaxed);
  atomic_thread_fence(std::memory_order_seq_cst);
}

typedef void (*int_func)(int);

uint64_t getnanos() {
  struct timespec result;
  if (clock_gettime(CLOCK_PROCESS_CPUTIME_ID, &result) != 0) {
	std::cerr << "clock_gettime() failed\n";
	exit(1);
  }
  return (uint64_t)result.tv_nsec + 1'000'000'000 * (uint64_t)result.tv_sec;
}

int_func tests[N_TESTS] = { test1, test2, test3, test4 };
const char *test_names[N_TESTS] =
	{ "control", "release store", "store with release fence", "store with two fences" };
uint64_t total_time[N_TESTS] = { 0 };

int main(int argc, char **argv) {
  struct timespec res;
  if (clock_getres(CLOCK_PROCESS_CPUTIME_ID, &res) != 0) {
	std::cerr << "clock_getres() failed\n";
	exit(1);
  } else {
	std::cout << "nsec resolution = " << res.tv_nsec << std::endl;
  }
  if (argc == 2 && argv[1][0] == 'r') {
	// Run tests in reverse order.
	for (int i = 0; i < N_TESTS / 2; ++i) {
  	std::swap(tests[i], tests[N_TESTS - 1 - i]);
  	std::swap(test_names[i], test_names[N_TESTS - 1 - i]);
	}
  }
  for (int i = 0; i < OUTER_ITERS; ++i) {
	// Alternate tests to minimize bias due to thermal throttling.
	for (int j = 0; j < N_TESTS; ++j) {
  	uint64_t start_time = getnanos();
  	for (int k = 1; k <= INNER_ITERS; ++k) {
    	tests[j](k); // Provides memory accesses between tests.
  	}
  	// Ignore first iteration for all tests. The first iteration of the first test is
  	// empirically slightly slower.
  	if (i != 0) {
    	total_time[j] += getnanos() - start_time;
  	}
  	if ((tests[j] == test1 ? the_volatile : the_atomic.load()) != INNER_ITERS) {
    	std::cerr << "result check failed, test = " << j << ", " << the_volatile << std::endl;
    	exit(1);
  	}
	}
  }
  for (int i = 0; i < N_TESTS; ++i) {
	double nsecs_per_iter = (double) total_time[i] / INNER_ITERS / (OUTER_ITERS - 1);
	std::cout << test_names[i] << " took " << nsecs_per_iter << " nseconds per iteration\n";
  }
  exit(0);
}

[4] Hans Boehm Raw Microbenchmark Results
Intel(R) Core(TM) i7-8650U (sanity check only):

hboehm@hboehm-glaptop0:~/tests$ ./a.out
nsec resolution = 1
control took 1.34812 nseconds per iteration
release store took 1.30038 nseconds per iteration
store with release fence took 1.2933 nseconds per iteration
store with two fences took 18.0474 nseconds per iteration

Cortex A53 (Raspberry pi)
hboehm@rpi3-20210823:~/tests$ ./a.out
nsec resolution = 1
control took 7.15224 nseconds per iteration
release store took 10.7282 nseconds per iteration
store with release fence took 7.15221 nseconds per iteration
store with two fences took 10.013 nseconds per iteration
hboehm@rpi3-20210823:~/tests$ ./a.out -r
nsec resolution = 1
control took 7.15227 nseconds per iteration
release store took 10.7282 nseconds per iteration
store with release fence took 7.16133 nseconds per iteration
store with two fences took 10.034 nseconds per iteration

Cortex A55 (Pixel 6 Pro)

raven:/data/tmp # taskset 0f ./release-timer
nsec resolution = 1
control took 2.77965 nseconds per iteration
release store took 8.89654 nseconds per iteration
store with release fence took 4.44787 nseconds per iteration
store with two fences took 7.78331 nseconds per iteration
raven:/data/tmp # taskset 0f ./release-timer -r                                                                 	 
nsec resolution = 1
control took 2.78024 nseconds per iteration
release store took 8.89574 nseconds per iteration
store with release fence took 4.44844 nseconds per iteration
store with two fences took 7.78428 nseconds per iteration

Cortex A76 (Pixel 6 Pro)
raven:/data/tmp # taskset 30 ./release-timer -r                                                                 	 
nsec resolution = 1
control took 1.77686 nseconds per iteration
release store took 1.81081 nseconds per iteration
store with release fence took 5.3301 nseconds per iteration
store with two fences took 8.88346 nseconds per iteration
raven:/data/tmp # taskset 30 ./release-timer                                                                   	 
nsec resolution = 1
control took 1.78021 nseconds per iteration
release store took 1.86095 nseconds per iteration
store with release fence took 5.33088 nseconds per iteration
store with two fences took 8.88462 nseconds per iteration

Cortex X1 (Pixel 6 Pro)
raven:/data/tmp # taskset c0 ./release-timer                                                                   	 
nsec resolution = 1
control took 2.14252 nseconds per iteration
release store took 2.14258 nseconds per iteration
store with release fence took 4.32982 nseconds per iteration
store with two fences took 7.05234 nseconds per iteration
raven:/data/tmp # taskset c0 ./release-timer -r                                                                 	 
nsec resolution = 1
control took 2.14254 nseconds per iteration
release store took 2.14251 nseconds per iteration
store with release fence took 4.3273 nseconds per iteration
store with two fences took 7.05239 nseconds per iteration

[5] Hans Boehm almost-all-load Microbenchmark:
// Copyright 2023 Google LLC.
// SPDX-License-Identifier: Apache-2.0

#include <atomic>
#include <iostream>
#include <time.h>

static constexpr int INNER_ITERS = 10'000'000;
static constexpr int OUTER_ITERS = 20;
static constexpr int N_TESTS = 4;

volatile int the_volatile(17);
std::atomic<int> the_atomic(17);

int test1() {
  return the_volatile;
}

int test2() {
  return the_atomic.load(std::memory_order_acquire);
}

int test3() {
  int result = the_atomic.load(std::memory_order_relaxed);
  atomic_thread_fence(std::memory_order_acquire);
  return result;
}

int test4() {
  atomic_thread_fence(std::memory_order_seq_cst);
  int result = the_atomic.load(std::memory_order_relaxed);
  atomic_thread_fence(std::memory_order_acquire);
  return result;
}

typedef int (*int_func)();

uint64_t getnanos() {
  struct timespec result;
  if (clock_gettime(CLOCK_PROCESS_CPUTIME_ID, &result) != 0) {
	std::cerr << "clock_gettime() failed\n";
	exit(1);
  }
  return (uint64_t)result.tv_nsec + 1'000'000'000 * (uint64_t)result.tv_sec;
}

int_func tests[N_TESTS] = { test1, test2, test3, test4 };
const char *test_names[N_TESTS] =
	{ "control", "acquire load", "load with acquire fence", "load with two fences" };
uint64_t total_time[N_TESTS] = { 0 };

uint sum, last_sum = 0;

int main(int argc, char **argv) {
  struct timespec res;
  if (clock_getres(CLOCK_PROCESS_CPUTIME_ID, &res) != 0) {
	std::cerr << "clock_getres() failed\n";
	exit(1);
  } else {
	std::cout << "nsec resolution = " << res.tv_nsec << std::endl;
  }
  if (argc == 2 && argv[1][0] == 'r') {
	// Run tests in reverse order.
	for (int i = 0; i < N_TESTS / 2; ++i) {
  	std::swap(tests[i], tests[N_TESTS - 1 - i]);
  	std::swap(test_names[i], test_names[N_TESTS - 1 - i]);
	}
  }
  for (int i = 0; i < OUTER_ITERS; ++i) {
	// Alternate tests to minimize bias due to thermal throttling.
	for (int j = 0; j < N_TESTS; ++j) {
  	sum = 0;
  	uint64_t start_time = getnanos();
  	for (int k = 0; k < INNER_ITERS; ++k) {
    	sum += tests[j](); // Provides memory accesses between tests.
  	}
  	// Ignore first iteration for all tests. The first iteration of the first test is
  	// empirically slightly slower.
  	if (i != 0) {
    	total_time[j] += getnanos() - start_time;
  	}
  	if (sum == 0 || last_sum != 0 && sum != last_sum) {
    	std::cerr << "result check failed";
    	exit(1);
  	}
  	last_sum = sum;
	}
  }
  for (int i = 0; i < N_TESTS; ++i) {
	double nsecs_per_iter = (double) total_time[i] / INNER_ITERS / (OUTER_ITERS - 1);
	std::cout << test_names[i] << " took " << nsecs_per_iter << " nseconds per iteration\n";
  }
  exit(0);
}

Patrick O'Neill (11):
  RISC-V: Eliminate SYNC memory models
  RISC-V: Enforce Libatomic LR/SC SEQ_CST
  RISC-V: Enforce subword atomic LR/SC SEQ_CST
  RISC-V: Enforce atomic compare_exchange SEQ_CST
  RISC-V: Add AMO release bits
  RISC-V: Strengthen atomic stores
  RISC-V: Eliminate AMO op fences
  RISC-V: Weaken LR/SC pairs
  RISC-V: Weaken mem_thread_fence
  RISC-V: Weaken atomic loads
  RISC-V: Table A.6 conformance tests

 gcc/config/riscv/riscv-protos.h               |   3 +
 gcc/config/riscv/riscv.cc                     |  66 ++++--
 gcc/config/riscv/sync.md                      | 194 ++++++++++++------
 .../riscv/amo-table-a-6-amo-add-1.c           |   8 +
 .../riscv/amo-table-a-6-amo-add-2.c           |   8 +
 .../riscv/amo-table-a-6-amo-add-3.c           |   8 +
 .../riscv/amo-table-a-6-amo-add-4.c           |   8 +
 .../riscv/amo-table-a-6-amo-add-5.c           |   8 +
 .../riscv/amo-table-a-6-compare-exchange-1.c  |  10 +
 .../riscv/amo-table-a-6-compare-exchange-2.c  |  10 +
 .../riscv/amo-table-a-6-compare-exchange-3.c  |  10 +
 .../riscv/amo-table-a-6-compare-exchange-4.c  |  10 +
 .../riscv/amo-table-a-6-compare-exchange-5.c  |  10 +
 .../riscv/amo-table-a-6-compare-exchange-6.c  |  11 +
 .../riscv/amo-table-a-6-compare-exchange-7.c  |  10 +
 .../gcc.target/riscv/amo-table-a-6-fence-1.c  |   8 +
 .../gcc.target/riscv/amo-table-a-6-fence-2.c  |  10 +
 .../gcc.target/riscv/amo-table-a-6-fence-3.c  |  10 +
 .../gcc.target/riscv/amo-table-a-6-fence-4.c  |  10 +
 .../gcc.target/riscv/amo-table-a-6-fence-5.c  |  10 +
 .../gcc.target/riscv/amo-table-a-6-load-1.c   |   9 +
 .../gcc.target/riscv/amo-table-a-6-load-2.c   |  11 +
 .../gcc.target/riscv/amo-table-a-6-load-3.c   |  11 +
 .../gcc.target/riscv/amo-table-a-6-store-1.c  |   9 +
 .../gcc.target/riscv/amo-table-a-6-store-2.c  |  11 +
 .../riscv/amo-table-a-6-store-compat-3.c      |  11 +
 .../riscv/amo-table-a-6-subword-amo-add-1.c   |   9 +
 .../riscv/amo-table-a-6-subword-amo-add-2.c   |   9 +
 .../riscv/amo-table-a-6-subword-amo-add-3.c   |   9 +
 .../riscv/amo-table-a-6-subword-amo-add-4.c   |   9 +
 .../riscv/amo-table-a-6-subword-amo-add-5.c   |   9 +
 gcc/testsuite/gcc.target/riscv/pr89835.c      |   9 +
 libgcc/config/riscv/atomic.c                  |   4 +-
 33 files changed, 467 insertions(+), 75 deletions(-)
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-1.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-2.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-3.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-4.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-5.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-1.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-2.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-3.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-4.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-5.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-6.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-7.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-1.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-2.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-3.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-4.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-5.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-1.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-2.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-3.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-1.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-2.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-compat-3.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-1.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-2.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-3.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-4.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-5.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/pr89835.c

-- 
2.34.1

[PATCH v5 01/11] RISC-V: Eliminate SYNC memory models

[Patrick O'Neill]

Remove references to MEMMODEL_SYNC_* models by converting via
memmodel_base().

2023-04-27 Patrick O'Neill <patrick@rivosinc.com>

gcc/ChangeLog:

	* config/riscv/riscv.cc: Remove MEMMODEL_SYNC_* cases and
	sanitize memmodel input with memmodel_base.

Signed-off-by: Patrick O'Neill <patrick@rivosinc.com>
---
 gcc/config/riscv/riscv.cc | 11 +++--------
 1 file changed, 3 insertions(+), 8 deletions(-)

diff --git a/gcc/config/riscv/riscv.cc b/gcc/config/riscv/riscv.cc
index 1529855a2b4..02eb5125ac1 100644
--- a/gcc/config/riscv/riscv.cc
+++ b/gcc/config/riscv/riscv.cc
@@ -4299,14 +4299,11 @@ riscv_memmodel_needs_amo_acquire (enum memmodel model)
     {
       case MEMMODEL_ACQ_REL:
       case MEMMODEL_SEQ_CST:
-      case MEMMODEL_SYNC_SEQ_CST:
       case MEMMODEL_ACQUIRE:
       case MEMMODEL_CONSUME:
-      case MEMMODEL_SYNC_ACQUIRE:
 	return true;
 
       case MEMMODEL_RELEASE:
-      case MEMMODEL_SYNC_RELEASE:
       case MEMMODEL_RELAXED:
 	return false;
 
@@ -4325,14 +4322,11 @@ riscv_memmodel_needs_release_fence (enum memmodel model)
     {
       case MEMMODEL_ACQ_REL:
       case MEMMODEL_SEQ_CST:
-      case MEMMODEL_SYNC_SEQ_CST:
       case MEMMODEL_RELEASE:
-      case MEMMODEL_SYNC_RELEASE:
 	return true;
 
       case MEMMODEL_ACQUIRE:
       case MEMMODEL_CONSUME:
-      case MEMMODEL_SYNC_ACQUIRE:
       case MEMMODEL_RELAXED:
 	return false;
 
@@ -4371,6 +4365,7 @@ riscv_print_operand (FILE *file, rtx op, int letter)
     }
   machine_mode mode = GET_MODE (op);
   enum rtx_code code = GET_CODE (op);
+  const enum memmodel model = memmodel_base (INTVAL (op));
 
   switch (letter)
     {
@@ -4508,12 +4503,12 @@ riscv_print_operand (FILE *file, rtx op, int letter)
       break;
 
     case 'A':
-      if (riscv_memmodel_needs_amo_acquire ((enum memmodel) INTVAL (op)))
+      if (riscv_memmodel_needs_amo_acquire (model))
 	fputs (".aq", file);
       break;
 
     case 'F':
-      if (riscv_memmodel_needs_release_fence ((enum memmodel) INTVAL (op)))
+      if (riscv_memmodel_needs_release_fence (model))
 	fputs ("fence iorw,ow; ", file);
       break;
 
-- 
2.34.1

[PATCH v5 02/11] RISC-V: Enforce Libatomic LR/SC SEQ_CST

[Patrick O'Neill]

Replace LR.aq/SC.rl pairs with the SEQ_CST LR.aqrl/SC.rl pairs
recommended by table A.6 of the ISA manual.

2023-04-27 Patrick O'Neill <patrick@rivosinc.com>

libgcc/ChangeLog:

	* config/riscv/atomic.c: Change LR.aq/SC.rl pairs into
	sequentially consistent LR.aqrl/SC.rl pairs.

Signed-off-by: Patrick O'Neill <patrick@rivosinc.com>
---
 libgcc/config/riscv/atomic.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/libgcc/config/riscv/atomic.c b/libgcc/config/riscv/atomic.c
index 573d163ea04..bd2b033132b 100644
--- a/libgcc/config/riscv/atomic.c
+++ b/libgcc/config/riscv/atomic.c
@@ -41,7 +41,7 @@ see the files COPYING3 and COPYING.RUNTIME respectively.  If not, see
     unsigned old, tmp1, tmp2;						\
 									\
     asm volatile ("1:\n\t"						\
-		  "lr.w.aq %[old], %[mem]\n\t"				\
+		  "lr.w.aqrl %[old], %[mem]\n\t"			\
 		  #insn " %[tmp1], %[old], %[value]\n\t"		\
 		  invert						\
 		  "and %[tmp1], %[tmp1], %[mask]\n\t"			\
@@ -75,7 +75,7 @@ see the files COPYING3 and COPYING.RUNTIME respectively.  If not, see
     unsigned old, tmp1;							\
 									\
     asm volatile ("1:\n\t"						\
-		  "lr.w.aq %[old], %[mem]\n\t"				\
+		  "lr.w.aqrl %[old], %[mem]\n\t"			\
 		  "and %[tmp1], %[old], %[mask]\n\t"			\
 		  "bne %[tmp1], %[o], 1f\n\t"				\
 		  "and %[tmp1], %[old], %[not_mask]\n\t"		\
-- 
2.34.1

[PATCH v5 03/11] RISC-V: Enforce subword atomic LR/SC SEQ_CST

[Patrick O'Neill]

Replace LR.aq/SC.rl pairs with the SEQ_CST LR.aqrl/SC.rl pairs
recommended by table A.6 of the ISA manual.

2023-04-27 Patrick O'Neill <patrick@rivosinc.com>

gcc/ChangeLog:

	* config/riscv/sync.md: Change LR.aq/SC.rl pairs into
	sequentially consistent LR.aqrl/SC.rl pairs.

Signed-off-by: Patrick O'Neill <patrick@rivosinc.com>
---
v5 Changelog:
* Add this patch to address the added inline subword atomic sequences.
---
 gcc/config/riscv/sync.md | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/gcc/config/riscv/sync.md b/gcc/config/riscv/sync.md
index 19274528262..0c83ef04607 100644
--- a/gcc/config/riscv/sync.md
+++ b/gcc/config/riscv/sync.md
@@ -109,7 +109,7 @@
   "TARGET_ATOMIC && TARGET_INLINE_SUBWORD_ATOMIC"
   {
     return "1:\;"
-	   "lr.w.aq\t%0, %1\;"
+	   "lr.w.aqrl\t%0, %1\;"
 	   "<insn>\t%5, %0, %2\;"
 	   "and\t%5, %5, %3\;"
 	   "and\t%6, %0, %4\;"
@@ -173,7 +173,7 @@
   "TARGET_ATOMIC && TARGET_INLINE_SUBWORD_ATOMIC"
   {
     return "1:\;"
-	   "lr.w.aq\t%0, %1\;"
+	   "lr.w.aqrl\t%0, %1\;"
 	   "and\t%5, %0, %2\;"
 	   "not\t%5, %5\;"
 	   "and\t%5, %5, %3\;"
@@ -278,7 +278,7 @@
   "TARGET_ATOMIC && TARGET_INLINE_SUBWORD_ATOMIC"
   {
     return "1:\;"
-	   "lr.w.aq\t%0, %1\;"
+	   "lr.w.aqrl\t%0, %1\;"
 	   "and\t%4, %0, %3\;"
 	   "or\t%4, %4, %2\;"
 	   "sc.w.rl\t%4, %4, %1\;"
@@ -443,7 +443,7 @@
   "TARGET_ATOMIC && TARGET_INLINE_SUBWORD_ATOMIC"
   {
     return "1:\;"
-	   "lr.w.aq\t%0, %1\;"
+	   "lr.w.aqrl\t%0, %1\;"
 	   "and\t%6, %0, %4\;"
 	   "bne\t%6, %z2, 1f\;"
 	   "and\t%6, %0, %5\;"
-- 
2.34.1

[PATCH v5 04/11] RISC-V: Enforce atomic compare_exchange SEQ_CST

[Patrick O'Neill]

This patch enforces SEQ_CST for atomic compare_exchange ops.

Replace Fence/LR.aq/SC.aq pairs with SEQ_CST LR.aqrl/SC.rl pairs
recommended by table A.6 of the ISA manual.

2023-04-27 Patrick O'Neill <patrick@rivosinc.com>

gcc/ChangeLog:

	* config/riscv/sync.md: Change FENCE/LR.aq/SC.aq into
	sequentially consistent LR.aqrl/SC.rl pair.

Signed-off-by: Patrick O'Neill <patrick@rivosinc.com>
---
 gcc/config/riscv/sync.md | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)

diff --git a/gcc/config/riscv/sync.md b/gcc/config/riscv/sync.md
index 0c83ef04607..5620d6ffa58 100644
--- a/gcc/config/riscv/sync.md
+++ b/gcc/config/riscv/sync.md
@@ -297,9 +297,16 @@
 	 UNSPEC_COMPARE_AND_SWAP))
    (clobber (match_scratch:GPR 6 "=&r"))]
   "TARGET_ATOMIC"
-  "%F5 1: lr.<amo>%A5 %0,%1; bne %0,%z2,1f; sc.<amo>%A4 %6,%z3,%1; bnez %6,1b; 1:"
+  {
+    return "1:\;"
+	   "lr.<amo>.aqrl\t%0,%1\;"
+	   "bne\t%0,%z2,1f\;"
+	   "sc.<amo>.rl\t%6,%z3,%1\;"
+	   "bnez\t%6,1b\;"
+	   "1:";
+  }
   [(set_attr "type" "atomic")
-   (set (attr "length") (const_int 20))])
+   (set (attr "length") (const_int 16))])
 
 (define_expand "atomic_compare_and_swap<mode>"
   [(match_operand:SI 0 "register_operand" "")   ;; bool output
-- 
2.34.1

[PATCH v5 05/11] RISC-V: Add AMO release bits

[Patrick O'Neill]

This patch sets the relevant .rl bits on amo operations.

2023-04-27 Patrick O'Neill <patrick@rivosinc.com>

gcc/ChangeLog:

	* config/riscv/riscv.cc (riscv_print_operand): change behavior
	of %A to include release bits.

Signed-off-by: Patrick O'Neill <patrick@rivosinc.com>
---
 gcc/config/riscv/riscv.cc | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/gcc/config/riscv/riscv.cc b/gcc/config/riscv/riscv.cc
index 02eb5125ac1..d46781d8981 100644
--- a/gcc/config/riscv/riscv.cc
+++ b/gcc/config/riscv/riscv.cc
@@ -4503,8 +4503,13 @@ riscv_print_operand (FILE *file, rtx op, int letter)
       break;
 
     case 'A':
-      if (riscv_memmodel_needs_amo_acquire (model))
+      if (riscv_memmodel_needs_amo_acquire (model)
+	  && riscv_memmodel_needs_release_fence (model))
+	fputs (".aqrl", file);
+      else if (riscv_memmodel_needs_amo_acquire (model))
 	fputs (".aq", file);
+      else if (riscv_memmodel_needs_release_fence (model))
+	fputs (".rl", file);
       break;
 
     case 'F':
-- 
2.34.1

[PATCH v5 06/11] RISC-V: Strengthen atomic stores

[Patrick O'Neill]

This change makes atomic stores strictly stronger than table A.6 of the
ISA manual. This mapping makes the overall patchset compatible with
table A.7 as well.

2023-04-27 Patrick O'Neill <patrick@rivosinc.com>

	PR 89835

gcc/ChangeLog:

	* config/riscv/sync.md:

gcc/testsuite/ChangeLog:

	* gcc.target/riscv/pr89835.c: New test.

Signed-off-by: Patrick O'Neill <patrick@rivosinc.com>
---
 gcc/config/riscv/sync.md                 | 21 ++++++++++++++++++---
 gcc/testsuite/gcc.target/riscv/pr89835.c |  9 +++++++++
 2 files changed, 27 insertions(+), 3 deletions(-)
 create mode 100644 gcc/testsuite/gcc.target/riscv/pr89835.c

diff --git a/gcc/config/riscv/sync.md b/gcc/config/riscv/sync.md
index 5620d6ffa58..1acb78a9ae4 100644
--- a/gcc/config/riscv/sync.md
+++ b/gcc/config/riscv/sync.md
@@ -56,7 +56,9 @@
 
 ;; Atomic memory operations.
 
-;; Implement atomic stores with amoswap.  Fall back to fences for atomic loads.
+;; Implement atomic stores with conservative fences.  Fall back to fences for
+;; atomic loads.
+;; This allows us to be compatible with the ISA manual Table A.6 and Table A.7.
 (define_insn "atomic_store<mode>"
   [(set (match_operand:GPR 0 "memory_operand" "=A")
     (unspec_volatile:GPR
@@ -64,9 +66,22 @@
        (match_operand:SI 2 "const_int_operand")]      ;; model
       UNSPEC_ATOMIC_STORE))]
   "TARGET_ATOMIC"
-  "%F2amoswap.<amo>%A2 zero,%z1,%0"
+  {
+    enum memmodel model = (enum memmodel) INTVAL (operands[2]);
+    model = memmodel_base (model);
+
+    if (model == MEMMODEL_SEQ_CST)
+      return "fence\trw,w\;"
+	     "s<amo>\t%z1,%0\;"
+	     "fence\trw,rw";
+    if (model == MEMMODEL_RELEASE)
+      return "fence\trw,w\;"
+	     "s<amo>\t%z1,%0";
+    else
+      return "s<amo>\t%z1,%0";
+  }
   [(set_attr "type" "atomic")
-   (set (attr "length") (const_int 8))])
+   (set (attr "length") (const_int 12))])
 
 (define_insn "atomic_<atomic_optab><mode>"
   [(set (match_operand:GPR 0 "memory_operand" "+A")
diff --git a/gcc/testsuite/gcc.target/riscv/pr89835.c b/gcc/testsuite/gcc.target/riscv/pr89835.c
new file mode 100644
index 00000000000..ab190e11b60
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/pr89835.c
@@ -0,0 +1,9 @@
+/* { dg-do compile } */
+/* Verify that relaxed atomic stores use simple store instuctions.  */
+/* { dg-final { scan-assembler-not "amoswap" } } */
+
+void
+foo(int bar, int baz)
+{
+  __atomic_store_n(&bar, baz, __ATOMIC_RELAXED);
+}
-- 
2.34.1

[PATCH v5 07/11] RISC-V: Eliminate AMO op fences

[Patrick O'Neill]

Atomic operations with the appropriate bits set already enfore release
semantics. Remove unnecessary release fences from atomic ops.

This change brings AMO ops in line with table A.6 of the ISA manual.

2023-04-27 Patrick O'Neill <patrick@rivosinc.com>

gcc/ChangeLog:

	* config/riscv/riscv.cc
	(riscv_memmodel_needs_amo_release): Change function name.
	(riscv_print_operand): Remove unneeded %F case.
	* config/riscv/sync.md: Remove unneeded fences.

Signed-off-by: Patrick O'Neill <patrick@rivosinc.com>
---
 gcc/config/riscv/riscv.cc | 16 +++++-----------
 gcc/config/riscv/sync.md  | 12 ++++++------
 2 files changed, 11 insertions(+), 17 deletions(-)

diff --git a/gcc/config/riscv/riscv.cc b/gcc/config/riscv/riscv.cc
index d46781d8981..9eba03ac189 100644
--- a/gcc/config/riscv/riscv.cc
+++ b/gcc/config/riscv/riscv.cc
@@ -4312,11 +4312,11 @@ riscv_memmodel_needs_amo_acquire (enum memmodel model)
     }
 }
 
-/* Return true if a FENCE should be emitted to before a memory access to
-   implement the release portion of memory model MODEL.  */
+/* Return true if the .RL suffix should be added to an AMO to implement the
+   release portion of memory model MODEL.  */
 
 static bool
-riscv_memmodel_needs_release_fence (enum memmodel model)
+riscv_memmodel_needs_amo_release (enum memmodel model)
 {
   switch (model)
     {
@@ -4342,7 +4342,6 @@ riscv_memmodel_needs_release_fence (enum memmodel model)
    'R'	Print the low-part relocation associated with OP.
    'C'	Print the integer branch condition for comparison OP.
    'A'	Print the atomic operation suffix for memory model OP.
-   'F'	Print a FENCE if the memory model requires a release.
    'z'	Print x0 if OP is zero, otherwise print OP normally.
    'i'	Print i if the operand is not a register.
    'S'	Print shift-index of single-bit mask OP.
@@ -4504,19 +4503,14 @@ riscv_print_operand (FILE *file, rtx op, int letter)
 
     case 'A':
       if (riscv_memmodel_needs_amo_acquire (model)
-	  && riscv_memmodel_needs_release_fence (model))
+	  && riscv_memmodel_needs_amo_release (model))
 	fputs (".aqrl", file);
       else if (riscv_memmodel_needs_amo_acquire (model))
 	fputs (".aq", file);
-      else if (riscv_memmodel_needs_release_fence (model))
+      else if (riscv_memmodel_needs_amo_release (model))
 	fputs (".rl", file);
       break;
 
-    case 'F':
-      if (riscv_memmodel_needs_release_fence (model))
-	fputs ("fence iorw,ow; ", file);
-      break;
-
     case 'i':
       if (code != REG)
         fputs ("i", file);
diff --git a/gcc/config/riscv/sync.md b/gcc/config/riscv/sync.md
index 1acb78a9ae4..9a3b57bd09f 100644
--- a/gcc/config/riscv/sync.md
+++ b/gcc/config/riscv/sync.md
@@ -91,9 +91,9 @@
 	   (match_operand:SI 2 "const_int_operand")] ;; model
 	 UNSPEC_SYNC_OLD_OP))]
   "TARGET_ATOMIC"
-  "%F2amo<insn>.<amo>%A2 zero,%z1,%0"
+  "amo<insn>.<amo>%A2\tzero,%z1,%0"
   [(set_attr "type" "atomic")
-   (set (attr "length") (const_int 8))])
+   (set (attr "length") (const_int 4))])
 
 (define_insn "atomic_fetch_<atomic_optab><mode>"
   [(set (match_operand:GPR 0 "register_operand" "=&r")
@@ -105,9 +105,9 @@
 	   (match_operand:SI 3 "const_int_operand")] ;; model
 	 UNSPEC_SYNC_OLD_OP))]
   "TARGET_ATOMIC"
-  "%F3amo<insn>.<amo>%A3 %0,%z2,%1"
+  "amo<insn>.<amo>%A3\t%0,%z2,%1"
   [(set_attr "type" "atomic")
-   (set (attr "length") (const_int 8))])
+   (set (attr "length") (const_int 4))])
 
 (define_insn "subword_atomic_fetch_strong_<atomic_optab>"
   [(set (match_operand:SI 0 "register_operand" "=&r")		   ;; old value at mem
@@ -247,9 +247,9 @@
    (set (match_dup 1)
 	(match_operand:GPR 2 "register_operand" "0"))]
   "TARGET_ATOMIC"
-  "%F3amoswap.<amo>%A3 %0,%z2,%1"
+  "amoswap.<amo>%A3\t%0,%z2,%1"
   [(set_attr "type" "atomic")
-   (set (attr "length") (const_int 8))])
+   (set (attr "length") (const_int 4))])
 
 (define_expand "atomic_exchange<mode>"
   [(match_operand:SHORT 0 "register_operand") ;; old value at mem
-- 
2.34.1

[PATCH v5 08/11] RISC-V: Weaken LR/SC pairs

[Patrick O'Neill]

Introduce the %I and %J flags for setting the .aqrl bits on LR/SC pairs
as needed.

Atomic compare and exchange ops provide success and failure memory
models. C++17 and later place no restrictions on the relative strength
of each model, so ensure we cover both by using a model that enforces
the ordering of both given models.

This change brings LR/SC ops in line with table A.6 of the ISA manual.

2023-04-27 Patrick O'Neill <patrick@rivosinc.com>

gcc/ChangeLog:

	* config/riscv/riscv-protos.h (riscv_union_memmodels): Expose
	riscv_union_memmodels function to sync.md.
	* config/riscv/riscv.cc (riscv_union_memmodels): Add function to
	get the union of two memmodels in sync.md.
	(riscv_print_operand): Add %I and %J flags that output the
	optimal LR/SC flag bits for a given memory model.
	* config/riscv/sync.md: Remove static .aqrl bits on LR op/.rl
	bits on SC op and replace with optimized %I, %J flags.

Signed-off-by: Patrick O'Neill <patrick@rivosinc.com>
---
v3 Changelog:
* Consolidate tests in [PATCH v3 10/10]
---
v5 Changelog:
* Also optimize subword LR/SC ops based on given memory model.
---
 gcc/config/riscv/riscv-protos.h |   3 +
 gcc/config/riscv/riscv.cc       |  44 ++++++++++++
 gcc/config/riscv/sync.md        | 114 +++++++++++++++++++-------------
 3 files changed, 114 insertions(+), 47 deletions(-)

diff --git a/gcc/config/riscv/riscv-protos.h b/gcc/config/riscv/riscv-protos.h
index f87661bde2c..5fa9e1122ab 100644
--- a/gcc/config/riscv/riscv-protos.h
+++ b/gcc/config/riscv/riscv-protos.h
@@ -22,6 +22,8 @@ along with GCC; see the file COPYING3.  If not see
 #ifndef GCC_RISCV_PROTOS_H
 #define GCC_RISCV_PROTOS_H
 
+#include "memmodel.h"
+
 /* Symbol types we understand.  The order of this list must match that of
    the unspec enum in riscv.md, subsequent to UNSPEC_ADDRESS_FIRST.  */
 enum riscv_symbol_type {
@@ -81,6 +83,7 @@ extern bool riscv_v_ext_vector_mode_p (machine_mode);
 extern bool riscv_shamt_matches_mask_p (int, HOST_WIDE_INT);
 extern void riscv_subword_address (rtx, rtx *, rtx *, rtx *, rtx *);
 extern void riscv_lshift_subword (machine_mode, rtx, rtx, rtx *);
+extern enum memmodel riscv_union_memmodels (enum memmodel, enum memmodel);
 
 /* Routines implemented in riscv-c.cc.  */
 void riscv_cpu_cpp_builtins (cpp_reader *);
diff --git a/gcc/config/riscv/riscv.cc b/gcc/config/riscv/riscv.cc
index 9eba03ac189..69e9b2aa548 100644
--- a/gcc/config/riscv/riscv.cc
+++ b/gcc/config/riscv/riscv.cc
@@ -4289,6 +4289,36 @@ riscv_print_operand_reloc (FILE *file, rtx op, bool hi_reloc)
   fputc (')', file);
 }
 
+/* Return the memory model that encapuslates both given models.  */
+
+enum memmodel
+riscv_union_memmodels (enum memmodel model1, enum memmodel model2)
+{
+  model1 = memmodel_base (model1);
+  model2 = memmodel_base (model2);
+
+  enum memmodel weaker = model1 <= model2 ? model1: model2;
+  enum memmodel stronger = model1 > model2 ? model1: model2;
+
+  switch (stronger)
+    {
+      case MEMMODEL_SEQ_CST:
+      case MEMMODEL_ACQ_REL:
+	return stronger;
+      case MEMMODEL_RELEASE:
+	if (weaker == MEMMODEL_ACQUIRE || weaker == MEMMODEL_CONSUME)
+	  return MEMMODEL_ACQ_REL;
+	else
+	  return stronger;
+      case MEMMODEL_ACQUIRE:
+      case MEMMODEL_CONSUME:
+      case MEMMODEL_RELAXED:
+	return stronger;
+      default:
+	gcc_unreachable ();
+    }
+}
+
 /* Return true if the .AQ suffix should be added to an AMO to implement the
    acquire portion of memory model MODEL.  */
 
@@ -4342,6 +4372,8 @@ riscv_memmodel_needs_amo_release (enum memmodel model)
    'R'	Print the low-part relocation associated with OP.
    'C'	Print the integer branch condition for comparison OP.
    'A'	Print the atomic operation suffix for memory model OP.
+   'I'	Print the LR suffix for memory model OP.
+   'J'	Print the SC suffix for memory model OP.
    'z'	Print x0 if OP is zero, otherwise print OP normally.
    'i'	Print i if the operand is not a register.
    'S'	Print shift-index of single-bit mask OP.
@@ -4511,6 +4543,18 @@ riscv_print_operand (FILE *file, rtx op, int letter)
 	fputs (".rl", file);
       break;
 
+    case 'I':
+      if (model == MEMMODEL_SEQ_CST)
+	fputs (".aqrl", file);
+      else if (riscv_memmodel_needs_amo_acquire (model))
+	fputs (".aq", file);
+      break;
+
+    case 'J':
+      if (riscv_memmodel_needs_amo_release (model))
+	fputs (".rl", file);
+      break;
+
     case 'i':
       if (code != REG)
         fputs ("i", file);
diff --git a/gcc/config/riscv/sync.md b/gcc/config/riscv/sync.md
index 9a3b57bd09f..3e6345e83a3 100644
--- a/gcc/config/riscv/sync.md
+++ b/gcc/config/riscv/sync.md
@@ -116,21 +116,22 @@
 	(unspec_volatile:SI
 	  [(any_atomic:SI (match_dup 1)
 		     (match_operand:SI 2 "register_operand" "rI")) ;; value for op
-	   (match_operand:SI 3 "register_operand" "rI")]	   ;; mask
+	   (match_operand:SI 3 "const_int_operand")]		   ;; model
 	 UNSPEC_SYNC_OLD_OP_SUBWORD))
-    (match_operand:SI 4 "register_operand" "rI")		   ;; not_mask
-    (clobber (match_scratch:SI 5 "=&r"))			   ;; tmp_1
-    (clobber (match_scratch:SI 6 "=&r"))]			   ;; tmp_2
+    (match_operand:SI 4 "register_operand" "rI")		   ;; mask
+    (match_operand:SI 5 "register_operand" "rI")		   ;; not_mask
+    (clobber (match_scratch:SI 6 "=&r"))			   ;; tmp_1
+    (clobber (match_scratch:SI 7 "=&r"))]			   ;; tmp_2
   "TARGET_ATOMIC && TARGET_INLINE_SUBWORD_ATOMIC"
   {
     return "1:\;"
-	   "lr.w.aqrl\t%0, %1\;"
-	   "<insn>\t%5, %0, %2\;"
-	   "and\t%5, %5, %3\;"
-	   "and\t%6, %0, %4\;"
-	   "or\t%6, %6, %5\;"
-	   "sc.w.rl\t%5, %6, %1\;"
-	   "bnez\t%5, 1b";
+	   "lr.w%I3\t%0, %1\;"
+	   "<insn>\t%6, %0, %2\;"
+	   "and\t%6, %6, %4\;"
+	   "and\t%7, %0, %5\;"
+	   "or\t%7, %7, %6\;"
+	   "sc.w%J3\t%6, %7, %1\;"
+	   "bnez\t%6, 1b";
   }
   [(set (attr "length") (const_int 28))])
 
@@ -151,6 +152,7 @@
   rtx old = gen_reg_rtx (SImode);
   rtx mem = operands[1];
   rtx value = operands[2];
+  rtx model = operands[3];
   rtx aligned_mem = gen_reg_rtx (SImode);
   rtx shift = gen_reg_rtx (SImode);
   rtx mask = gen_reg_rtx (SImode);
@@ -162,7 +164,7 @@
   riscv_lshift_subword (<MODE>mode, value, shift, &shifted_value);
 
   emit_insn (gen_subword_atomic_fetch_strong_nand (old, aligned_mem,
-						   shifted_value,
+						   shifted_value, model,
 						   mask, not_mask));
 
   emit_move_insn (old, gen_rtx_ASHIFTRT (SImode, old,
@@ -180,22 +182,23 @@
 	(unspec_volatile:SI
 	  [(not:SI (and:SI (match_dup 1)
 			   (match_operand:SI 2 "register_operand" "rI"))) ;; value for op
-	   (match_operand:SI 3 "register_operand" "rI")]		  ;; mask
+	   (match_operand:SI 3 "const_int_operand")]			  ;; mask
 	 UNSPEC_SYNC_OLD_OP_SUBWORD))
-    (match_operand:SI 4 "register_operand" "rI")			  ;; not_mask
-    (clobber (match_scratch:SI 5 "=&r"))				  ;; tmp_1
-    (clobber (match_scratch:SI 6 "=&r"))]				  ;; tmp_2
+    (match_operand:SI 4 "register_operand" "rI")			  ;; mask
+    (match_operand:SI 5 "register_operand" "rI")			  ;; not_mask
+    (clobber (match_scratch:SI 6 "=&r"))				  ;; tmp_1
+    (clobber (match_scratch:SI 7 "=&r"))]				  ;; tmp_2
   "TARGET_ATOMIC && TARGET_INLINE_SUBWORD_ATOMIC"
   {
     return "1:\;"
-	   "lr.w.aqrl\t%0, %1\;"
-	   "and\t%5, %0, %2\;"
-	   "not\t%5, %5\;"
-	   "and\t%5, %5, %3\;"
-	   "and\t%6, %0, %4\;"
-	   "or\t%6, %6, %5\;"
-	   "sc.w.rl\t%5, %6, %1\;"
-	   "bnez\t%5, 1b";
+	   "lr.w%I3\t%0, %1\;"
+	   "and\t%6, %0, %2\;"
+	   "not\t%6, %6\;"
+	   "and\t%6, %6, %4\;"
+	   "and\t%7, %0, %5\;"
+	   "or\t%7, %7, %6\;"
+	   "sc.w%J3\t%6, %7, %1\;"
+	   "bnez\t%6, 1b";
   }
   [(set (attr "length") (const_int 32))])
 
@@ -216,6 +219,7 @@
   rtx old = gen_reg_rtx (SImode);
   rtx mem = operands[1];
   rtx value = operands[2];
+  rtx model = operands[3];
   rtx aligned_mem = gen_reg_rtx (SImode);
   rtx shift = gen_reg_rtx (SImode);
   rtx mask = gen_reg_rtx (SImode);
@@ -228,7 +232,8 @@
 
   emit_insn (gen_subword_atomic_fetch_strong_<atomic_optab> (old, aligned_mem,
 							     shifted_value,
-							     mask, not_mask));
+							     model, mask,
+							     not_mask));
 
   emit_move_insn (old, gen_rtx_ASHIFTRT (SImode, old,
 					 gen_lowpart (QImode, shift)));
@@ -261,6 +266,7 @@
   rtx old = gen_reg_rtx (SImode);
   rtx mem = operands[1];
   rtx value = operands[2];
+  rtx model = operands[3];
   rtx aligned_mem = gen_reg_rtx (SImode);
   rtx shift = gen_reg_rtx (SImode);
   rtx mask = gen_reg_rtx (SImode);
@@ -272,7 +278,8 @@
   riscv_lshift_subword (<MODE>mode, value, shift, &shifted_value);
 
   emit_insn (gen_subword_atomic_exchange_strong (old, aligned_mem,
-						 shifted_value, not_mask));
+						 shifted_value, model,
+						 not_mask));
 
   emit_move_insn (old, gen_rtx_ASHIFTRT (SImode, old,
 					 gen_lowpart (QImode, shift)));
@@ -286,18 +293,19 @@
 	(match_operand:SI 1 "memory_operand" "+A"))	 ;; mem location
    (set (match_dup 1)
 	(unspec_volatile:SI
-	  [(match_operand:SI 2 "reg_or_0_operand" "rI")  ;; value
-	   (match_operand:SI 3 "reg_or_0_operand" "rI")] ;; not_mask
+	  [(match_operand:SI 2 "reg_or_0_operand" "rI")	 ;; value
+	   (match_operand:SI 3 "const_int_operand")]	 ;; model
       UNSPEC_SYNC_EXCHANGE_SUBWORD))
-    (clobber (match_scratch:SI 4 "=&r"))]		 ;; tmp_1
+    (match_operand:SI 4 "reg_or_0_operand" "rI")	 ;; not_mask
+    (clobber (match_scratch:SI 5 "=&r"))]		 ;; tmp_1
   "TARGET_ATOMIC && TARGET_INLINE_SUBWORD_ATOMIC"
   {
     return "1:\;"
-	   "lr.w.aqrl\t%0, %1\;"
-	   "and\t%4, %0, %3\;"
-	   "or\t%4, %4, %2\;"
-	   "sc.w.rl\t%4, %4, %1\;"
-	   "bnez\t%4, 1b";
+	   "lr.w%I3\t%0, %1\;"
+	   "and\t%5, %0, %4\;"
+	   "or\t%5, %5, %2\;"
+	   "sc.w%J3\t%5, %5, %1\;"
+	   "bnez\t%5, 1b";
   }
   [(set (attr "length") (const_int 20))])
 
@@ -313,10 +321,15 @@
    (clobber (match_scratch:GPR 6 "=&r"))]
   "TARGET_ATOMIC"
   {
+    enum memmodel model_success = (enum memmodel) INTVAL (operands[4]);
+    enum memmodel model_failure = (enum memmodel) INTVAL (operands[5]);
+    /* Find the union of the two memory models so we can satisfy both success
+       and failure memory models.  */
+    operands[5] = GEN_INT (riscv_union_memmodels (model_success, model_failure));
     return "1:\;"
-	   "lr.<amo>.aqrl\t%0,%1\;"
+	   "lr.<amo>%I5\t%0,%1\;"
 	   "bne\t%0,%z2,1f\;"
-	   "sc.<amo>.rl\t%6,%z3,%1\;"
+	   "sc.<amo>%J5\t%6,%z3,%1\;"
 	   "bnez\t%6,1b\;"
 	   "1:";
   }
@@ -440,9 +453,15 @@
   emit_move_insn (shifted_o, gen_rtx_AND (SImode, shifted_o, mask));
   emit_move_insn (shifted_n, gen_rtx_AND (SImode, shifted_n, mask));
 
+  enum memmodel model_success = (enum memmodel) INTVAL (operands[4]);
+  enum memmodel model_failure = (enum memmodel) INTVAL (operands[5]);
+  /* Find the union of the two memory models so we can satisfy both success
+     and failure memory models.  */
+  rtx model = GEN_INT (riscv_union_memmodels (model_success, model_failure));
+
   emit_insn (gen_subword_atomic_cas_strong (old, aligned_mem,
 					    shifted_o, shifted_n,
-					    mask, not_mask));
+					    model, mask, not_mask));
 
   emit_move_insn (old, gen_rtx_ASHIFTRT (SImode, old,
 					 gen_lowpart (QImode, shift)));
@@ -459,19 +478,20 @@
 	(unspec_volatile:SI [(match_operand:SI 2 "reg_or_0_operand" "rJ")  ;; expected value
 			     (match_operand:SI 3 "reg_or_0_operand" "rJ")] ;; desired value
 	 UNSPEC_COMPARE_AND_SWAP_SUBWORD))
-	(match_operand:SI 4 "register_operand" "rI")			   ;; mask
-	(match_operand:SI 5 "register_operand" "rI")			   ;; not_mask
-	(clobber (match_scratch:SI 6 "=&r"))]				   ;; tmp_1
+	(match_operand:SI 4 "const_int_operand")			   ;; model
+	(match_operand:SI 5 "register_operand" "rI")			   ;; mask
+	(match_operand:SI 6 "register_operand" "rI")			   ;; not_mask
+	(clobber (match_scratch:SI 7 "=&r"))]				   ;; tmp_1
   "TARGET_ATOMIC && TARGET_INLINE_SUBWORD_ATOMIC"
   {
     return "1:\;"
-	   "lr.w.aqrl\t%0, %1\;"
-	   "and\t%6, %0, %4\;"
-	   "bne\t%6, %z2, 1f\;"
-	   "and\t%6, %0, %5\;"
-	   "or\t%6, %6, %3\;"
-	   "sc.w.rl\t%6, %6, %1\;"
-	   "bnez\t%6, 1b\;"
+	   "lr.w%I4\t%0, %1\;"
+	   "and\t%7, %0, %5\;"
+	   "bne\t%7, %z2, 1f\;"
+	   "and\t%7, %0, %6\;"
+	   "or\t%7, %7, %3\;"
+	   "sc.w%J4\t%7, %7, %1\;"
+	   "bnez\t%7, 1b\;"
 	   "1:";
   }
   [(set (attr "length") (const_int 28))])
-- 
2.34.1

[PATCH v5 09/11] RISC-V: Weaken mem_thread_fence

[Patrick O'Neill]

This change brings atomic fences in line with table A.6 of the ISA
manual.

Relax mem_thread_fence according to the memmodel given.

2023-04-27 Patrick O'Neill <patrick@rivosinc.com>

gcc/ChangeLog:

	* config/riscv/sync.md (mem_thread_fence_1): Change fence
	depending on the given memory model.

Signed-off-by: Patrick O'Neill <patrick@rivosinc.com>
---
v3 Changelog:
* Consolidate tests in [PATCH v3 10/10]
* Remove helper functions
---
 gcc/config/riscv/sync.md | 16 +++++++++++++---
 1 file changed, 13 insertions(+), 3 deletions(-)

diff --git a/gcc/config/riscv/sync.md b/gcc/config/riscv/sync.md
index 3e6345e83a3..ba132d8a1ce 100644
--- a/gcc/config/riscv/sync.md
+++ b/gcc/config/riscv/sync.md
@@ -45,14 +45,24 @@
   DONE;
 })
 
-;; Until the RISC-V memory model (hence its mapping from C++) is finalized,
-;; conservatively emit a full FENCE.
 (define_insn "mem_thread_fence_1"
   [(set (match_operand:BLK 0 "" "")
 	(unspec:BLK [(match_dup 0)] UNSPEC_MEMORY_BARRIER))
    (match_operand:SI 1 "const_int_operand" "")] ;; model
   ""
-  "fence\tiorw,iorw")
+  {
+    enum memmodel model = (enum memmodel) INTVAL (operands[1]);
+    model = memmodel_base (model);
+    if (model == MEMMODEL_SEQ_CST)
+	return "fence\trw,rw";
+    else if (model == MEMMODEL_ACQ_REL)
+	return "fence.tso";
+    else if (model == MEMMODEL_ACQUIRE)
+	return "fence\tr,rw";
+    else if (model == MEMMODEL_RELEASE)
+	return "fence\trw,w";
+  }
+  [(set (attr "length") (const_int 4))])
 
 ;; Atomic memory operations.
 
-- 
2.34.1

[PATCH v5 10/11] RISC-V: Weaken atomic loads

[Patrick O'Neill]

This change brings atomic loads in line with table A.6 of the ISA
manual.

2023-04-27 Patrick O'Neill <patrick@rivosinc.com>

gcc/ChangeLog:

	* config/riscv/sync.md (atomic_load<mode>): Implement atomic
	load mapping.

Signed-off-by: Patrick O'Neill <patrick@rivosinc.com>
---
 gcc/config/riscv/sync.md | 28 ++++++++++++++++++++++++++--
 1 file changed, 26 insertions(+), 2 deletions(-)

diff --git a/gcc/config/riscv/sync.md b/gcc/config/riscv/sync.md
index ba132d8a1ce..6e7c762ac57 100644
--- a/gcc/config/riscv/sync.md
+++ b/gcc/config/riscv/sync.md
@@ -26,6 +26,7 @@
   UNSPEC_SYNC_OLD_OP_SUBWORD
   UNSPEC_SYNC_EXCHANGE
   UNSPEC_SYNC_EXCHANGE_SUBWORD
+  UNSPEC_ATOMIC_LOAD
   UNSPEC_ATOMIC_STORE
   UNSPEC_MEMORY_BARRIER
 ])
@@ -66,8 +67,31 @@
 
 ;; Atomic memory operations.
 
-;; Implement atomic stores with conservative fences.  Fall back to fences for
-;; atomic loads.
+(define_insn "atomic_load<mode>"
+  [(set (match_operand:GPR 0 "register_operand" "=r")
+    (unspec_volatile:GPR
+      [(match_operand:GPR 1 "memory_operand" "A")
+       (match_operand:SI 2 "const_int_operand")]      ;; model
+      UNSPEC_ATOMIC_LOAD))]
+  "TARGET_ATOMIC"
+  {
+    enum memmodel model = (enum memmodel) INTVAL (operands[2]);
+    model = memmodel_base (model);
+
+    if (model == MEMMODEL_SEQ_CST)
+      return "fence\trw,rw\;"
+	     "l<amo>\t%0,%1\;"
+	     "fence\tr,rw";
+    if (model == MEMMODEL_ACQUIRE)
+      return "l<amo>\t%0,%1\;"
+	     "fence\tr,rw";
+    else
+      return "l<amo>\t%0,%1";
+  }
+  [(set_attr "type" "atomic")
+   (set (attr "length") (const_int 12))])
+
+;; Implement atomic stores with conservative fences.
 ;; This allows us to be compatible with the ISA manual Table A.6 and Table A.7.
 (define_insn "atomic_store<mode>"
   [(set (match_operand:GPR 0 "memory_operand" "=A")
-- 
2.34.1

[PATCH v5 11/11] RISC-V: Table A.6 conformance tests

[Patrick O'Neill]

These tests cover basic cases to ensure the atomic mappings follow the
strengthened Table A.6 mappings that are compatible with Table A.7.

2023-04-27 Patrick O'Neill <patrick@rivosinc.com>

gcc/testsuite/ChangeLog:

	* gcc.target/riscv/amo-table-a-6-amo-add-1.c: New test.
	* gcc.target/riscv/amo-table-a-6-amo-add-2.c: New test.
	* gcc.target/riscv/amo-table-a-6-amo-add-3.c: New test.
	* gcc.target/riscv/amo-table-a-6-amo-add-4.c: New test.
	* gcc.target/riscv/amo-table-a-6-amo-add-5.c: New test.
	* gcc.target/riscv/amo-table-a-6-compare-exchange-1.c: New test.
	* gcc.target/riscv/amo-table-a-6-compare-exchange-2.c: New test.
	* gcc.target/riscv/amo-table-a-6-compare-exchange-3.c: New test.
	* gcc.target/riscv/amo-table-a-6-compare-exchange-4.c: New test.
	* gcc.target/riscv/amo-table-a-6-compare-exchange-5.c: New test.
	* gcc.target/riscv/amo-table-a-6-compare-exchange-6.c: New test.
	* gcc.target/riscv/amo-table-a-6-compare-exchange-7.c: New test.
	* gcc.target/riscv/amo-table-a-6-fence-1.c: New test.
	* gcc.target/riscv/amo-table-a-6-fence-2.c: New test.
	* gcc.target/riscv/amo-table-a-6-fence-3.c: New test.
	* gcc.target/riscv/amo-table-a-6-fence-4.c: New test.
	* gcc.target/riscv/amo-table-a-6-fence-5.c: New test.
	* gcc.target/riscv/amo-table-a-6-load-1.c: New test.
	* gcc.target/riscv/amo-table-a-6-load-2.c: New test.
	* gcc.target/riscv/amo-table-a-6-load-3.c: New test.
	* gcc.target/riscv/amo-table-a-6-store-1.c: New test.
	* gcc.target/riscv/amo-table-a-6-store-2.c: New test.
	* gcc.target/riscv/amo-table-a-6-store-compat-3.c: New test.
	* gcc.target/riscv/amo-table-a-6-subword-amo-add-1.c: New test.
	* gcc.target/riscv/amo-table-a-6-subword-amo-add-2.c: New test.
	* gcc.target/riscv/amo-table-a-6-subword-amo-add-3.c: New test.
	* gcc.target/riscv/amo-table-a-6-subword-amo-add-4.c: New test.
	* gcc.target/riscv/amo-table-a-6-subword-amo-add-5.c: New test.

Signed-off-by: Patrick O'Neill <patrick@rivosinc.com>
---
 .../gcc.target/riscv/amo-table-a-6-amo-add-1.c        |  8 ++++++++
 .../gcc.target/riscv/amo-table-a-6-amo-add-2.c        |  8 ++++++++
 .../gcc.target/riscv/amo-table-a-6-amo-add-3.c        |  8 ++++++++
 .../gcc.target/riscv/amo-table-a-6-amo-add-4.c        |  8 ++++++++
 .../gcc.target/riscv/amo-table-a-6-amo-add-5.c        |  8 ++++++++
 .../riscv/amo-table-a-6-compare-exchange-1.c          | 10 ++++++++++
 .../riscv/amo-table-a-6-compare-exchange-2.c          | 10 ++++++++++
 .../riscv/amo-table-a-6-compare-exchange-3.c          | 10 ++++++++++
 .../riscv/amo-table-a-6-compare-exchange-4.c          | 10 ++++++++++
 .../riscv/amo-table-a-6-compare-exchange-5.c          | 10 ++++++++++
 .../riscv/amo-table-a-6-compare-exchange-6.c          | 11 +++++++++++
 .../riscv/amo-table-a-6-compare-exchange-7.c          | 10 ++++++++++
 .../gcc.target/riscv/amo-table-a-6-fence-1.c          |  8 ++++++++
 .../gcc.target/riscv/amo-table-a-6-fence-2.c          | 10 ++++++++++
 .../gcc.target/riscv/amo-table-a-6-fence-3.c          | 10 ++++++++++
 .../gcc.target/riscv/amo-table-a-6-fence-4.c          | 10 ++++++++++
 .../gcc.target/riscv/amo-table-a-6-fence-5.c          | 10 ++++++++++
 gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-1.c |  9 +++++++++
 gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-2.c | 11 +++++++++++
 gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-3.c | 11 +++++++++++
 .../gcc.target/riscv/amo-table-a-6-store-1.c          |  9 +++++++++
 .../gcc.target/riscv/amo-table-a-6-store-2.c          | 11 +++++++++++
 .../gcc.target/riscv/amo-table-a-6-store-compat-3.c   | 11 +++++++++++
 .../riscv/amo-table-a-6-subword-amo-add-1.c           |  9 +++++++++
 .../riscv/amo-table-a-6-subword-amo-add-2.c           |  9 +++++++++
 .../riscv/amo-table-a-6-subword-amo-add-3.c           |  9 +++++++++
 .../riscv/amo-table-a-6-subword-amo-add-4.c           |  9 +++++++++
 .../riscv/amo-table-a-6-subword-amo-add-5.c           |  9 +++++++++
 28 files changed, 266 insertions(+)
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-1.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-2.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-3.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-4.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-5.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-1.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-2.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-3.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-4.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-5.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-6.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-7.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-1.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-2.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-3.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-4.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-5.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-1.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-2.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-3.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-1.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-2.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-compat-3.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-1.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-2.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-3.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-4.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-5.c

diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-1.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-1.c
new file mode 100644
index 00000000000..cb044f78fcf
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-1.c
@@ -0,0 +1,8 @@
+/* { dg-do compile } */
+/* Verify that atomic op mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler "amoadd.w\t" } } */
+
+void
+foo (int* bar, int* baz) {
+  __atomic_add_fetch(bar, baz, __ATOMIC_RELAXED);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-2.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-2.c
new file mode 100644
index 00000000000..c8445321989
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-2.c
@@ -0,0 +1,8 @@
+/* { dg-do compile } */
+/* Verify that atomic op mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler "amoadd.w.aq\t" } } */
+
+void
+foo (int* bar, int* baz) {
+  __atomic_add_fetch(bar, baz, __ATOMIC_ACQUIRE);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-3.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-3.c
new file mode 100644
index 00000000000..dfec3020c91
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-3.c
@@ -0,0 +1,8 @@
+/* { dg-do compile } */
+/* Verify that atomic op mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler "amoadd.w.rl\t" } } */
+
+void
+foo (int* bar, int* baz) {
+  __atomic_add_fetch(bar, baz, __ATOMIC_RELEASE);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-4.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-4.c
new file mode 100644
index 00000000000..b9f90e199b6
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-4.c
@@ -0,0 +1,8 @@
+/* { dg-do compile } */
+/* Verify that atomic op mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler "amoadd.w.aqrl\t" } } */
+
+void
+foo (int* bar, int* baz) {
+  __atomic_add_fetch(bar, baz, __ATOMIC_ACQ_REL);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-5.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-5.c
new file mode 100644
index 00000000000..94c4bec933b
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-5.c
@@ -0,0 +1,8 @@
+/* { dg-do compile } */
+/* Verify that atomic op mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler "amoadd.w.aqrl\t" } } */
+
+void
+foo (int* bar, int* baz) {
+  __atomic_add_fetch(bar, baz, __ATOMIC_SEQ_CST);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-1.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-1.c
new file mode 100644
index 00000000000..a9141cde48f
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-1.c
@@ -0,0 +1,10 @@
+/* { dg-do compile } */
+/* Verify that compare exchange mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler-times "lr.w\t" 1 } } */
+/* { dg-final { scan-assembler-times "sc.w\t" 1 } } */
+
+void
+foo (int bar, int baz, int qux)
+{
+  __atomic_compare_exchange_n(&bar, &baz, qux, 1, __ATOMIC_RELAXED, __ATOMIC_RELAXED);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-2.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-2.c
new file mode 100644
index 00000000000..b1ebb20e2f3
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-2.c
@@ -0,0 +1,10 @@
+/* { dg-do compile } */
+/* Verify that compare exchange mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler-times "lr.w.aq\t" 1 } } */
+/* { dg-final { scan-assembler-times "sc.w\t" 1 } } */
+
+void
+foo (int bar, int baz, int qux)
+{
+  __atomic_compare_exchange_n(&bar, &baz, qux, 1, __ATOMIC_CONSUME, __ATOMIC_CONSUME);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-3.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-3.c
new file mode 100644
index 00000000000..47d8d02f7e8
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-3.c
@@ -0,0 +1,10 @@
+/* { dg-do compile } */
+/* Verify that compare exchange mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler-times "lr.w.aq\t" 1 } } */
+/* { dg-final { scan-assembler-times "sc.w\t" 1 } } */
+
+void
+foo (int bar, int baz, int qux)
+{
+  __atomic_compare_exchange_n(&bar, &baz, qux, 1, __ATOMIC_ACQUIRE, __ATOMIC_ACQUIRE);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-4.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-4.c
new file mode 100644
index 00000000000..af6e1d69c75
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-4.c
@@ -0,0 +1,10 @@
+/* { dg-do compile } */
+/* Verify that compare exchange mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler-times "lr.w\t" 1 } } */
+/* { dg-final { scan-assembler-times "sc.w.rl\t" 1 } } */
+
+void
+foo (int bar, int baz, int qux)
+{
+  __atomic_compare_exchange_n(&bar, &baz, qux, 1, __ATOMIC_RELEASE, __ATOMIC_RELAXED);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-5.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-5.c
new file mode 100644
index 00000000000..ceb5660b6af
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-5.c
@@ -0,0 +1,10 @@
+/* { dg-do compile } */
+/* Verify that compare exchange mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler-times "lr.w.aqrl\t" 1 } } */
+/* { dg-final { scan-assembler-times "sc.w.rl\t" 1 } } */
+
+void
+foo (int bar, int baz, int qux)
+{
+  __atomic_compare_exchange_n(&bar, &baz, qux, 1, __ATOMIC_SEQ_CST, __ATOMIC_SEQ_CST);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-6.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-6.c
new file mode 100644
index 00000000000..7b012fb1288
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-6.c
@@ -0,0 +1,11 @@
+/* { dg-do compile } */
+/* Verify that compare exchange mappings match Table A.6's recommended mapping.  */
+/* Mixed mappings need to be unioned.  */
+/* { dg-final { scan-assembler-times "lr.w.aq\t" 1 } } */
+/* { dg-final { scan-assembler-times "sc.w.rl\t" 1 } } */
+
+void
+foo (int bar, int baz, int qux)
+{
+  __atomic_compare_exchange_n(&bar, &baz, qux, 1, __ATOMIC_RELEASE, __ATOMIC_ACQUIRE);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-7.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-7.c
new file mode 100644
index 00000000000..5adec6f7a19
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-7.c
@@ -0,0 +1,10 @@
+/* { dg-do compile } */
+/* Verify that compare exchange mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler-times "lr.w.aqrl\t" 1 } } */
+/* { dg-final { scan-assembler-times "sc.w.rl\t" 1 } } */
+
+void
+foo (int bar, int baz, int qux)
+{
+  __atomic_compare_exchange_n(&bar, &baz, qux, 1, __ATOMIC_SEQ_CST, __ATOMIC_RELAXED);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-1.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-1.c
new file mode 100644
index 00000000000..b8c28013ef3
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-1.c
@@ -0,0 +1,8 @@
+/* { dg-do compile } */
+/* Verify that fence mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler-not "fence\t" } } */
+/* { dg-final { scan-assembler-not "fence.tso" } } */
+
+int main() {
+  __atomic_thread_fence(__ATOMIC_RELAXED);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-2.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-2.c
new file mode 100644
index 00000000000..117f9036e39
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-2.c
@@ -0,0 +1,10 @@
+/* { dg-do compile } */
+/* Verify that fence mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler-times "fence\tr,rw" 1 } } */
+/* { dg-final { scan-assembler-not "fence\trw,w" } } */
+/* { dg-final { scan-assembler-not "fence.tso" } } */
+/* { dg-final { scan-assembler-not "fence\trw,rw" } } */
+
+int main() {
+  __atomic_thread_fence(__ATOMIC_ACQUIRE);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-3.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-3.c
new file mode 100644
index 00000000000..4b6dd7a9aa7
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-3.c
@@ -0,0 +1,10 @@
+/* { dg-do compile } */
+/* Verify that fence mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler-times "fence\trw,w" 1 } } */
+/* { dg-final { scan-assembler-not "fence\tr,rw" } } */
+/* { dg-final { scan-assembler-not "fence.tso" } } */
+/* { dg-final { scan-assembler-not "fence\trw,rw" } } */
+
+int main() {
+  __atomic_thread_fence(__ATOMIC_RELEASE);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-4.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-4.c
new file mode 100644
index 00000000000..d40d3bc37db
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-4.c
@@ -0,0 +1,10 @@
+/* { dg-do compile } */
+/* Verify that fence mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler-times "fence.tso" 1 } } */
+/* { dg-final { scan-assembler-not "fence\trw,w" } } */
+/* { dg-final { scan-assembler-not "fence\tr,rw" } } */
+/* { dg-final { scan-assembler-not "fence\trw,rw" } } */
+
+int main() {
+  __atomic_thread_fence(__ATOMIC_ACQ_REL);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-5.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-5.c
new file mode 100644
index 00000000000..71f76c27789
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-5.c
@@ -0,0 +1,10 @@
+/* { dg-do compile } */
+/* Verify that fence mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler-times "fence\trw,rw" 1 } } */
+/* { dg-final { scan-assembler-not "fence.tso" } } */
+/* { dg-final { scan-assembler-not "fence\trw,w" } } */
+/* { dg-final { scan-assembler-not "fence\tr,rw" } } */
+
+int main() {
+  __atomic_thread_fence(__ATOMIC_SEQ_CST);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-1.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-1.c
new file mode 100644
index 00000000000..8278198072e
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-1.c
@@ -0,0 +1,9 @@
+/* { dg-do compile } */
+/* Verify that load mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler-not "fence\t" } } */
+/* { dg-final { scan-assembler-not "fence.tso" } } */
+
+void
+foo (int* bar, int* baz) {
+  __atomic_load(bar, baz, __ATOMIC_RELAXED);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-2.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-2.c
new file mode 100644
index 00000000000..84b6cc542ac
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-2.c
@@ -0,0 +1,11 @@
+/* { dg-do compile } */
+/* Verify that load mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler-times "fence\tr,rw" 1 } } */
+/* { dg-final { scan-assembler-not "fence\trw,w" } } */
+/* { dg-final { scan-assembler-not "fence\trw,rw" } } */
+/* { dg-final { scan-assembler-not "fence.tso" } } */
+
+void
+foo (int* bar, int* baz) {
+  __atomic_load(bar, baz, __ATOMIC_ACQUIRE);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-3.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-3.c
new file mode 100644
index 00000000000..3f15041d117
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-3.c
@@ -0,0 +1,11 @@
+/* { dg-do compile } */
+/* Verify that load mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler-times "fence\tr,rw" 1 } } */
+/* { dg-final { scan-assembler-times "fence\trw,rw" 1 } } */
+/* { dg-final { scan-assembler-not "fence\trw,w" } } */
+/* { dg-final { scan-assembler-not "fence.tso" } } */
+
+void
+foo (int* bar, int* baz) {
+  __atomic_load(bar, baz, __ATOMIC_SEQ_CST);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-1.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-1.c
new file mode 100644
index 00000000000..c200bd1d11d
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-1.c
@@ -0,0 +1,9 @@
+/* { dg-do compile } */
+/* Verify that store mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler-not "fence\t" } } */
+/* { dg-final { scan-assembler-not "fence.tso" } } */
+
+void
+foo (int* bar, int* baz) {
+  __atomic_store(bar, baz, __ATOMIC_RELAXED);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-2.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-2.c
new file mode 100644
index 00000000000..1cf366b5986
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-2.c
@@ -0,0 +1,11 @@
+/* { dg-do compile } */
+/* Verify that store mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler-times "fence\trw,w" 1 } } */
+/* { dg-final { scan-assembler-not "fence\tr,rw" } } */
+/* { dg-final { scan-assembler-not "fence\trw,rw" } } */
+/* { dg-final { scan-assembler-not "fence.tso" } } */
+
+void
+foo (int* bar, int* baz) {
+  __atomic_store(bar, baz, __ATOMIC_RELEASE);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-compat-3.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-compat-3.c
new file mode 100644
index 00000000000..288e1493156
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-compat-3.c
@@ -0,0 +1,11 @@
+/* { dg-do compile } */
+/* Verify that store mapping are compatible with Table A.6 & A.7.  */
+/* { dg-final { scan-assembler-times "fence\trw,w" 1 } } */
+/* { dg-final { scan-assembler-times "fence\trw,rw" 1 } } */
+/* { dg-final { scan-assembler-not "fence\tr,rw" } } */
+/* { dg-final { scan-assembler-not "fence.tso" } } */
+
+void
+foo (int* bar, int* baz) {
+  __atomic_store(bar, baz, __ATOMIC_SEQ_CST);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-1.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-1.c
new file mode 100644
index 00000000000..9efa25ddf26
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-1.c
@@ -0,0 +1,9 @@
+/* { dg-do compile } */
+/* Verify that subword atomic op mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler-times "lr.w\t" 1 } } */
+/* { dg-final { scan-assembler-times "sc.w\t" 1 } } */
+
+void
+foo (short* bar, short* baz) {
+  __atomic_add_fetch(bar, baz, __ATOMIC_RELAXED);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-2.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-2.c
new file mode 100644
index 00000000000..536f7e9228f
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-2.c
@@ -0,0 +1,9 @@
+/* { dg-do compile } */
+/* Verify that subword atomic op mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler-times "lr.w.aq\t" 1 } } */
+/* { dg-final { scan-assembler-times "sc.w\t" 1 } } */
+
+void
+foo (short* bar, short* baz) {
+  __atomic_add_fetch(bar, baz, __ATOMIC_ACQUIRE);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-3.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-3.c
new file mode 100644
index 00000000000..69c2324a1f7
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-3.c
@@ -0,0 +1,9 @@
+/* { dg-do compile } */
+/* Verify that subword atomic op mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler-times "lr.w\t" 1 } } */
+/* { dg-final { scan-assembler-times "sc.w.rl\t" 1 } } */
+
+void
+foo (short* bar, short* baz) {
+  __atomic_add_fetch(bar, baz, __ATOMIC_RELEASE);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-4.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-4.c
new file mode 100644
index 00000000000..a0779b4c421
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-4.c
@@ -0,0 +1,9 @@
+/* { dg-do compile } */
+/* Verify that subword atomic op mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler-times "lr.w.aq\t" 1 } } */
+/* { dg-final { scan-assembler-times "sc.w.rl\t" 1 } } */
+
+void
+foo (short* bar, short* baz) {
+  __atomic_add_fetch(bar, baz, __ATOMIC_ACQ_REL);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-5.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-5.c
new file mode 100644
index 00000000000..f88901dc717
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-5.c
@@ -0,0 +1,9 @@
+/* { dg-do compile } */
+/* Verify that subword atomic op mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler-times "lr.w.aqrl\t" 1 } } */
+/* { dg-final { scan-assembler-times "sc.w.rl\t" 1 } } */
+
+void
+foo (short* bar, short* baz) {
+  __atomic_add_fetch(bar, baz, __ATOMIC_SEQ_CST);
+}
-- 
2.34.1

Re: [PATCH v5 00/11] RISC-V: Implement ISA Manual Table A.6 Mappings

[Andrea Parri]

On Thu, Apr 27, 2023 at 09:22:50AM -0700, Patrick O'Neill wrote:
> This patchset aims to make the RISCV atomics implementation stronger
> than the recommended mapping present in table A.6 of the ISA manual.
> https://github.com/riscv/riscv-isa-manual/blob/c7cf84547b3aefacab5463add1734c1602b67a49/src/memory.tex#L1083-L1157 
> 
> Context
> ---------
> GCC defined RISC-V mappings [1] before the Memory Model task group
> finalized their work and provided the ISA Manual Table A.6/A.7 mappings[2].
> 
> For at least a year now, we've known that the mappings were different,
> but it wasn't clear if these unique mappings had correctness issues.
> 
> Andrea Parri found an issue with the GCC mappings, showing that
> atomic_compare_exchange_weak_explicit(-,-,-,release,relaxed) mappings do
> not enforce release ordering guarantees. (Meaning the GCC mappings have
> a correctness issue).
>   https://inbox.sourceware.org/gcc-patches/Y1GbJuhcBFpPGJQ0@andrea/ 
> 
> Why not A.6?
> ---------
> We can update our mappings now, so the obvious choice would be to
> implement Table A.6 (what LLVM implements/ISA manual recommends).
> 
> The reason why that isn't the best path forward for GCC is due to a
> proposal by Hans Boehm to add L{d|w|b|h}.aq/rl and S{d|w|b|h}.aq/rl.
> 
> For context, there is discussion about fast-tracking the addition of
> these instructions. The RISCV architectural review committee supports
> adopting a "new and common atomics ABI for gcc and LLVM toochains ...
> that assumes the addition of the preceding instructions”. That common
> ABI is likely to be A.7.
>   https://lists.riscv.org/g/tech-privileged/message/1284 
> 
> Transitioning from A.6 to A.7 will cause an ABI break. We can hedge
> against that risk by emitting a conservative fence after SEQ_CST stores
> to make the mapping compatible with both A.6 and A.7.
> 
> What does a mapping compatible with both A.6 & A.7 look like?
> ---------
> It is exactly the same as Table A.6, but SEQ_CST stores have a trailing
> fence rw,rw. It's strictly stronger than Table A.6.
> 
> Microbenchmark
> ---------
> Hans Boehm helpfully wrote a microbenchmark [3] that uses ARM to give a
> rough estimate for the performance benefits/penalties of the different
> mappings. The microbenchmark is single threaded and almost-write-only.
> This case seems unlikely but is useful for getting a rough idea of the
> workload that would be impacted the most.
> 
> Testcases
> -------
> Control: A simple volatile store. This is most similar to a relaxed
> store.
> Release Store: This is most similar to Sw.rl (one of the instructions in
> Hans' proposal).
> Store with release fence: This is most similar to the mapping present in
> Table A.6.
> Store with two fences: This is most similar to the compatibility mapping
> present in this patchset.
> 
> Machines
> -------
> Intel(R) Core(TM) i7-8650U (sanity check only): x86 TSO
> Cortex A53 (Raspberry pi): ARM In order core
> Cortex A55 (Pixel 6 Pro): ARM In order core
> Cortex A76 (Pixel 6 Pro): ARM Out of order core
> Cortex X1 (Pixel 6 Pro): ARM Out of order core
> 
> Microbenchmark Results [4]
> --------
> Units are nsecs per iteration.
> 
> Sanity check
> Machine    	   CONTROL   REL_STORE   STORE_REL_FENCE   STORE_TWO_FENCE
> -------    	   -------   ---------   ---------------   ---------------
> Intel i7-8650U 1.34812   1.30038     1.2933            18.0474
> 
> 
> Machine    	CONTROL   REL_STORE   STORE_REL_FENCE   STORE_TWO_FENCE
> -------    	-------   ---------   ---------------   ---------------
> Cortex A53 	7.15224   10.7282     7.15221           10.013
> Cortex A55 	2.77965   8.89654     4.44787           7.78331
> Cortex A76 	1.78021   1.86095     5.33088           8.88462
> Cortex X1  	2.14252   2.14258     4.32982           7.05234
> 
> Reordered tests (using -r flag on microbenchmark)
> Machine    	CONTROL   REL_STORE   STORE_REL_FENCE   STORE_TWO_FENCE
> -------    	-------   ---------   ---------------   ---------------
> Cortex A53 	7.15227   10.7282     7.16113           10.034
> Cortex A55 	2.78024   8.89574     4.44844           7.78428
> Cortex A76 	1.77686   1.81081     5.3301            8.88346
> Cortex X1  	2.14254   2.14251     4.3273            7.05239
> 
> Benchmark Interpretation
> --------
> As expected, out of order machines are significantly faster with the
> REL_STORE mappings. Unexpectedly, the in-order machines are
> significantly slower with REL_STORE rather than REL_STORE_FENCE.
> 
> Most machines in the wild are expected to use Table A.7 once the
> instructions are introduced. 
> Incurring this added cost now will make it easier for compiled RISC-V
> binaries to transition to the A.7 memory model mapping.
> 
> The performance benefits of moving to A.7 can be more clearly seen using
> an almost-all-load microbenchmark (included on page 3 of Hans’
> proposal). The code for that microbenchmark is attached below [5].
>   https://lists.riscv.org/g/tech-unprivileged/attachment/382/0/load-acquire110422.pdf 
>   https://lists.riscv.org/g/tech-unprivileged/topic/92916241 
> 
> Caveats
> --------
> This is a very synthetic microbenchmark that represents what is expected
> to be a very unlikely workload. Nevertheless, it's helpful to see the
> worst-case price we are paying for compatibility. 
> 
> “All times include an entire loop iteration, indirect dispatch and all.
> The benchmark alternates tests, but does not lock CPU frequency. Since a
> single core was in use, I expect this was running at basically full
> speed. Any throttling affected everything more or less uniformly.”
> - Hans Boehm
> 
> Patchset overview
> --------
> Patch 1 simplifies the memmodel to ignore MEMMODEL_SYNC_* cases (legacy
> cases that aren't handled differently for RISC-V).
> Patches 2-6 make the mappings strictly stronger.
> Patches 7-9 weaken the mappings to be in line with table A.6 of the ISA
> manual.
> Patch 11 adds some basic conformance tests to ensure the implemented
> mapping matches table A.6 with stronger SEQ_CST stores.
> 
> Conformance test cases notes
> --------
> The conformance tests in this patch are a good sanity check but do not
> guarantee exactly following Table A.6. It checks that the right
> instructions are emitted (ex. fence rw,r) but not the order of those
> instructions.
> 
> LLVM mapping notes
> --------
> LLVM emits corresponding fences for atomic_signal_fence instructions.
> This seems to be an oversight since AFAIK atomic_signal_fence acts as a
> compiler directive. GCC does not emit any fences for atomic_signal_fence
> instructions.
> 
> Future work
> --------
> There still remains some work to be done in this space after this
> patchset fixes the correctness of the GCC mappings. 
> * Look into explicitly handling subword loads/stores.
> * Look into using AMOSWAP.rl for store words/doubles.
> * L{b|h|w|d}.aq/rl & S{b|h|w|d}.aq/rl support once ratified.
> * zTSO mappings.
> 
> Prior Patchsets
> --------
> Patchset v1:
>   https://gcc.gnu.org/pipermail/gcc-patches/2022-April/592950.html 
> 
> Patchset v2:
>   https://gcc.gnu.org/pipermail/gcc-patches/2023-April/615264.html 
> 
> Patchset v3:
>   https://gcc.gnu.org/pipermail/gcc-patches/2023-April/615431.html 
> 
> Patchset v4:
>   https://gcc.gnu.org/pipermail/gcc-patches/2023-April/615748.html 
> 
> Changelogs
> --------
> Changes for v2:
> * Use memmodel_base rather than a custom simplify_memmodel function
>   (Inspired by Christoph Muellner's patch 1/9)
> * Move instruction styling change from [v1 5/7] to [v2 3/8] to reduce
>   [v2 6/8]'s complexity
> * Eliminated %K flag for atomic store introduced in v1 in favor of
>   if/else
> * Rebase/test
> 
> Changes for v3:
> * Use a trailing fence for atomic stores to be compatible with table A.7
> * Emit an optimized fence r,rw following a SEQ_CST load
> * Consolidate tests in [PATCH v3 10/10]
> * Add tests for basic A.6 conformance
> 
> Changes for v4:
> * Update cover letter to cover more of the reasoning behind moving to a
>   compatibility mapping
> * Improve conformance testcases patch assertions and add new
>   compare-exchange testcases
> 
> Changes for v5:
> * Update cover letter to cover more context and reasoning behind moving
>   to a compatibility mapping
> * Rebase to include the subword-atomic cases introduced here:
>   https://gcc.gnu.org/pipermail/gcc-patches/2023-April/616080.html
> * Add basic amo-add subword atomic testcases
> * Reformat changelogs
> * Fix misc. whitespace issues
> 
> [1] GCC port with mappings merged 06 Feb 2017
>   https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=09cae7507d9e88f2b05cf3a9404bf181e65ccbac
> 
> [2] A.6 mappings added to ISA manual 12 Dec 2017
> https://github.com/riscv/riscv-isa-manual/commit/9da1a115bcc4fe327f35acceb851d4850d12e9fa 
> 
> [3] Hans Boehm almost-all-store Microbenchmark:
> // Copyright 2023 Google LLC.
> // SPDX-License-Identifier: Apache-2.0
> 
> #include <atomic>
> #include <iostream>
> #include <time.h>
> 
> static constexpr int INNER_ITERS = 10'000'000;
> static constexpr int OUTER_ITERS = 20;
> static constexpr int N_TESTS = 4;
> 
> volatile int the_volatile(17);
> std::atomic<int> the_atomic(17);
> 
> void test1(int i) {
>   the_volatile = i;
> }
> 
> void test2(int i) {
>   the_atomic.store(i, std::memory_order_release);
> }
> 
> void test3(int i) {
>   atomic_thread_fence(std::memory_order_release);
>   the_atomic.store(i, std::memory_order_relaxed);
> }
> 
> void test4(int i) {
>   atomic_thread_fence(std::memory_order_release);
>   the_atomic.store(i, std::memory_order_relaxed);
>   atomic_thread_fence(std::memory_order_seq_cst);
> }
> 
> typedef void (*int_func)(int);
> 
> uint64_t getnanos() {
>   struct timespec result;
>   if (clock_gettime(CLOCK_PROCESS_CPUTIME_ID, &result) != 0) {
> 	std::cerr << "clock_gettime() failed\n";
> 	exit(1);
>   }
>   return (uint64_t)result.tv_nsec + 1'000'000'000 * (uint64_t)result.tv_sec;
> }
> 
> int_func tests[N_TESTS] = { test1, test2, test3, test4 };
> const char *test_names[N_TESTS] =
> 	{ "control", "release store", "store with release fence", "store with two fences" };
> uint64_t total_time[N_TESTS] = { 0 };
> 
> int main(int argc, char **argv) {
>   struct timespec res;
>   if (clock_getres(CLOCK_PROCESS_CPUTIME_ID, &res) != 0) {
> 	std::cerr << "clock_getres() failed\n";
> 	exit(1);
>   } else {
> 	std::cout << "nsec resolution = " << res.tv_nsec << std::endl;
>   }
>   if (argc == 2 && argv[1][0] == 'r') {
> 	// Run tests in reverse order.
> 	for (int i = 0; i < N_TESTS / 2; ++i) {
>   	std::swap(tests[i], tests[N_TESTS - 1 - i]);
>   	std::swap(test_names[i], test_names[N_TESTS - 1 - i]);
> 	}
>   }
>   for (int i = 0; i < OUTER_ITERS; ++i) {
> 	// Alternate tests to minimize bias due to thermal throttling.
> 	for (int j = 0; j < N_TESTS; ++j) {
>   	uint64_t start_time = getnanos();
>   	for (int k = 1; k <= INNER_ITERS; ++k) {
>     	tests[j](k); // Provides memory accesses between tests.
>   	}
>   	// Ignore first iteration for all tests. The first iteration of the first test is
>   	// empirically slightly slower.
>   	if (i != 0) {
>     	total_time[j] += getnanos() - start_time;
>   	}
>   	if ((tests[j] == test1 ? the_volatile : the_atomic.load()) != INNER_ITERS) {
>     	std::cerr << "result check failed, test = " << j << ", " << the_volatile << std::endl;
>     	exit(1);
>   	}
> 	}
>   }
>   for (int i = 0; i < N_TESTS; ++i) {
> 	double nsecs_per_iter = (double) total_time[i] / INNER_ITERS / (OUTER_ITERS - 1);
> 	std::cout << test_names[i] << " took " << nsecs_per_iter << " nseconds per iteration\n";
>   }
>   exit(0);
> }
> 
> [4] Hans Boehm Raw Microbenchmark Results
> Intel(R) Core(TM) i7-8650U (sanity check only):
> 
> hboehm@hboehm-glaptop0:~/tests$ ./a.out
> nsec resolution = 1
> control took 1.34812 nseconds per iteration
> release store took 1.30038 nseconds per iteration
> store with release fence took 1.2933 nseconds per iteration
> store with two fences took 18.0474 nseconds per iteration
> 
> Cortex A53 (Raspberry pi)
> hboehm@rpi3-20210823:~/tests$ ./a.out
> nsec resolution = 1
> control took 7.15224 nseconds per iteration
> release store took 10.7282 nseconds per iteration
> store with release fence took 7.15221 nseconds per iteration
> store with two fences took 10.013 nseconds per iteration
> hboehm@rpi3-20210823:~/tests$ ./a.out -r
> nsec resolution = 1
> control took 7.15227 nseconds per iteration
> release store took 10.7282 nseconds per iteration
> store with release fence took 7.16133 nseconds per iteration
> store with two fences took 10.034 nseconds per iteration
> 
> Cortex A55 (Pixel 6 Pro)
> 
> raven:/data/tmp # taskset 0f ./release-timer
> nsec resolution = 1
> control took 2.77965 nseconds per iteration
> release store took 8.89654 nseconds per iteration
> store with release fence took 4.44787 nseconds per iteration
> store with two fences took 7.78331 nseconds per iteration
> raven:/data/tmp # taskset 0f ./release-timer -r                                                                 	 
> nsec resolution = 1
> control took 2.78024 nseconds per iteration
> release store took 8.89574 nseconds per iteration
> store with release fence took 4.44844 nseconds per iteration
> store with two fences took 7.78428 nseconds per iteration
> 
> Cortex A76 (Pixel 6 Pro)
> raven:/data/tmp # taskset 30 ./release-timer -r                                                                 	 
> nsec resolution = 1
> control took 1.77686 nseconds per iteration
> release store took 1.81081 nseconds per iteration
> store with release fence took 5.3301 nseconds per iteration
> store with two fences took 8.88346 nseconds per iteration
> raven:/data/tmp # taskset 30 ./release-timer                                                                   	 
> nsec resolution = 1
> control took 1.78021 nseconds per iteration
> release store took 1.86095 nseconds per iteration
> store with release fence took 5.33088 nseconds per iteration
> store with two fences took 8.88462 nseconds per iteration
> 
> Cortex X1 (Pixel 6 Pro)
> raven:/data/tmp # taskset c0 ./release-timer                                                                   	 
> nsec resolution = 1
> control took 2.14252 nseconds per iteration
> release store took 2.14258 nseconds per iteration
> store with release fence took 4.32982 nseconds per iteration
> store with two fences took 7.05234 nseconds per iteration
> raven:/data/tmp # taskset c0 ./release-timer -r                                                                 	 
> nsec resolution = 1
> control took 2.14254 nseconds per iteration
> release store took 2.14251 nseconds per iteration
> store with release fence took 4.3273 nseconds per iteration
> store with two fences took 7.05239 nseconds per iteration
> 
> [5] Hans Boehm almost-all-load Microbenchmark:
> // Copyright 2023 Google LLC.
> // SPDX-License-Identifier: Apache-2.0
> 
> #include <atomic>
> #include <iostream>
> #include <time.h>
> 
> static constexpr int INNER_ITERS = 10'000'000;
> static constexpr int OUTER_ITERS = 20;
> static constexpr int N_TESTS = 4;
> 
> volatile int the_volatile(17);
> std::atomic<int> the_atomic(17);
> 
> int test1() {
>   return the_volatile;
> }
> 
> int test2() {
>   return the_atomic.load(std::memory_order_acquire);
> }
> 
> int test3() {
>   int result = the_atomic.load(std::memory_order_relaxed);
>   atomic_thread_fence(std::memory_order_acquire);
>   return result;
> }
> 
> int test4() {
>   atomic_thread_fence(std::memory_order_seq_cst);
>   int result = the_atomic.load(std::memory_order_relaxed);
>   atomic_thread_fence(std::memory_order_acquire);
>   return result;
> }
> 
> typedef int (*int_func)();
> 
> uint64_t getnanos() {
>   struct timespec result;
>   if (clock_gettime(CLOCK_PROCESS_CPUTIME_ID, &result) != 0) {
> 	std::cerr << "clock_gettime() failed\n";
> 	exit(1);
>   }
>   return (uint64_t)result.tv_nsec + 1'000'000'000 * (uint64_t)result.tv_sec;
> }
> 
> int_func tests[N_TESTS] = { test1, test2, test3, test4 };
> const char *test_names[N_TESTS] =
> 	{ "control", "acquire load", "load with acquire fence", "load with two fences" };
> uint64_t total_time[N_TESTS] = { 0 };
> 
> uint sum, last_sum = 0;
> 
> int main(int argc, char **argv) {
>   struct timespec res;
>   if (clock_getres(CLOCK_PROCESS_CPUTIME_ID, &res) != 0) {
> 	std::cerr << "clock_getres() failed\n";
> 	exit(1);
>   } else {
> 	std::cout << "nsec resolution = " << res.tv_nsec << std::endl;
>   }
>   if (argc == 2 && argv[1][0] == 'r') {
> 	// Run tests in reverse order.
> 	for (int i = 0; i < N_TESTS / 2; ++i) {
>   	std::swap(tests[i], tests[N_TESTS - 1 - i]);
>   	std::swap(test_names[i], test_names[N_TESTS - 1 - i]);
> 	}
>   }
>   for (int i = 0; i < OUTER_ITERS; ++i) {
> 	// Alternate tests to minimize bias due to thermal throttling.
> 	for (int j = 0; j < N_TESTS; ++j) {
>   	sum = 0;
>   	uint64_t start_time = getnanos();
>   	for (int k = 0; k < INNER_ITERS; ++k) {
>     	sum += tests[j](); // Provides memory accesses between tests.
>   	}
>   	// Ignore first iteration for all tests. The first iteration of the first test is
>   	// empirically slightly slower.
>   	if (i != 0) {
>     	total_time[j] += getnanos() - start_time;
>   	}
>   	if (sum == 0 || last_sum != 0 && sum != last_sum) {
>     	std::cerr << "result check failed";
>     	exit(1);
>   	}
>   	last_sum = sum;
> 	}
>   }
>   for (int i = 0; i < N_TESTS; ++i) {
> 	double nsecs_per_iter = (double) total_time[i] / INNER_ITERS / (OUTER_ITERS - 1);
> 	std::cout << test_names[i] << " took " << nsecs_per_iter << " nseconds per iteration\n";
>   }
>   exit(0);
> }
> 
> Patrick O'Neill (11):
>   RISC-V: Eliminate SYNC memory models
>   RISC-V: Enforce Libatomic LR/SC SEQ_CST
>   RISC-V: Enforce subword atomic LR/SC SEQ_CST
>   RISC-V: Enforce atomic compare_exchange SEQ_CST
>   RISC-V: Add AMO release bits
>   RISC-V: Strengthen atomic stores
>   RISC-V: Eliminate AMO op fences
>   RISC-V: Weaken LR/SC pairs
>   RISC-V: Weaken mem_thread_fence
>   RISC-V: Weaken atomic loads
>   RISC-V: Table A.6 conformance tests
> 
>  gcc/config/riscv/riscv-protos.h               |   3 +
>  gcc/config/riscv/riscv.cc                     |  66 ++++--
>  gcc/config/riscv/sync.md                      | 194 ++++++++++++------
>  .../riscv/amo-table-a-6-amo-add-1.c           |   8 +
>  .../riscv/amo-table-a-6-amo-add-2.c           |   8 +
>  .../riscv/amo-table-a-6-amo-add-3.c           |   8 +
>  .../riscv/amo-table-a-6-amo-add-4.c           |   8 +
>  .../riscv/amo-table-a-6-amo-add-5.c           |   8 +
>  .../riscv/amo-table-a-6-compare-exchange-1.c  |  10 +
>  .../riscv/amo-table-a-6-compare-exchange-2.c  |  10 +
>  .../riscv/amo-table-a-6-compare-exchange-3.c  |  10 +
>  .../riscv/amo-table-a-6-compare-exchange-4.c  |  10 +
>  .../riscv/amo-table-a-6-compare-exchange-5.c  |  10 +
>  .../riscv/amo-table-a-6-compare-exchange-6.c  |  11 +
>  .../riscv/amo-table-a-6-compare-exchange-7.c  |  10 +
>  .../gcc.target/riscv/amo-table-a-6-fence-1.c  |   8 +
>  .../gcc.target/riscv/amo-table-a-6-fence-2.c  |  10 +
>  .../gcc.target/riscv/amo-table-a-6-fence-3.c  |  10 +
>  .../gcc.target/riscv/amo-table-a-6-fence-4.c  |  10 +
>  .../gcc.target/riscv/amo-table-a-6-fence-5.c  |  10 +
>  .../gcc.target/riscv/amo-table-a-6-load-1.c   |   9 +
>  .../gcc.target/riscv/amo-table-a-6-load-2.c   |  11 +
>  .../gcc.target/riscv/amo-table-a-6-load-3.c   |  11 +
>  .../gcc.target/riscv/amo-table-a-6-store-1.c  |   9 +
>  .../gcc.target/riscv/amo-table-a-6-store-2.c  |  11 +
>  .../riscv/amo-table-a-6-store-compat-3.c      |  11 +
>  .../riscv/amo-table-a-6-subword-amo-add-1.c   |   9 +
>  .../riscv/amo-table-a-6-subword-amo-add-2.c   |   9 +
>  .../riscv/amo-table-a-6-subword-amo-add-3.c   |   9 +
>  .../riscv/amo-table-a-6-subword-amo-add-4.c   |   9 +
>  .../riscv/amo-table-a-6-subword-amo-add-5.c   |   9 +
>  gcc/testsuite/gcc.target/riscv/pr89835.c      |   9 +
>  libgcc/config/riscv/atomic.c                  |   4 +-
>  33 files changed, 467 insertions(+), 75 deletions(-)
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-1.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-2.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-3.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-4.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-5.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-1.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-2.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-3.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-4.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-5.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-6.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-7.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-1.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-2.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-3.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-4.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-5.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-1.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-2.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-3.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-1.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-2.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-compat-3.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-1.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-2.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-3.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-4.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-5.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/pr89835.c

These changes address and fix all the issues I reported/I'm aware of,
thank you!

Tested-by: Andrea Parri <andrea@rivosinc.com>

  Andrea

Re: [PATCH v5 00/11] RISC-V: Implement ISA Manual Table A.6 Mappings

[Jeff Law]

On 4/27/23 10:22, Patrick O'Neill wrote:
> This patchset aims to make the RISCV atomics implementation stronger
> than the recommended mapping present in table A.6 of the ISA manual.
> https://github.com/riscv/riscv-isa-manual/blob/c7cf84547b3aefacab5463add1734c1602b67a49/src/memory.tex#L1083-L1157
> 
> Context
> ---------
> GCC defined RISC-V mappings [1] before the Memory Model task group
> finalized their work and provided the ISA Manual Table A.6/A.7 mappings[2].
> 
> For at least a year now, we've known that the mappings were different,
> but it wasn't clear if these unique mappings had correctness issues.
> 
> Andrea Parri found an issue with the GCC mappings, showing that
> atomic_compare_exchange_weak_explicit(-,-,-,release,relaxed) mappings do
> not enforce release ordering guarantees. (Meaning the GCC mappings have
> a correctness issue).
>    https://inbox.sourceware.org/gcc-patches/Y1GbJuhcBFpPGJQ0@andrea/
Right.  I recall this discussion, but thanks for the back reference.

> 
> Why not A.6?
> ---------
> We can update our mappings now, so the obvious choice would be to
> implement Table A.6 (what LLVM implements/ISA manual recommends).
> 
> The reason why that isn't the best path forward for GCC is due to a
> proposal by Hans Boehm to add L{d|w|b|h}.aq/rl and S{d|w|b|h}.aq/rl.
> 
> For context, there is discussion about fast-tracking the addition of
> these instructions. The RISCV architectural review committee supports
> adopting a "new and common atomics ABI for gcc and LLVM toochains ...
> that assumes the addition of the preceding instructions”. That common
> ABI is likely to be A.7.
>    https://lists.riscv.org/g/tech-privileged/message/1284
> 
> Transitioning from A.6 to A.7 will cause an ABI break. We can hedge
> against that risk by emitting a conservative fence after SEQ_CST stores
> to make the mapping compatible with both A.6 and A.7.
So I like that we can have compatible sequences across A.6 and A.7.  Of 
course the concern is performance ;-)


> 
> What does a mapping compatible with both A.6 & A.7 look like?
> ---------
> It is exactly the same as Table A.6, but SEQ_CST stores have a trailing
> fence rw,rw. It's strictly stronger than Table A.6.
Right.  So my worry here is silicon that is either already available or 
coming online shortly.   Those implementations simply aren't going to be 
able to use the A.7 mapping, so they pay a penalty.  Does it make sense 
to have the compatibility fences conditional?



> 
> Benchmark Interpretation
> --------
> As expected, out of order machines are significantly faster with the
> REL_STORE mappings. Unexpectedly, the in-order machines are
> significantly slower with REL_STORE rather than REL_STORE_FENCE.
Yea, that's a bit of a surprise.

> 
> Most machines in the wild are expected to use Table A.7 once the
> instructions are introduced.
> Incurring this added cost now will make it easier for compiled RISC-V
> binaries to transition to the A.7 memory model mapping.
> 
> The performance benefits of moving to A.7 can be more clearly seen using
> an almost-all-load microbenchmark (included on page 3 of Hans’
> proposal). The code for that microbenchmark is attached below [5].
>    https://lists.riscv.org/g/tech-unprivileged/attachment/382/0/load-acquire110422.pdf
>    https://lists.riscv.org/g/tech-unprivileged/topic/92916241
Yea.  I'm not questioning the value of the new instructions that are on 
the horizon, just the value of trying to make everything A.7 compatible.


> 
> Conformance test cases notes
> --------
> The conformance tests in this patch are a good sanity check but do not
> guarantee exactly following Table A.6. It checks that the right
> instructions are emitted (ex. fence rw,r) but not the order of those
> instructions.
Note there is a way to check ordering as well.  You might look at the 
check-function-bodies approach.  I think there are some recent examples 
in the gcc risc-v specific tests.


> 
> LLVM mapping notes
> --------
> LLVM emits corresponding fences for atomic_signal_fence instructions.
> This seems to be an oversight since AFAIK atomic_signal_fence acts as a
> compiler directive. GCC does not emit any fences for atomic_signal_fence
> instructions.
This starts to touch on a larger concern.  Specifically I'd really like 
the two compilers to be compatible in terms of the code they generate 
for the various atomics.

What I worry about is code being written (by design or accident) that is 
dependent on the particular behavior of one compiler and then if that 
code gets built with the other compiler, and we end up different 
behavior.  Worse yet, if/when this happens, it's likely to be tough to 
expose, reproduce & debug.

Do you have any sense of where Clang/LLVM is going to go WRT providing 
an A.6 mapping that is compatible with A.7 by using the additional fences?


Jeff

Re: [PATCH v5 01/11] RISC-V: Eliminate SYNC memory models

[Jeff Law]

On 4/27/23 10:22, Patrick O'Neill wrote:
> Remove references to MEMMODEL_SYNC_* models by converting via
> memmodel_base().
> 
> 2023-04-27 Patrick O'Neill <patrick@rivosinc.com>
> 
> gcc/ChangeLog:
> 
> 	* config/riscv/riscv.cc: Remove MEMMODEL_SYNC_* cases and
> 	sanitize memmodel input with memmodel_base.
OK.  Not sure if you want to commit it now or wait for the full set to 
get ACK'd (since there are some questions on the trailing sync approach).

Jeff

Re: [PATCH v5 00/11] RISC-V: Implement ISA Manual Table A.6 Mappings

[Palmer Dabbelt]

On Fri, 28 Apr 2023 09:14:00 PDT (-0700), jeffreyalaw@gmail.com wrote:
>
>
> On 4/27/23 10:22, Patrick O'Neill wrote:
>> This patchset aims to make the RISCV atomics implementation stronger
>> than the recommended mapping present in table A.6 of the ISA manual.
>> https://github.com/riscv/riscv-isa-manual/blob/c7cf84547b3aefacab5463add1734c1602b67a49/src/memory.tex#L1083-L1157
>>
>> Context
>> ---------
>> GCC defined RISC-V mappings [1] before the Memory Model task group
>> finalized their work and provided the ISA Manual Table A.6/A.7 mappings[2].
>>
>> For at least a year now, we've known that the mappings were different,
>> but it wasn't clear if these unique mappings had correctness issues.
>>
>> Andrea Parri found an issue with the GCC mappings, showing that
>> atomic_compare_exchange_weak_explicit(-,-,-,release,relaxed) mappings do
>> not enforce release ordering guarantees. (Meaning the GCC mappings have
>> a correctness issue).
>>    https://inbox.sourceware.org/gcc-patches/Y1GbJuhcBFpPGJQ0@andrea/
> Right.  I recall this discussion, but thanks for the back reference.

Yep, and it's an important one: that's why we're calling the change a 
bug fix and dropping the current GCC mappings.  If we didn't have the 
bug we'd be talking about an ABI break, and since the GCC mappings 
predate the ISA mappings we'd likely need an additional compatibility 
mode.

So I guess we're lucky that we have a concurrency bug.  I think it's the 
first time I've said that ;)

>> Why not A.6?
>> ---------
>> We can update our mappings now, so the obvious choice would be to
>> implement Table A.6 (what LLVM implements/ISA manual recommends).
>>
>> The reason why that isn't the best path forward for GCC is due to a
>> proposal by Hans Boehm to add L{d|w|b|h}.aq/rl and S{d|w|b|h}.aq/rl.
>>
>> For context, there is discussion about fast-tracking the addition of
>> these instructions. The RISCV architectural review committee supports
>> adopting a "new and common atomics ABI for gcc and LLVM toochains ...
>> that assumes the addition of the preceding instructions”. That common
>> ABI is likely to be A.7.
>>    https://lists.riscv.org/g/tech-privileged/message/1284
>>
>> Transitioning from A.6 to A.7 will cause an ABI break. We can hedge
>> against that risk by emitting a conservative fence after SEQ_CST stores
>> to make the mapping compatible with both A.6 and A.7.
> So I like that we can have compatible sequences across A.6 and A.7.  Of
> course the concern is performance ;-)
>
>
>>
>> What does a mapping compatible with both A.6 & A.7 look like?
>> ---------
>> It is exactly the same as Table A.6, but SEQ_CST stores have a trailing
>> fence rw,rw. It's strictly stronger than Table A.6.
> Right.  So my worry here is silicon that is either already available or
> coming online shortly.   Those implementations simply aren't going to be
> able to use the A.7 mapping, so they pay a penalty.  Does it make sense
> to have the compatibility fences conditional?

IIRC this was discussed somewhere in some thread, but I think there's 
really three ABIs that could be implemented here (ignoring the current 
GCC mappings as they're broken):

* ABI compatible with the current mappings in the ISA manual (A.6).  
  This will presumably perform best on extant hardware, given that it's 
  what the words in the PDF say to do.
* ABI compatible with the proposed mappings for the ISA manual (A.7).  
  This may perform better on new hardware.
* ABI compatible with both A.6 and A.7.  This is likely slow on both new 
  and old hardware, but allows cross-linking.  If there's no performance 
  issues this would be the only mode we need, but that seems unlikely.

IMO those should be encoded somewhere in the ELF.  I'd just do it as two 
bits in the header, but last time I proposed header bits the psABI folks 
wanted to do something different.  I don't think where we encode this 
matters all that much, but if we're doing to treat these as real 
long-term ABIs we should have some way to encode that.

There's also the orthogonal axis of whether we use the new instructions.  
Those aren't in specs yet so I think we can hold off on them for a bit, 
but they're the whole point of doing the ABI break so we should at least 
think them over.  I think we're OK because we've just split out the ABI 
from the ISA here, but I'm not sure if I'm missing something.

Now that I wrote that, though, I remember talking to Patrick about it 
and we drew a bunch of stuff on the whiteboard and then got confused.  
So sorry if I'm just out of the loop here...

>
>
>
>>
>> Benchmark Interpretation
>> --------
>> As expected, out of order machines are significantly faster with the
>> REL_STORE mappings. Unexpectedly, the in-order machines are
>> significantly slower with REL_STORE rather than REL_STORE_FENCE.
> Yea, that's a bit of a surprise.
>
>>
>> Most machines in the wild are expected to use Table A.7 once the
>> instructions are introduced.
>> Incurring this added cost now will make it easier for compiled RISC-V
>> binaries to transition to the A.7 memory model mapping.
>>
>> The performance benefits of moving to A.7 can be more clearly seen using
>> an almost-all-load microbenchmark (included on page 3 of Hans’
>> proposal). The code for that microbenchmark is attached below [5].
>>    https://lists.riscv.org/g/tech-unprivileged/attachment/382/0/load-acquire110422.pdf
>>    https://lists.riscv.org/g/tech-unprivileged/topic/92916241
> Yea.  I'm not questioning the value of the new instructions that are on
> the horizon, just the value of trying to make everything A.7 compatible.
>
>
>>
>> Conformance test cases notes
>> --------
>> The conformance tests in this patch are a good sanity check but do not
>> guarantee exactly following Table A.6. It checks that the right
>> instructions are emitted (ex. fence rw,r) but not the order of those
>> instructions.
> Note there is a way to check ordering as well.  You might look at the
> check-function-bodies approach.  I think there are some recent examples
> in the gcc risc-v specific tests.
>
>
>>
>> LLVM mapping notes
>> --------
>> LLVM emits corresponding fences for atomic_signal_fence instructions.
>> This seems to be an oversight since AFAIK atomic_signal_fence acts as a
>> compiler directive. GCC does not emit any fences for atomic_signal_fence
>> instructions.
> This starts to touch on a larger concern.  Specifically I'd really like
> the two compilers to be compatible in terms of the code they generate
> for the various atomics.
>
> What I worry about is code being written (by design or accident) that is
> dependent on the particular behavior of one compiler and then if that
> code gets built with the other compiler, and we end up different
> behavior.  Worse yet, if/when this happens, it's likely to be tough to
> expose, reproduce & debug.
>
> Do you have any sense of where Clang/LLVM is going to go WRT providing
> an A.6 mapping that is compatible with A.7 by using the additional fences?
>
>
> Jeff

Re: [PATCH v5 02/11] RISC-V: Enforce Libatomic LR/SC SEQ_CST

[Jeff Law]

On 4/27/23 10:22, Patrick O'Neill wrote:
> Replace LR.aq/SC.rl pairs with the SEQ_CST LR.aqrl/SC.rl pairs
> recommended by table A.6 of the ISA manual.
> 
> 2023-04-27 Patrick O'Neill <patrick@rivosinc.com>
> 
> libgcc/ChangeLog:
> 
> 	* config/riscv/atomic.c: Change LR.aq/SC.rl pairs into
> 	sequentially consistent LR.aqrl/SC.rl pairs.
OK.  When you install this, make sure you also install #3 of the kit 
which mirrors these changes for the inline subword atomics.

jeff

Re: [PATCH v5 04/11] RISC-V: Enforce atomic compare_exchange SEQ_CST

[Jeff Law]

On 4/27/23 10:22, Patrick O'Neill wrote:
> This patch enforces SEQ_CST for atomic compare_exchange ops.
> 
> Replace Fence/LR.aq/SC.aq pairs with SEQ_CST LR.aqrl/SC.rl pairs
> recommended by table A.6 of the ISA manual.
> 
> 2023-04-27 Patrick O'Neill <patrick@rivosinc.com>
> 
> gcc/ChangeLog:
> 
> 	* config/riscv/sync.md: Change FENCE/LR.aq/SC.aq into
> 	sequentially consistent LR.aqrl/SC.rl pair.
OK.  Note that generally you should note which pattern you're changing 
in a ChangeLog entry, similar to how we note the function being changed. 
  So something like this might be better:

	* config/riscv/sync.md (atomic_cas_value_strong<mode>): ...

Jeff

Re: [PATCH v5 05/11] RISC-V: Add AMO release bits

[Jeff Law]

On 4/27/23 10:22, Patrick O'Neill wrote:
> This patch sets the relevant .rl bits on amo operations.
> 
> 2023-04-27 Patrick O'Neill <patrick@rivosinc.com>
> 
> gcc/ChangeLog:
> 
> 	* config/riscv/riscv.cc (riscv_print_operand): change behavior
> 	of %A to include release bits.
Capitalize "change" in the ChangeLog entry.  OK with that nit fixed.

jeff

Re: [PATCH v5 06/11] RISC-V: Strengthen atomic stores

[Jeff Law]

On 4/27/23 10:22, Patrick O'Neill wrote:
> This change makes atomic stores strictly stronger than table A.6 of the
> ISA manual. This mapping makes the overall patchset compatible with
> table A.7 as well.
> 
> 2023-04-27 Patrick O'Neill <patrick@rivosinc.com>
> 
> 	PR 89835
Should be "PR target/89835"

> 
> gcc/ChangeLog:
> 
> 	* config/riscv/sync.md:
Needs some text here :-)


I'm not objecting to this patch, but I think we've got an option 
question about whether or not this approach is too expensive for 
existing or soon arriving implementations.

If the decision on that topic is to just pay the cost, then this patch 
is fine.  If we decide to make compatibility optional to avoid the 
additional cost, then this will need suitable adjustments.

Jeff

Re: [PATCH v5 07/11] RISC-V: Eliminate AMO op fences

[Jeff Law]

On 4/27/23 10:22, Patrick O'Neill wrote:
> Atomic operations with the appropriate bits set already enfore release
> semantics. Remove unnecessary release fences from atomic ops.
> 
> This change brings AMO ops in line with table A.6 of the ISA manual.
> 
> 2023-04-27 Patrick O'Neill <patrick@rivosinc.com>
> 
> gcc/ChangeLog:
> 
> 	* config/riscv/riscv.cc
> 	(riscv_memmodel_needs_amo_release): Change function name.
> 	(riscv_print_operand): Remove unneeded %F case.
> 	* config/riscv/sync.md: Remove unneeded fences.
OK.  Though note this depends on a resolution of patch #6.  You could 
potentially leave the %F support in riscv_print_operand and install the 
rest of this patch while we settle the question around #6.

Jeff

Re: [PATCH v5 06/11] RISC-V: Strengthen atomic stores

[Palmer Dabbelt]

On Fri, 28 Apr 2023 10:40:15 PDT (-0700), jeffreyalaw@gmail.com wrote:
>
>
> On 4/27/23 10:22, Patrick O'Neill wrote:
>> This change makes atomic stores strictly stronger than table A.6 of the
>> ISA manual. This mapping makes the overall patchset compatible with
>> table A.7 as well.
>>
>> 2023-04-27 Patrick O'Neill <patrick@rivosinc.com>
>>
>> 	PR 89835
> Should be "PR target/89835"
>
>>
>> gcc/ChangeLog:
>>
>> 	* config/riscv/sync.md:
> Needs some text here :-)
>
>
> I'm not objecting to this patch, but I think we've got an option
> question about whether or not this approach is too expensive for
> existing or soon arriving implementations.
>
> If the decision on that topic is to just pay the cost, then this patch
> is fine.  If we decide to make compatibility optional to avoid the
> additional cost, then this will need suitable adjustments.

IMO the only hardware that's going to be here by gcc-14 and to have 
enough concurrency for these to matter is the Ventana stuff.  I think 
you're the only one who can figure out if these are slow, at least until 
that stuff is availiable outside the lab.

So are they too slow for you?

>
> Jeff

Re: [PATCH v5 00/11] RISC-V: Implement ISA Manual Table A.6 Mappings

[Patrick O'Neill]

On 4/28/23 09:29, Palmer Dabbelt wrote:
> On Fri, 28 Apr 2023 09:14:00 PDT (-0700), jeffreyalaw@gmail.com wrote:
>>
>>
>> On 4/27/23 10:22, Patrick O'Neill wrote:
>>> This patchset aims to make the RISCV atomics implementation stronger
>>> than the recommended mapping present in table A.6 of the ISA manual.
>>> https://github.com/riscv/riscv-isa-manual/blob/c7cf84547b3aefacab5463add1734c1602b67a49/src/memory.tex#L1083-L1157 
>>>
>>>
>>> Context
>>> ---------
>>> GCC defined RISC-V mappings [1] before the Memory Model task group
>>> finalized their work and provided the ISA Manual Table A.6/A.7 
>>> mappings[2].
>>>
>>> For at least a year now, we've known that the mappings were different,
>>> but it wasn't clear if these unique mappings had correctness issues.
>>>
>>> Andrea Parri found an issue with the GCC mappings, showing that
>>> atomic_compare_exchange_weak_explicit(-,-,-,release,relaxed) 
>>> mappings do
>>> not enforce release ordering guarantees. (Meaning the GCC mappings have
>>> a correctness issue).
>>> https://inbox.sourceware.org/gcc-patches/Y1GbJuhcBFpPGJQ0@andrea/
>> Right.  I recall this discussion, but thanks for the back reference.
>
> Yep, and it's an important one: that's why we're calling the change a 
> bug fix and dropping the current GCC mappings.  If we didn't have the 
> bug we'd be talking about an ABI break, and since the GCC mappings 
> predate the ISA mappings we'd likely need an additional compatibility 
> mode.
>
> So I guess we're lucky that we have a concurrency bug.  I think it's 
> the first time I've said that ;)
>
>>> Why not A.6?
>>> ---------
>>> We can update our mappings now, so the obvious choice would be to
>>> implement Table A.6 (what LLVM implements/ISA manual recommends).
>>>
>>> The reason why that isn't the best path forward for GCC is due to a
>>> proposal by Hans Boehm to add L{d|w|b|h}.aq/rl and S{d|w|b|h}.aq/rl.
>>>
>>> For context, there is discussion about fast-tracking the addition of
>>> these instructions. The RISCV architectural review committee supports
>>> adopting a "new and common atomics ABI for gcc and LLVM toochains ...
>>> that assumes the addition of the preceding instructions”. That common
>>> ABI is likely to be A.7.
>>>    https://lists.riscv.org/g/tech-privileged/message/1284
>>>
>>> Transitioning from A.6 to A.7 will cause an ABI break. We can hedge
>>> against that risk by emitting a conservative fence after SEQ_CST stores
>>> to make the mapping compatible with both A.6 and A.7.
>> So I like that we can have compatible sequences across A.6 and A.7.  Of
>> course the concern is performance ;-)
>>
>>
>>>
>>> What does a mapping compatible with both A.6 & A.7 look like?
>>> ---------
>>> It is exactly the same as Table A.6, but SEQ_CST stores have a trailing
>>> fence rw,rw. It's strictly stronger than Table A.6.
>> Right.  So my worry here is silicon that is either already available or
>> coming online shortly.   Those implementations simply aren't going to be
>> able to use the A.7 mapping, so they pay a penalty.  Does it make sense
>> to have the compatibility fences conditional?
>
> IIRC this was discussed somewhere in some thread, but I think there's 
> really three ABIs that could be implemented here (ignoring the current 
> GCC mappings as they're broken):
>
> * ABI compatible with the current mappings in the ISA manual (A.6).  
>  This will presumably perform best on extant hardware, given that it's 
>  what the words in the PDF say to do.
> * ABI compatible with the proposed mappings for the ISA manual (A.7).  
>  This may perform better on new hardware.
> * ABI compatible with both A.6 and A.7.  This is likely slow on both 
> new  and old hardware, but allows cross-linking.  If there's no 
> performance  issues this would be the only mode we need, but that 
> seems unlikely.
>
> IMO those should be encoded somewhere in the ELF.  I'd just do it as 
> two bits in the header, but last time I proposed header bits the psABI 
> folks wanted to do something different.  I don't think where we encode 
> this matters all that much, but if we're doing to treat these as real 
> long-term ABIs we should have some way to encode that.
>
> There's also the orthogonal axis of whether we use the new 
> instructions.  Those aren't in specs yet so I think we can hold off on 
> them for a bit, but they're the whole point of doing the ABI break so 
> we should at least think them over.  I think we're OK because we've 
> just split out the ABI from the ISA here, but I'm not sure if I'm 
> missing something.
>
> Now that I wrote that, though, I remember talking to Patrick about it 
> and we drew a bunch of stuff on the whiteboard and then got confused.  
> So sorry if I'm just out of the loop here...
This looks up-to-date with how I understand it.
>>
>>>
>>> Benchmark Interpretation
>>> --------
>>> As expected, out of order machines are significantly faster with the
>>> REL_STORE mappings. Unexpectedly, the in-order machines are
>>> significantly slower with REL_STORE rather than REL_STORE_FENCE.
>> Yea, that's a bit of a surprise.
>>
>>>
>>> Most machines in the wild are expected to use Table A.7 once the
>>> instructions are introduced.
>>> Incurring this added cost now will make it easier for compiled RISC-V
>>> binaries to transition to the A.7 memory model mapping.
>>>
>>> The performance benefits of moving to A.7 can be more clearly seen 
>>> using
>>> an almost-all-load microbenchmark (included on page 3 of Hans’
>>> proposal). The code for that microbenchmark is attached below [5].
>>> https://lists.riscv.org/g/tech-unprivileged/attachment/382/0/load-acquire110422.pdf
>>>    https://lists.riscv.org/g/tech-unprivileged/topic/92916241
>> Yea.  I'm not questioning the value of the new instructions that are on
>> the horizon, just the value of trying to make everything A.7 compatible.
>>
>>
>>>
>>> Conformance test cases notes
>>> --------
>>> The conformance tests in this patch are a good sanity check but do not
>>> guarantee exactly following Table A.6. It checks that the right
>>> instructions are emitted (ex. fence rw,r) but not the order of those
>>> instructions.
>> Note there is a way to check ordering as well.  You might look at the
>> check-function-bodies approach.  I think there are some recent examples
>> in the gcc risc-v specific tests.
I'll check that out - thank you!
>>>
>>> LLVM mapping notes
>>> --------
>>> LLVM emits corresponding fences for atomic_signal_fence instructions.
>>> This seems to be an oversight since AFAIK atomic_signal_fence acts as a
>>> compiler directive. GCC does not emit any fences for 
>>> atomic_signal_fence
>>> instructions.
>> This starts to touch on a larger concern.  Specifically I'd really like
>> the two compilers to be compatible in terms of the code they generate
>> for the various atomics.
>>
>> What I worry about is code being written (by design or accident) that is
>> dependent on the particular behavior of one compiler and then if that
>> code gets built with the other compiler, and we end up different
>> behavior.  Worse yet, if/when this happens, it's likely to be tough to
>> expose, reproduce & debug.
Agreed.

I'll open an issue with LLVM and see what they have to say about this
particular behavior. Ideally we'd have perfectly compatible compilers
(for atomic ops) by the end of this :)

AFAICT GCC hasn't ever been emitting fences for these instructions.
(& This behavior isn't touched by the patchset).
>>
>> Do you have any sense of where Clang/LLVM is going to go WRT providing
>> an A.6 mapping that is compatible with A.7 by using the additional 
>> fences?
I don't - Based on how LLVM initially waited for A.6, I'd speculate that
they would wait for an official compatibility mapping to be added the
PSABI doc or ISA manual before implementing it.

I imagine there will be demand for an ABI-compatible mapping so the ABI
break has a transition plan, but the timeline for LLVM isn't clear to me.

Patrick
>>
>> Jeff

Re: [PATCH v5 08/11] RISC-V: Weaken LR/SC pairs

[Jeff Law]

On 4/27/23 10:22, Patrick O'Neill wrote:
> Introduce the %I and %J flags for setting the .aqrl bits on LR/SC pairs
> as needed.
> 
> Atomic compare and exchange ops provide success and failure memory
> models. C++17 and later place no restrictions on the relative strength
> of each model, so ensure we cover both by using a model that enforces
> the ordering of both given models.
> 
> This change brings LR/SC ops in line with table A.6 of the ISA manual.
> 
> 2023-04-27 Patrick O'Neill <patrick@rivosinc.com>
> 
> gcc/ChangeLog:
> 
> 	* config/riscv/riscv-protos.h (riscv_union_memmodels): Expose
> 	riscv_union_memmodels function to sync.md.
> 	* config/riscv/riscv.cc (riscv_union_memmodels): Add function to
> 	get the union of two memmodels in sync.md.
> 	(riscv_print_operand): Add %I and %J flags that output the
> 	optimal LR/SC flag bits for a given memory model.
> 	* config/riscv/sync.md: Remove static .aqrl bits on LR op/.rl
> 	bits on SC op and replace with optimized %I, %J flags.
OK.

Note for the future.  Operands don't have to appear in-order in a 
define_insn.  So the kind of reordering you did here may not have been 
strictly necessary.   As you found out, when you renumber the operands, 
you have to adjust the assembly template, which can be error prone. 
Knowing that I checked them pretty closely and they look right to me.



Jeff


> 
> Signed-off-by: Patrick O'Neill <patrick@rivosinc.com>
> ---
> v3 Changelog:
> * Consolidate tests in [PATCH v3 10/10]
> ---
> v5 Changelog:
> * Also optimize subword LR/SC ops based on given memory model.
> ---
>   gcc/config/riscv/riscv-protos.h |   3 +
>   gcc/config/riscv/riscv.cc       |  44 ++++++++++++
>   gcc/config/riscv/sync.md        | 114 +++++++++++++++++++-------------
>   3 files changed, 114 insertions(+), 47 deletions(-)
> 
> diff --git a/gcc/config/riscv/riscv-protos.h b/gcc/config/riscv/riscv-protos.h
> index f87661bde2c..5fa9e1122ab 100644
> --- a/gcc/config/riscv/riscv-protos.h
> +++ b/gcc/config/riscv/riscv-protos.h
> @@ -22,6 +22,8 @@ along with GCC; see the file COPYING3.  If not see
>   #ifndef GCC_RISCV_PROTOS_H
>   #define GCC_RISCV_PROTOS_H
>   
> +#include "memmodel.h"
> +
>   /* Symbol types we understand.  The order of this list must match that of
>      the unspec enum in riscv.md, subsequent to UNSPEC_ADDRESS_FIRST.  */
>   enum riscv_symbol_type {
> @@ -81,6 +83,7 @@ extern bool riscv_v_ext_vector_mode_p (machine_mode);
>   extern bool riscv_shamt_matches_mask_p (int, HOST_WIDE_INT);
>   extern void riscv_subword_address (rtx, rtx *, rtx *, rtx *, rtx *);
>   extern void riscv_lshift_subword (machine_mode, rtx, rtx, rtx *);
> +extern enum memmodel riscv_union_memmodels (enum memmodel, enum memmodel);
>   
>   /* Routines implemented in riscv-c.cc.  */
>   void riscv_cpu_cpp_builtins (cpp_reader *);
> diff --git a/gcc/config/riscv/riscv.cc b/gcc/config/riscv/riscv.cc
> index 9eba03ac189..69e9b2aa548 100644
> --- a/gcc/config/riscv/riscv.cc
> +++ b/gcc/config/riscv/riscv.cc
> @@ -4289,6 +4289,36 @@ riscv_print_operand_reloc (FILE *file, rtx op, bool hi_reloc)
>     fputc (')', file);
>   }
>   
> +/* Return the memory model that encapuslates both given models.  */
> +
> +enum memmodel
> +riscv_union_memmodels (enum memmodel model1, enum memmodel model2)
> +{
> +  model1 = memmodel_base (model1);
> +  model2 = memmodel_base (model2);
> +
> +  enum memmodel weaker = model1 <= model2 ? model1: model2;
> +  enum memmodel stronger = model1 > model2 ? model1: model2;
> +
> +  switch (stronger)
> +    {
> +      case MEMMODEL_SEQ_CST:
> +      case MEMMODEL_ACQ_REL:
> +	return stronger;
> +      case MEMMODEL_RELEASE:
> +	if (weaker == MEMMODEL_ACQUIRE || weaker == MEMMODEL_CONSUME)
> +	  return MEMMODEL_ACQ_REL;
> +	else
> +	  return stronger;
> +      case MEMMODEL_ACQUIRE:
> +      case MEMMODEL_CONSUME:
> +      case MEMMODEL_RELAXED:
> +	return stronger;
> +      default:
> +	gcc_unreachable ();
> +    }
> +}
> +
>   /* Return true if the .AQ suffix should be added to an AMO to implement the
>      acquire portion of memory model MODEL.  */
>   
> @@ -4342,6 +4372,8 @@ riscv_memmodel_needs_amo_release (enum memmodel model)
>      'R'	Print the low-part relocation associated with OP.
>      'C'	Print the integer branch condition for comparison OP.
>      'A'	Print the atomic operation suffix for memory model OP.
> +   'I'	Print the LR suffix for memory model OP.
> +   'J'	Print the SC suffix for memory model OP.
>      'z'	Print x0 if OP is zero, otherwise print OP normally.
>      'i'	Print i if the operand is not a register.
>      'S'	Print shift-index of single-bit mask OP.
> @@ -4511,6 +4543,18 @@ riscv_print_operand (FILE *file, rtx op, int letter)
>   	fputs (".rl", file);
>         break;
>   
> +    case 'I':
> +      if (model == MEMMODEL_SEQ_CST)
> +	fputs (".aqrl", file);
> +      else if (riscv_memmodel_needs_amo_acquire (model))
> +	fputs (".aq", file);
> +      break;
> +
> +    case 'J':
> +      if (riscv_memmodel_needs_amo_release (model))
> +	fputs (".rl", file);
> +      break;
> +
>       case 'i':
>         if (code != REG)
>           fputs ("i", file);
> diff --git a/gcc/config/riscv/sync.md b/gcc/config/riscv/sync.md
> index 9a3b57bd09f..3e6345e83a3 100644
> --- a/gcc/config/riscv/sync.md
> +++ b/gcc/config/riscv/sync.md
> @@ -116,21 +116,22 @@
>   	(unspec_volatile:SI
>   	  [(any_atomic:SI (match_dup 1)
>   		     (match_operand:SI 2 "register_operand" "rI")) ;; value for op
> -	   (match_operand:SI 3 "register_operand" "rI")]	   ;; mask
> +	   (match_operand:SI 3 "const_int_operand")]		   ;; model
>   	 UNSPEC_SYNC_OLD_OP_SUBWORD))
> -    (match_operand:SI 4 "register_operand" "rI")		   ;; not_mask
> -    (clobber (match_scratch:SI 5 "=&r"))			   ;; tmp_1
> -    (clobber (match_scratch:SI 6 "=&r"))]			   ;; tmp_2
> +    (match_operand:SI 4 "register_operand" "rI")		   ;; mask
> +    (match_operand:SI 5 "register_operand" "rI")		   ;; not_mask
> +    (clobber (match_scratch:SI 6 "=&r"))			   ;; tmp_1
> +    (clobber (match_scratch:SI 7 "=&r"))]			   ;; tmp_2
>     "TARGET_ATOMIC && TARGET_INLINE_SUBWORD_ATOMIC"
>     {
>       return "1:\;"
> -	   "lr.w.aqrl\t%0, %1\;"
> -	   "<insn>\t%5, %0, %2\;"
> -	   "and\t%5, %5, %3\;"
> -	   "and\t%6, %0, %4\;"
> -	   "or\t%6, %6, %5\;"
> -	   "sc.w.rl\t%5, %6, %1\;"
> -	   "bnez\t%5, 1b";
> +	   "lr.w%I3\t%0, %1\;"
> +	   "<insn>\t%6, %0, %2\;"
> +	   "and\t%6, %6, %4\;"
> +	   "and\t%7, %0, %5\;"
> +	   "or\t%7, %7, %6\;"
> +	   "sc.w%J3\t%6, %7, %1\;"
> +	   "bnez\t%6, 1b";
>     }
>     [(set (attr "length") (const_int 28))])
>   
> @@ -151,6 +152,7 @@
>     rtx old = gen_reg_rtx (SImode);
>     rtx mem = operands[1];
>     rtx value = operands[2];
> +  rtx model = operands[3];
>     rtx aligned_mem = gen_reg_rtx (SImode);
>     rtx shift = gen_reg_rtx (SImode);
>     rtx mask = gen_reg_rtx (SImode);
> @@ -162,7 +164,7 @@
>     riscv_lshift_subword (<MODE>mode, value, shift, &shifted_value);
>   
>     emit_insn (gen_subword_atomic_fetch_strong_nand (old, aligned_mem,
> -						   shifted_value,
> +						   shifted_value, model,
>   						   mask, not_mask));
>   
>     emit_move_insn (old, gen_rtx_ASHIFTRT (SImode, old,
> @@ -180,22 +182,23 @@
>   	(unspec_volatile:SI
>   	  [(not:SI (and:SI (match_dup 1)
>   			   (match_operand:SI 2 "register_operand" "rI"))) ;; value for op
> -	   (match_operand:SI 3 "register_operand" "rI")]		  ;; mask
> +	   (match_operand:SI 3 "const_int_operand")]			  ;; mask
>   	 UNSPEC_SYNC_OLD_OP_SUBWORD))
> -    (match_operand:SI 4 "register_operand" "rI")			  ;; not_mask
> -    (clobber (match_scratch:SI 5 "=&r"))				  ;; tmp_1
> -    (clobber (match_scratch:SI 6 "=&r"))]				  ;; tmp_2
> +    (match_operand:SI 4 "register_operand" "rI")			  ;; mask
> +    (match_operand:SI 5 "register_operand" "rI")			  ;; not_mask
> +    (clobber (match_scratch:SI 6 "=&r"))				  ;; tmp_1
> +    (clobber (match_scratch:SI 7 "=&r"))]				  ;; tmp_2
>     "TARGET_ATOMIC && TARGET_INLINE_SUBWORD_ATOMIC"
>     {
>       return "1:\;"
> -	   "lr.w.aqrl\t%0, %1\;"
> -	   "and\t%5, %0, %2\;"
> -	   "not\t%5, %5\;"
> -	   "and\t%5, %5, %3\;"
> -	   "and\t%6, %0, %4\;"
> -	   "or\t%6, %6, %5\;"
> -	   "sc.w.rl\t%5, %6, %1\;"
> -	   "bnez\t%5, 1b";
> +	   "lr.w%I3\t%0, %1\;"
> +	   "and\t%6, %0, %2\;"
> +	   "not\t%6, %6\;"
> +	   "and\t%6, %6, %4\;"
> +	   "and\t%7, %0, %5\;"
> +	   "or\t%7, %7, %6\;"
> +	   "sc.w%J3\t%6, %7, %1\;"
> +	   "bnez\t%6, 1b";
>     }
>     [(set (attr "length") (const_int 32))])
>   
> @@ -216,6 +219,7 @@
>     rtx old = gen_reg_rtx (SImode);
>     rtx mem = operands[1];
>     rtx value = operands[2];
> +  rtx model = operands[3];
>     rtx aligned_mem = gen_reg_rtx (SImode);
>     rtx shift = gen_reg_rtx (SImode);
>     rtx mask = gen_reg_rtx (SImode);
> @@ -228,7 +232,8 @@
>   
>     emit_insn (gen_subword_atomic_fetch_strong_<atomic_optab> (old, aligned_mem,
>   							     shifted_value,
> -							     mask, not_mask));
> +							     model, mask,
> +							     not_mask));
>   
>     emit_move_insn (old, gen_rtx_ASHIFTRT (SImode, old,
>   					 gen_lowpart (QImode, shift)));
> @@ -261,6 +266,7 @@
>     rtx old = gen_reg_rtx (SImode);
>     rtx mem = operands[1];
>     rtx value = operands[2];
> +  rtx model = operands[3];
>     rtx aligned_mem = gen_reg_rtx (SImode);
>     rtx shift = gen_reg_rtx (SImode);
>     rtx mask = gen_reg_rtx (SImode);
> @@ -272,7 +278,8 @@
>     riscv_lshift_subword (<MODE>mode, value, shift, &shifted_value);
>   
>     emit_insn (gen_subword_atomic_exchange_strong (old, aligned_mem,
> -						 shifted_value, not_mask));
> +						 shifted_value, model,
> +						 not_mask));
>   
>     emit_move_insn (old, gen_rtx_ASHIFTRT (SImode, old,
>   					 gen_lowpart (QImode, shift)));
> @@ -286,18 +293,19 @@
>   	(match_operand:SI 1 "memory_operand" "+A"))	 ;; mem location
>      (set (match_dup 1)
>   	(unspec_volatile:SI
> -	  [(match_operand:SI 2 "reg_or_0_operand" "rI")  ;; value
> -	   (match_operand:SI 3 "reg_or_0_operand" "rI")] ;; not_mask
> +	  [(match_operand:SI 2 "reg_or_0_operand" "rI")	 ;; value
> +	   (match_operand:SI 3 "const_int_operand")]	 ;; model
>         UNSPEC_SYNC_EXCHANGE_SUBWORD))
> -    (clobber (match_scratch:SI 4 "=&r"))]		 ;; tmp_1
> +    (match_operand:SI 4 "reg_or_0_operand" "rI")	 ;; not_mask
> +    (clobber (match_scratch:SI 5 "=&r"))]		 ;; tmp_1
>     "TARGET_ATOMIC && TARGET_INLINE_SUBWORD_ATOMIC"
>     {
>       return "1:\;"
> -	   "lr.w.aqrl\t%0, %1\;"
> -	   "and\t%4, %0, %3\;"
> -	   "or\t%4, %4, %2\;"
> -	   "sc.w.rl\t%4, %4, %1\;"
> -	   "bnez\t%4, 1b";
> +	   "lr.w%I3\t%0, %1\;"
> +	   "and\t%5, %0, %4\;"
> +	   "or\t%5, %5, %2\;"
> +	   "sc.w%J3\t%5, %5, %1\;"
> +	   "bnez\t%5, 1b";
>     }
>     [(set (attr "length") (const_int 20))])
>   
> @@ -313,10 +321,15 @@
>      (clobber (match_scratch:GPR 6 "=&r"))]
>     "TARGET_ATOMIC"
>     {
> +    enum memmodel model_success = (enum memmodel) INTVAL (operands[4]);
> +    enum memmodel model_failure = (enum memmodel) INTVAL (operands[5]);
> +    /* Find the union of the two memory models so we can satisfy both success
> +       and failure memory models.  */
> +    operands[5] = GEN_INT (riscv_union_memmodels (model_success, model_failure));
>       return "1:\;"
> -	   "lr.<amo>.aqrl\t%0,%1\;"
> +	   "lr.<amo>%I5\t%0,%1\;"
>   	   "bne\t%0,%z2,1f\;"
> -	   "sc.<amo>.rl\t%6,%z3,%1\;"
> +	   "sc.<amo>%J5\t%6,%z3,%1\;"
>   	   "bnez\t%6,1b\;"
>   	   "1:";
>     }
> @@ -440,9 +453,15 @@
>     emit_move_insn (shifted_o, gen_rtx_AND (SImode, shifted_o, mask));
>     emit_move_insn (shifted_n, gen_rtx_AND (SImode, shifted_n, mask));
>   
> +  enum memmodel model_success = (enum memmodel) INTVAL (operands[4]);
> +  enum memmodel model_failure = (enum memmodel) INTVAL (operands[5]);
> +  /* Find the union of the two memory models so we can satisfy both success
> +     and failure memory models.  */
> +  rtx model = GEN_INT (riscv_union_memmodels (model_success, model_failure));
> +
>     emit_insn (gen_subword_atomic_cas_strong (old, aligned_mem,
>   					    shifted_o, shifted_n,
> -					    mask, not_mask));
> +					    model, mask, not_mask));
>   
>     emit_move_insn (old, gen_rtx_ASHIFTRT (SImode, old,
>   					 gen_lowpart (QImode, shift)));
> @@ -459,19 +478,20 @@
>   	(unspec_volatile:SI [(match_operand:SI 2 "reg_or_0_operand" "rJ")  ;; expected value
>   			     (match_operand:SI 3 "reg_or_0_operand" "rJ")] ;; desired value
>   	 UNSPEC_COMPARE_AND_SWAP_SUBWORD))
> -	(match_operand:SI 4 "register_operand" "rI")			   ;; mask
> -	(match_operand:SI 5 "register_operand" "rI")			   ;; not_mask
> -	(clobber (match_scratch:SI 6 "=&r"))]				   ;; tmp_1
> +	(match_operand:SI 4 "const_int_operand")			   ;; model
> +	(match_operand:SI 5 "register_operand" "rI")			   ;; mask
> +	(match_operand:SI 6 "register_operand" "rI")			   ;; not_mask
> +	(clobber (match_scratch:SI 7 "=&r"))]				   ;; tmp_1
>     "TARGET_ATOMIC && TARGET_INLINE_SUBWORD_ATOMIC"
>     {
>       return "1:\;"
> -	   "lr.w.aqrl\t%0, %1\;"
> -	   "and\t%6, %0, %4\;"
> -	   "bne\t%6, %z2, 1f\;"
> -	   "and\t%6, %0, %5\;"
> -	   "or\t%6, %6, %3\;"
> -	   "sc.w.rl\t%6, %6, %1\;"
> -	   "bnez\t%6, 1b\;"
> +	   "lr.w%I4\t%0, %1\;"
> +	   "and\t%7, %0, %5\;"
> +	   "bne\t%7, %z2, 1f\;"
> +	   "and\t%7, %0, %6\;"
> +	   "or\t%7, %7, %3\;"
> +	   "sc.w%J4\t%7, %7, %1\;"
> +	   "bnez\t%7, 1b\;"
>   	   "1:";
>     }
>     [(set (attr "length") (const_int 28))])

Re: [PATCH v5 09/11] RISC-V: Weaken mem_thread_fence

[Jeff Law]

On 4/27/23 10:22, Patrick O'Neill wrote:
> This change brings atomic fences in line with table A.6 of the ISA
> manual.
> 
> Relax mem_thread_fence according to the memmodel given.
> 
> 2023-04-27 Patrick O'Neill <patrick@rivosinc.com>
> 
> gcc/ChangeLog:
> 
> 	* config/riscv/sync.md (mem_thread_fence_1): Change fence
> 	depending on the given memory model.
OK
jeff

Re: [PATCH v5 10/11] RISC-V: Weaken atomic loads

[Jeff Law]

On 4/27/23 10:23, Patrick O'Neill wrote:
> This change brings atomic loads in line with table A.6 of the ISA
> manual.
> 
> 2023-04-27 Patrick O'Neill <patrick@rivosinc.com>
> 
> gcc/ChangeLog:
> 
> 	* config/riscv/sync.md (atomic_load<mode>): Implement atomic
> 	load mapping.
OK.
jeff

Re: [PATCH v5 11/11] RISC-V: Table A.6 conformance tests

[Jeff Law]

On 4/27/23 10:23, Patrick O'Neill wrote:
> These tests cover basic cases to ensure the atomic mappings follow the
> strengthened Table A.6 mappings that are compatible with Table A.7.
> 
> 2023-04-27 Patrick O'Neill <patrick@rivosinc.com>
> 
> gcc/testsuite/ChangeLog:
> 
> 	* gcc.target/riscv/amo-table-a-6-amo-add-1.c: New test.
> 	* gcc.target/riscv/amo-table-a-6-amo-add-2.c: New test.
> 	* gcc.target/riscv/amo-table-a-6-amo-add-3.c: New test.
> 	* gcc.target/riscv/amo-table-a-6-amo-add-4.c: New test.
> 	* gcc.target/riscv/amo-table-a-6-amo-add-5.c: New test.
> 	* gcc.target/riscv/amo-table-a-6-compare-exchange-1.c: New test.
> 	* gcc.target/riscv/amo-table-a-6-compare-exchange-2.c: New test.
> 	* gcc.target/riscv/amo-table-a-6-compare-exchange-3.c: New test.
> 	* gcc.target/riscv/amo-table-a-6-compare-exchange-4.c: New test.
> 	* gcc.target/riscv/amo-table-a-6-compare-exchange-5.c: New test.
> 	* gcc.target/riscv/amo-table-a-6-compare-exchange-6.c: New test.
> 	* gcc.target/riscv/amo-table-a-6-compare-exchange-7.c: New test.
> 	* gcc.target/riscv/amo-table-a-6-fence-1.c: New test.
> 	* gcc.target/riscv/amo-table-a-6-fence-2.c: New test.
> 	* gcc.target/riscv/amo-table-a-6-fence-3.c: New test.
> 	* gcc.target/riscv/amo-table-a-6-fence-4.c: New test.
> 	* gcc.target/riscv/amo-table-a-6-fence-5.c: New test.
> 	* gcc.target/riscv/amo-table-a-6-load-1.c: New test.
> 	* gcc.target/riscv/amo-table-a-6-load-2.c: New test.
> 	* gcc.target/riscv/amo-table-a-6-load-3.c: New test.
> 	* gcc.target/riscv/amo-table-a-6-store-1.c: New test.
> 	* gcc.target/riscv/amo-table-a-6-store-2.c: New test.
> 	* gcc.target/riscv/amo-table-a-6-store-compat-3.c: New test.
> 	* gcc.target/riscv/amo-table-a-6-subword-amo-add-1.c: New test.
> 	* gcc.target/riscv/amo-table-a-6-subword-amo-add-2.c: New test.
> 	* gcc.target/riscv/amo-table-a-6-subword-amo-add-3.c: New test.
> 	* gcc.target/riscv/amo-table-a-6-subword-amo-add-4.c: New test.
> 	* gcc.target/riscv/amo-table-a-6-subword-amo-add-5.c: New test.
OK.  And as mentioned earlier, there is a framework where you can verify 
ordering as well.  Your call whether or not you want to switch to that 
form.  If you do choose to checking ordering, consider that patch 
pre-approved.

Jeff

Re: [PATCH v5 00/11] RISC-V: Implement ISA Manual Table A.6 Mappings

[Patrick O'Neill]

On 4/28/23 10:44, Patrick O'Neill wrote:
> On 4/28/23 09:29, Palmer Dabbelt wrote:
>> On Fri, 28 Apr 2023 09:14:00 PDT (-0700), jeffreyalaw@gmail.com wrote:
>>> On 4/27/23 10:22, Patrick O'Neill wrote:
...
>>>> LLVM mapping notes
>>>> --------
>>>> LLVM emits corresponding fences for atomic_signal_fence instructions.
>>>> This seems to be an oversight since AFAIK atomic_signal_fence acts 
>>>> as a
>>>> compiler directive. GCC does not emit any fences for 
>>>> atomic_signal_fence
>>>> instructions.
>>> This starts to touch on a larger concern.  Specifically I'd really like
>>> the two compilers to be compatible in terms of the code they generate
>>> for the various atomics.
>>>
>>> What I worry about is code being written (by design or accident) 
>>> that is
>>> dependent on the particular behavior of one compiler and then if that
>>> code gets built with the other compiler, and we end up different
>>> behavior.  Worse yet, if/when this happens, it's likely to be tough to
>>> expose, reproduce & debug.
> Agreed.
>
> I'll open an issue with LLVM and see what they have to say about this
> particular behavior. Ideally we'd have perfectly compatible compilers
> (for atomic ops) by the end of this :)
>
> AFAICT GCC hasn't ever been emitting fences for these instructions.
> (& This behavior isn't touched by the patchset).
I re-ran the set of tests I was using and couldn't replicate LLVM's
behavior that was noted here. I think I mixed had up atomic_thread_fence
with atomic_signal_fence at some point.

That was the only difference I could find during my testing of this
patchset (other than the strengthened SEQ_CST store), so I think
GCC/LLVM atomics will be fully compatible once this patchset is applied.

Re: [PATCH v5 06/11] RISC-V: Strengthen atomic stores

[Hans Boehm]

[-- Attachment #1: Type: text/plain, Size: 1884 bytes --]

The concern with making the new behavior non-default is of course that the
generated code will eventually end up on an A.7-capable platform.

An A.6-classic option for compiling code that will never run on a newer
machine seems OK. But I'm not sure that seq_cst stores are dynamically
frequent enough in C++ code for this to be worth the trouble. Unlike loads,
they are also costly on x86, programmers may also have been somewhat
trained to avoid them where possible. (And probably where not possible, too
:-( )

Hans

On Fri, Apr 28, 2023 at 10:43 AM Palmer Dabbelt <palmer@rivosinc.com> wrote:

> On Fri, 28 Apr 2023 10:40:15 PDT (-0700), jeffreyalaw@gmail.com wrote:
> >
> >
> > On 4/27/23 10:22, Patrick O'Neill wrote:
> >> This change makes atomic stores strictly stronger than table A.6 of the
> >> ISA manual. This mapping makes the overall patchset compatible with
> >> table A.7 as well.
> >>
> >> 2023-04-27 Patrick O'Neill <patrick@rivosinc.com>
> >>
> >>      PR 89835
> > Should be "PR target/89835"
> >
> >>
> >> gcc/ChangeLog:
> >>
> >>      * config/riscv/sync.md:
> > Needs some text here :-)
> >
> >
> > I'm not objecting to this patch, but I think we've got an option
> > question about whether or not this approach is too expensive for
> > existing or soon arriving implementations.
> >
> > If the decision on that topic is to just pay the cost, then this patch
> > is fine.  If we decide to make compatibility optional to avoid the
> > additional cost, then this will need suitable adjustments.
>
> IMO the only hardware that's going to be here by gcc-14 and to have
> enough concurrency for these to matter is the Ventana stuff.  I think
> you're the only one who can figure out if these are slow, at least until
> that stuff is availiable outside the lab.
>
> So are they too slow for you?
>
> >
> > Jeff
>

Re: [PATCH v5 06/11] RISC-V: Strengthen atomic stores

[Hans Boehm]

[-- Attachment #1: Type: text/plain, Size: 2200 bytes --]

The RISC-V psABI pull request is at
https://github.com/riscv-non-isa/riscv-elf-psabi-doc/pull/378 . Pointers to
Paul Kirth's corresponding LLVM patches are also there.

On Fri, Apr 28, 2023 at 2:42 PM Hans Boehm <hboehm@google.com> wrote:

> The concern with making the new behavior non-default is of course that the
> generated code will eventually end up on an A.7-capable platform.
>
> An A.6-classic option for compiling code that will never run on a newer
> machine seems OK. But I'm not sure that seq_cst stores are dynamically
> frequent enough in C++ code for this to be worth the trouble. Unlike loads,
> they are also costly on x86, programmers may also have been somewhat
> trained to avoid them where possible. (And probably where not possible, too
> :-( )
>
> Hans
>
> On Fri, Apr 28, 2023 at 10:43 AM Palmer Dabbelt <palmer@rivosinc.com>
> wrote:
>
>> On Fri, 28 Apr 2023 10:40:15 PDT (-0700), jeffreyalaw@gmail.com wrote:
>> >
>> >
>> > On 4/27/23 10:22, Patrick O'Neill wrote:
>> >> This change makes atomic stores strictly stronger than table A.6 of the
>> >> ISA manual. This mapping makes the overall patchset compatible with
>> >> table A.7 as well.
>> >>
>> >> 2023-04-27 Patrick O'Neill <patrick@rivosinc.com>
>> >>
>> >>      PR 89835
>> > Should be "PR target/89835"
>> >
>> >>
>> >> gcc/ChangeLog:
>> >>
>> >>      * config/riscv/sync.md:
>> > Needs some text here :-)
>> >
>> >
>> > I'm not objecting to this patch, but I think we've got an option
>> > question about whether or not this approach is too expensive for
>> > existing or soon arriving implementations.
>> >
>> > If the decision on that topic is to just pay the cost, then this patch
>> > is fine.  If we decide to make compatibility optional to avoid the
>> > additional cost, then this will need suitable adjustments.
>>
>> IMO the only hardware that's going to be here by gcc-14 and to have
>> enough concurrency for these to matter is the Ventana stuff.  I think
>> you're the only one who can figure out if these are slow, at least until
>> that stuff is availiable outside the lab.
>>
>> So are they too slow for you?
>>
>> >
>> > Jeff
>>
>

Re: [PATCH v5 00/11] RISC-V: Implement ISA Manual Table A.6 Mappings

[Jeff Law]

On 4/28/23 12:45, Hans Boehm wrote:
> We're certainly pushing for the same ABI (A.6 + trailing fence on store) 
> in LLVM as well. I'm about to upload a pull request for the psABI 
> document that describes this version of the ABI, and a bit of the 
> rationale for it. I'll attach the current draft here.
> 
> I agree that compatibility is critical here, not just across llvm and 
> gcc, but also with other language implementations. That's part of the 
> reason to get this correct asap.
> 
> I believe that standardizing on A.6 + trailing fence on store, though 
> initially suboptimal, is by far the best bet to get us to an efficient 
> ABI in the long term. I expect the A.7 ABI to perform well. A.6, even 
> without the trailing store fence, has annoyingly expensive seq_cst 
> loads, which I would really like to get away from.
Thanks for the additional info.  This stuff is well outside my area of 
expertise, so having someone with your background to give key insights 
is definitely appreciated.

jeff

Re: [PATCH v5 06/11] RISC-V: Strengthen atomic stores

[Jeff Law]

On 4/28/23 15:42, Hans Boehm wrote:
> The concern with making the new behavior non-default is of course that 
> the generated code will eventually end up on an A.7-capable platform.
Yea, certainly a significant concern in general, though probably not for 
Ventana.  I expect we'll have folks rebuilding as they go from V1 to V2 
due to the ventanacondops -> condops change.


> 
> An A.6-classic option for compiling code that will never run on a newer 
> machine seems OK. But I'm not sure that seq_cst stores are dynamically 
> frequent enough in C++ code for this to be worth the trouble. 
> Unlike loads, they are also costly on x86, programmers may also have 
> been somewhat trained to avoid them where possible. (And probably where 
> not possible, too :-( )
I was more worried about the kernel, perhaps because I'm less familiar 
with it these days than ever.  WRT C++, I can ping Jon@RedHat, he's got 
a great sense what's in the standard library obviously, but also a 
reasonable sense of what folks are doing in the wild.

Jeff

Re: [PATCH v5 06/11] RISC-V: Strengthen atomic stores

[Patrick O'Neill]

Discussed in the patchworks meeting with Jeff Law and decided to move
forward with the trailing fence compatibility approach. If the trailing
fence becomes a performance issue and people want to generate A.6 code,
we'll need a PSABI change to identify which mapping a binary uses. We'll
cross that bridge when/if we get to it.

Patrick

On 4/27/23 09:22, Patrick O'Neill wrote:
> This change makes atomic stores strictly stronger than table A.6 of the
> ISA manual. This mapping makes the overall patchset compatible with
> table A.7 as well.
>
> 2023-04-27 Patrick O'Neill <patrick@rivosinc.com>
>
> 	PR 89835
>
> gcc/ChangeLog:
>
> 	* config/riscv/sync.md:
>
> gcc/testsuite/ChangeLog:
>
> 	* gcc.target/riscv/pr89835.c: New test.
>
> Signed-off-by: Patrick O'Neill <patrick@rivosinc.com>

[Committed 01/11] RISC-V: Eliminate SYNC memory models

[Patrick O'Neill]

On 4/28/23 09:23, Jeff Law wrote:
> On 4/27/23 10:22, Patrick O'Neill wrote:
>> Remove references to MEMMODEL_SYNC_* models by converting via
>> memmodel_base().
>>
>> 2023-04-27 Patrick O'Neill <patrick@rivosinc.com>
>>
>> gcc/ChangeLog:
>>
>>     * config/riscv/riscv.cc: Remove MEMMODEL_SYNC_* cases and
>>     sanitize memmodel input with memmodel_base.
> OK.  Not sure if you want to commit it now or wait for the full set to 
> get ACK'd (since there are some questions on the trailing sync approach).
>
> Jeff

Committed.

Patrick

[Committed 02/11] RISC-V: Enforce Libatomic LR/SC SEQ_CST

[Patrick O'Neill]

On 4/28/23 09:50, Jeff Law wrote:
>
>
> On 4/27/23 10:22, Patrick O'Neill wrote:
>> Replace LR.aq/SC.rl pairs with the SEQ_CST LR.aqrl/SC.rl pairs
>> recommended by table A.6 of the ISA manual.
>>
>> 2023-04-27 Patrick O'Neill <patrick@rivosinc.com>
>>
>> libgcc/ChangeLog:
>>
>>     * config/riscv/atomic.c: Change LR.aq/SC.rl pairs into
>>     sequentially consistent LR.aqrl/SC.rl pairs.
> OK.  When you install this, make sure you also install #3 of the kit 
> which mirrors these changes for the inline subword atomics.
>
> jeff

Committed.


Patrick

[Committed 03/11] RISC-V: Enforce subword atomic LR/SC SEQ_CST

[Patrick O'Neill]

On 4/27/23 09:22, Patrick O'Neill wrote:
> Replace LR.aq/SC.rl pairs with the SEQ_CST LR.aqrl/SC.rl pairs
> recommended by table A.6 of the ISA manual.
>
> 2023-04-27 Patrick O'Neill <patrick@rivosinc.com>
>
> gcc/ChangeLog:
>
> 	* config/riscv/sync.md: Change LR.aq/SC.rl pairs into
> 	sequentially consistent LR.aqrl/SC.rl pairs.
>
> Signed-off-by: Patrick O'Neill <patrick@rivosinc.com>
Acked by Jeff Law in reply to [PATCH v5 02/11]
https://inbox.sourceware.org/gcc-patches/ce2e7d52-046d-90d7-baa2-11b299d5034d@gmail.com/

Committed.

Patrick

[Committed 04/11] RISC-V: Enforce atomic compare_exchange SEQ_CST

[Patrick O'Neill]

On 4/28/23 10:23, Jeff Law wrote:
>
>
> On 4/27/23 10:22, Patrick O'Neill wrote:
>> This patch enforces SEQ_CST for atomic compare_exchange ops.
>>
>> Replace Fence/LR.aq/SC.aq pairs with SEQ_CST LR.aqrl/SC.rl pairs
>> recommended by table A.6 of the ISA manual.
>>
>> 2023-04-27 Patrick O'Neill <patrick@rivosinc.com>
>>
>> gcc/ChangeLog:
>>
>>     * config/riscv/sync.md: Change FENCE/LR.aq/SC.aq into
>>     sequentially consistent LR.aqrl/SC.rl pair.
> OK.  Note that generally you should note which pattern you're changing 
> in a ChangeLog entry, similar to how we note the function being 
> changed.  So something like this might be better:
>
>     * config/riscv/sync.md (atomic_cas_value_strong<mode>): ...
>
> Jeff

Edited ChangeLog and committed:

gcc/ChangeLog:

     * config/riscv/sync.md (atomic_cas_value_strong<mode>): Change
     FENCE/LR.aq/SC.aq into sequentially consistent LR.aqrl/SC.rl
     pair.

Patrick

Re: [PATCH v5 05/11] RISC-V: Add AMO release bits

[Patrick O'Neill]

On 4/28/23 10:34, Jeff Law wrote:
>
>
> On 4/27/23 10:22, Patrick O'Neill wrote:
>> This patch sets the relevant .rl bits on amo operations.
>>
>> 2023-04-27 Patrick O'Neill <patrick@rivosinc.com>
>>
>> gcc/ChangeLog:
>>
>>     * config/riscv/riscv.cc (riscv_print_operand): change behavior
>>     of %A to include release bits.
> Capitalize "change" in the ChangeLog entry.  OK with that nit fixed.
>
> jeff

Capitalized "change" and committed.

gcc/ChangeLog:

     * config/riscv/riscv.cc (riscv_print_operand): Change behavior
     of %A to include release bits.

Patrick

[Committed 06/11] RISC-V: Strengthen atomic stores

[Patrick O'Neill]

On 4/28/23 10:40, Jeff Law wrote:
>
>
> On 4/27/23 10:22, Patrick O'Neill wrote:
>> This change makes atomic stores strictly stronger than table A.6 of the
>> ISA manual. This mapping makes the overall patchset compatible with
>> table A.7 as well.
>>
>> 2023-04-27 Patrick O'Neill <patrick@rivosinc.com>
>>
>>     PR 89835
> Should be "PR target/89835"
>
>>
>> gcc/ChangeLog:
>>
>>     * config/riscv/sync.md:
> Needs some text here :-)
>
>
> I'm not objecting to this patch, but I think we've got an option 
> question about whether or not this approach is too expensive for 
> existing or soon arriving implementations.
>
> If the decision on that topic is to just pay the cost, then this patch 
> is fine.  If we decide to make compatibility optional to avoid the 
> additional cost, then this will need suitable adjustments.
>
> Jeff

Acked in Patchworks meeting:
https://inbox.sourceware.org/gcc-patches/c53ac2b2-4edf-34c6-a935-3b31644c9864@rivosinc.com/

Updated changelog and committed:

     PR target/89835

gcc/ChangeLog:

     * config/riscv/sync.md (atomic_store<mode>): Use simple store
     instruction in combination with fence(s).

gcc/testsuite/ChangeLog:

     * gcc.target/riscv/pr89835.c: New test.

Patrick

[Committed 07/11] RISC-V: Eliminate AMO op fences

[Patrick O'Neill]

On 4/28/23 10:43, Jeff Law wrote:
>
>
> On 4/27/23 10:22, Patrick O'Neill wrote:
>> Atomic operations with the appropriate bits set already enfore release
>> semantics. Remove unnecessary release fences from atomic ops.
>>
>> This change brings AMO ops in line with table A.6 of the ISA manual.
>>
>> 2023-04-27 Patrick O'Neill <patrick@rivosinc.com>
>>
>> gcc/ChangeLog:
>>
>>     * config/riscv/riscv.cc
>>     (riscv_memmodel_needs_amo_release): Change function name.
>>     (riscv_print_operand): Remove unneeded %F case.
>>     * config/riscv/sync.md: Remove unneeded fences.
> OK.  Though note this depends on a resolution of patch #6.  You could 
> potentially leave the %F support in riscv_print_operand and install 
> the rest of this patch while we settle the question around #6.
>
> Jeff

Committed.

Patrick

[Committed 08/11] RISC-V: Weaken LR/SC pairs

[Patrick O'Neill]

On 4/28/23 10:56, Jeff Law wrote:
>
>
> On 4/27/23 10:22, Patrick O'Neill wrote:
>> Introduce the %I and %J flags for setting the .aqrl bits on LR/SC pairs
>> as needed.
>>
>> Atomic compare and exchange ops provide success and failure memory
>> models. C++17 and later place no restrictions on the relative strength
>> of each model, so ensure we cover both by using a model that enforces
>> the ordering of both given models.
>>
>> This change brings LR/SC ops in line with table A.6 of the ISA manual.
>>
>> 2023-04-27 Patrick O'Neill <patrick@rivosinc.com>
>>
>> gcc/ChangeLog:
>>
>>     * config/riscv/riscv-protos.h (riscv_union_memmodels): Expose
>>     riscv_union_memmodels function to sync.md.
>>     * config/riscv/riscv.cc (riscv_union_memmodels): Add function to
>>     get the union of two memmodels in sync.md.
>>     (riscv_print_operand): Add %I and %J flags that output the
>>     optimal LR/SC flag bits for a given memory model.
>>     * config/riscv/sync.md: Remove static .aqrl bits on LR op/.rl
>>     bits on SC op and replace with optimized %I, %J flags.
> OK.
>
> Note for the future.  Operands don't have to appear in-order in a 
> define_insn.  So the kind of reordering you did here may not have been 
> strictly necessary.   As you found out, when you renumber the 
> operands, you have to adjust the assembly template, which can be error 
> prone. Knowing that I checked them pretty closely and they look right 
> to me.
>
>
>
> Jeff

Committed.

Patrick

[Committed 09/11] RISC-V: Weaken mem_thread_fence

[Patrick O'Neill]

On 4/28/23 11:00, Jeff Law wrote:
>
>
> On 4/27/23 10:22, Patrick O'Neill wrote:
>> This change brings atomic fences in line with table A.6 of the ISA
>> manual.
>>
>> Relax mem_thread_fence according to the memmodel given.
>>
>> 2023-04-27 Patrick O'Neill <patrick@rivosinc.com>
>>
>> gcc/ChangeLog:
>>
>>     * config/riscv/sync.md (mem_thread_fence_1): Change fence
>>     depending on the given memory model.
> OK
> jeff

Committed.

Patrick

[Committed 10/11] RISC-V: Weaken atomic loads

[Patrick O'Neill]

On 4/28/23 11:04, Jeff Law wrote:
>
>
> On 4/27/23 10:23, Patrick O'Neill wrote:
>> This change brings atomic loads in line with table A.6 of the ISA
>> manual.
>>
>> 2023-04-27 Patrick O'Neill <patrick@rivosinc.com>
>>
>> gcc/ChangeLog:
>>
>>     * config/riscv/sync.md (atomic_load<mode>): Implement atomic
>>     load mapping.
> OK.
> jeff

Committed.

Patrick

[Committed 11/11] RISC-V: Table A.6 conformance tests

[Patrick O'Neill]

Updated the amo/load/store/fence tests to use check-function-bodies to
ensure ordering. This is especially important for Load/Store where
we want to ensure the correct fence is emitted in the correct spot.

Compare exchange & subword amo ops still use scan-assembler-times.

The change to check-function-bodies was pre-approved by Jeff Law.

Committed.

Patrick

---

These tests cover basic cases to ensure the atomic mappings follow the
strengthened Table A.6 mappings that are compatible with Table A.7.

2023-04-27 Patrick O'Neill <patrick@rivosinc.com>

gcc/testsuite/ChangeLog:

	* gcc.target/riscv/amo-table-a-6-amo-add-1.c: New test.
	* gcc.target/riscv/amo-table-a-6-amo-add-2.c: New test.
	* gcc.target/riscv/amo-table-a-6-amo-add-3.c: New test.
	* gcc.target/riscv/amo-table-a-6-amo-add-4.c: New test.
	* gcc.target/riscv/amo-table-a-6-amo-add-5.c: New test.
	* gcc.target/riscv/amo-table-a-6-compare-exchange-1.c: New test.
	* gcc.target/riscv/amo-table-a-6-compare-exchange-2.c: New test.
	* gcc.target/riscv/amo-table-a-6-compare-exchange-3.c: New test.
	* gcc.target/riscv/amo-table-a-6-compare-exchange-4.c: New test.
	* gcc.target/riscv/amo-table-a-6-compare-exchange-5.c: New test.
	* gcc.target/riscv/amo-table-a-6-compare-exchange-6.c: New test.
	* gcc.target/riscv/amo-table-a-6-compare-exchange-7.c: New test.
	* gcc.target/riscv/amo-table-a-6-fence-1.c: New test.
	* gcc.target/riscv/amo-table-a-6-fence-2.c: New test.
	* gcc.target/riscv/amo-table-a-6-fence-3.c: New test.
	* gcc.target/riscv/amo-table-a-6-fence-4.c: New test.
	* gcc.target/riscv/amo-table-a-6-fence-5.c: New test.
	* gcc.target/riscv/amo-table-a-6-load-1.c: New test.
	* gcc.target/riscv/amo-table-a-6-load-2.c: New test.
	* gcc.target/riscv/amo-table-a-6-load-3.c: New test.
	* gcc.target/riscv/amo-table-a-6-store-1.c: New test.
	* gcc.target/riscv/amo-table-a-6-store-2.c: New test.
	* gcc.target/riscv/amo-table-a-6-store-compat-3.c: New test.
	* gcc.target/riscv/amo-table-a-6-subword-amo-add-1.c: New test.
	* gcc.target/riscv/amo-table-a-6-subword-amo-add-2.c: New test.
	* gcc.target/riscv/amo-table-a-6-subword-amo-add-3.c: New test.
	* gcc.target/riscv/amo-table-a-6-subword-amo-add-4.c: New test.
	* gcc.target/riscv/amo-table-a-6-subword-amo-add-5.c: New test.

Signed-off-by: Patrick O'Neill <patrick@rivosinc.com>
---
 .../gcc.target/riscv/amo-table-a-6-amo-add-1.c | 15 +++++++++++++++
 .../gcc.target/riscv/amo-table-a-6-amo-add-2.c | 15 +++++++++++++++
 .../gcc.target/riscv/amo-table-a-6-amo-add-3.c | 15 +++++++++++++++
 .../gcc.target/riscv/amo-table-a-6-amo-add-4.c | 15 +++++++++++++++
 .../gcc.target/riscv/amo-table-a-6-amo-add-5.c | 15 +++++++++++++++
 .../riscv/amo-table-a-6-compare-exchange-1.c   |  9 +++++++++
 .../riscv/amo-table-a-6-compare-exchange-2.c   |  9 +++++++++
 .../riscv/amo-table-a-6-compare-exchange-3.c   |  9 +++++++++
 .../riscv/amo-table-a-6-compare-exchange-4.c   |  9 +++++++++
 .../riscv/amo-table-a-6-compare-exchange-5.c   |  9 +++++++++
 .../riscv/amo-table-a-6-compare-exchange-6.c   | 10 ++++++++++
 .../riscv/amo-table-a-6-compare-exchange-7.c   |  9 +++++++++
 .../gcc.target/riscv/amo-table-a-6-fence-1.c   | 14 ++++++++++++++
 .../gcc.target/riscv/amo-table-a-6-fence-2.c   | 15 +++++++++++++++
 .../gcc.target/riscv/amo-table-a-6-fence-3.c   | 15 +++++++++++++++
 .../gcc.target/riscv/amo-table-a-6-fence-4.c   | 15 +++++++++++++++
 .../gcc.target/riscv/amo-table-a-6-fence-5.c   | 15 +++++++++++++++
 .../gcc.target/riscv/amo-table-a-6-load-1.c    | 16 ++++++++++++++++
 .../gcc.target/riscv/amo-table-a-6-load-2.c    | 17 +++++++++++++++++
 .../gcc.target/riscv/amo-table-a-6-load-3.c    | 18 ++++++++++++++++++
 .../gcc.target/riscv/amo-table-a-6-store-1.c   | 16 ++++++++++++++++
 .../gcc.target/riscv/amo-table-a-6-store-2.c   | 17 +++++++++++++++++
 .../riscv/amo-table-a-6-store-compat-3.c       | 18 ++++++++++++++++++
 .../riscv/amo-table-a-6-subword-amo-add-1.c    |  9 +++++++++
 .../riscv/amo-table-a-6-subword-amo-add-2.c    |  9 +++++++++
 .../riscv/amo-table-a-6-subword-amo-add-3.c    |  9 +++++++++
 .../riscv/amo-table-a-6-subword-amo-add-4.c    |  9 +++++++++
 .../riscv/amo-table-a-6-subword-amo-add-5.c    |  9 +++++++++
 28 files changed, 360 insertions(+)
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-1.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-2.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-3.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-4.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-5.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-1.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-2.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-3.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-4.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-5.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-6.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-7.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-1.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-2.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-3.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-4.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-5.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-1.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-2.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-3.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-1.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-2.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-compat-3.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-1.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-2.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-3.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-4.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-5.c

diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-1.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-1.c
new file mode 100644
index 00000000000..071a33928fe
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-1.c
@@ -0,0 +1,15 @@
+/* { dg-do compile } */
+/* Verify that atomic op mappings match Table A.6's recommended mapping.  */
+/* { dg-options "-O3" } */
+/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */
+/* { dg-final { check-function-bodies "**" "" } } */
+
+/*
+** foo:
+**	amoadd\.w\tzero,a1,0\(a0\)
+**	ret
+*/
+void foo (int* bar, int* baz)
+{
+  __atomic_add_fetch(bar, baz, __ATOMIC_RELAXED);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-2.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-2.c
new file mode 100644
index 00000000000..d6b2d91db2a
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-2.c
@@ -0,0 +1,15 @@
+/* { dg-do compile } */
+/* Verify that atomic op mappings match Table A.6's recommended mapping.  */
+/* { dg-options "-O3" } */
+/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */
+/* { dg-final { check-function-bodies "**" "" } } */
+
+/*
+** foo:
+**	amoadd\.w\.aq\tzero,a1,0\(a0\)
+**	ret
+*/
+void foo (int* bar, int* baz)
+{
+  __atomic_add_fetch(bar, baz, __ATOMIC_ACQUIRE);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-3.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-3.c
new file mode 100644
index 00000000000..68a69ed8b78
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-3.c
@@ -0,0 +1,15 @@
+/* { dg-do compile } */
+/* Verify that atomic op mappings match Table A.6's recommended mapping.  */
+/* { dg-options "-O3" } */
+/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */
+/* { dg-final { check-function-bodies "**" "" } } */
+
+/*
+** foo:
+**	amoadd\.w\.rl\tzero,a1,0\(a0\)
+**	ret
+*/
+void foo (int* bar, int* baz)
+{
+  __atomic_add_fetch(bar, baz, __ATOMIC_RELEASE);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-4.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-4.c
new file mode 100644
index 00000000000..b5cac4c4797
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-4.c
@@ -0,0 +1,15 @@
+/* { dg-do compile } */
+/* Verify that atomic op mappings match Table A.6's recommended mapping.  */
+/* { dg-options "-O3" } */
+/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */
+/* { dg-final { check-function-bodies "**" "" } } */
+
+/*
+** foo:
+**	amoadd\.w\.aqrl\tzero,a1,0\(a0\)
+**	ret
+*/
+void foo (int* bar, int* baz)
+{
+  __atomic_add_fetch(bar, baz, __ATOMIC_ACQ_REL);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-5.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-5.c
new file mode 100644
index 00000000000..268e58cb95f
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-5.c
@@ -0,0 +1,15 @@
+/* { dg-do compile } */
+/* Verify that atomic op mappings match Table A.6's recommended mapping.  */
+/* { dg-options "-O3" } */
+/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */
+/* { dg-final { check-function-bodies "**" "" } } */
+
+/*
+** foo:
+**	amoadd\.w\.aqrl\tzero,a1,0\(a0\)
+**	ret
+*/
+void foo (int* bar, int* baz)
+{
+  __atomic_add_fetch(bar, baz, __ATOMIC_SEQ_CST);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-1.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-1.c
new file mode 100644
index 00000000000..8349e7a69ac
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-1.c
@@ -0,0 +1,9 @@
+/* { dg-do compile } */
+/* Verify that compare exchange mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler-times "lr.w\t" 1 } } */
+/* { dg-final { scan-assembler-times "sc.w\t" 1 } } */
+
+void foo (int bar, int baz, int qux)
+{
+  __atomic_compare_exchange_n(&bar, &baz, qux, 1, __ATOMIC_RELAXED, __ATOMIC_RELAXED);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-2.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-2.c
new file mode 100644
index 00000000000..bf30b298b4b
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-2.c
@@ -0,0 +1,9 @@
+/* { dg-do compile } */
+/* Verify that compare exchange mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler-times "lr.w.aq\t" 1 } } */
+/* { dg-final { scan-assembler-times "sc.w\t" 1 } } */
+
+void foo (int bar, int baz, int qux)
+{
+  __atomic_compare_exchange_n(&bar, &baz, qux, 1, __ATOMIC_CONSUME, __ATOMIC_CONSUME);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-3.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-3.c
new file mode 100644
index 00000000000..41444ec95e9
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-3.c
@@ -0,0 +1,9 @@
+/* { dg-do compile } */
+/* Verify that compare exchange mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler-times "lr.w.aq\t" 1 } } */
+/* { dg-final { scan-assembler-times "sc.w\t" 1 } } */
+
+void foo (int bar, int baz, int qux)
+{
+  __atomic_compare_exchange_n(&bar, &baz, qux, 1, __ATOMIC_ACQUIRE, __ATOMIC_ACQUIRE);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-4.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-4.c
new file mode 100644
index 00000000000..dc2d7bd300d
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-4.c
@@ -0,0 +1,9 @@
+/* { dg-do compile } */
+/* Verify that compare exchange mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler-times "lr.w\t" 1 } } */
+/* { dg-final { scan-assembler-times "sc.w.rl\t" 1 } } */
+
+void foo (int bar, int baz, int qux)
+{
+  __atomic_compare_exchange_n(&bar, &baz, qux, 1, __ATOMIC_RELEASE, __ATOMIC_RELAXED);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-5.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-5.c
new file mode 100644
index 00000000000..53246210900
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-5.c
@@ -0,0 +1,9 @@
+/* { dg-do compile } */
+/* Verify that compare exchange mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler-times "lr.w.aqrl\t" 1 } } */
+/* { dg-final { scan-assembler-times "sc.w.rl\t" 1 } } */
+
+void foo (int bar, int baz, int qux)
+{
+  __atomic_compare_exchange_n(&bar, &baz, qux, 1, __ATOMIC_SEQ_CST, __ATOMIC_SEQ_CST);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-6.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-6.c
new file mode 100644
index 00000000000..1376ac2a95b
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-6.c
@@ -0,0 +1,10 @@
+/* { dg-do compile } */
+/* Verify that compare exchange mappings match Table A.6's recommended mapping.  */
+/* Mixed mappings need to be unioned.  */
+/* { dg-final { scan-assembler-times "lr.w.aq\t" 1 } } */
+/* { dg-final { scan-assembler-times "sc.w.rl\t" 1 } } */
+
+void foo (int bar, int baz, int qux)
+{
+  __atomic_compare_exchange_n(&bar, &baz, qux, 1, __ATOMIC_RELEASE, __ATOMIC_ACQUIRE);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-7.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-7.c
new file mode 100644
index 00000000000..98083cbae08
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-7.c
@@ -0,0 +1,9 @@
+/* { dg-do compile } */
+/* Verify that compare exchange mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler-times "lr.w.aqrl\t" 1 } } */
+/* { dg-final { scan-assembler-times "sc.w.rl\t" 1 } } */
+
+void foo (int bar, int baz, int qux)
+{
+  __atomic_compare_exchange_n(&bar, &baz, qux, 1, __ATOMIC_SEQ_CST, __ATOMIC_RELAXED);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-1.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-1.c
new file mode 100644
index 00000000000..bf590489c39
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-1.c
@@ -0,0 +1,14 @@
+/* { dg-do compile } */
+/* Verify that fence mappings match Table A.6's recommended mapping.  */
+/* { dg-options "-O3" } */
+/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */
+/* { dg-final { check-function-bodies "**" "" } } */
+
+/*
+** foo:
+**	ret
+*/
+void foo()
+{
+  __atomic_thread_fence(__ATOMIC_RELAXED);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-2.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-2.c
new file mode 100644
index 00000000000..9848f8cae31
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-2.c
@@ -0,0 +1,15 @@
+/* { dg-do compile } */
+/* Verify that fence mappings match Table A.6's recommended mapping.  */
+/* { dg-options "-O3" } */
+/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */
+/* { dg-final { check-function-bodies "**" "" } } */
+
+/*
+** foo:
+**	fence\tr,rw
+**	ret
+*/
+void foo()
+{
+  __atomic_thread_fence(__ATOMIC_ACQUIRE);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-3.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-3.c
new file mode 100644
index 00000000000..3c3ce6e0d18
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-3.c
@@ -0,0 +1,15 @@
+/* { dg-do compile } */
+/* Verify that fence mappings match Table A.6's recommended mapping.  */
+/* { dg-options "-O3" } */
+/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */
+/* { dg-final { check-function-bodies "**" "" } } */
+
+/*
+** foo:
+**	fence\trw,w
+**	ret
+*/
+void foo()
+{
+  __atomic_thread_fence(__ATOMIC_RELEASE);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-4.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-4.c
new file mode 100644
index 00000000000..12d71717085
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-4.c
@@ -0,0 +1,15 @@
+/* { dg-do compile } */
+/* Verify that fence mappings match Table A.6's recommended mapping.  */
+/* { dg-options "-O3" } */
+/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */
+/* { dg-final { check-function-bodies "**" "" } } */
+
+/*
+** foo:
+**	fence\.tso
+**	ret
+*/
+void foo()
+{
+  __atomic_thread_fence(__ATOMIC_ACQ_REL);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-5.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-5.c
new file mode 100644
index 00000000000..9567b604c2c
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-5.c
@@ -0,0 +1,15 @@
+/* { dg-do compile } */
+/* Verify that fence mappings match Table A.6's recommended mapping.  */
+/* { dg-options "-O3" } */
+/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */
+/* { dg-final { check-function-bodies "**" "" } } */
+
+/*
+** foo:
+**	fence\trw,rw
+**	ret
+*/
+void foo()
+{
+  __atomic_thread_fence(__ATOMIC_SEQ_CST);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-1.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-1.c
new file mode 100644
index 00000000000..3c79035e46d
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-1.c
@@ -0,0 +1,16 @@
+/* { dg-do compile } */
+/* Verify that load mappings match Table A.6's recommended mapping.  */
+/* { dg-options "-O3" } */
+/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */
+/* { dg-final { check-function-bodies "**" "" } } */
+
+/*
+** foo:
+**	lw\ta[0-9]+,0\(a0\)
+**	sw\ta[0-9]+,0\(a1\)
+**	ret
+*/
+void foo (int* bar, int* baz)
+{
+  __atomic_load(bar, baz, __ATOMIC_RELAXED);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-2.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-2.c
new file mode 100644
index 00000000000..7d74841846f
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-2.c
@@ -0,0 +1,17 @@
+/* { dg-do compile } */
+/* Verify that load mappings match Table A.6's recommended mapping.  */
+/* { dg-options "-O3" } */
+/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */
+/* { dg-final { check-function-bodies "**" "" } } */
+
+/*
+** foo:
+**	lw\ta[0-9]+,0\(a0\)
+**	fence\tr,rw
+**	sw\ta[0-9]+,0\(a1\)
+**	ret
+*/
+void foo (int* bar, int* baz)
+{
+  __atomic_load(bar, baz, __ATOMIC_ACQUIRE);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-3.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-3.c
new file mode 100644
index 00000000000..ab95fa660d2
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-3.c
@@ -0,0 +1,18 @@
+/* { dg-do compile } */
+/* Verify that load mappings match Table A.6's recommended mapping.  */
+/* { dg-options "-O3" } */
+/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */
+/* { dg-final { check-function-bodies "**" "" } } */
+
+/*
+** foo:
+**	fence\trw,rw
+**	lw\ta[0-9]+,0\(a0\)
+**	fence\tr,rw
+**	sw\ta[0-9]+,0\(a1\)
+**	ret
+*/
+void foo (int* bar, int* baz)
+{
+  __atomic_load(bar, baz, __ATOMIC_SEQ_CST);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-1.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-1.c
new file mode 100644
index 00000000000..d852fddf03d
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-1.c
@@ -0,0 +1,16 @@
+/* { dg-do compile } */
+/* Verify that store mappings match Table A.6's recommended mapping.  */
+/* { dg-options "-O3" } */
+/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */
+/* { dg-final { check-function-bodies "**" "" } } */
+
+/*
+** foo:
+**	lw\ta[0-9]+,0\(a1\)
+**	sw\ta[0-9]+,0\(a0\)
+**	ret
+*/
+void foo (int* bar, int* baz)
+{
+  __atomic_store(bar, baz, __ATOMIC_RELAXED);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-2.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-2.c
new file mode 100644
index 00000000000..ccb5e2af7cc
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-2.c
@@ -0,0 +1,17 @@
+/* { dg-do compile } */
+/* Verify that store mappings match Table A.6's recommended mapping.  */
+/* { dg-options "-O3" } */
+/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */
+/* { dg-final { check-function-bodies "**" "" } } */
+
+/*
+** foo:
+**	lw\ta[0-9]+,0\(a1\)
+**	fence\trw,w
+**	sw\ta[0-9]+,0\(a0\)
+**	ret
+*/
+void foo (int* bar, int* baz)
+{
+  __atomic_store(bar, baz, __ATOMIC_RELEASE);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-compat-3.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-compat-3.c
new file mode 100644
index 00000000000..761889f18cf
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-compat-3.c
@@ -0,0 +1,18 @@
+/* { dg-do compile } */
+/* Verify that store mapping are compatible with Table A.6 & A.7.  */
+/* { dg-options "-O3" } */
+/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */
+/* { dg-final { check-function-bodies "**" "" } } */
+
+/*
+** foo:
+**	lw\ta[0-9]+,0\(a1\)
+**	fence\trw,w
+**	sw\ta[0-9]+,0\(a0\)
+**	fence\trw,rw
+**	ret
+*/
+void foo (int* bar, int* baz)
+{
+  __atomic_store(bar, baz, __ATOMIC_SEQ_CST);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-1.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-1.c
new file mode 100644
index 00000000000..d7d887dd181
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-1.c
@@ -0,0 +1,9 @@
+/* { dg-do compile } */
+/* Verify that subword atomic op mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler-times "lr.w\t" 1 } } */
+/* { dg-final { scan-assembler-times "sc.w\t" 1 } } */
+
+void foo (short* bar, short* baz)
+{
+  __atomic_add_fetch(bar, baz, __ATOMIC_RELAXED);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-2.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-2.c
new file mode 100644
index 00000000000..897bad26ebd
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-2.c
@@ -0,0 +1,9 @@
+/* { dg-do compile } */
+/* Verify that subword atomic op mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler-times "lr.w.aq\t" 1 } } */
+/* { dg-final { scan-assembler-times "sc.w\t" 1 } } */
+
+void foo (short* bar, short* baz)
+{
+  __atomic_add_fetch(bar, baz, __ATOMIC_ACQUIRE);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-3.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-3.c
new file mode 100644
index 00000000000..79efca2839a
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-3.c
@@ -0,0 +1,9 @@
+/* { dg-do compile } */
+/* Verify that subword atomic op mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler-times "lr.w\t" 1 } } */
+/* { dg-final { scan-assembler-times "sc.w.rl\t" 1 } } */
+
+void foo (short* bar, short* baz)
+{
+  __atomic_add_fetch(bar, baz, __ATOMIC_RELEASE);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-4.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-4.c
new file mode 100644
index 00000000000..772ac1be6eb
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-4.c
@@ -0,0 +1,9 @@
+/* { dg-do compile } */
+/* Verify that subword atomic op mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler-times "lr.w.aq\t" 1 } } */
+/* { dg-final { scan-assembler-times "sc.w.rl\t" 1 } } */
+
+void foo (short* bar, short* baz)
+{
+  __atomic_add_fetch(bar, baz, __ATOMIC_ACQ_REL);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-5.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-5.c
new file mode 100644
index 00000000000..b0bec66990e
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-5.c
@@ -0,0 +1,9 @@
+/* { dg-do compile } */
+/* Verify that subword atomic op mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler-times "lr.w.aqrl\t" 1 } } */
+/* { dg-final { scan-assembler-times "sc.w.rl\t" 1 } } */
+
+void foo (short* bar, short* baz)
+{
+  __atomic_add_fetch(bar, baz, __ATOMIC_SEQ_CST);
+}
-- 
2.25.1

Re: [PATCH v5 09/11] RISC-V: Weaken mem_thread_fence

[Andreas Schwab]

../../gcc/config/riscv/sync.md: In function 'const char* output_479(rtx_def**, rtx_insn*)':
../../gcc/config/riscv/sync.md:66:1: error: control reaches end of non-void function [-Werror=return-type]
   66 |   [(set (attr "length") (const_int 4))])
      | ^

-- 
Andreas Schwab, SUSE Labs, schwab@suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

Re: [PATCH v5 09/11] RISC-V: Weaken mem_thread_fence

[Martin Liška]

On 5/3/23 14:18, Andreas Schwab via Gcc-patches wrote:
> ../../gcc/config/riscv/sync.md: In function 'const char* output_479(rtx_def**, rtx_insn*)':
> ../../gcc/config/riscv/sync.md:66:1: error: control reaches end of non-void function [-Werror=return-type]
>    66 |   [(set (attr "length") (const_int 4))])
>       | ^
> 

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109713

[gcc13 backport 00/12] RISC-V: Implement ISA Manual Table A.6 Mappings

[Patrick O'Neill]

Discussed during the weekly RISC-V GCC meeting[1] and pre-approved by
Jeff Law.
If there aren't any objections I'll commit this cherry-picked series
on Thursday (July 27th).

Patchset on trunk:
https://inbox.sourceware.org/gcc-patches/20230427162301.1151333-1-patrick@rivosinc.com/
First commit: f37a36bce81b50a43ec1613c1d08d803642f7506

Also includes bugfix from:
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109713
commit: 4bd434fbfc7865961a8e10d7e9601b28765ce7be

[1] https://inbox.sourceware.org/gcc/mhng-b7423fca-67ec-4ce4-9694-4e062632ceb0@palmer-ri-x1c9/T/#t

Martin Liska (1):
  riscv: fix error: control reaches end of non-void function

Patrick O'Neill (11):
  RISC-V: Eliminate SYNC memory models
  RISC-V: Enforce Libatomic LR/SC SEQ_CST
  RISC-V: Enforce subword atomic LR/SC SEQ_CST
  RISC-V: Enforce atomic compare_exchange SEQ_CST
  RISC-V: Add AMO release bits
  RISC-V: Strengthen atomic stores
  RISC-V: Eliminate AMO op fences
  RISC-V: Weaken LR/SC pairs
  RISC-V: Weaken mem_thread_fence
  RISC-V: Weaken atomic loads
  RISC-V: Table A.6 conformance tests

 gcc/config/riscv/riscv-protos.h               |   3 +
 gcc/config/riscv/riscv.cc                     |  66 ++++--
 gcc/config/riscv/sync.md                      | 196 ++++++++++++------
 .../riscv/amo-table-a-6-amo-add-1.c           |  15 ++
 .../riscv/amo-table-a-6-amo-add-2.c           |  15 ++
 .../riscv/amo-table-a-6-amo-add-3.c           |  15 ++
 .../riscv/amo-table-a-6-amo-add-4.c           |  15 ++
 .../riscv/amo-table-a-6-amo-add-5.c           |  15 ++
 .../riscv/amo-table-a-6-compare-exchange-1.c  |   9 +
 .../riscv/amo-table-a-6-compare-exchange-2.c  |   9 +
 .../riscv/amo-table-a-6-compare-exchange-3.c  |   9 +
 .../riscv/amo-table-a-6-compare-exchange-4.c  |   9 +
 .../riscv/amo-table-a-6-compare-exchange-5.c  |   9 +
 .../riscv/amo-table-a-6-compare-exchange-6.c  |  10 +
 .../riscv/amo-table-a-6-compare-exchange-7.c  |   9 +
 .../gcc.target/riscv/amo-table-a-6-fence-1.c  |  14 ++
 .../gcc.target/riscv/amo-table-a-6-fence-2.c  |  15 ++
 .../gcc.target/riscv/amo-table-a-6-fence-3.c  |  15 ++
 .../gcc.target/riscv/amo-table-a-6-fence-4.c  |  15 ++
 .../gcc.target/riscv/amo-table-a-6-fence-5.c  |  15 ++
 .../gcc.target/riscv/amo-table-a-6-load-1.c   |  16 ++
 .../gcc.target/riscv/amo-table-a-6-load-2.c   |  17 ++
 .../gcc.target/riscv/amo-table-a-6-load-3.c   |  18 ++
 .../gcc.target/riscv/amo-table-a-6-store-1.c  |  16 ++
 .../gcc.target/riscv/amo-table-a-6-store-2.c  |  17 ++
 .../riscv/amo-table-a-6-store-compat-3.c      |  18 ++
 .../riscv/amo-table-a-6-subword-amo-add-1.c   |   9 +
 .../riscv/amo-table-a-6-subword-amo-add-2.c   |   9 +
 .../riscv/amo-table-a-6-subword-amo-add-3.c   |   9 +
 .../riscv/amo-table-a-6-subword-amo-add-4.c   |   9 +
 .../riscv/amo-table-a-6-subword-amo-add-5.c   |   9 +
 gcc/testsuite/gcc.target/riscv/pr89835.c      |   9 +
 libgcc/config/riscv/atomic.c                  |   4 +-
 33 files changed, 563 insertions(+), 75 deletions(-)
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-1.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-2.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-3.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-4.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-5.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-1.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-2.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-3.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-4.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-5.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-6.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-7.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-1.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-2.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-3.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-4.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-5.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-1.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-2.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-3.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-1.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-2.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-compat-3.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-1.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-2.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-3.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-4.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-5.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/pr89835.c

-- 
2.34.1

[gcc13 backport 01/12] RISC-V: Eliminate SYNC memory models

[Patrick O'Neill]

Remove references to MEMMODEL_SYNC_* models by converting via
memmodel_base().

2023-04-27 Patrick O'Neill <patrick@rivosinc.com>

gcc/ChangeLog:

	* config/riscv/riscv.cc: Remove MEMMODEL_SYNC_* cases and
	sanitize memmodel input with memmodel_base.

Signed-off-by: Patrick O'Neill <patrick@rivosinc.com>
---
 gcc/config/riscv/riscv.cc | 11 +++--------
 1 file changed, 3 insertions(+), 8 deletions(-)

diff --git a/gcc/config/riscv/riscv.cc b/gcc/config/riscv/riscv.cc
index 9c626904e89..11b897aca5c 100644
--- a/gcc/config/riscv/riscv.cc
+++ b/gcc/config/riscv/riscv.cc
@@ -4294,14 +4294,11 @@ riscv_memmodel_needs_amo_acquire (enum memmodel model)
     {
       case MEMMODEL_ACQ_REL:
       case MEMMODEL_SEQ_CST:
-      case MEMMODEL_SYNC_SEQ_CST:
       case MEMMODEL_ACQUIRE:
       case MEMMODEL_CONSUME:
-      case MEMMODEL_SYNC_ACQUIRE:
 	return true;
 
       case MEMMODEL_RELEASE:
-      case MEMMODEL_SYNC_RELEASE:
       case MEMMODEL_RELAXED:
 	return false;
 
@@ -4320,14 +4317,11 @@ riscv_memmodel_needs_release_fence (enum memmodel model)
     {
       case MEMMODEL_ACQ_REL:
       case MEMMODEL_SEQ_CST:
-      case MEMMODEL_SYNC_SEQ_CST:
       case MEMMODEL_RELEASE:
-      case MEMMODEL_SYNC_RELEASE:
 	return true;
 
       case MEMMODEL_ACQUIRE:
       case MEMMODEL_CONSUME:
-      case MEMMODEL_SYNC_ACQUIRE:
       case MEMMODEL_RELAXED:
 	return false;
 
@@ -4366,6 +4360,7 @@ riscv_print_operand (FILE *file, rtx op, int letter)
     }
   machine_mode mode = GET_MODE (op);
   enum rtx_code code = GET_CODE (op);
+  const enum memmodel model = memmodel_base (INTVAL (op));
 
   switch (letter)
     {
@@ -4503,12 +4498,12 @@ riscv_print_operand (FILE *file, rtx op, int letter)
       break;
 
     case 'A':
-      if (riscv_memmodel_needs_amo_acquire ((enum memmodel) INTVAL (op)))
+      if (riscv_memmodel_needs_amo_acquire (model))
 	fputs (".aq", file);
       break;
 
     case 'F':
-      if (riscv_memmodel_needs_release_fence ((enum memmodel) INTVAL (op)))
+      if (riscv_memmodel_needs_release_fence (model))
 	fputs ("fence iorw,ow; ", file);
       break;
 
-- 
2.34.1

[gcc13 backport 02/12] RISC-V: Enforce Libatomic LR/SC SEQ_CST

[Patrick O'Neill]

Replace LR.aq/SC.rl pairs with the SEQ_CST LR.aqrl/SC.rl pairs
recommended by table A.6 of the ISA manual.

2023-04-27 Patrick O'Neill <patrick@rivosinc.com>

libgcc/ChangeLog:

	* config/riscv/atomic.c: Change LR.aq/SC.rl pairs into
	sequentially consistent LR.aqrl/SC.rl pairs.

Signed-off-by: Patrick O'Neill <patrick@rivosinc.com>
---
 libgcc/config/riscv/atomic.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/libgcc/config/riscv/atomic.c b/libgcc/config/riscv/atomic.c
index 573d163ea04..bd2b033132b 100644
--- a/libgcc/config/riscv/atomic.c
+++ b/libgcc/config/riscv/atomic.c
@@ -41,7 +41,7 @@ see the files COPYING3 and COPYING.RUNTIME respectively.  If not, see
     unsigned old, tmp1, tmp2;						\
 									\
     asm volatile ("1:\n\t"						\
-		  "lr.w.aq %[old], %[mem]\n\t"				\
+		  "lr.w.aqrl %[old], %[mem]\n\t"			\
 		  #insn " %[tmp1], %[old], %[value]\n\t"		\
 		  invert						\
 		  "and %[tmp1], %[tmp1], %[mask]\n\t"			\
@@ -75,7 +75,7 @@ see the files COPYING3 and COPYING.RUNTIME respectively.  If not, see
     unsigned old, tmp1;							\
 									\
     asm volatile ("1:\n\t"						\
-		  "lr.w.aq %[old], %[mem]\n\t"				\
+		  "lr.w.aqrl %[old], %[mem]\n\t"			\
 		  "and %[tmp1], %[old], %[mask]\n\t"			\
 		  "bne %[tmp1], %[o], 1f\n\t"				\
 		  "and %[tmp1], %[old], %[not_mask]\n\t"		\
-- 
2.34.1

[gcc13 backport 03/12] RISC-V: Enforce subword atomic LR/SC SEQ_CST

[Patrick O'Neill]

Replace LR.aq/SC.rl pairs with the SEQ_CST LR.aqrl/SC.rl pairs
recommended by table A.6 of the ISA manual.

2023-04-27 Patrick O'Neill <patrick@rivosinc.com>

gcc/ChangeLog:

	* config/riscv/sync.md: Change LR.aq/SC.rl pairs into
	sequentially consistent LR.aqrl/SC.rl pairs.

Signed-off-by: Patrick O'Neill <patrick@rivosinc.com>
---
 gcc/config/riscv/sync.md | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/gcc/config/riscv/sync.md b/gcc/config/riscv/sync.md
index 19274528262..0c83ef04607 100644
--- a/gcc/config/riscv/sync.md
+++ b/gcc/config/riscv/sync.md
@@ -109,7 +109,7 @@
   "TARGET_ATOMIC && TARGET_INLINE_SUBWORD_ATOMIC"
   {
     return "1:\;"
-	   "lr.w.aq\t%0, %1\;"
+	   "lr.w.aqrl\t%0, %1\;"
 	   "<insn>\t%5, %0, %2\;"
 	   "and\t%5, %5, %3\;"
 	   "and\t%6, %0, %4\;"
@@ -173,7 +173,7 @@
   "TARGET_ATOMIC && TARGET_INLINE_SUBWORD_ATOMIC"
   {
     return "1:\;"
-	   "lr.w.aq\t%0, %1\;"
+	   "lr.w.aqrl\t%0, %1\;"
 	   "and\t%5, %0, %2\;"
 	   "not\t%5, %5\;"
 	   "and\t%5, %5, %3\;"
@@ -278,7 +278,7 @@
   "TARGET_ATOMIC && TARGET_INLINE_SUBWORD_ATOMIC"
   {
     return "1:\;"
-	   "lr.w.aq\t%0, %1\;"
+	   "lr.w.aqrl\t%0, %1\;"
 	   "and\t%4, %0, %3\;"
 	   "or\t%4, %4, %2\;"
 	   "sc.w.rl\t%4, %4, %1\;"
@@ -443,7 +443,7 @@
   "TARGET_ATOMIC && TARGET_INLINE_SUBWORD_ATOMIC"
   {
     return "1:\;"
-	   "lr.w.aq\t%0, %1\;"
+	   "lr.w.aqrl\t%0, %1\;"
 	   "and\t%6, %0, %4\;"
 	   "bne\t%6, %z2, 1f\;"
 	   "and\t%6, %0, %5\;"
-- 
2.34.1

[gcc13 backport 04/12] RISC-V: Enforce atomic compare_exchange SEQ_CST

[Patrick O'Neill]

This patch enforces SEQ_CST for atomic compare_exchange ops.

Replace Fence/LR.aq/SC.aq pairs with SEQ_CST LR.aqrl/SC.rl pairs
recommended by table A.6 of the ISA manual.

2023-04-27 Patrick O'Neill <patrick@rivosinc.com>

gcc/ChangeLog:

	* config/riscv/sync.md (atomic_cas_value_strong<mode>): Change
	FENCE/LR.aq/SC.aq into sequentially consistent LR.aqrl/SC.rl
	pair.

Signed-off-by: Patrick O'Neill <patrick@rivosinc.com>
---
 gcc/config/riscv/sync.md | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)

diff --git a/gcc/config/riscv/sync.md b/gcc/config/riscv/sync.md
index 0c83ef04607..5620d6ffa58 100644
--- a/gcc/config/riscv/sync.md
+++ b/gcc/config/riscv/sync.md
@@ -297,9 +297,16 @@
 	 UNSPEC_COMPARE_AND_SWAP))
    (clobber (match_scratch:GPR 6 "=&r"))]
   "TARGET_ATOMIC"
-  "%F5 1: lr.<amo>%A5 %0,%1; bne %0,%z2,1f; sc.<amo>%A4 %6,%z3,%1; bnez %6,1b; 1:"
+  {
+    return "1:\;"
+	   "lr.<amo>.aqrl\t%0,%1\;"
+	   "bne\t%0,%z2,1f\;"
+	   "sc.<amo>.rl\t%6,%z3,%1\;"
+	   "bnez\t%6,1b\;"
+	   "1:";
+  }
   [(set_attr "type" "atomic")
-   (set (attr "length") (const_int 20))])
+   (set (attr "length") (const_int 16))])
 
 (define_expand "atomic_compare_and_swap<mode>"
   [(match_operand:SI 0 "register_operand" "")   ;; bool output
-- 
2.34.1

[gcc13 backport 05/12] RISC-V: Add AMO release bits

[Patrick O'Neill]

This patch sets the relevant .rl bits on amo operations.

2023-04-27 Patrick O'Neill <patrick@rivosinc.com>

gcc/ChangeLog:

	* config/riscv/riscv.cc (riscv_print_operand): Change behavior
	of %A to include release bits.

Signed-off-by: Patrick O'Neill <patrick@rivosinc.com>
---
 gcc/config/riscv/riscv.cc | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/gcc/config/riscv/riscv.cc b/gcc/config/riscv/riscv.cc
index 11b897aca5c..df55c427b1b 100644
--- a/gcc/config/riscv/riscv.cc
+++ b/gcc/config/riscv/riscv.cc
@@ -4498,8 +4498,13 @@ riscv_print_operand (FILE *file, rtx op, int letter)
       break;
 
     case 'A':
-      if (riscv_memmodel_needs_amo_acquire (model))
+      if (riscv_memmodel_needs_amo_acquire (model)
+	  && riscv_memmodel_needs_release_fence (model))
+	fputs (".aqrl", file);
+      else if (riscv_memmodel_needs_amo_acquire (model))
 	fputs (".aq", file);
+      else if (riscv_memmodel_needs_release_fence (model))
+	fputs (".rl", file);
       break;
 
     case 'F':
-- 
2.34.1

[gcc13 backport 06/12] RISC-V: Strengthen atomic stores

[Patrick O'Neill]

This change makes atomic stores strictly stronger than table A.6 of the
ISA manual. This mapping makes the overall patchset compatible with
table A.7 as well.

2023-04-27 Patrick O'Neill <patrick@rivosinc.com>

	PR target/89835

gcc/ChangeLog:

	* config/riscv/sync.md (atomic_store<mode>): Use simple store
	instruction in combination with fence(s).

gcc/testsuite/ChangeLog:

	* gcc.target/riscv/pr89835.c: New test.

Signed-off-by: Patrick O'Neill <patrick@rivosinc.com>
---
 gcc/config/riscv/sync.md                 | 21 ++++++++++++++++++---
 gcc/testsuite/gcc.target/riscv/pr89835.c |  9 +++++++++
 2 files changed, 27 insertions(+), 3 deletions(-)
 create mode 100644 gcc/testsuite/gcc.target/riscv/pr89835.c

diff --git a/gcc/config/riscv/sync.md b/gcc/config/riscv/sync.md
index 5620d6ffa58..1acb78a9ae4 100644
--- a/gcc/config/riscv/sync.md
+++ b/gcc/config/riscv/sync.md
@@ -56,7 +56,9 @@
 
 ;; Atomic memory operations.
 
-;; Implement atomic stores with amoswap.  Fall back to fences for atomic loads.
+;; Implement atomic stores with conservative fences.  Fall back to fences for
+;; atomic loads.
+;; This allows us to be compatible with the ISA manual Table A.6 and Table A.7.
 (define_insn "atomic_store<mode>"
   [(set (match_operand:GPR 0 "memory_operand" "=A")
     (unspec_volatile:GPR
@@ -64,9 +66,22 @@
        (match_operand:SI 2 "const_int_operand")]      ;; model
       UNSPEC_ATOMIC_STORE))]
   "TARGET_ATOMIC"
-  "%F2amoswap.<amo>%A2 zero,%z1,%0"
+  {
+    enum memmodel model = (enum memmodel) INTVAL (operands[2]);
+    model = memmodel_base (model);
+
+    if (model == MEMMODEL_SEQ_CST)
+      return "fence\trw,w\;"
+	     "s<amo>\t%z1,%0\;"
+	     "fence\trw,rw";
+    if (model == MEMMODEL_RELEASE)
+      return "fence\trw,w\;"
+	     "s<amo>\t%z1,%0";
+    else
+      return "s<amo>\t%z1,%0";
+  }
   [(set_attr "type" "atomic")
-   (set (attr "length") (const_int 8))])
+   (set (attr "length") (const_int 12))])
 
 (define_insn "atomic_<atomic_optab><mode>"
   [(set (match_operand:GPR 0 "memory_operand" "+A")
diff --git a/gcc/testsuite/gcc.target/riscv/pr89835.c b/gcc/testsuite/gcc.target/riscv/pr89835.c
new file mode 100644
index 00000000000..ab190e11b60
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/pr89835.c
@@ -0,0 +1,9 @@
+/* { dg-do compile } */
+/* Verify that relaxed atomic stores use simple store instuctions.  */
+/* { dg-final { scan-assembler-not "amoswap" } } */
+
+void
+foo(int bar, int baz)
+{
+  __atomic_store_n(&bar, baz, __ATOMIC_RELAXED);
+}
-- 
2.34.1

[gcc13 backport 07/12] RISC-V: Eliminate AMO op fences

[Patrick O'Neill]

Atomic operations with the appropriate bits set already enfore release
semantics. Remove unnecessary release fences from atomic ops.

This change brings AMO ops in line with table A.6 of the ISA manual.

2023-04-27 Patrick O'Neill <patrick@rivosinc.com>

gcc/ChangeLog:

	* config/riscv/riscv.cc
	(riscv_memmodel_needs_amo_release): Change function name.
	(riscv_print_operand): Remove unneeded %F case.
	* config/riscv/sync.md: Remove unneeded fences.

Signed-off-by: Patrick O'Neill <patrick@rivosinc.com>
---
 gcc/config/riscv/riscv.cc | 16 +++++-----------
 gcc/config/riscv/sync.md  | 12 ++++++------
 2 files changed, 11 insertions(+), 17 deletions(-)

diff --git a/gcc/config/riscv/riscv.cc b/gcc/config/riscv/riscv.cc
index df55c427b1b..951f6b5cf42 100644
--- a/gcc/config/riscv/riscv.cc
+++ b/gcc/config/riscv/riscv.cc
@@ -4307,11 +4307,11 @@ riscv_memmodel_needs_amo_acquire (enum memmodel model)
     }
 }
 
-/* Return true if a FENCE should be emitted to before a memory access to
-   implement the release portion of memory model MODEL.  */
+/* Return true if the .RL suffix should be added to an AMO to implement the
+   release portion of memory model MODEL.  */
 
 static bool
-riscv_memmodel_needs_release_fence (enum memmodel model)
+riscv_memmodel_needs_amo_release (enum memmodel model)
 {
   switch (model)
     {
@@ -4337,7 +4337,6 @@ riscv_memmodel_needs_release_fence (enum memmodel model)
    'R'	Print the low-part relocation associated with OP.
    'C'	Print the integer branch condition for comparison OP.
    'A'	Print the atomic operation suffix for memory model OP.
-   'F'	Print a FENCE if the memory model requires a release.
    'z'	Print x0 if OP is zero, otherwise print OP normally.
    'i'	Print i if the operand is not a register.
    'S'	Print shift-index of single-bit mask OP.
@@ -4499,19 +4498,14 @@ riscv_print_operand (FILE *file, rtx op, int letter)
 
     case 'A':
       if (riscv_memmodel_needs_amo_acquire (model)
-	  && riscv_memmodel_needs_release_fence (model))
+	  && riscv_memmodel_needs_amo_release (model))
 	fputs (".aqrl", file);
       else if (riscv_memmodel_needs_amo_acquire (model))
 	fputs (".aq", file);
-      else if (riscv_memmodel_needs_release_fence (model))
+      else if (riscv_memmodel_needs_amo_release (model))
 	fputs (".rl", file);
       break;
 
-    case 'F':
-      if (riscv_memmodel_needs_release_fence (model))
-	fputs ("fence iorw,ow; ", file);
-      break;
-
     case 'i':
       if (code != REG)
         fputs ("i", file);
diff --git a/gcc/config/riscv/sync.md b/gcc/config/riscv/sync.md
index 1acb78a9ae4..9a3b57bd09f 100644
--- a/gcc/config/riscv/sync.md
+++ b/gcc/config/riscv/sync.md
@@ -91,9 +91,9 @@
 	   (match_operand:SI 2 "const_int_operand")] ;; model
 	 UNSPEC_SYNC_OLD_OP))]
   "TARGET_ATOMIC"
-  "%F2amo<insn>.<amo>%A2 zero,%z1,%0"
+  "amo<insn>.<amo>%A2\tzero,%z1,%0"
   [(set_attr "type" "atomic")
-   (set (attr "length") (const_int 8))])
+   (set (attr "length") (const_int 4))])
 
 (define_insn "atomic_fetch_<atomic_optab><mode>"
   [(set (match_operand:GPR 0 "register_operand" "=&r")
@@ -105,9 +105,9 @@
 	   (match_operand:SI 3 "const_int_operand")] ;; model
 	 UNSPEC_SYNC_OLD_OP))]
   "TARGET_ATOMIC"
-  "%F3amo<insn>.<amo>%A3 %0,%z2,%1"
+  "amo<insn>.<amo>%A3\t%0,%z2,%1"
   [(set_attr "type" "atomic")
-   (set (attr "length") (const_int 8))])
+   (set (attr "length") (const_int 4))])
 
 (define_insn "subword_atomic_fetch_strong_<atomic_optab>"
   [(set (match_operand:SI 0 "register_operand" "=&r")		   ;; old value at mem
@@ -247,9 +247,9 @@
    (set (match_dup 1)
 	(match_operand:GPR 2 "register_operand" "0"))]
   "TARGET_ATOMIC"
-  "%F3amoswap.<amo>%A3 %0,%z2,%1"
+  "amoswap.<amo>%A3\t%0,%z2,%1"
   [(set_attr "type" "atomic")
-   (set (attr "length") (const_int 8))])
+   (set (attr "length") (const_int 4))])
 
 (define_expand "atomic_exchange<mode>"
   [(match_operand:SHORT 0 "register_operand") ;; old value at mem
-- 
2.34.1

[gcc13 backport 08/12] RISC-V: Weaken LR/SC pairs

[Patrick O'Neill]

Introduce the %I and %J flags for setting the .aqrl bits on LR/SC pairs
as needed.

Atomic compare and exchange ops provide success and failure memory
models. C++17 and later place no restrictions on the relative strength
of each model, so ensure we cover both by using a model that enforces
the ordering of both given models.

This change brings LR/SC ops in line with table A.6 of the ISA manual.

2023-04-27 Patrick O'Neill <patrick@rivosinc.com>

gcc/ChangeLog:

	* config/riscv/riscv-protos.h (riscv_union_memmodels): Expose
	riscv_union_memmodels function to sync.md.
	* config/riscv/riscv.cc (riscv_union_memmodels): Add function to
	get the union of two memmodels in sync.md.
	(riscv_print_operand): Add %I and %J flags that output the
	optimal LR/SC flag bits for a given memory model.
	* config/riscv/sync.md: Remove static .aqrl bits on LR op/.rl
	bits on SC op and replace with optimized %I, %J flags.

Signed-off-by: Patrick O'Neill <patrick@rivosinc.com>
---
 gcc/config/riscv/riscv-protos.h |   3 +
 gcc/config/riscv/riscv.cc       |  44 ++++++++++++
 gcc/config/riscv/sync.md        | 114 +++++++++++++++++++-------------
 3 files changed, 114 insertions(+), 47 deletions(-)

diff --git a/gcc/config/riscv/riscv-protos.h b/gcc/config/riscv/riscv-protos.h
index 02b33e02020..b5616fb3e88 100644
--- a/gcc/config/riscv/riscv-protos.h
+++ b/gcc/config/riscv/riscv-protos.h
@@ -22,6 +22,8 @@ along with GCC; see the file COPYING3.  If not see
 #ifndef GCC_RISCV_PROTOS_H
 #define GCC_RISCV_PROTOS_H
 
+#include "memmodel.h"
+
 /* Symbol types we understand.  The order of this list must match that of
    the unspec enum in riscv.md, subsequent to UNSPEC_ADDRESS_FIRST.  */
 enum riscv_symbol_type {
@@ -81,6 +83,7 @@ extern bool riscv_v_ext_vector_mode_p (machine_mode);
 extern bool riscv_shamt_matches_mask_p (int, HOST_WIDE_INT);
 extern void riscv_subword_address (rtx, rtx *, rtx *, rtx *, rtx *);
 extern void riscv_lshift_subword (machine_mode, rtx, rtx, rtx *);
+extern enum memmodel riscv_union_memmodels (enum memmodel, enum memmodel);
 
 /* Routines implemented in riscv-c.cc.  */
 void riscv_cpu_cpp_builtins (cpp_reader *);
diff --git a/gcc/config/riscv/riscv.cc b/gcc/config/riscv/riscv.cc
index 951f6b5cf42..59899268918 100644
--- a/gcc/config/riscv/riscv.cc
+++ b/gcc/config/riscv/riscv.cc
@@ -4284,6 +4284,36 @@ riscv_print_operand_reloc (FILE *file, rtx op, bool hi_reloc)
   fputc (')', file);
 }
 
+/* Return the memory model that encapuslates both given models.  */
+
+enum memmodel
+riscv_union_memmodels (enum memmodel model1, enum memmodel model2)
+{
+  model1 = memmodel_base (model1);
+  model2 = memmodel_base (model2);
+
+  enum memmodel weaker = model1 <= model2 ? model1: model2;
+  enum memmodel stronger = model1 > model2 ? model1: model2;
+
+  switch (stronger)
+    {
+      case MEMMODEL_SEQ_CST:
+      case MEMMODEL_ACQ_REL:
+	return stronger;
+      case MEMMODEL_RELEASE:
+	if (weaker == MEMMODEL_ACQUIRE || weaker == MEMMODEL_CONSUME)
+	  return MEMMODEL_ACQ_REL;
+	else
+	  return stronger;
+      case MEMMODEL_ACQUIRE:
+      case MEMMODEL_CONSUME:
+      case MEMMODEL_RELAXED:
+	return stronger;
+      default:
+	gcc_unreachable ();
+    }
+}
+
 /* Return true if the .AQ suffix should be added to an AMO to implement the
    acquire portion of memory model MODEL.  */
 
@@ -4337,6 +4367,8 @@ riscv_memmodel_needs_amo_release (enum memmodel model)
    'R'	Print the low-part relocation associated with OP.
    'C'	Print the integer branch condition for comparison OP.
    'A'	Print the atomic operation suffix for memory model OP.
+   'I'	Print the LR suffix for memory model OP.
+   'J'	Print the SC suffix for memory model OP.
    'z'	Print x0 if OP is zero, otherwise print OP normally.
    'i'	Print i if the operand is not a register.
    'S'	Print shift-index of single-bit mask OP.
@@ -4506,6 +4538,18 @@ riscv_print_operand (FILE *file, rtx op, int letter)
 	fputs (".rl", file);
       break;
 
+    case 'I':
+      if (model == MEMMODEL_SEQ_CST)
+	fputs (".aqrl", file);
+      else if (riscv_memmodel_needs_amo_acquire (model))
+	fputs (".aq", file);
+      break;
+
+    case 'J':
+      if (riscv_memmodel_needs_amo_release (model))
+	fputs (".rl", file);
+      break;
+
     case 'i':
       if (code != REG)
         fputs ("i", file);
diff --git a/gcc/config/riscv/sync.md b/gcc/config/riscv/sync.md
index 9a3b57bd09f..3e6345e83a3 100644
--- a/gcc/config/riscv/sync.md
+++ b/gcc/config/riscv/sync.md
@@ -116,21 +116,22 @@
 	(unspec_volatile:SI
 	  [(any_atomic:SI (match_dup 1)
 		     (match_operand:SI 2 "register_operand" "rI")) ;; value for op
-	   (match_operand:SI 3 "register_operand" "rI")]	   ;; mask
+	   (match_operand:SI 3 "const_int_operand")]		   ;; model
 	 UNSPEC_SYNC_OLD_OP_SUBWORD))
-    (match_operand:SI 4 "register_operand" "rI")		   ;; not_mask
-    (clobber (match_scratch:SI 5 "=&r"))			   ;; tmp_1
-    (clobber (match_scratch:SI 6 "=&r"))]			   ;; tmp_2
+    (match_operand:SI 4 "register_operand" "rI")		   ;; mask
+    (match_operand:SI 5 "register_operand" "rI")		   ;; not_mask
+    (clobber (match_scratch:SI 6 "=&r"))			   ;; tmp_1
+    (clobber (match_scratch:SI 7 "=&r"))]			   ;; tmp_2
   "TARGET_ATOMIC && TARGET_INLINE_SUBWORD_ATOMIC"
   {
     return "1:\;"
-	   "lr.w.aqrl\t%0, %1\;"
-	   "<insn>\t%5, %0, %2\;"
-	   "and\t%5, %5, %3\;"
-	   "and\t%6, %0, %4\;"
-	   "or\t%6, %6, %5\;"
-	   "sc.w.rl\t%5, %6, %1\;"
-	   "bnez\t%5, 1b";
+	   "lr.w%I3\t%0, %1\;"
+	   "<insn>\t%6, %0, %2\;"
+	   "and\t%6, %6, %4\;"
+	   "and\t%7, %0, %5\;"
+	   "or\t%7, %7, %6\;"
+	   "sc.w%J3\t%6, %7, %1\;"
+	   "bnez\t%6, 1b";
   }
   [(set (attr "length") (const_int 28))])
 
@@ -151,6 +152,7 @@
   rtx old = gen_reg_rtx (SImode);
   rtx mem = operands[1];
   rtx value = operands[2];
+  rtx model = operands[3];
   rtx aligned_mem = gen_reg_rtx (SImode);
   rtx shift = gen_reg_rtx (SImode);
   rtx mask = gen_reg_rtx (SImode);
@@ -162,7 +164,7 @@
   riscv_lshift_subword (<MODE>mode, value, shift, &shifted_value);
 
   emit_insn (gen_subword_atomic_fetch_strong_nand (old, aligned_mem,
-						   shifted_value,
+						   shifted_value, model,
 						   mask, not_mask));
 
   emit_move_insn (old, gen_rtx_ASHIFTRT (SImode, old,
@@ -180,22 +182,23 @@
 	(unspec_volatile:SI
 	  [(not:SI (and:SI (match_dup 1)
 			   (match_operand:SI 2 "register_operand" "rI"))) ;; value for op
-	   (match_operand:SI 3 "register_operand" "rI")]		  ;; mask
+	   (match_operand:SI 3 "const_int_operand")]			  ;; mask
 	 UNSPEC_SYNC_OLD_OP_SUBWORD))
-    (match_operand:SI 4 "register_operand" "rI")			  ;; not_mask
-    (clobber (match_scratch:SI 5 "=&r"))				  ;; tmp_1
-    (clobber (match_scratch:SI 6 "=&r"))]				  ;; tmp_2
+    (match_operand:SI 4 "register_operand" "rI")			  ;; mask
+    (match_operand:SI 5 "register_operand" "rI")			  ;; not_mask
+    (clobber (match_scratch:SI 6 "=&r"))				  ;; tmp_1
+    (clobber (match_scratch:SI 7 "=&r"))]				  ;; tmp_2
   "TARGET_ATOMIC && TARGET_INLINE_SUBWORD_ATOMIC"
   {
     return "1:\;"
-	   "lr.w.aqrl\t%0, %1\;"
-	   "and\t%5, %0, %2\;"
-	   "not\t%5, %5\;"
-	   "and\t%5, %5, %3\;"
-	   "and\t%6, %0, %4\;"
-	   "or\t%6, %6, %5\;"
-	   "sc.w.rl\t%5, %6, %1\;"
-	   "bnez\t%5, 1b";
+	   "lr.w%I3\t%0, %1\;"
+	   "and\t%6, %0, %2\;"
+	   "not\t%6, %6\;"
+	   "and\t%6, %6, %4\;"
+	   "and\t%7, %0, %5\;"
+	   "or\t%7, %7, %6\;"
+	   "sc.w%J3\t%6, %7, %1\;"
+	   "bnez\t%6, 1b";
   }
   [(set (attr "length") (const_int 32))])
 
@@ -216,6 +219,7 @@
   rtx old = gen_reg_rtx (SImode);
   rtx mem = operands[1];
   rtx value = operands[2];
+  rtx model = operands[3];
   rtx aligned_mem = gen_reg_rtx (SImode);
   rtx shift = gen_reg_rtx (SImode);
   rtx mask = gen_reg_rtx (SImode);
@@ -228,7 +232,8 @@
 
   emit_insn (gen_subword_atomic_fetch_strong_<atomic_optab> (old, aligned_mem,
 							     shifted_value,
-							     mask, not_mask));
+							     model, mask,
+							     not_mask));
 
   emit_move_insn (old, gen_rtx_ASHIFTRT (SImode, old,
 					 gen_lowpart (QImode, shift)));
@@ -261,6 +266,7 @@
   rtx old = gen_reg_rtx (SImode);
   rtx mem = operands[1];
   rtx value = operands[2];
+  rtx model = operands[3];
   rtx aligned_mem = gen_reg_rtx (SImode);
   rtx shift = gen_reg_rtx (SImode);
   rtx mask = gen_reg_rtx (SImode);
@@ -272,7 +278,8 @@
   riscv_lshift_subword (<MODE>mode, value, shift, &shifted_value);
 
   emit_insn (gen_subword_atomic_exchange_strong (old, aligned_mem,
-						 shifted_value, not_mask));
+						 shifted_value, model,
+						 not_mask));
 
   emit_move_insn (old, gen_rtx_ASHIFTRT (SImode, old,
 					 gen_lowpart (QImode, shift)));
@@ -286,18 +293,19 @@
 	(match_operand:SI 1 "memory_operand" "+A"))	 ;; mem location
    (set (match_dup 1)
 	(unspec_volatile:SI
-	  [(match_operand:SI 2 "reg_or_0_operand" "rI")  ;; value
-	   (match_operand:SI 3 "reg_or_0_operand" "rI")] ;; not_mask
+	  [(match_operand:SI 2 "reg_or_0_operand" "rI")	 ;; value
+	   (match_operand:SI 3 "const_int_operand")]	 ;; model
       UNSPEC_SYNC_EXCHANGE_SUBWORD))
-    (clobber (match_scratch:SI 4 "=&r"))]		 ;; tmp_1
+    (match_operand:SI 4 "reg_or_0_operand" "rI")	 ;; not_mask
+    (clobber (match_scratch:SI 5 "=&r"))]		 ;; tmp_1
   "TARGET_ATOMIC && TARGET_INLINE_SUBWORD_ATOMIC"
   {
     return "1:\;"
-	   "lr.w.aqrl\t%0, %1\;"
-	   "and\t%4, %0, %3\;"
-	   "or\t%4, %4, %2\;"
-	   "sc.w.rl\t%4, %4, %1\;"
-	   "bnez\t%4, 1b";
+	   "lr.w%I3\t%0, %1\;"
+	   "and\t%5, %0, %4\;"
+	   "or\t%5, %5, %2\;"
+	   "sc.w%J3\t%5, %5, %1\;"
+	   "bnez\t%5, 1b";
   }
   [(set (attr "length") (const_int 20))])
 
@@ -313,10 +321,15 @@
    (clobber (match_scratch:GPR 6 "=&r"))]
   "TARGET_ATOMIC"
   {
+    enum memmodel model_success = (enum memmodel) INTVAL (operands[4]);
+    enum memmodel model_failure = (enum memmodel) INTVAL (operands[5]);
+    /* Find the union of the two memory models so we can satisfy both success
+       and failure memory models.  */
+    operands[5] = GEN_INT (riscv_union_memmodels (model_success, model_failure));
     return "1:\;"
-	   "lr.<amo>.aqrl\t%0,%1\;"
+	   "lr.<amo>%I5\t%0,%1\;"
 	   "bne\t%0,%z2,1f\;"
-	   "sc.<amo>.rl\t%6,%z3,%1\;"
+	   "sc.<amo>%J5\t%6,%z3,%1\;"
 	   "bnez\t%6,1b\;"
 	   "1:";
   }
@@ -440,9 +453,15 @@
   emit_move_insn (shifted_o, gen_rtx_AND (SImode, shifted_o, mask));
   emit_move_insn (shifted_n, gen_rtx_AND (SImode, shifted_n, mask));
 
+  enum memmodel model_success = (enum memmodel) INTVAL (operands[4]);
+  enum memmodel model_failure = (enum memmodel) INTVAL (operands[5]);
+  /* Find the union of the two memory models so we can satisfy both success
+     and failure memory models.  */
+  rtx model = GEN_INT (riscv_union_memmodels (model_success, model_failure));
+
   emit_insn (gen_subword_atomic_cas_strong (old, aligned_mem,
 					    shifted_o, shifted_n,
-					    mask, not_mask));
+					    model, mask, not_mask));
 
   emit_move_insn (old, gen_rtx_ASHIFTRT (SImode, old,
 					 gen_lowpart (QImode, shift)));
@@ -459,19 +478,20 @@
 	(unspec_volatile:SI [(match_operand:SI 2 "reg_or_0_operand" "rJ")  ;; expected value
 			     (match_operand:SI 3 "reg_or_0_operand" "rJ")] ;; desired value
 	 UNSPEC_COMPARE_AND_SWAP_SUBWORD))
-	(match_operand:SI 4 "register_operand" "rI")			   ;; mask
-	(match_operand:SI 5 "register_operand" "rI")			   ;; not_mask
-	(clobber (match_scratch:SI 6 "=&r"))]				   ;; tmp_1
+	(match_operand:SI 4 "const_int_operand")			   ;; model
+	(match_operand:SI 5 "register_operand" "rI")			   ;; mask
+	(match_operand:SI 6 "register_operand" "rI")			   ;; not_mask
+	(clobber (match_scratch:SI 7 "=&r"))]				   ;; tmp_1
   "TARGET_ATOMIC && TARGET_INLINE_SUBWORD_ATOMIC"
   {
     return "1:\;"
-	   "lr.w.aqrl\t%0, %1\;"
-	   "and\t%6, %0, %4\;"
-	   "bne\t%6, %z2, 1f\;"
-	   "and\t%6, %0, %5\;"
-	   "or\t%6, %6, %3\;"
-	   "sc.w.rl\t%6, %6, %1\;"
-	   "bnez\t%6, 1b\;"
+	   "lr.w%I4\t%0, %1\;"
+	   "and\t%7, %0, %5\;"
+	   "bne\t%7, %z2, 1f\;"
+	   "and\t%7, %0, %6\;"
+	   "or\t%7, %7, %3\;"
+	   "sc.w%J4\t%7, %7, %1\;"
+	   "bnez\t%7, 1b\;"
 	   "1:";
   }
   [(set (attr "length") (const_int 28))])
-- 
2.34.1

[gcc13 backport 09/12] RISC-V: Weaken mem_thread_fence

[Patrick O'Neill]

This change brings atomic fences in line with table A.6 of the ISA
manual.

Relax mem_thread_fence according to the memmodel given.

2023-04-27 Patrick O'Neill <patrick@rivosinc.com>

gcc/ChangeLog:

	* config/riscv/sync.md (mem_thread_fence_1): Change fence
	depending on the given memory model.

Signed-off-by: Patrick O'Neill <patrick@rivosinc.com>
---
 gcc/config/riscv/sync.md | 16 +++++++++++++---
 1 file changed, 13 insertions(+), 3 deletions(-)

diff --git a/gcc/config/riscv/sync.md b/gcc/config/riscv/sync.md
index 3e6345e83a3..ba132d8a1ce 100644
--- a/gcc/config/riscv/sync.md
+++ b/gcc/config/riscv/sync.md
@@ -45,14 +45,24 @@
   DONE;
 })
 
-;; Until the RISC-V memory model (hence its mapping from C++) is finalized,
-;; conservatively emit a full FENCE.
 (define_insn "mem_thread_fence_1"
   [(set (match_operand:BLK 0 "" "")
 	(unspec:BLK [(match_dup 0)] UNSPEC_MEMORY_BARRIER))
    (match_operand:SI 1 "const_int_operand" "")] ;; model
   ""
-  "fence\tiorw,iorw")
+  {
+    enum memmodel model = (enum memmodel) INTVAL (operands[1]);
+    model = memmodel_base (model);
+    if (model == MEMMODEL_SEQ_CST)
+	return "fence\trw,rw";
+    else if (model == MEMMODEL_ACQ_REL)
+	return "fence.tso";
+    else if (model == MEMMODEL_ACQUIRE)
+	return "fence\tr,rw";
+    else if (model == MEMMODEL_RELEASE)
+	return "fence\trw,w";
+  }
+  [(set (attr "length") (const_int 4))])
 
 ;; Atomic memory operations.
 
-- 
2.34.1

[gcc13 backport 10/12] RISC-V: Weaken atomic loads

[Patrick O'Neill]

This change brings atomic loads in line with table A.6 of the ISA
manual.

2023-04-27 Patrick O'Neill <patrick@rivosinc.com>

gcc/ChangeLog:

	* config/riscv/sync.md (atomic_load<mode>): Implement atomic
	load mapping.

Signed-off-by: Patrick O'Neill <patrick@rivosinc.com>
---
 gcc/config/riscv/sync.md | 28 ++++++++++++++++++++++++++--
 1 file changed, 26 insertions(+), 2 deletions(-)

diff --git a/gcc/config/riscv/sync.md b/gcc/config/riscv/sync.md
index ba132d8a1ce..6e7c762ac57 100644
--- a/gcc/config/riscv/sync.md
+++ b/gcc/config/riscv/sync.md
@@ -26,6 +26,7 @@
   UNSPEC_SYNC_OLD_OP_SUBWORD
   UNSPEC_SYNC_EXCHANGE
   UNSPEC_SYNC_EXCHANGE_SUBWORD
+  UNSPEC_ATOMIC_LOAD
   UNSPEC_ATOMIC_STORE
   UNSPEC_MEMORY_BARRIER
 ])
@@ -66,8 +67,31 @@
 
 ;; Atomic memory operations.
 
-;; Implement atomic stores with conservative fences.  Fall back to fences for
-;; atomic loads.
+(define_insn "atomic_load<mode>"
+  [(set (match_operand:GPR 0 "register_operand" "=r")
+    (unspec_volatile:GPR
+      [(match_operand:GPR 1 "memory_operand" "A")
+       (match_operand:SI 2 "const_int_operand")]      ;; model
+      UNSPEC_ATOMIC_LOAD))]
+  "TARGET_ATOMIC"
+  {
+    enum memmodel model = (enum memmodel) INTVAL (operands[2]);
+    model = memmodel_base (model);
+
+    if (model == MEMMODEL_SEQ_CST)
+      return "fence\trw,rw\;"
+	     "l<amo>\t%0,%1\;"
+	     "fence\tr,rw";
+    if (model == MEMMODEL_ACQUIRE)
+      return "l<amo>\t%0,%1\;"
+	     "fence\tr,rw";
+    else
+      return "l<amo>\t%0,%1";
+  }
+  [(set_attr "type" "atomic")
+   (set (attr "length") (const_int 12))])
+
+;; Implement atomic stores with conservative fences.
 ;; This allows us to be compatible with the ISA manual Table A.6 and Table A.7.
 (define_insn "atomic_store<mode>"
   [(set (match_operand:GPR 0 "memory_operand" "=A")
-- 
2.34.1

[gcc13 backport 11/12] RISC-V: Table A.6 conformance tests

[Patrick O'Neill]

These tests cover basic cases to ensure the atomic mappings follow the
strengthened Table A.6 mappings that are compatible with Table A.7.

2023-04-27 Patrick O'Neill <patrick@rivosinc.com>

gcc/testsuite/ChangeLog:

	* gcc.target/riscv/amo-table-a-6-amo-add-1.c: New test.
	* gcc.target/riscv/amo-table-a-6-amo-add-2.c: New test.
	* gcc.target/riscv/amo-table-a-6-amo-add-3.c: New test.
	* gcc.target/riscv/amo-table-a-6-amo-add-4.c: New test.
	* gcc.target/riscv/amo-table-a-6-amo-add-5.c: New test.
	* gcc.target/riscv/amo-table-a-6-compare-exchange-1.c: New test.
	* gcc.target/riscv/amo-table-a-6-compare-exchange-2.c: New test.
	* gcc.target/riscv/amo-table-a-6-compare-exchange-3.c: New test.
	* gcc.target/riscv/amo-table-a-6-compare-exchange-4.c: New test.
	* gcc.target/riscv/amo-table-a-6-compare-exchange-5.c: New test.
	* gcc.target/riscv/amo-table-a-6-compare-exchange-6.c: New test.
	* gcc.target/riscv/amo-table-a-6-compare-exchange-7.c: New test.
	* gcc.target/riscv/amo-table-a-6-fence-1.c: New test.
	* gcc.target/riscv/amo-table-a-6-fence-2.c: New test.
	* gcc.target/riscv/amo-table-a-6-fence-3.c: New test.
	* gcc.target/riscv/amo-table-a-6-fence-4.c: New test.
	* gcc.target/riscv/amo-table-a-6-fence-5.c: New test.
	* gcc.target/riscv/amo-table-a-6-load-1.c: New test.
	* gcc.target/riscv/amo-table-a-6-load-2.c: New test.
	* gcc.target/riscv/amo-table-a-6-load-3.c: New test.
	* gcc.target/riscv/amo-table-a-6-store-1.c: New test.
	* gcc.target/riscv/amo-table-a-6-store-2.c: New test.
	* gcc.target/riscv/amo-table-a-6-store-compat-3.c: New test.
	* gcc.target/riscv/amo-table-a-6-subword-amo-add-1.c: New test.
	* gcc.target/riscv/amo-table-a-6-subword-amo-add-2.c: New test.
	* gcc.target/riscv/amo-table-a-6-subword-amo-add-3.c: New test.
	* gcc.target/riscv/amo-table-a-6-subword-amo-add-4.c: New test.
	* gcc.target/riscv/amo-table-a-6-subword-amo-add-5.c: New test.

Signed-off-by: Patrick O'Neill <patrick@rivosinc.com>
---
 .../gcc.target/riscv/amo-table-a-6-amo-add-1.c | 15 +++++++++++++++
 .../gcc.target/riscv/amo-table-a-6-amo-add-2.c | 15 +++++++++++++++
 .../gcc.target/riscv/amo-table-a-6-amo-add-3.c | 15 +++++++++++++++
 .../gcc.target/riscv/amo-table-a-6-amo-add-4.c | 15 +++++++++++++++
 .../gcc.target/riscv/amo-table-a-6-amo-add-5.c | 15 +++++++++++++++
 .../riscv/amo-table-a-6-compare-exchange-1.c   |  9 +++++++++
 .../riscv/amo-table-a-6-compare-exchange-2.c   |  9 +++++++++
 .../riscv/amo-table-a-6-compare-exchange-3.c   |  9 +++++++++
 .../riscv/amo-table-a-6-compare-exchange-4.c   |  9 +++++++++
 .../riscv/amo-table-a-6-compare-exchange-5.c   |  9 +++++++++
 .../riscv/amo-table-a-6-compare-exchange-6.c   | 10 ++++++++++
 .../riscv/amo-table-a-6-compare-exchange-7.c   |  9 +++++++++
 .../gcc.target/riscv/amo-table-a-6-fence-1.c   | 14 ++++++++++++++
 .../gcc.target/riscv/amo-table-a-6-fence-2.c   | 15 +++++++++++++++
 .../gcc.target/riscv/amo-table-a-6-fence-3.c   | 15 +++++++++++++++
 .../gcc.target/riscv/amo-table-a-6-fence-4.c   | 15 +++++++++++++++
 .../gcc.target/riscv/amo-table-a-6-fence-5.c   | 15 +++++++++++++++
 .../gcc.target/riscv/amo-table-a-6-load-1.c    | 16 ++++++++++++++++
 .../gcc.target/riscv/amo-table-a-6-load-2.c    | 17 +++++++++++++++++
 .../gcc.target/riscv/amo-table-a-6-load-3.c    | 18 ++++++++++++++++++
 .../gcc.target/riscv/amo-table-a-6-store-1.c   | 16 ++++++++++++++++
 .../gcc.target/riscv/amo-table-a-6-store-2.c   | 17 +++++++++++++++++
 .../riscv/amo-table-a-6-store-compat-3.c       | 18 ++++++++++++++++++
 .../riscv/amo-table-a-6-subword-amo-add-1.c    |  9 +++++++++
 .../riscv/amo-table-a-6-subword-amo-add-2.c    |  9 +++++++++
 .../riscv/amo-table-a-6-subword-amo-add-3.c    |  9 +++++++++
 .../riscv/amo-table-a-6-subword-amo-add-4.c    |  9 +++++++++
 .../riscv/amo-table-a-6-subword-amo-add-5.c    |  9 +++++++++
 28 files changed, 360 insertions(+)
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-1.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-2.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-3.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-4.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-5.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-1.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-2.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-3.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-4.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-5.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-6.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-7.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-1.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-2.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-3.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-4.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-5.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-1.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-2.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-3.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-1.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-2.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-compat-3.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-1.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-2.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-3.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-4.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-5.c

diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-1.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-1.c
new file mode 100644
index 00000000000..071a33928fe
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-1.c
@@ -0,0 +1,15 @@
+/* { dg-do compile } */
+/* Verify that atomic op mappings match Table A.6's recommended mapping.  */
+/* { dg-options "-O3" } */
+/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */
+/* { dg-final { check-function-bodies "**" "" } } */
+
+/*
+** foo:
+**	amoadd\.w\tzero,a1,0\(a0\)
+**	ret
+*/
+void foo (int* bar, int* baz)
+{
+  __atomic_add_fetch(bar, baz, __ATOMIC_RELAXED);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-2.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-2.c
new file mode 100644
index 00000000000..d6b2d91db2a
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-2.c
@@ -0,0 +1,15 @@
+/* { dg-do compile } */
+/* Verify that atomic op mappings match Table A.6's recommended mapping.  */
+/* { dg-options "-O3" } */
+/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */
+/* { dg-final { check-function-bodies "**" "" } } */
+
+/*
+** foo:
+**	amoadd\.w\.aq\tzero,a1,0\(a0\)
+**	ret
+*/
+void foo (int* bar, int* baz)
+{
+  __atomic_add_fetch(bar, baz, __ATOMIC_ACQUIRE);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-3.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-3.c
new file mode 100644
index 00000000000..68a69ed8b78
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-3.c
@@ -0,0 +1,15 @@
+/* { dg-do compile } */
+/* Verify that atomic op mappings match Table A.6's recommended mapping.  */
+/* { dg-options "-O3" } */
+/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */
+/* { dg-final { check-function-bodies "**" "" } } */
+
+/*
+** foo:
+**	amoadd\.w\.rl\tzero,a1,0\(a0\)
+**	ret
+*/
+void foo (int* bar, int* baz)
+{
+  __atomic_add_fetch(bar, baz, __ATOMIC_RELEASE);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-4.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-4.c
new file mode 100644
index 00000000000..b5cac4c4797
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-4.c
@@ -0,0 +1,15 @@
+/* { dg-do compile } */
+/* Verify that atomic op mappings match Table A.6's recommended mapping.  */
+/* { dg-options "-O3" } */
+/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */
+/* { dg-final { check-function-bodies "**" "" } } */
+
+/*
+** foo:
+**	amoadd\.w\.aqrl\tzero,a1,0\(a0\)
+**	ret
+*/
+void foo (int* bar, int* baz)
+{
+  __atomic_add_fetch(bar, baz, __ATOMIC_ACQ_REL);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-5.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-5.c
new file mode 100644
index 00000000000..268e58cb95f
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-5.c
@@ -0,0 +1,15 @@
+/* { dg-do compile } */
+/* Verify that atomic op mappings match Table A.6's recommended mapping.  */
+/* { dg-options "-O3" } */
+/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */
+/* { dg-final { check-function-bodies "**" "" } } */
+
+/*
+** foo:
+**	amoadd\.w\.aqrl\tzero,a1,0\(a0\)
+**	ret
+*/
+void foo (int* bar, int* baz)
+{
+  __atomic_add_fetch(bar, baz, __ATOMIC_SEQ_CST);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-1.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-1.c
new file mode 100644
index 00000000000..8349e7a69ac
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-1.c
@@ -0,0 +1,9 @@
+/* { dg-do compile } */
+/* Verify that compare exchange mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler-times "lr.w\t" 1 } } */
+/* { dg-final { scan-assembler-times "sc.w\t" 1 } } */
+
+void foo (int bar, int baz, int qux)
+{
+  __atomic_compare_exchange_n(&bar, &baz, qux, 1, __ATOMIC_RELAXED, __ATOMIC_RELAXED);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-2.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-2.c
new file mode 100644
index 00000000000..bf30b298b4b
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-2.c
@@ -0,0 +1,9 @@
+/* { dg-do compile } */
+/* Verify that compare exchange mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler-times "lr.w.aq\t" 1 } } */
+/* { dg-final { scan-assembler-times "sc.w\t" 1 } } */
+
+void foo (int bar, int baz, int qux)
+{
+  __atomic_compare_exchange_n(&bar, &baz, qux, 1, __ATOMIC_CONSUME, __ATOMIC_CONSUME);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-3.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-3.c
new file mode 100644
index 00000000000..41444ec95e9
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-3.c
@@ -0,0 +1,9 @@
+/* { dg-do compile } */
+/* Verify that compare exchange mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler-times "lr.w.aq\t" 1 } } */
+/* { dg-final { scan-assembler-times "sc.w\t" 1 } } */
+
+void foo (int bar, int baz, int qux)
+{
+  __atomic_compare_exchange_n(&bar, &baz, qux, 1, __ATOMIC_ACQUIRE, __ATOMIC_ACQUIRE);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-4.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-4.c
new file mode 100644
index 00000000000..dc2d7bd300d
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-4.c
@@ -0,0 +1,9 @@
+/* { dg-do compile } */
+/* Verify that compare exchange mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler-times "lr.w\t" 1 } } */
+/* { dg-final { scan-assembler-times "sc.w.rl\t" 1 } } */
+
+void foo (int bar, int baz, int qux)
+{
+  __atomic_compare_exchange_n(&bar, &baz, qux, 1, __ATOMIC_RELEASE, __ATOMIC_RELAXED);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-5.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-5.c
new file mode 100644
index 00000000000..53246210900
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-5.c
@@ -0,0 +1,9 @@
+/* { dg-do compile } */
+/* Verify that compare exchange mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler-times "lr.w.aqrl\t" 1 } } */
+/* { dg-final { scan-assembler-times "sc.w.rl\t" 1 } } */
+
+void foo (int bar, int baz, int qux)
+{
+  __atomic_compare_exchange_n(&bar, &baz, qux, 1, __ATOMIC_SEQ_CST, __ATOMIC_SEQ_CST);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-6.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-6.c
new file mode 100644
index 00000000000..1376ac2a95b
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-6.c
@@ -0,0 +1,10 @@
+/* { dg-do compile } */
+/* Verify that compare exchange mappings match Table A.6's recommended mapping.  */
+/* Mixed mappings need to be unioned.  */
+/* { dg-final { scan-assembler-times "lr.w.aq\t" 1 } } */
+/* { dg-final { scan-assembler-times "sc.w.rl\t" 1 } } */
+
+void foo (int bar, int baz, int qux)
+{
+  __atomic_compare_exchange_n(&bar, &baz, qux, 1, __ATOMIC_RELEASE, __ATOMIC_ACQUIRE);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-7.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-7.c
new file mode 100644
index 00000000000..98083cbae08
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-7.c
@@ -0,0 +1,9 @@
+/* { dg-do compile } */
+/* Verify that compare exchange mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler-times "lr.w.aqrl\t" 1 } } */
+/* { dg-final { scan-assembler-times "sc.w.rl\t" 1 } } */
+
+void foo (int bar, int baz, int qux)
+{
+  __atomic_compare_exchange_n(&bar, &baz, qux, 1, __ATOMIC_SEQ_CST, __ATOMIC_RELAXED);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-1.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-1.c
new file mode 100644
index 00000000000..bf590489c39
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-1.c
@@ -0,0 +1,14 @@
+/* { dg-do compile } */
+/* Verify that fence mappings match Table A.6's recommended mapping.  */
+/* { dg-options "-O3" } */
+/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */
+/* { dg-final { check-function-bodies "**" "" } } */
+
+/*
+** foo:
+**	ret
+*/
+void foo()
+{
+  __atomic_thread_fence(__ATOMIC_RELAXED);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-2.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-2.c
new file mode 100644
index 00000000000..9848f8cae31
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-2.c
@@ -0,0 +1,15 @@
+/* { dg-do compile } */
+/* Verify that fence mappings match Table A.6's recommended mapping.  */
+/* { dg-options "-O3" } */
+/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */
+/* { dg-final { check-function-bodies "**" "" } } */
+
+/*
+** foo:
+**	fence\tr,rw
+**	ret
+*/
+void foo()
+{
+  __atomic_thread_fence(__ATOMIC_ACQUIRE);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-3.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-3.c
new file mode 100644
index 00000000000..3c3ce6e0d18
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-3.c
@@ -0,0 +1,15 @@
+/* { dg-do compile } */
+/* Verify that fence mappings match Table A.6's recommended mapping.  */
+/* { dg-options "-O3" } */
+/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */
+/* { dg-final { check-function-bodies "**" "" } } */
+
+/*
+** foo:
+**	fence\trw,w
+**	ret
+*/
+void foo()
+{
+  __atomic_thread_fence(__ATOMIC_RELEASE);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-4.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-4.c
new file mode 100644
index 00000000000..12d71717085
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-4.c
@@ -0,0 +1,15 @@
+/* { dg-do compile } */
+/* Verify that fence mappings match Table A.6's recommended mapping.  */
+/* { dg-options "-O3" } */
+/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */
+/* { dg-final { check-function-bodies "**" "" } } */
+
+/*
+** foo:
+**	fence\.tso
+**	ret
+*/
+void foo()
+{
+  __atomic_thread_fence(__ATOMIC_ACQ_REL);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-5.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-5.c
new file mode 100644
index 00000000000..9567b604c2c
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-5.c
@@ -0,0 +1,15 @@
+/* { dg-do compile } */
+/* Verify that fence mappings match Table A.6's recommended mapping.  */
+/* { dg-options "-O3" } */
+/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */
+/* { dg-final { check-function-bodies "**" "" } } */
+
+/*
+** foo:
+**	fence\trw,rw
+**	ret
+*/
+void foo()
+{
+  __atomic_thread_fence(__ATOMIC_SEQ_CST);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-1.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-1.c
new file mode 100644
index 00000000000..3c79035e46d
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-1.c
@@ -0,0 +1,16 @@
+/* { dg-do compile } */
+/* Verify that load mappings match Table A.6's recommended mapping.  */
+/* { dg-options "-O3" } */
+/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */
+/* { dg-final { check-function-bodies "**" "" } } */
+
+/*
+** foo:
+**	lw\ta[0-9]+,0\(a0\)
+**	sw\ta[0-9]+,0\(a1\)
+**	ret
+*/
+void foo (int* bar, int* baz)
+{
+  __atomic_load(bar, baz, __ATOMIC_RELAXED);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-2.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-2.c
new file mode 100644
index 00000000000..7d74841846f
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-2.c
@@ -0,0 +1,17 @@
+/* { dg-do compile } */
+/* Verify that load mappings match Table A.6's recommended mapping.  */
+/* { dg-options "-O3" } */
+/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */
+/* { dg-final { check-function-bodies "**" "" } } */
+
+/*
+** foo:
+**	lw\ta[0-9]+,0\(a0\)
+**	fence\tr,rw
+**	sw\ta[0-9]+,0\(a1\)
+**	ret
+*/
+void foo (int* bar, int* baz)
+{
+  __atomic_load(bar, baz, __ATOMIC_ACQUIRE);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-3.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-3.c
new file mode 100644
index 00000000000..ab95fa660d2
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-3.c
@@ -0,0 +1,18 @@
+/* { dg-do compile } */
+/* Verify that load mappings match Table A.6's recommended mapping.  */
+/* { dg-options "-O3" } */
+/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */
+/* { dg-final { check-function-bodies "**" "" } } */
+
+/*
+** foo:
+**	fence\trw,rw
+**	lw\ta[0-9]+,0\(a0\)
+**	fence\tr,rw
+**	sw\ta[0-9]+,0\(a1\)
+**	ret
+*/
+void foo (int* bar, int* baz)
+{
+  __atomic_load(bar, baz, __ATOMIC_SEQ_CST);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-1.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-1.c
new file mode 100644
index 00000000000..d852fddf03d
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-1.c
@@ -0,0 +1,16 @@
+/* { dg-do compile } */
+/* Verify that store mappings match Table A.6's recommended mapping.  */
+/* { dg-options "-O3" } */
+/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */
+/* { dg-final { check-function-bodies "**" "" } } */
+
+/*
+** foo:
+**	lw\ta[0-9]+,0\(a1\)
+**	sw\ta[0-9]+,0\(a0\)
+**	ret
+*/
+void foo (int* bar, int* baz)
+{
+  __atomic_store(bar, baz, __ATOMIC_RELAXED);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-2.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-2.c
new file mode 100644
index 00000000000..ccb5e2af7cc
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-2.c
@@ -0,0 +1,17 @@
+/* { dg-do compile } */
+/* Verify that store mappings match Table A.6's recommended mapping.  */
+/* { dg-options "-O3" } */
+/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */
+/* { dg-final { check-function-bodies "**" "" } } */
+
+/*
+** foo:
+**	lw\ta[0-9]+,0\(a1\)
+**	fence\trw,w
+**	sw\ta[0-9]+,0\(a0\)
+**	ret
+*/
+void foo (int* bar, int* baz)
+{
+  __atomic_store(bar, baz, __ATOMIC_RELEASE);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-compat-3.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-compat-3.c
new file mode 100644
index 00000000000..761889f18cf
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-compat-3.c
@@ -0,0 +1,18 @@
+/* { dg-do compile } */
+/* Verify that store mapping are compatible with Table A.6 & A.7.  */
+/* { dg-options "-O3" } */
+/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */
+/* { dg-final { check-function-bodies "**" "" } } */
+
+/*
+** foo:
+**	lw\ta[0-9]+,0\(a1\)
+**	fence\trw,w
+**	sw\ta[0-9]+,0\(a0\)
+**	fence\trw,rw
+**	ret
+*/
+void foo (int* bar, int* baz)
+{
+  __atomic_store(bar, baz, __ATOMIC_SEQ_CST);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-1.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-1.c
new file mode 100644
index 00000000000..d7d887dd181
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-1.c
@@ -0,0 +1,9 @@
+/* { dg-do compile } */
+/* Verify that subword atomic op mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler-times "lr.w\t" 1 } } */
+/* { dg-final { scan-assembler-times "sc.w\t" 1 } } */
+
+void foo (short* bar, short* baz)
+{
+  __atomic_add_fetch(bar, baz, __ATOMIC_RELAXED);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-2.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-2.c
new file mode 100644
index 00000000000..897bad26ebd
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-2.c
@@ -0,0 +1,9 @@
+/* { dg-do compile } */
+/* Verify that subword atomic op mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler-times "lr.w.aq\t" 1 } } */
+/* { dg-final { scan-assembler-times "sc.w\t" 1 } } */
+
+void foo (short* bar, short* baz)
+{
+  __atomic_add_fetch(bar, baz, __ATOMIC_ACQUIRE);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-3.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-3.c
new file mode 100644
index 00000000000..79efca2839a
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-3.c
@@ -0,0 +1,9 @@
+/* { dg-do compile } */
+/* Verify that subword atomic op mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler-times "lr.w\t" 1 } } */
+/* { dg-final { scan-assembler-times "sc.w.rl\t" 1 } } */
+
+void foo (short* bar, short* baz)
+{
+  __atomic_add_fetch(bar, baz, __ATOMIC_RELEASE);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-4.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-4.c
new file mode 100644
index 00000000000..772ac1be6eb
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-4.c
@@ -0,0 +1,9 @@
+/* { dg-do compile } */
+/* Verify that subword atomic op mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler-times "lr.w.aq\t" 1 } } */
+/* { dg-final { scan-assembler-times "sc.w.rl\t" 1 } } */
+
+void foo (short* bar, short* baz)
+{
+  __atomic_add_fetch(bar, baz, __ATOMIC_ACQ_REL);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-5.c b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-5.c
new file mode 100644
index 00000000000..b0bec66990e
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-5.c
@@ -0,0 +1,9 @@
+/* { dg-do compile } */
+/* Verify that subword atomic op mappings match Table A.6's recommended mapping.  */
+/* { dg-final { scan-assembler-times "lr.w.aqrl\t" 1 } } */
+/* { dg-final { scan-assembler-times "sc.w.rl\t" 1 } } */
+
+void foo (short* bar, short* baz)
+{
+  __atomic_add_fetch(bar, baz, __ATOMIC_SEQ_CST);
+}
-- 
2.34.1

[gcc13 backport 12/12] riscv: fix error: control reaches end of non-void function

[Patrick O'Neill]

From: Martin Liska <mliska@suse.cz>

Fixes:
gcc/config/riscv/sync.md:66:1: error: control reaches end of non-void function [-Werror=return-type]
66 |   [(set (attr "length") (const_int 4))])
   | ^

	PR target/109713

gcc/ChangeLog:

	* config/riscv/sync.md: Add gcc_unreachable to a switch.
---
 gcc/config/riscv/sync.md | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/gcc/config/riscv/sync.md b/gcc/config/riscv/sync.md
index 6e7c762ac57..9fc626267de 100644
--- a/gcc/config/riscv/sync.md
+++ b/gcc/config/riscv/sync.md
@@ -62,6 +62,8 @@
 	return "fence\tr,rw";
     else if (model == MEMMODEL_RELEASE)
 	return "fence\trw,w";
+    else
+	gcc_unreachable ();
   }
   [(set (attr "length") (const_int 4))])
 
-- 
2.34.1

Re: [gcc13 backport 00/12] RISC-V: Implement ISA Manual Table A.6 Mappings

[Jakub Jelinek]

On Tue, Jul 25, 2023 at 11:01:54AM -0700, Patrick O'Neill wrote:
> Discussed during the weekly RISC-V GCC meeting[1] and pre-approved by
> Jeff Law.
> If there aren't any objections I'll commit this cherry-picked series
> on Thursday (July 27th).

Please don't before 13.2 will be released, the branch is frozen and none of
this seems to be a release blocker.

	Jakub

Re: [gcc13 backport 00/12] RISC-V: Implement ISA Manual Table A.6 Mappings

[Palmer Dabbelt]

On Tue, 25 Jul 2023 11:01:54 PDT (-0700), Patrick O'Neill wrote:
> Discussed during the weekly RISC-V GCC meeting[1] and pre-approved by
> Jeff Law.
> If there aren't any objections I'll commit this cherry-picked series
> on Thursday (July 27th).

+Jakub

According to the "GCC 13.1.1 Status Report (2023-07-20)", it looks like 
we're frozen for 13.2 and thus would need a release maintainer to sign 
off on anything we backport until 13.2 is released.

I'm not opposed to the backport, but it does looks like we're down to no 
P1 regressions which means we might release very soon.  So we should at 
least make sure this gets through all the tests and such.  It's kind of 
splitting hairs as this is a pretty bad set of bugs we're fixing and 
distros are probably going to just backport it anyway, so not sure what 
the right answer is.

> Patchset on trunk:
> https://inbox.sourceware.org/gcc-patches/20230427162301.1151333-1-patrick@rivosinc.com/
> First commit: f37a36bce81b50a43ec1613c1d08d803642f7506
>
> Also includes bugfix from:
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109713
> commit: 4bd434fbfc7865961a8e10d7e9601b28765ce7be
>
> [1] https://inbox.sourceware.org/gcc/mhng-b7423fca-67ec-4ce4-9694-4e062632ceb0@palmer-ri-x1c9/T/#t
>
> Martin Liska (1):
>   riscv: fix error: control reaches end of non-void function
>
> Patrick O'Neill (11):
>   RISC-V: Eliminate SYNC memory models
>   RISC-V: Enforce Libatomic LR/SC SEQ_CST
>   RISC-V: Enforce subword atomic LR/SC SEQ_CST
>   RISC-V: Enforce atomic compare_exchange SEQ_CST
>   RISC-V: Add AMO release bits
>   RISC-V: Strengthen atomic stores
>   RISC-V: Eliminate AMO op fences
>   RISC-V: Weaken LR/SC pairs
>   RISC-V: Weaken mem_thread_fence
>   RISC-V: Weaken atomic loads
>   RISC-V: Table A.6 conformance tests
>
>  gcc/config/riscv/riscv-protos.h               |   3 +
>  gcc/config/riscv/riscv.cc                     |  66 ++++--
>  gcc/config/riscv/sync.md                      | 196 ++++++++++++------
>  .../riscv/amo-table-a-6-amo-add-1.c           |  15 ++
>  .../riscv/amo-table-a-6-amo-add-2.c           |  15 ++
>  .../riscv/amo-table-a-6-amo-add-3.c           |  15 ++
>  .../riscv/amo-table-a-6-amo-add-4.c           |  15 ++
>  .../riscv/amo-table-a-6-amo-add-5.c           |  15 ++
>  .../riscv/amo-table-a-6-compare-exchange-1.c  |   9 +
>  .../riscv/amo-table-a-6-compare-exchange-2.c  |   9 +
>  .../riscv/amo-table-a-6-compare-exchange-3.c  |   9 +
>  .../riscv/amo-table-a-6-compare-exchange-4.c  |   9 +
>  .../riscv/amo-table-a-6-compare-exchange-5.c  |   9 +
>  .../riscv/amo-table-a-6-compare-exchange-6.c  |  10 +
>  .../riscv/amo-table-a-6-compare-exchange-7.c  |   9 +
>  .../gcc.target/riscv/amo-table-a-6-fence-1.c  |  14 ++
>  .../gcc.target/riscv/amo-table-a-6-fence-2.c  |  15 ++
>  .../gcc.target/riscv/amo-table-a-6-fence-3.c  |  15 ++
>  .../gcc.target/riscv/amo-table-a-6-fence-4.c  |  15 ++
>  .../gcc.target/riscv/amo-table-a-6-fence-5.c  |  15 ++
>  .../gcc.target/riscv/amo-table-a-6-load-1.c   |  16 ++
>  .../gcc.target/riscv/amo-table-a-6-load-2.c   |  17 ++
>  .../gcc.target/riscv/amo-table-a-6-load-3.c   |  18 ++
>  .../gcc.target/riscv/amo-table-a-6-store-1.c  |  16 ++
>  .../gcc.target/riscv/amo-table-a-6-store-2.c  |  17 ++
>  .../riscv/amo-table-a-6-store-compat-3.c      |  18 ++
>  .../riscv/amo-table-a-6-subword-amo-add-1.c   |   9 +
>  .../riscv/amo-table-a-6-subword-amo-add-2.c   |   9 +
>  .../riscv/amo-table-a-6-subword-amo-add-3.c   |   9 +
>  .../riscv/amo-table-a-6-subword-amo-add-4.c   |   9 +
>  .../riscv/amo-table-a-6-subword-amo-add-5.c   |   9 +
>  gcc/testsuite/gcc.target/riscv/pr89835.c      |   9 +
>  libgcc/config/riscv/atomic.c                  |   4 +-
>  33 files changed, 563 insertions(+), 75 deletions(-)
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-1.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-2.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-3.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-4.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-5.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-1.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-2.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-3.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-4.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-5.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-6.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-7.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-1.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-2.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-3.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-4.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-5.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-1.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-2.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-3.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-1.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-2.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-compat-3.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-1.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-2.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-3.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-4.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-5.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/pr89835.c

Re: [gcc13 backport 00/12] RISC-V: Implement ISA Manual Table A.6 Mappings

[Palmer Dabbelt]

On Tue, 25 Jul 2023 12:50:48 PDT (-0700), jakub@redhat.com wrote:
> On Tue, Jul 25, 2023 at 11:01:54AM -0700, Patrick O'Neill wrote:
>> Discussed during the weekly RISC-V GCC meeting[1] and pre-approved by
>> Jeff Law.
>> If there aren't any objections I'll commit this cherry-picked series
>> on Thursday (July 27th).
>
> Please don't before 13.2 will be released, the branch is frozen and none of
> this seems to be a release blocker.

Sorry I missed this.  IMO it's fine to wait, this has been broken for 
5-10 years so we can wait another cycle ;)

>
> 	Jakub

Re: [gcc13 backport 00/12] RISC-V: Implement ISA Manual Table A.6 Mappings

[Jeff Law]

On 7/25/23 13:50, Jakub Jelinek wrote:
> On Tue, Jul 25, 2023 at 11:01:54AM -0700, Patrick O'Neill wrote:
>> Discussed during the weekly RISC-V GCC meeting[1] and pre-approved by
>> Jeff Law.
>> If there aren't any objections I'll commit this cherry-picked series
>> on Thursday (July 27th).
> 
> Please don't before 13.2 will be released, the branch is frozen and none of
> this seems to be a release blocker.
Ugh.  Missed the boat :(

I could make an argument for inclusion given the strong desire to have 
compatible mappings across the toolchains and alignment with the RVI 
specs -- but I won't.  As Palmer has indicated, it's been broken for a 
while and we can manage that breakage.




jeff

Re: [gcc13 backport 00/12] RISC-V: Implement ISA Manual Table A.6 Mappings

[Palmer Dabbelt]

On Tue, 25 Jul 2023 14:02:24 PDT (-0700), jeffreyalaw@gmail.com wrote:
>
>
> On 7/25/23 13:50, Jakub Jelinek wrote:
>> On Tue, Jul 25, 2023 at 11:01:54AM -0700, Patrick O'Neill wrote:
>>> Discussed during the weekly RISC-V GCC meeting[1] and pre-approved by
>>> Jeff Law.
>>> If there aren't any objections I'll commit this cherry-picked series
>>> on Thursday (July 27th).
>>
>> Please don't before 13.2 will be released, the branch is frozen and none of
>> this seems to be a release blocker.
> Ugh.  Missed the boat :(
>
> I could make an argument for inclusion given the strong desire to have
> compatible mappings across the toolchains and alignment with the RVI
> specs -- but I won't.  As Palmer has indicated, it's been broken for a
> while and we can manage that breakage.

I think if we just merge it right after 13.2 and indicate that distros 
doing long-term binary builds before 13.3 backport the patches we should 
be fine.  I think that's just Debian right now, so while it's an 
important set of bugs to get fixed it's just the single user.

It's certainly a bummer to miss 13.2, but we've just got ourselves to 
blame for forgetting about the backport ;)

>
>
>
>
> jeff

Re: [gcc13 backport 12/12] riscv: fix error: control reaches end of non-void function

[Kito Cheng]

OK for backport :)

On Wed, Jul 26, 2023 at 2:11 AM Patrick O'Neill <patrick@rivosinc.com> wrote:
>
> From: Martin Liska <mliska@suse.cz>
>
> Fixes:
> gcc/config/riscv/sync.md:66:1: error: control reaches end of non-void function [-Werror=return-type]
> 66 |   [(set (attr "length") (const_int 4))])
>    | ^
>
>         PR target/109713
>
> gcc/ChangeLog:
>
>         * config/riscv/sync.md: Add gcc_unreachable to a switch.
> ---
>  gcc/config/riscv/sync.md | 2 ++
>  1 file changed, 2 insertions(+)
>
> diff --git a/gcc/config/riscv/sync.md b/gcc/config/riscv/sync.md
> index 6e7c762ac57..9fc626267de 100644
> --- a/gcc/config/riscv/sync.md
> +++ b/gcc/config/riscv/sync.md
> @@ -62,6 +62,8 @@
>         return "fence\tr,rw";
>      else if (model == MEMMODEL_RELEASE)
>         return "fence\trw,w";
> +    else
> +       gcc_unreachable ();
>    }
>    [(set (attr "length") (const_int 4))])
>
> --
> 2.34.1
>

Re: [gcc13 backport 12/12] riscv: fix error: control reaches end of non-void function

[Patrick O'Neill]

This final patch fixes an error introduced by patch 9/12 in this series.
I'll backport alongside the other patches once the 13 branch is unfrozen :)

On 7/25/23 18:22, Kito Cheng wrote:
> OK for backport :)
>
> On Wed, Jul 26, 2023 at 2:11 AM Patrick O'Neill <patrick@rivosinc.com> wrote:
>> From: Martin Liska <mliska@suse.cz>
>>
>> Fixes:
>> gcc/config/riscv/sync.md:66:1: error: control reaches end of non-void function [-Werror=return-type]
>> 66 |   [(set (attr "length") (const_int 4))])
>>     | ^
>>
>>          PR target/109713
>>
>> gcc/ChangeLog:
>>
>>          * config/riscv/sync.md: Add gcc_unreachable to a switch.
>> ---
>>   gcc/config/riscv/sync.md | 2 ++
>>   1 file changed, 2 insertions(+)
>>
>> diff --git a/gcc/config/riscv/sync.md b/gcc/config/riscv/sync.md
>> index 6e7c762ac57..9fc626267de 100644
>> --- a/gcc/config/riscv/sync.md
>> +++ b/gcc/config/riscv/sync.md
>> @@ -62,6 +62,8 @@
>>          return "fence\tr,rw";
>>       else if (model == MEMMODEL_RELEASE)
>>          return "fence\trw,w";
>> +    else
>> +       gcc_unreachable ();
>>     }
>>     [(set (attr "length") (const_int 4))])
>>
>> --
>> 2.34.1
>>

[Committed] RISC-V: Implement ISA Manual Table A.6 Mappings

[Patrick O'Neill]

GCC 13.2 released[2] so I merged the series now that the branch is unfrozen.

Thanks,
Patrick

[2] https://inbox.sourceware.org/gcc/ZMJeq%2FY5SN+7i8a+@tucnak/T/#u

On 7/25/23 11:01, Patrick O'Neill wrote:
> Discussed during the weekly RISC-V GCC meeting[1] and pre-approved by
> Jeff Law.
> If there aren't any objections I'll commit this cherry-picked series
> on Thursday (July 27th).
>
> Patchset on trunk:
> https://inbox.sourceware.org/gcc-patches/20230427162301.1151333-1-patrick@rivosinc.com/
> First commit: f37a36bce81b50a43ec1613c1d08d803642f7506
>
> Also includes bugfix from:
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109713
> commit: 4bd434fbfc7865961a8e10d7e9601b28765ce7be
>
> [1] https://inbox.sourceware.org/gcc/mhng-b7423fca-67ec-4ce4-9694-4e062632ceb0@palmer-ri-x1c9/T/#t
>
> Martin Liska (1):
>    riscv: fix error: control reaches end of non-void function
>
> Patrick O'Neill (11):
>    RISC-V: Eliminate SYNC memory models
>    RISC-V: Enforce Libatomic LR/SC SEQ_CST
>    RISC-V: Enforce subword atomic LR/SC SEQ_CST
>    RISC-V: Enforce atomic compare_exchange SEQ_CST
>    RISC-V: Add AMO release bits
>    RISC-V: Strengthen atomic stores
>    RISC-V: Eliminate AMO op fences
>    RISC-V: Weaken LR/SC pairs
>    RISC-V: Weaken mem_thread_fence
>    RISC-V: Weaken atomic loads
>    RISC-V: Table A.6 conformance tests
>
>   gcc/config/riscv/riscv-protos.h               |   3 +
>   gcc/config/riscv/riscv.cc                     |  66 ++++--
>   gcc/config/riscv/sync.md                      | 196 ++++++++++++------
>   .../riscv/amo-table-a-6-amo-add-1.c           |  15 ++
>   .../riscv/amo-table-a-6-amo-add-2.c           |  15 ++
>   .../riscv/amo-table-a-6-amo-add-3.c           |  15 ++
>   .../riscv/amo-table-a-6-amo-add-4.c           |  15 ++
>   .../riscv/amo-table-a-6-amo-add-5.c           |  15 ++
>   .../riscv/amo-table-a-6-compare-exchange-1.c  |   9 +
>   .../riscv/amo-table-a-6-compare-exchange-2.c  |   9 +
>   .../riscv/amo-table-a-6-compare-exchange-3.c  |   9 +
>   .../riscv/amo-table-a-6-compare-exchange-4.c  |   9 +
>   .../riscv/amo-table-a-6-compare-exchange-5.c  |   9 +
>   .../riscv/amo-table-a-6-compare-exchange-6.c  |  10 +
>   .../riscv/amo-table-a-6-compare-exchange-7.c  |   9 +
>   .../gcc.target/riscv/amo-table-a-6-fence-1.c  |  14 ++
>   .../gcc.target/riscv/amo-table-a-6-fence-2.c  |  15 ++
>   .../gcc.target/riscv/amo-table-a-6-fence-3.c  |  15 ++
>   .../gcc.target/riscv/amo-table-a-6-fence-4.c  |  15 ++
>   .../gcc.target/riscv/amo-table-a-6-fence-5.c  |  15 ++
>   .../gcc.target/riscv/amo-table-a-6-load-1.c   |  16 ++
>   .../gcc.target/riscv/amo-table-a-6-load-2.c   |  17 ++
>   .../gcc.target/riscv/amo-table-a-6-load-3.c   |  18 ++
>   .../gcc.target/riscv/amo-table-a-6-store-1.c  |  16 ++
>   .../gcc.target/riscv/amo-table-a-6-store-2.c  |  17 ++
>   .../riscv/amo-table-a-6-store-compat-3.c      |  18 ++
>   .../riscv/amo-table-a-6-subword-amo-add-1.c   |   9 +
>   .../riscv/amo-table-a-6-subword-amo-add-2.c   |   9 +
>   .../riscv/amo-table-a-6-subword-amo-add-3.c   |   9 +
>   .../riscv/amo-table-a-6-subword-amo-add-4.c   |   9 +
>   .../riscv/amo-table-a-6-subword-amo-add-5.c   |   9 +
>   gcc/testsuite/gcc.target/riscv/pr89835.c      |   9 +
>   libgcc/config/riscv/atomic.c                  |   4 +-
>   33 files changed, 563 insertions(+), 75 deletions(-)
>   create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-1.c
>   create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-2.c
>   create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-3.c
>   create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-4.c
>   create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-amo-add-5.c
>   create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-1.c
>   create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-2.c
>   create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-3.c
>   create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-4.c
>   create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-5.c
>   create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-6.c
>   create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-compare-exchange-7.c
>   create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-1.c
>   create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-2.c
>   create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-3.c
>   create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-4.c
>   create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-fence-5.c
>   create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-1.c
>   create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-2.c
>   create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-load-3.c
>   create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-1.c
>   create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-2.c
>   create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-store-compat-3.c
>   create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-1.c
>   create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-2.c
>   create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-3.c
>   create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-4.c
>   create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-a-6-subword-amo-add-5.c
>   create mode 100644 gcc/testsuite/gcc.target/riscv/pr89835.c
>