[PATCH] RISC-V: Add support for subword atomic loads/stores

[PATCH] RISC-V: Add support for subword atomic loads/stores

[Patrick O'Neill]

Andrea Parri recently pointed out that we were emitting overly conservative
fences for seq_cst atomic loads/stores. This adds support for the optimized
fences specified in the PSABI:
https://github.com/riscv-non-isa/riscv-elf-psabi-doc/blob/2092568f7896ceaa1ec0f02569b19eaa42cd51c9/riscv-atomic.adoc

gcc/ChangeLog:

	* config/riscv/sync-rvwmo.md: Add support for subword fenced
	loads/stores.
	* config/riscv/sync-ztso.md: Ditto.
	* config/riscv/sync.md: Ditto.

gcc/testsuite/ChangeLog:

	* gcc.target/riscv/amo/amo-table-a-6-load-1.c: Increase test coverage to
	include longs, shorts, chars, and bools.
	* gcc.target/riscv/amo/amo-table-a-6-load-2.c: Ditto.
	* gcc.target/riscv/amo/amo-table-a-6-load-3.c: Ditto.
	* gcc.target/riscv/amo/amo-table-a-6-store-1.c: Ditto.
	* gcc.target/riscv/amo/amo-table-a-6-store-2.c: Ditto.
	* gcc.target/riscv/amo/amo-table-a-6-store-compat-3.c: Ditto.
	* gcc.target/riscv/amo/amo-table-ztso-load-1.c: Ditto.
	* gcc.target/riscv/amo/amo-table-ztso-load-2.c: Ditto.
	* gcc.target/riscv/amo/amo-table-ztso-load-3.c: Ditto.
	* gcc.target/riscv/amo/amo-table-ztso-store-1.c: Ditto.
	* gcc.target/riscv/amo/amo-table-ztso-store-2.c: Ditto.
	* gcc.target/riscv/amo/amo-table-ztso-store-3.c: Ditto.

Signed-off-by: Patrick O'Neill <patrick@rivosinc.com>
---
Andrea Parri's comment: https://inbox.sourceware.org/gcc-patches/ZmeXGRF+zDktFqaY@andrea/
---
 gcc/config/riscv/sync-rvwmo.md                | 24 ++++----
 gcc/config/riscv/sync-ztso.md                 | 20 +++----
 gcc/config/riscv/sync.md                      |  8 +--
 .../riscv/amo/amo-table-a-6-load-1.c          | 48 +++++++++++++++-
 .../riscv/amo/amo-table-a-6-load-2.c          | 52 ++++++++++++++++-
 .../riscv/amo/amo-table-a-6-load-3.c          | 56 ++++++++++++++++++-
 .../riscv/amo/amo-table-a-6-store-1.c         | 48 +++++++++++++++-
 .../riscv/amo/amo-table-a-6-store-2.c         | 52 ++++++++++++++++-
 .../riscv/amo/amo-table-a-6-store-compat-3.c  | 56 ++++++++++++++++++-
 .../riscv/amo/amo-table-ztso-load-1.c         | 48 +++++++++++++++-
 .../riscv/amo/amo-table-ztso-load-2.c         | 48 +++++++++++++++-
 .../riscv/amo/amo-table-ztso-load-3.c         | 52 ++++++++++++++++-
 .../riscv/amo/amo-table-ztso-store-1.c        | 48 +++++++++++++++-
 .../riscv/amo/amo-table-ztso-store-2.c        | 48 +++++++++++++++-
 .../riscv/amo/amo-table-ztso-store-3.c        | 52 ++++++++++++++++-
 15 files changed, 610 insertions(+), 50 deletions(-)

diff --git a/gcc/config/riscv/sync-rvwmo.md b/gcc/config/riscv/sync-rvwmo.md
index e639a1e2392..5db94c8c27f 100644
--- a/gcc/config/riscv/sync-rvwmo.md
+++ b/gcc/config/riscv/sync-rvwmo.md
@@ -47,9 +47,9 @@
 ;; Atomic memory operations.

 (define_insn "atomic_load_rvwmo<mode>"
-  [(set (match_operand:GPR 0 "register_operand" "=r")
-	(unspec_volatile:GPR
-	    [(match_operand:GPR 1 "memory_operand" "A")
+  [(set (match_operand:ANYI 0 "register_operand" "=r")
+	(unspec_volatile:ANYI
+	    [(match_operand:ANYI 1 "memory_operand" "A")
 	     (match_operand:SI 2 "const_int_operand")]  ;; model
 	 UNSPEC_ATOMIC_LOAD))]
   "!TARGET_ZTSO"
@@ -59,13 +59,13 @@

     if (model == MEMMODEL_SEQ_CST)
       return "fence\trw,rw\;"
-	     "l<amo>\t%0,%1\;"
+	     "<load>\t%0,%1\;"
 	     "fence\tr,rw";
     if (model == MEMMODEL_ACQUIRE)
-      return "l<amo>\t%0,%1\;"
+      return "<load>\t%0,%1\;"
 	     "fence\tr,rw";
     else
-      return "l<amo>\t%0,%1";
+      return "<load>\t%0,%1";
   }
   [(set_attr "type" "multi")
    (set (attr "length") (const_int 12))])
@@ -73,9 +73,9 @@
 ;; Implement atomic stores with conservative fences.
 ;; This allows us to be compatible with the ISA manual Table A.6 and Table A.7.
 (define_insn "atomic_store_rvwmo<mode>"
-  [(set (match_operand:GPR 0 "memory_operand" "=A")
-	(unspec_volatile:GPR
-	    [(match_operand:GPR 1 "reg_or_0_operand" "rJ")
+  [(set (match_operand:ANYI 0 "memory_operand" "=A")
+	(unspec_volatile:ANYI
+	    [(match_operand:ANYI 1 "reg_or_0_operand" "rJ")
 	     (match_operand:SI 2 "const_int_operand")]  ;; model
 	 UNSPEC_ATOMIC_STORE))]
   "!TARGET_ZTSO"
@@ -85,13 +85,13 @@

     if (model == MEMMODEL_SEQ_CST)
       return "fence\trw,w\;"
-	     "s<amo>\t%z1,%0\;"
+	     "<store>\t%z1,%0\;"
 	     "fence\trw,rw";
     if (model == MEMMODEL_RELEASE)
       return "fence\trw,w\;"
-	     "s<amo>\t%z1,%0";
+	     "<store>\t%z1,%0";
     else
-      return "s<amo>\t%z1,%0";
+      return "<store>\t%z1,%0";
   }
   [(set_attr "type" "multi")
    (set (attr "length") (const_int 12))])
diff --git a/gcc/config/riscv/sync-ztso.md b/gcc/config/riscv/sync-ztso.md
index 0a866d2906b..f99a21b45ca 100644
--- a/gcc/config/riscv/sync-ztso.md
+++ b/gcc/config/riscv/sync-ztso.md
@@ -41,9 +41,9 @@
 ;; Atomic memory operations.

 (define_insn "atomic_load_ztso<mode>"
-  [(set (match_operand:GPR 0 "register_operand" "=r")
-	(unspec_volatile:GPR
-	    [(match_operand:GPR 1 "memory_operand" "A")
+  [(set (match_operand:ANYI 0 "register_operand" "=r")
+	(unspec_volatile:ANYI
+	    [(match_operand:ANYI 1 "memory_operand" "A")
 	     (match_operand:SI 2 "const_int_operand")]  ;; model
 	 UNSPEC_ATOMIC_LOAD))]
   "TARGET_ZTSO"
@@ -53,17 +53,17 @@

     if (model == MEMMODEL_SEQ_CST)
       return "fence\trw,rw\;"
-	     "l<amo>\t%0,%1";
+	     "<load>\t%0,%1";
     else
-      return "l<amo>\t%0,%1";
+      return "<load>\t%0,%1";
   }
   [(set_attr "type" "multi")
    (set (attr "length") (const_int 12))])

 (define_insn "atomic_store_ztso<mode>"
-  [(set (match_operand:GPR 0 "memory_operand" "=A")
-	(unspec_volatile:GPR
-	    [(match_operand:GPR 1 "reg_or_0_operand" "rJ")
+  [(set (match_operand:ANYI 0 "memory_operand" "=A")
+	(unspec_volatile:ANYI
+	    [(match_operand:ANYI 1 "reg_or_0_operand" "rJ")
 	     (match_operand:SI 2 "const_int_operand")]  ;; model
 	 UNSPEC_ATOMIC_STORE))]
   "TARGET_ZTSO"
@@ -72,10 +72,10 @@
     model = memmodel_base (model);

     if (model == MEMMODEL_SEQ_CST)
-      return "s<amo>\t%z1,%0\;"
+      return "<store>\t%z1,%0\;"
 	     "fence\trw,rw";
     else
-      return "s<amo>\t%z1,%0";
+      return "<store>\t%z1,%0";
   }
   [(set_attr "type" "multi")
    (set (attr "length") (const_int 8))])
diff --git a/gcc/config/riscv/sync.md b/gcc/config/riscv/sync.md
index 4df9d0b5a5f..4784375449f 100644
--- a/gcc/config/riscv/sync.md
+++ b/gcc/config/riscv/sync.md
@@ -57,8 +57,8 @@
 ;; Atomic memory operations.

 (define_expand "atomic_load<mode>"
-  [(match_operand:GPR 0 "register_operand")
-   (match_operand:GPR 1 "memory_operand")
+  [(match_operand:ANYI 0 "register_operand")
+   (match_operand:ANYI 1 "memory_operand")
    (match_operand:SI 2 "const_int_operand")] ;; model
   ""
   {
@@ -72,8 +72,8 @@
   })

 (define_expand "atomic_store<mode>"
-  [(match_operand:GPR 0 "memory_operand")
-   (match_operand:GPR 1 "reg_or_0_operand")
+  [(match_operand:ANYI 0 "memory_operand")
+   (match_operand:ANYI 1 "reg_or_0_operand")
    (match_operand:SI 2 "const_int_operand")] ;; model
   ""
   {
diff --git a/gcc/testsuite/gcc.target/riscv/amo/amo-table-a-6-load-1.c b/gcc/testsuite/gcc.target/riscv/amo/amo-table-a-6-load-1.c
index 53dd5234452..dc98c7d5ed1 100644
--- a/gcc/testsuite/gcc.target/riscv/amo/amo-table-a-6-load-1.c
+++ b/gcc/testsuite/gcc.target/riscv/amo/amo-table-a-6-load-1.c
@@ -5,12 +5,56 @@
 /* { dg-final { check-function-bodies "**" "" } } */

 /*
-** foo:
+** atomic_load_long_relaxed:
+**	ld\t[atx][0-9]+,0\(a0\)
+**	sd\t[atx][0-9]+,0\(a1\)
+**	ret
+*/
+void atomic_load_long_relaxed (long* bar, long* baz)
+{
+  __atomic_load(bar, baz, __ATOMIC_RELAXED);
+}
+
+/*
+** atomic_load_int_relaxed:
 **	lw\t[atx][0-9]+,0\(a0\)
 **	sw\t[atx][0-9]+,0\(a1\)
 **	ret
 */
-void foo (int* bar, int* baz)
+void atomic_load_int_relaxed (int* bar, int* baz)
+{
+  __atomic_load(bar, baz, __ATOMIC_RELAXED);
+}
+
+/*
+** atomic_load_short_relaxed:
+**	lh\t[atx][0-9]+,0\(a0\)
+**	sh\t[atx][0-9]+,0\(a1\)
+**	ret
+*/
+void atomic_load_short_relaxed (short* bar, short* baz)
+{
+  __atomic_load(bar, baz, __ATOMIC_RELAXED);
+}
+
+/*
+** atomic_load_char_relaxed:
+**	lb\t[atx][0-9]+,0\(a0\)
+**	sb\t[atx][0-9]+,0\(a1\)
+**	ret
+*/
+void atomic_load_char_relaxed (char* bar, char* baz)
+{
+  __atomic_load(bar, baz, __ATOMIC_RELAXED);
+}
+
+/*
+** atomic_load_bool_relaxed:
+**	lb\t[atx][0-9]+,0\(a0\)
+**	sb\t[atx][0-9]+,0\(a1\)
+**	ret
+*/
+void atomic_load_bool_relaxed (_Bool* bar, _Bool* baz)
 {
   __atomic_load(bar, baz, __ATOMIC_RELAXED);
 }
diff --git a/gcc/testsuite/gcc.target/riscv/amo/amo-table-a-6-load-2.c b/gcc/testsuite/gcc.target/riscv/amo/amo-table-a-6-load-2.c
index dda0f541515..abb841ddad6 100644
--- a/gcc/testsuite/gcc.target/riscv/amo/amo-table-a-6-load-2.c
+++ b/gcc/testsuite/gcc.target/riscv/amo/amo-table-a-6-load-2.c
@@ -5,13 +5,61 @@
 /* { dg-final { check-function-bodies "**" "" } } */

 /*
-** foo:
+** atomic_load_long_acquire:
+**	ld\t[atx][0-9]+,0\(a0\)
+**	fence\tr,rw
+**	sd\t[atx][0-9]+,0\(a1\)
+**	ret
+*/
+void atomic_load_long_acquire (long* bar, long* baz)
+{
+  __atomic_load(bar, baz, __ATOMIC_ACQUIRE);
+}
+
+/*
+** atomic_load_int_acquire:
 **	lw\t[atx][0-9]+,0\(a0\)
 **	fence\tr,rw
 **	sw\t[atx][0-9]+,0\(a1\)
 **	ret
 */
-void foo (int* bar, int* baz)
+void atomic_load_int_acquire (int* bar, int* baz)
+{
+  __atomic_load(bar, baz, __ATOMIC_ACQUIRE);
+}
+
+/*
+** atomic_load_short_acquire:
+**	lh\t[atx][0-9]+,0\(a0\)
+**	fence\tr,rw
+**	sh\t[atx][0-9]+,0\(a1\)
+**	ret
+*/
+void atomic_load_short_acquire (short* bar, short* baz)
+{
+  __atomic_load(bar, baz, __ATOMIC_ACQUIRE);
+}
+
+/*
+** atomic_load_char_acquire:
+**	lb\t[atx][0-9]+,0\(a0\)
+**	fence\tr,rw
+**	sb\t[atx][0-9]+,0\(a1\)
+**	ret
+*/
+void atomic_load_char_acquire (char* bar, char* baz)
+{
+  __atomic_load(bar, baz, __ATOMIC_ACQUIRE);
+}
+
+/*
+** atomic_load_bool_acquire:
+**	lb\t[atx][0-9]+,0\(a0\)
+**	fence\tr,rw
+**	sb\t[atx][0-9]+,0\(a1\)
+**	ret
+*/
+void atomic_load_bool_acquire (_Bool* bar, _Bool* baz)
 {
   __atomic_load(bar, baz, __ATOMIC_ACQUIRE);
 }
diff --git a/gcc/testsuite/gcc.target/riscv/amo/amo-table-a-6-load-3.c b/gcc/testsuite/gcc.target/riscv/amo/amo-table-a-6-load-3.c
index 3279557fa4a..654f890aac0 100644
--- a/gcc/testsuite/gcc.target/riscv/amo/amo-table-a-6-load-3.c
+++ b/gcc/testsuite/gcc.target/riscv/amo/amo-table-a-6-load-3.c
@@ -5,14 +5,66 @@
 /* { dg-final { check-function-bodies "**" "" } } */

 /*
-** foo:
+** atomic_load_long_seq_cst:
+**	fence\trw,rw
+**	ld\t[atx][0-9]+,0\(a0\)
+**	fence\tr,rw
+**	sd\t[atx][0-9]+,0\(a1\)
+**	ret
+*/
+void atomic_load_long_seq_cst (long* bar, long* baz)
+{
+  __atomic_load(bar, baz, __ATOMIC_SEQ_CST);
+}
+
+/*
+** atomic_load_int_seq_cst:
 **	fence\trw,rw
 **	lw\t[atx][0-9]+,0\(a0\)
 **	fence\tr,rw
 **	sw\t[atx][0-9]+,0\(a1\)
 **	ret
 */
-void foo (int* bar, int* baz)
+void atomic_load_int_seq_cst (int* bar, int* baz)
+{
+  __atomic_load(bar, baz, __ATOMIC_SEQ_CST);
+}
+
+/*
+** atomic_load_short_seq_cst:
+**	fence\trw,rw
+**	lh\t[atx][0-9]+,0\(a0\)
+**	fence\tr,rw
+**	sh\t[atx][0-9]+,0\(a1\)
+**	ret
+*/
+void atomic_load_short_seq_cst (short* bar, short* baz)
+{
+  __atomic_load(bar, baz, __ATOMIC_SEQ_CST);
+}
+
+/*
+** atomic_load_char_seq_cst:
+**	fence\trw,rw
+**	lb\t[atx][0-9]+,0\(a0\)
+**	fence\tr,rw
+**	sb\t[atx][0-9]+,0\(a1\)
+**	ret
+*/
+void atomic_load_char_seq_cst (char* bar, char* baz)
+{
+  __atomic_load(bar, baz, __ATOMIC_SEQ_CST);
+}
+
+/*
+** atomic_load_bool_seq_cst:
+**	fence\trw,rw
+**	lb\t[atx][0-9]+,0\(a0\)
+**	fence\tr,rw
+**	sb\t[atx][0-9]+,0\(a1\)
+**	ret
+*/
+void atomic_load_bool_seq_cst (_Bool* bar, _Bool* baz)
 {
   __atomic_load(bar, baz, __ATOMIC_SEQ_CST);
 }
diff --git a/gcc/testsuite/gcc.target/riscv/amo/amo-table-a-6-store-1.c b/gcc/testsuite/gcc.target/riscv/amo/amo-table-a-6-store-1.c
index 6b05429520b..a1a33ccb454 100644
--- a/gcc/testsuite/gcc.target/riscv/amo/amo-table-a-6-store-1.c
+++ b/gcc/testsuite/gcc.target/riscv/amo/amo-table-a-6-store-1.c
@@ -5,12 +5,56 @@
 /* { dg-final { check-function-bodies "**" "" } } */

 /*
-** foo:
+** atomic_store_long_relaxed:
+**	ld\t[atx][0-9]+,0\(a1\)
+**	sd\t[atx][0-9]+,0\(a0\)
+**	ret
+*/
+void atomic_store_long_relaxed (long* bar, long* baz)
+{
+  __atomic_store(bar, baz, __ATOMIC_RELAXED);
+}
+
+/*
+** atomic_store_int_relaxed:
 **	lw\t[atx][0-9]+,0\(a1\)
 **	sw\t[atx][0-9]+,0\(a0\)
 **	ret
 */
-void foo (int* bar, int* baz)
+void atomic_store_int_relaxed (int* bar, int* baz)
+{
+  __atomic_store(bar, baz, __ATOMIC_RELAXED);
+}
+
+/*
+** atomic_store_short_relaxed:
+**	lhu\t[atx][0-9]+,0\(a1\)
+**	sh\t[atx][0-9]+,0\(a0\)
+**	ret
+*/
+void atomic_store_short_relaxed (short* bar, short* baz)
+{
+  __atomic_store(bar, baz, __ATOMIC_RELAXED);
+}
+
+/*
+** atomic_store_char_relaxed:
+**	lbu\t[atx][0-9]+,0\(a1\)
+**	sb\t[atx][0-9]+,0\(a0\)
+**	ret
+*/
+void atomic_store_char_relaxed (char* bar, char* baz)
+{
+  __atomic_store(bar, baz, __ATOMIC_RELAXED);
+}
+
+/*
+** atomic_store_bool_relaxed:
+**	lbu\t[atx][0-9]+,0\(a1\)
+**	sb\t[atx][0-9]+,0\(a0\)
+**	ret
+*/
+void atomic_store_bool_relaxed (_Bool* bar, _Bool* baz)
 {
   __atomic_store(bar, baz, __ATOMIC_RELAXED);
 }
diff --git a/gcc/testsuite/gcc.target/riscv/amo/amo-table-a-6-store-2.c b/gcc/testsuite/gcc.target/riscv/amo/amo-table-a-6-store-2.c
index 1ad7dede931..5c9451fa324 100644
--- a/gcc/testsuite/gcc.target/riscv/amo/amo-table-a-6-store-2.c
+++ b/gcc/testsuite/gcc.target/riscv/amo/amo-table-a-6-store-2.c
@@ -5,13 +5,61 @@
 /* { dg-final { check-function-bodies "**" "" } } */

 /*
-** foo:
+** atomic_store_long_release:
+**	ld\t[atx][0-9]+,0\(a1\)
+**	fence\trw,w
+**	sd\t[atx][0-9]+,0\(a0\)
+**	ret
+*/
+void atomic_store_long_release (long* bar, long* baz)
+{
+  __atomic_store(bar, baz, __ATOMIC_RELEASE);
+}
+
+/*
+** atomic_store_int_release:
 **	lw\t[atx][0-9]+,0\(a1\)
 **	fence\trw,w
 **	sw\t[atx][0-9]+,0\(a0\)
 **	ret
 */
-void foo (int* bar, int* baz)
+void atomic_store_int_release (int* bar, int* baz)
+{
+  __atomic_store(bar, baz, __ATOMIC_RELEASE);
+}
+
+/*
+** atomic_store_short_release:
+**	lhu\t[atx][0-9]+,0\(a1\)
+**	fence\trw,w
+**	sh\t[atx][0-9]+,0\(a0\)
+**	ret
+*/
+void atomic_store_short_release (short* bar, short* baz)
+{
+  __atomic_store(bar, baz, __ATOMIC_RELEASE);
+}
+
+/*
+** atomic_store_char_release:
+**	lbu\t[atx][0-9]+,0\(a1\)
+**	fence\trw,w
+**	sb\t[atx][0-9]+,0\(a0\)
+**	ret
+*/
+void atomic_store_char_release (char* bar, char* baz)
+{
+  __atomic_store(bar, baz, __ATOMIC_RELEASE);
+}
+
+/*
+** atomic_store_bool_release:
+**	lbu\t[atx][0-9]+,0\(a1\)
+**	fence\trw,w
+**	sb\t[atx][0-9]+,0\(a0\)
+**	ret
+*/
+void atomic_store_bool_release (_Bool* bar, _Bool* baz)
 {
   __atomic_store(bar, baz, __ATOMIC_RELEASE);
 }
diff --git a/gcc/testsuite/gcc.target/riscv/amo/amo-table-a-6-store-compat-3.c b/gcc/testsuite/gcc.target/riscv/amo/amo-table-a-6-store-compat-3.c
index b16b2058413..52d53b9104f 100644
--- a/gcc/testsuite/gcc.target/riscv/amo/amo-table-a-6-store-compat-3.c
+++ b/gcc/testsuite/gcc.target/riscv/amo/amo-table-a-6-store-compat-3.c
@@ -5,14 +5,66 @@
 /* { dg-final { check-function-bodies "**" "" } } */

 /*
-** foo:
+** atomic_store_long_seq_cst:
+**	ld\t[atx][0-9]+,0\(a1\)
+**	fence\trw,w
+**	sd\t[atx][0-9]+,0\(a0\)
+**	fence\trw,rw
+**	ret
+*/
+void atomic_store_long_seq_cst (long* bar, long* baz)
+{
+  __atomic_store(bar, baz, __ATOMIC_SEQ_CST);
+}
+
+/*
+** atomic_store_int_seq_cst:
 **	lw\t[atx][0-9]+,0\(a1\)
 **	fence\trw,w
 **	sw\t[atx][0-9]+,0\(a0\)
 **	fence\trw,rw
 **	ret
 */
-void foo (int* bar, int* baz)
+void atomic_store_int_seq_cst (int* bar, int* baz)
+{
+  __atomic_store(bar, baz, __ATOMIC_SEQ_CST);
+}
+
+/*
+** atomic_store_short_seq_cst:
+**	lhu\t[atx][0-9]+,0\(a1\)
+**	fence\trw,w
+**	sh\t[atx][0-9]+,0\(a0\)
+**	fence\trw,rw
+**	ret
+*/
+void atomic_store_short_seq_cst (short* bar, short* baz)
+{
+  __atomic_store(bar, baz, __ATOMIC_SEQ_CST);
+}
+
+/*
+** atomic_store_char_seq_cst:
+**	lbu\t[atx][0-9]+,0\(a1\)
+**	fence\trw,w
+**	sb\t[atx][0-9]+,0\(a0\)
+**	fence\trw,rw
+**	ret
+*/
+void atomic_store_char_seq_cst (char* bar, char* baz)
+{
+  __atomic_store(bar, baz, __ATOMIC_SEQ_CST);
+}
+
+/*
+** atomic_store_bool_seq_cst:
+**	lbu\t[atx][0-9]+,0\(a1\)
+**	fence\trw,w
+**	sb\t[atx][0-9]+,0\(a0\)
+**	fence\trw,rw
+**	ret
+*/
+void atomic_store_bool_seq_cst (_Bool* bar, _Bool* baz)
 {
   __atomic_store(bar, baz, __ATOMIC_SEQ_CST);
 }
diff --git a/gcc/testsuite/gcc.target/riscv/amo/amo-table-ztso-load-1.c b/gcc/testsuite/gcc.target/riscv/amo/amo-table-ztso-load-1.c
index ebb0a2e1d38..b3e263e777d 100644
--- a/gcc/testsuite/gcc.target/riscv/amo/amo-table-ztso-load-1.c
+++ b/gcc/testsuite/gcc.target/riscv/amo/amo-table-ztso-load-1.c
@@ -6,12 +6,56 @@
 /* { dg-final { check-function-bodies "**" "" } } */

 /*
-** foo:
+** atomic_load_long_relaxed:
+**	ld\t[atx][0-9]+,0\(a0\)
+**	sd\t[atx][0-9]+,0\(a1\)
+**	ret
+*/
+void atomic_load_long_relaxed (long* bar, long* baz)
+{
+  __atomic_load(bar, baz, __ATOMIC_RELAXED);
+}
+
+/*
+** atomic_load_int_relaxed:
 **	lw\t[atx][0-9]+,0\(a0\)
 **	sw\t[atx][0-9]+,0\(a1\)
 **	ret
 */
-void foo (int* bar, int* baz)
+void atomic_load_int_relaxed (int* bar, int* baz)
+{
+  __atomic_load(bar, baz, __ATOMIC_RELAXED);
+}
+
+/*
+** atomic_load_short_relaxed:
+**	lh\t[atx][0-9]+,0\(a0\)
+**	sh\t[atx][0-9]+,0\(a1\)
+**	ret
+*/
+void atomic_load_short_relaxed (short* bar, short* baz)
+{
+  __atomic_load(bar, baz, __ATOMIC_RELAXED);
+}
+
+/*
+** atomic_load_char_relaxed:
+**	lb\t[atx][0-9]+,0\(a0\)
+**	sb\t[atx][0-9]+,0\(a1\)
+**	ret
+*/
+void atomic_load_char_relaxed (char* bar, char* baz)
+{
+  __atomic_load(bar, baz, __ATOMIC_RELAXED);
+}
+
+/*
+** atomic_load_bool_relaxed:
+**	lb\t[atx][0-9]+,0\(a0\)
+**	sb\t[atx][0-9]+,0\(a1\)
+**	ret
+*/
+void atomic_load_bool_relaxed (_Bool* bar, _Bool* baz)
 {
   __atomic_load(bar, baz, __ATOMIC_RELAXED);
 }
diff --git a/gcc/testsuite/gcc.target/riscv/amo/amo-table-ztso-load-2.c b/gcc/testsuite/gcc.target/riscv/amo/amo-table-ztso-load-2.c
index c88c4be5aea..15ecccf473b 100644
--- a/gcc/testsuite/gcc.target/riscv/amo/amo-table-ztso-load-2.c
+++ b/gcc/testsuite/gcc.target/riscv/amo/amo-table-ztso-load-2.c
@@ -6,12 +6,56 @@
 /* { dg-final { check-function-bodies "**" "" } } */

 /*
-** foo:
+** atomic_load_long_acquire:
+**	ld\t[atx][0-9]+,0\(a0\)
+**	sd\t[atx][0-9]+,0\(a1\)
+**	ret
+*/
+void atomic_load_long_acquire (long* bar, long* baz)
+{
+  __atomic_load(bar, baz, __ATOMIC_ACQUIRE);
+}
+
+/*
+** atomic_load_int_acquire:
 **	lw\t[atx][0-9]+,0\(a0\)
 **	sw\t[atx][0-9]+,0\(a1\)
 **	ret
 */
-void foo (int* bar, int* baz)
+void atomic_load_int_acquire (int* bar, int* baz)
+{
+  __atomic_load(bar, baz, __ATOMIC_ACQUIRE);
+}
+
+/*
+** atomic_load_short_acquire:
+**	lh\t[atx][0-9]+,0\(a0\)
+**	sh\t[atx][0-9]+,0\(a1\)
+**	ret
+*/
+void atomic_load_short_acquire (short* bar, short* baz)
+{
+  __atomic_load(bar, baz, __ATOMIC_ACQUIRE);
+}
+
+/*
+** atomic_load_char_acquire:
+**	lb\t[atx][0-9]+,0\(a0\)
+**	sb\t[atx][0-9]+,0\(a1\)
+**	ret
+*/
+void atomic_load_char_acquire (char* bar, char* baz)
+{
+  __atomic_load(bar, baz, __ATOMIC_ACQUIRE);
+}
+
+/*
+** atomic_load_bool_acquire:
+**	lb\t[atx][0-9]+,0\(a0\)
+**	sb\t[atx][0-9]+,0\(a1\)
+**	ret
+*/
+void atomic_load_bool_acquire (_Bool* bar, _Bool* baz)
 {
   __atomic_load(bar, baz, __ATOMIC_ACQUIRE);
 }
diff --git a/gcc/testsuite/gcc.target/riscv/amo/amo-table-ztso-load-3.c b/gcc/testsuite/gcc.target/riscv/amo/amo-table-ztso-load-3.c
index 8713729c378..ca56f233042 100644
--- a/gcc/testsuite/gcc.target/riscv/amo/amo-table-ztso-load-3.c
+++ b/gcc/testsuite/gcc.target/riscv/amo/amo-table-ztso-load-3.c
@@ -6,13 +6,61 @@
 /* { dg-final { check-function-bodies "**" "" } } */

 /*
-** foo:
+** atomic_load_long_seq_cst:
+**	fence\trw,rw
+**	ld\t[atx][0-9]+,0\(a0\)
+**	sd\t[atx][0-9]+,0\(a1\)
+**	ret
+*/
+void atomic_load_long_seq_cst (long* bar, long* baz)
+{
+  __atomic_load(bar, baz, __ATOMIC_SEQ_CST);
+}
+
+/*
+** atomic_load_int_seq_cst:
 **	fence\trw,rw
 **	lw\t[atx][0-9]+,0\(a0\)
 **	sw\t[atx][0-9]+,0\(a1\)
 **	ret
 */
-void foo (int* bar, int* baz)
+void atomic_load_int_seq_cst (int* bar, int* baz)
+{
+  __atomic_load(bar, baz, __ATOMIC_SEQ_CST);
+}
+
+/*
+** atomic_load_short_seq_cst:
+**	fence\trw,rw
+**	lh\t[atx][0-9]+,0\(a0\)
+**	sh\t[atx][0-9]+,0\(a1\)
+**	ret
+*/
+void atomic_load_short_seq_cst (short* bar, short* baz)
+{
+  __atomic_load(bar, baz, __ATOMIC_SEQ_CST);
+}
+
+/*
+** atomic_load_char_seq_cst:
+**	fence\trw,rw
+**	lb\t[atx][0-9]+,0\(a0\)
+**	sb\t[atx][0-9]+,0\(a1\)
+**	ret
+*/
+void atomic_load_char_seq_cst (char* bar, char* baz)
+{
+  __atomic_load(bar, baz, __ATOMIC_SEQ_CST);
+}
+
+/*
+** atomic_load_bool_seq_cst:
+**	fence\trw,rw
+**	lb\t[atx][0-9]+,0\(a0\)
+**	sb\t[atx][0-9]+,0\(a1\)
+**	ret
+*/
+void atomic_load_bool_seq_cst (_Bool* bar, _Bool* baz)
 {
   __atomic_load(bar, baz, __ATOMIC_SEQ_CST);
 }
diff --git a/gcc/testsuite/gcc.target/riscv/amo/amo-table-ztso-store-1.c b/gcc/testsuite/gcc.target/riscv/amo/amo-table-ztso-store-1.c
index ca8d5ed7515..b353f53d7b8 100644
--- a/gcc/testsuite/gcc.target/riscv/amo/amo-table-ztso-store-1.c
+++ b/gcc/testsuite/gcc.target/riscv/amo/amo-table-ztso-store-1.c
@@ -6,12 +6,56 @@
 /* { dg-final { check-function-bodies "**" "" } } */

 /*
-** foo:
+** atomic_store_long_relaxed:
+**	ld\t[atx][0-9]+,0\(a1\)
+**	sd\t[atx][0-9]+,0\(a0\)
+**	ret
+*/
+void atomic_store_long_relaxed (long* bar, long* baz)
+{
+  __atomic_store(bar, baz, __ATOMIC_RELAXED);
+}
+
+/*
+** atomic_store_int_relaxed:
 **	lw\t[atx][0-9]+,0\(a1\)
 **	sw\t[atx][0-9]+,0\(a0\)
 **	ret
 */
-void foo (int* bar, int* baz)
+void atomic_store_int_relaxed (int* bar, int* baz)
+{
+  __atomic_store(bar, baz, __ATOMIC_RELAXED);
+}
+
+/*
+** atomic_store_short_relaxed:
+**	lhu\t[atx][0-9]+,0\(a1\)
+**	sh\t[atx][0-9]+,0\(a0\)
+**	ret
+*/
+void atomic_store_short_relaxed (short* bar, short* baz)
+{
+  __atomic_store(bar, baz, __ATOMIC_RELAXED);
+}
+
+/*
+** atomic_store_char_relaxed:
+**	lbu\t[atx][0-9]+,0\(a1\)
+**	sb\t[atx][0-9]+,0\(a0\)
+**	ret
+*/
+void atomic_store_char_relaxed (char* bar, char* baz)
+{
+  __atomic_store(bar, baz, __ATOMIC_RELAXED);
+}
+
+/*
+** atomic_store_bool_relaxed:
+**	lbu\t[atx][0-9]+,0\(a1\)
+**	sb\t[atx][0-9]+,0\(a0\)
+**	ret
+*/
+void atomic_store_bool_relaxed (_Bool* bar, _Bool* baz)
 {
   __atomic_store(bar, baz, __ATOMIC_RELAXED);
 }
diff --git a/gcc/testsuite/gcc.target/riscv/amo/amo-table-ztso-store-2.c b/gcc/testsuite/gcc.target/riscv/amo/amo-table-ztso-store-2.c
index 23957198cfb..989e062bf75 100644
--- a/gcc/testsuite/gcc.target/riscv/amo/amo-table-ztso-store-2.c
+++ b/gcc/testsuite/gcc.target/riscv/amo/amo-table-ztso-store-2.c
@@ -6,12 +6,56 @@
 /* { dg-final { check-function-bodies "**" "" } } */

 /*
-** foo:
+** atomic_store_long_release:
+**	ld\t[atx][0-9]+,0\(a1\)
+**	sd\t[atx][0-9]+,0\(a0\)
+**	ret
+*/
+void atomic_store_long_release (long* bar, long* baz)
+{
+  __atomic_store(bar, baz, __ATOMIC_RELEASE);
+}
+
+/*
+** atomic_store_int_release:
 **	lw\t[atx][0-9]+,0\(a1\)
 **	sw\t[atx][0-9]+,0\(a0\)
 **	ret
 */
-void foo (int* bar, int* baz)
+void atomic_store_int_release (int* bar, int* baz)
+{
+  __atomic_store(bar, baz, __ATOMIC_RELEASE);
+}
+
+/*
+** atomic_store_short_release:
+**	lhu\t[atx][0-9]+,0\(a1\)
+**	sh\t[atx][0-9]+,0\(a0\)
+**	ret
+*/
+void atomic_store_short_release (short* bar, short* baz)
+{
+  __atomic_store(bar, baz, __ATOMIC_RELEASE);
+}
+
+/*
+** atomic_store_char_release:
+**	lbu\t[atx][0-9]+,0\(a1\)
+**	sb\t[atx][0-9]+,0\(a0\)
+**	ret
+*/
+void atomic_store_char_release (char* bar, char* baz)
+{
+  __atomic_store(bar, baz, __ATOMIC_RELEASE);
+}
+
+/*
+** atomic_store_bool_release:
+**	lbu\t[atx][0-9]+,0\(a1\)
+**	sb\t[atx][0-9]+,0\(a0\)
+**	ret
+*/
+void atomic_store_bool_release (_Bool* bar, _Bool* baz)
 {
   __atomic_store(bar, baz, __ATOMIC_RELEASE);
 }
diff --git a/gcc/testsuite/gcc.target/riscv/amo/amo-table-ztso-store-3.c b/gcc/testsuite/gcc.target/riscv/amo/amo-table-ztso-store-3.c
index 11c12f0ca1a..0c24dc60225 100644
--- a/gcc/testsuite/gcc.target/riscv/amo/amo-table-ztso-store-3.c
+++ b/gcc/testsuite/gcc.target/riscv/amo/amo-table-ztso-store-3.c
@@ -6,13 +6,61 @@
 /* { dg-final { check-function-bodies "**" "" } } */

 /*
-** foo:
+** atomic_store_long_seq_cst:
+**	ld\t[atx][0-9]+,0\(a1\)
+**	sd\t[atx][0-9]+,0\(a0\)
+**	fence\trw,rw
+**	ret
+*/
+void atomic_store_long_seq_cst (long* bar, long* baz)
+{
+  __atomic_store(bar, baz, __ATOMIC_SEQ_CST);
+}
+
+/*
+** atomic_store_int_seq_cst:
 **	lw\t[atx][0-9]+,0\(a1\)
 **	sw\t[atx][0-9]+,0\(a0\)
 **	fence\trw,rw
 **	ret
 */
-void foo (int* bar, int* baz)
+void atomic_store_int_seq_cst (int* bar, int* baz)
+{
+  __atomic_store(bar, baz, __ATOMIC_SEQ_CST);
+}
+
+/*
+** atomic_store_short_seq_cst:
+**	lhu\t[atx][0-9]+,0\(a1\)
+**	sh\t[atx][0-9]+,0\(a0\)
+**	fence\trw,rw
+**	ret
+*/
+void atomic_store_short_seq_cst (short* bar, short* baz)
+{
+  __atomic_store(bar, baz, __ATOMIC_SEQ_CST);
+}
+
+/*
+** atomic_store_char_seq_cst:
+**	lbu\t[atx][0-9]+,0\(a1\)
+**	sb\t[atx][0-9]+,0\(a0\)
+**	fence\trw,rw
+**	ret
+*/
+void atomic_store_char_seq_cst (char* bar, char* baz)
+{
+  __atomic_store(bar, baz, __ATOMIC_SEQ_CST);
+}
+
+/*
+** atomic_store_bool_seq_cst:
+**	lbu\t[atx][0-9]+,0\(a1\)
+**	sb\t[atx][0-9]+,0\(a0\)
+**	fence\trw,rw
+**	ret
+*/
+void atomic_store_bool_seq_cst (_Bool* bar, _Bool* baz)
 {
   __atomic_store(bar, baz, __ATOMIC_SEQ_CST);
 }
--
2.34.1

Re: [PATCH] RISC-V: Add support for subword atomic loads/stores

[Andrea Parri]

On Wed, Jun 12, 2024 at 05:10:13PM -0700, Patrick O'Neill wrote:
> Andrea Parri recently pointed out that we were emitting overly conservative
> fences for seq_cst atomic loads/stores. This adds support for the optimized
> fences specified in the PSABI:
> https://github.com/riscv-non-isa/riscv-elf-psabi-doc/blob/2092568f7896ceaa1ec0f02569b19eaa42cd51c9/riscv-atomic.adoc
> 
> gcc/ChangeLog:
> 
> 	* config/riscv/sync-rvwmo.md: Add support for subword fenced
> 	loads/stores.
> 	* config/riscv/sync-ztso.md: Ditto.
> 	* config/riscv/sync.md: Ditto.
> 
> gcc/testsuite/ChangeLog:
> 
> 	* gcc.target/riscv/amo/amo-table-a-6-load-1.c: Increase test coverage to
> 	include longs, shorts, chars, and bools.
> 	* gcc.target/riscv/amo/amo-table-a-6-load-2.c: Ditto.
> 	* gcc.target/riscv/amo/amo-table-a-6-load-3.c: Ditto.
> 	* gcc.target/riscv/amo/amo-table-a-6-store-1.c: Ditto.
> 	* gcc.target/riscv/amo/amo-table-a-6-store-2.c: Ditto.
> 	* gcc.target/riscv/amo/amo-table-a-6-store-compat-3.c: Ditto.
> 	* gcc.target/riscv/amo/amo-table-ztso-load-1.c: Ditto.
> 	* gcc.target/riscv/amo/amo-table-ztso-load-2.c: Ditto.
> 	* gcc.target/riscv/amo/amo-table-ztso-load-3.c: Ditto.
> 	* gcc.target/riscv/amo/amo-table-ztso-store-1.c: Ditto.
> 	* gcc.target/riscv/amo/amo-table-ztso-store-2.c: Ditto.
> 	* gcc.target/riscv/amo/amo-table-ztso-store-3.c: Ditto.
> 
> Signed-off-by: Patrick O'Neill <patrick@rivosinc.com>

Tested-by: Andrea Parri <andrea@rivosinc.com>

  Andrea

Re: [PATCH] RISC-V: Add support for subword atomic loads/stores

[Jeff Law]

On 6/12/24 6:10 PM, Patrick O'Neill wrote:
> Andrea Parri recently pointed out that we were emitting overly conservative
> fences for seq_cst atomic loads/stores. This adds support for the optimized
> fences specified in the PSABI:
> https://github.com/riscv-non-isa/riscv-elf-psabi-doc/blob/2092568f7896ceaa1ec0f02569b19eaa42cd51c9/riscv-atomic.adoc
> 
> gcc/ChangeLog:
> 
> 	* config/riscv/sync-rvwmo.md: Add support for subword fenced
> 	loads/stores.
> 	* config/riscv/sync-ztso.md: Ditto.
> 	* config/riscv/sync.md: Ditto.
> 
> gcc/testsuite/ChangeLog:
> 
> 	* gcc.target/riscv/amo/amo-table-a-6-load-1.c: Increase test coverage to
> 	include longs, shorts, chars, and bools.
> 	* gcc.target/riscv/amo/amo-table-a-6-load-2.c: Ditto.
> 	* gcc.target/riscv/amo/amo-table-a-6-load-3.c: Ditto.
> 	* gcc.target/riscv/amo/amo-table-a-6-store-1.c: Ditto.
> 	* gcc.target/riscv/amo/amo-table-a-6-store-2.c: Ditto.
> 	* gcc.target/riscv/amo/amo-table-a-6-store-compat-3.c: Ditto.
> 	* gcc.target/riscv/amo/amo-table-ztso-load-1.c: Ditto.
> 	* gcc.target/riscv/amo/amo-table-ztso-load-2.c: Ditto.
> 	* gcc.target/riscv/amo/amo-table-ztso-load-3.c: Ditto.
> 	* gcc.target/riscv/amo/amo-table-ztso-store-1.c: Ditto.
> 	* gcc.target/riscv/amo/amo-table-ztso-store-2.c: Ditto.
> 	* gcc.target/riscv/amo/amo-table-ztso-store-3.c: Ditto.
OK
jeff

Re: [PATCH] RISC-V: Add support for subword atomic loads/stores

[Patrick O'Neill]

On 6/13/24 12:58, Jeff Law wrote:
>
>
> On 6/12/24 6:10 PM, Patrick O'Neill wrote:
>> Andrea Parri recently pointed out that we were emitting overly 
>> conservative
>> fences for seq_cst atomic loads/stores. This adds support for the 
>> optimized
>> fences specified in the PSABI:
>> https://github.com/riscv-non-isa/riscv-elf-psabi-doc/blob/2092568f7896ceaa1ec0f02569b19eaa42cd51c9/riscv-atomic.adoc 
>>
>>
>> gcc/ChangeLog:
>>
>>     * config/riscv/sync-rvwmo.md: Add support for subword fenced
>>     loads/stores.
>>     * config/riscv/sync-ztso.md: Ditto.
>>     * config/riscv/sync.md: Ditto.
>>
>> gcc/testsuite/ChangeLog:
>>
>>     * gcc.target/riscv/amo/amo-table-a-6-load-1.c: Increase test 
>> coverage to
>>     include longs, shorts, chars, and bools.
>>     * gcc.target/riscv/amo/amo-table-a-6-load-2.c: Ditto.
>>     * gcc.target/riscv/amo/amo-table-a-6-load-3.c: Ditto.
>>     * gcc.target/riscv/amo/amo-table-a-6-store-1.c: Ditto.
>>     * gcc.target/riscv/amo/amo-table-a-6-store-2.c: Ditto.
>>     * gcc.target/riscv/amo/amo-table-a-6-store-compat-3.c: Ditto.
>>     * gcc.target/riscv/amo/amo-table-ztso-load-1.c: Ditto.
>>     * gcc.target/riscv/amo/amo-table-ztso-load-2.c: Ditto.
>>     * gcc.target/riscv/amo/amo-table-ztso-load-3.c: Ditto.
>>     * gcc.target/riscv/amo/amo-table-ztso-store-1.c: Ditto.
>>     * gcc.target/riscv/amo/amo-table-ztso-store-2.c: Ditto.
>>     * gcc.target/riscv/amo/amo-table-ztso-store-3.c: Ditto.
> OK
> jeff
>
Committed with a fixup to the long case to match both ld/sd or lw/sw
since that tripped up on rv32 targets. I resent the committed patch for
the archiver.

Patrick