[PATCH v3] RISC-V: Add Ztso atomic mappings

[Patrick O'Neill <patrick@rivosinc.com>]

The RISC-V Ztso extension currently has no effect on generated code.
With the additional ordering constraints guarenteed by Ztso, we can emit
more optimized atomic mappings than the RVWMO mappings.

This PR implements the Ztso psABI mappings[1].

[1] https://github.com/riscv-non-isa/riscv-elf-psabi-doc/pull/391

2023-08-08 Patrick O'Neill <patrick@rivosinc.com>

gcc/ChangeLog:

	* common/config/riscv/riscv-common.cc: Add Ztso and mark Ztso as
	dependent on 'a' extension.
	* config/riscv/riscv-opts.h (MASK_ZTSO): New mask.
	(TARGET_ZTSO): New target.
	* config/riscv/riscv.cc (riscv_memmodel_needs_amo_acquire): Add
	Ztso case.
	(riscv_memmodel_needs_amo_release): Add Ztso case.
	(riscv_print_operand): Add Ztso case for LR/SC annotations.
	* config/riscv/riscv.md: Import sync-rvwmo.md and sync-ztso.md.
	* config/riscv/riscv.opt: Add Ztso target variable.
	* config/riscv/sync.md (mem_thread_fence_1): Expand to RVWMO or
	Ztso specific insn.
	(atomic_load<mode>): Expand to RVWMO or Ztso specific insn.
	(atomic_store<mode>): Expand to RVWMO or Ztso specific insn.
	* config/riscv/sync-rvwmo.md: New file. Seperate out RVWMO
	specific load/store/fence mappings.
	* config/riscv/sync-ztso.md: New file. Seperate out Ztso
	specific load/store/fence mappings.

gcc/testsuite/ChangeLog:

	* gcc.target/riscv/amo-table-ztso-amo-add-1.c: New test.
	* gcc.target/riscv/amo-table-ztso-amo-add-2.c: New test.
	* gcc.target/riscv/amo-table-ztso-amo-add-3.c: New test.
	* gcc.target/riscv/amo-table-ztso-amo-add-4.c: New test.
	* gcc.target/riscv/amo-table-ztso-amo-add-5.c: New test.
	* gcc.target/riscv/amo-table-ztso-compare-exchange-1.c: New test.
	* gcc.target/riscv/amo-table-ztso-compare-exchange-2.c: New test.
	* gcc.target/riscv/amo-table-ztso-compare-exchange-3.c: New test.
	* gcc.target/riscv/amo-table-ztso-compare-exchange-4.c: New test.
	* gcc.target/riscv/amo-table-ztso-compare-exchange-5.c: New test.
	* gcc.target/riscv/amo-table-ztso-compare-exchange-6.c: New test.
	* gcc.target/riscv/amo-table-ztso-compare-exchange-7.c: New test.
	* gcc.target/riscv/amo-table-ztso-fence-1.c: New test.
	* gcc.target/riscv/amo-table-ztso-fence-2.c: New test.
	* gcc.target/riscv/amo-table-ztso-fence-3.c: New test.
	* gcc.target/riscv/amo-table-ztso-fence-4.c: New test.
	* gcc.target/riscv/amo-table-ztso-fence-5.c: New test.
	* gcc.target/riscv/amo-table-ztso-load-1.c: New test.
	* gcc.target/riscv/amo-table-ztso-load-2.c: New test.
	* gcc.target/riscv/amo-table-ztso-load-3.c: New test.
	* gcc.target/riscv/amo-table-ztso-store-1.c: New test.
	* gcc.target/riscv/amo-table-ztso-store-2.c: New test.
	* gcc.target/riscv/amo-table-ztso-store-3.c: New test.
	* gcc.target/riscv/amo-table-ztso-subword-amo-add-1.c: New test.
	* gcc.target/riscv/amo-table-ztso-subword-amo-add-2.c: New test.
	* gcc.target/riscv/amo-table-ztso-subword-amo-add-3.c: New test.
	* gcc.target/riscv/amo-table-ztso-subword-amo-add-4.c: New test.
	* gcc.target/riscv/amo-table-ztso-subword-amo-add-5.c: New test.

Signed-off-by: Patrick O'Neill <patrick@rivosinc.com>
---
V3:
- Added missing Ztso extension version in riscv-common.cc
- Reformatted patterns/expanders
- Fix minor formatting issues
---
 gcc/common/config/riscv/riscv-common.cc       |   6 +
 gcc/config/riscv/riscv-opts.h                 |   4 +
 gcc/config/riscv/riscv.cc                     |  20 +++-
 gcc/config/riscv/riscv.md                     |   2 +
 gcc/config/riscv/riscv.opt                    |   3 +
 gcc/config/riscv/sync-rvwmo.md                |  96 +++++++++++++++
 gcc/config/riscv/sync-ztso.md                 |  80 +++++++++++++
 gcc/config/riscv/sync.md                      | 111 ++++++------------
 .../riscv/amo-table-ztso-amo-add-1.c          |  15 +++
 .../riscv/amo-table-ztso-amo-add-2.c          |  15 +++
 .../riscv/amo-table-ztso-amo-add-3.c          |  15 +++
 .../riscv/amo-table-ztso-amo-add-4.c          |  15 +++
 .../riscv/amo-table-ztso-amo-add-5.c          |  15 +++
 .../riscv/amo-table-ztso-compare-exchange-1.c |  10 ++
 .../riscv/amo-table-ztso-compare-exchange-2.c |  10 ++
 .../riscv/amo-table-ztso-compare-exchange-3.c |  10 ++
 .../riscv/amo-table-ztso-compare-exchange-4.c |  10 ++
 .../riscv/amo-table-ztso-compare-exchange-5.c |  10 ++
 .../riscv/amo-table-ztso-compare-exchange-6.c |  10 ++
 .../riscv/amo-table-ztso-compare-exchange-7.c |  10 ++
 .../gcc.target/riscv/amo-table-ztso-fence-1.c |  14 +++
 .../gcc.target/riscv/amo-table-ztso-fence-2.c |  14 +++
 .../gcc.target/riscv/amo-table-ztso-fence-3.c |  14 +++
 .../gcc.target/riscv/amo-table-ztso-fence-4.c |  14 +++
 .../gcc.target/riscv/amo-table-ztso-fence-5.c |  15 +++
 .../gcc.target/riscv/amo-table-ztso-load-1.c  |  16 +++
 .../gcc.target/riscv/amo-table-ztso-load-2.c  |  16 +++
 .../gcc.target/riscv/amo-table-ztso-load-3.c  |  17 +++
 .../gcc.target/riscv/amo-table-ztso-store-1.c |  16 +++
 .../gcc.target/riscv/amo-table-ztso-store-2.c |  16 +++
 .../gcc.target/riscv/amo-table-ztso-store-3.c |  17 +++
 .../riscv/amo-table-ztso-subword-amo-add-1.c  |  10 ++
 .../riscv/amo-table-ztso-subword-amo-add-2.c  |  10 ++
 .../riscv/amo-table-ztso-subword-amo-add-3.c  |  10 ++
 .../riscv/amo-table-ztso-subword-amo-add-4.c  |  10 ++
 .../riscv/amo-table-ztso-subword-amo-add-5.c  |  10 ++
 36 files changed, 612 insertions(+), 74 deletions(-)
 create mode 100644 gcc/config/riscv/sync-rvwmo.md
 create mode 100644 gcc/config/riscv/sync-ztso.md
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-ztso-amo-add-1.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-ztso-amo-add-2.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-ztso-amo-add-3.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-ztso-amo-add-4.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-ztso-amo-add-5.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-ztso-compare-exchange-1.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-ztso-compare-exchange-2.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-ztso-compare-exchange-3.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-ztso-compare-exchange-4.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-ztso-compare-exchange-5.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-ztso-compare-exchange-6.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-ztso-compare-exchange-7.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-ztso-fence-1.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-ztso-fence-2.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-ztso-fence-3.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-ztso-fence-4.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-ztso-fence-5.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-ztso-load-1.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-ztso-load-2.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-ztso-load-3.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-ztso-store-1.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-ztso-store-2.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-ztso-store-3.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-ztso-subword-amo-add-1.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-ztso-subword-amo-add-2.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-ztso-subword-amo-add-3.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-ztso-subword-amo-add-4.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-ztso-subword-amo-add-5.c

diff --git a/gcc/common/config/riscv/riscv-common.cc b/gcc/common/config/riscv/riscv-common.cc
index 2eb8c7cadff..025dc4e0d92 100644
--- a/gcc/common/config/riscv/riscv-common.cc
+++ b/gcc/common/config/riscv/riscv-common.cc
@@ -71,6 +71,8 @@ static const riscv_implied_info_t riscv_implied_info[] =
   {"zks", "zksed"},
   {"zks", "zksh"},
 
+  {"ztso", "a"},
+
   {"v", "zvl128b"},
   {"v", "zve64d"},
 
@@ -218,6 +220,8 @@ static const struct riscv_ext_version riscv_ext_version_table[] =
   {"zkn",   ISA_SPEC_CLASS_NONE, 1, 0},
   {"zks",   ISA_SPEC_CLASS_NONE, 1, 0},
 
+  {"ztso",  ISA_SPEC_CLASS_NONE, 1, 0},
+
   {"zve32x", ISA_SPEC_CLASS_NONE, 1, 0},
   {"zve32f", ISA_SPEC_CLASS_NONE, 1, 0},
   {"zve32d", ISA_SPEC_CLASS_NONE, 1, 0},
@@ -1407,6 +1411,8 @@ static const riscv_ext_flag_table_t riscv_ext_flag_table[] =
   {"svinval", &gcc_options::x_riscv_sv_subext, MASK_SVINVAL},
   {"svnapot", &gcc_options::x_riscv_sv_subext, MASK_SVNAPOT},
 
+  {"ztso", &gcc_options::x_riscv_ztso_subext, MASK_ZTSO},
+
   {"xtheadba",      &gcc_options::x_riscv_xthead_subext, MASK_XTHEADBA},
   {"xtheadbb",      &gcc_options::x_riscv_xthead_subext, MASK_XTHEADBB},
   {"xtheadbs",      &gcc_options::x_riscv_xthead_subext, MASK_XTHEADBS},
diff --git a/gcc/config/riscv/riscv-opts.h b/gcc/config/riscv/riscv-opts.h
index 28d9b81bd80..d6d785d0075 100644
--- a/gcc/config/riscv/riscv-opts.h
+++ b/gcc/config/riscv/riscv-opts.h
@@ -152,6 +152,10 @@ enum riscv_entity
 #define TARGET_ZKSH   ((riscv_zk_subext & MASK_ZKSH) != 0)
 #define TARGET_ZKT    ((riscv_zk_subext & MASK_ZKT) != 0)
 
+#define MASK_ZTSO   (1 << 0)
+
+#define TARGET_ZTSO ((riscv_ztso_subext & MASK_ZTSO) != 0)
+
 #define MASK_VECTOR_ELEN_32    (1 << 0)
 #define MASK_VECTOR_ELEN_64    (1 << 1)
 #define MASK_VECTOR_ELEN_FP_32 (1 << 2)
diff --git a/gcc/config/riscv/riscv.cc b/gcc/config/riscv/riscv.cc
index f9b53d21d1b..0b48a8e618e 100644
--- a/gcc/config/riscv/riscv.cc
+++ b/gcc/config/riscv/riscv.cc
@@ -4765,6 +4765,10 @@ riscv_union_memmodels (enum memmodel model1, enum memmodel model2)
 static bool
 riscv_memmodel_needs_amo_acquire (enum memmodel model)
 {
+  /* ZTSO amo mappings require no annotations.  */
+  if (TARGET_ZTSO)
+    return false;
+
   switch (model)
     {
       case MEMMODEL_ACQ_REL:
@@ -4788,6 +4792,10 @@ riscv_memmodel_needs_amo_acquire (enum memmodel model)
 static bool
 riscv_memmodel_needs_amo_release (enum memmodel model)
 {
+  /* ZTSO amo mappings require no annotations.  */
+  if (TARGET_ZTSO)
+    return false;
+
   switch (model)
     {
       case MEMMODEL_ACQ_REL:
@@ -5007,7 +5015,10 @@ riscv_print_operand (FILE *file, rtx op, int letter)
 
     case 'I': {
       const enum memmodel model = memmodel_base (INTVAL (op));
-      if (model == MEMMODEL_SEQ_CST)
+      if (TARGET_ZTSO && model != MEMMODEL_SEQ_CST)
+	/* LR ops only have an annotation for SEQ_CST in the Ztso mapping.  */
+	break;
+      else if (model == MEMMODEL_SEQ_CST)
 	fputs (".aqrl", file);
       else if (riscv_memmodel_needs_amo_acquire (model))
 	fputs (".aq", file);
@@ -5016,7 +5027,12 @@ riscv_print_operand (FILE *file, rtx op, int letter)
 
     case 'J': {
       const enum memmodel model = memmodel_base (INTVAL (op));
-      if (riscv_memmodel_needs_amo_release (model))
+      if (TARGET_ZTSO && model == MEMMODEL_SEQ_CST)
+	/* SC ops only have an annotation for SEQ_CST in the Ztso mapping.  */
+	fputs (".rl", file);
+      else if (TARGET_ZTSO)
+	break;
+      else if (riscv_memmodel_needs_amo_release (model))
 	fputs (".rl", file);
       break;
     }
diff --git a/gcc/config/riscv/riscv.md b/gcc/config/riscv/riscv.md
index 688fd697255..b456fa6abb3 100644
--- a/gcc/config/riscv/riscv.md
+++ b/gcc/config/riscv/riscv.md
@@ -3321,6 +3321,8 @@
 (include "bitmanip.md")
 (include "crypto.md")
 (include "sync.md")
+(include "sync-rvwmo.md")
+(include "sync-ztso.md")
 (include "peephole.md")
 (include "pic.md")
 (include "generic.md")
diff --git a/gcc/config/riscv/riscv.opt b/gcc/config/riscv/riscv.opt
index 4dfd8f78ad5..717b51b5305 100644
--- a/gcc/config/riscv/riscv.opt
+++ b/gcc/config/riscv/riscv.opt
@@ -245,6 +245,9 @@ int riscv_zm_subext
 TargetVariable
 int riscv_sv_subext
 
+TargetVariable
+int riscv_ztso_subext
+
 TargetVariable
 int riscv_xthead_subext
 
diff --git a/gcc/config/riscv/sync-rvwmo.md b/gcc/config/riscv/sync-rvwmo.md
new file mode 100644
index 00000000000..1fc7cf16b5b
--- /dev/null
+++ b/gcc/config/riscv/sync-rvwmo.md
@@ -0,0 +1,96 @@
+;; Machine description for RISC-V atomic operations.
+;; Copyright (C) 2011-2023 Free Software Foundation, Inc.
+;; Contributed by Andrew Waterman (andrew@sifive.com).
+;; Based on MIPS target for GNU compiler.
+
+;; This file is part of GCC.
+
+;; GCC is free software; you can redistribute it and/or modify
+;; it under the terms of the GNU General Public License as published by
+;; the Free Software Foundation; either version 3, or (at your option)
+;; any later version.
+
+;; GCC is distributed in the hope that it will be useful,
+;; but WITHOUT ANY WARRANTY; without even the implied warranty of
+;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;; GNU General Public License for more details.
+
+;; You should have received a copy of the GNU General Public License
+;; along with GCC; see the file COPYING3.  If not see
+;; <http://www.gnu.org/licenses/>.
+
+;; Memory barrier.
+
+(define_insn "mem_thread_fence_rvwmo"
+  [(set (match_operand:BLK 0 "" "")
+	(unspec:BLK [(match_dup 0)] UNSPEC_MEMORY_BARRIER))
+   (match_operand:SI 1 "const_int_operand" "")]  ;; model
+  "!TARGET_ZTSO"
+  {
+    enum memmodel model = (enum memmodel) INTVAL (operands[1]);
+    model = memmodel_base (model);
+
+    if (model == MEMMODEL_SEQ_CST)
+	return "fence\trw,rw";
+    else if (model == MEMMODEL_ACQ_REL)
+	return "fence.tso";
+    else if (model == MEMMODEL_ACQUIRE)
+	return "fence\tr,rw";
+    else if (model == MEMMODEL_RELEASE)
+	return "fence\trw,w";
+    else
+	gcc_unreachable ();
+  }
+  [(set (attr "length") (const_int 4))])
+
+;; Atomic memory operations.
+
+(define_insn "atomic_load_rvwmo<mode>"
+  [(set (match_operand:GPR 0 "register_operand" "=r")
+	(unspec_volatile:GPR
+	    [(match_operand:GPR 1 "memory_operand" "A")
+	     (match_operand:SI 2 "const_int_operand")]  ;; model
+	 UNSPEC_ATOMIC_LOAD))]
+  "TARGET_ATOMIC && !TARGET_ZTSO"
+  {
+    enum memmodel model = (enum memmodel) INTVAL (operands[2]);
+    model = memmodel_base (model);
+
+    if (model == MEMMODEL_SEQ_CST)
+      return "fence\trw,rw\;"
+	     "l<amo>\t%0,%1\;"
+	     "fence\tr,rw";
+    if (model == MEMMODEL_ACQUIRE)
+      return "l<amo>\t%0,%1\;"
+	     "fence\tr,rw";
+    else
+      return "l<amo>\t%0,%1";
+  }
+  [(set_attr "type" "atomic")
+   (set (attr "length") (const_int 12))])
+
+;; Implement atomic stores with conservative fences.
+;; This allows us to be compatible with the ISA manual Table A.6 and Table A.7.
+(define_insn "atomic_store_rvwmo<mode>"
+  [(set (match_operand:GPR 0 "memory_operand" "=A")
+	(unspec_volatile:GPR
+	    [(match_operand:GPR 1 "reg_or_0_operand" "rJ")
+	     (match_operand:SI 2 "const_int_operand")]  ;; model
+	 UNSPEC_ATOMIC_STORE))]
+  "TARGET_ATOMIC && !TARGET_ZTSO"
+  {
+    enum memmodel model = (enum memmodel) INTVAL (operands[2]);
+    model = memmodel_base (model);
+
+    if (model == MEMMODEL_SEQ_CST)
+      return "fence\trw,w\;"
+	     "s<amo>\t%z1,%0\;"
+	     "fence\trw,rw";
+    if (model == MEMMODEL_RELEASE)
+      return "fence\trw,w\;"
+	     "s<amo>\t%z1,%0";
+    else
+      return "s<amo>\t%z1,%0";
+  }
+  [(set_attr "type" "atomic")
+   (set (attr "length") (const_int 12))])
diff --git a/gcc/config/riscv/sync-ztso.md b/gcc/config/riscv/sync-ztso.md
new file mode 100644
index 00000000000..91c2a48c069
--- /dev/null
+++ b/gcc/config/riscv/sync-ztso.md
@@ -0,0 +1,80 @@
+;; Machine description for RISC-V atomic operations.
+;; Copyright (C) 2011-2023 Free Software Foundation, Inc.
+;; Contributed by Andrew Waterman (andrew@sifive.com).
+;; Based on MIPS target for GNU compiler.
+
+;; This file is part of GCC.
+
+;; GCC is free software; you can redistribute it and/or modify
+;; it under the terms of the GNU General Public License as published by
+;; the Free Software Foundation; either version 3, or (at your option)
+;; any later version.
+
+;; GCC is distributed in the hope that it will be useful,
+;; but WITHOUT ANY WARRANTY; without even the implied warranty of
+;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;; GNU General Public License for more details.
+
+;; You should have received a copy of the GNU General Public License
+;; along with GCC; see the file COPYING3.  If not see
+;; <http://www.gnu.org/licenses/>.
+
+;; Memory barriers.
+
+(define_insn "mem_thread_fence_ztso"
+  [(set (match_operand:BLK 0 "" "")
+	(unspec:BLK [(match_dup 0)] UNSPEC_MEMORY_BARRIER))
+   (match_operand:SI 1 "const_int_operand" "")]  ;; model
+  "TARGET_ZTSO"
+  {
+    enum memmodel model = (enum memmodel) INTVAL (operands[1]);
+    model = memmodel_base (model);
+
+    if (model == MEMMODEL_SEQ_CST)
+	return "fence\trw,rw";
+    else
+	gcc_unreachable ();
+  }
+  [(set (attr "length") (const_int 4))])
+
+;; Atomic memory operations.
+
+(define_insn "atomic_load_ztso<mode>"
+  [(set (match_operand:GPR 0 "register_operand" "=r")
+	(unspec_volatile:GPR
+	    [(match_operand:GPR 1 "memory_operand" "A")
+	     (match_operand:SI 2 "const_int_operand")]  ;; model
+	 UNSPEC_ATOMIC_LOAD))]
+  "TARGET_ATOMIC && TARGET_ZTSO"
+  {
+    enum memmodel model = (enum memmodel) INTVAL (operands[2]);
+    model = memmodel_base (model);
+
+    if (model == MEMMODEL_SEQ_CST)
+      return "fence\trw,rw\;"
+	     "l<amo>\t%0,%1\;";
+    else
+      return "l<amo>\t%0,%1";
+  }
+  [(set_attr "type" "atomic")
+   (set (attr "length") (const_int 12))])
+
+(define_insn "atomic_store_ztso<mode>"
+  [(set (match_operand:GPR 0 "memory_operand" "=A")
+	(unspec_volatile:GPR
+	    [(match_operand:GPR 1 "reg_or_0_operand" "rJ")
+	     (match_operand:SI 2 "const_int_operand")]  ;; model
+	 UNSPEC_ATOMIC_STORE))]
+  "TARGET_ATOMIC && TARGET_ZTSO"
+  {
+    enum memmodel model = (enum memmodel) INTVAL (operands[2]);
+    model = memmodel_base (model);
+
+    if (model == MEMMODEL_SEQ_CST)
+      return "s<amo>\t%z1,%0\;"
+	     "fence\trw,rw";
+    else
+      return "s<amo>\t%z1,%0";
+  }
+  [(set_attr "type" "atomic")
+   (set (attr "length") (const_int 8))])
\ No newline at end of file
diff --git a/gcc/config/riscv/sync.md b/gcc/config/riscv/sync.md
index 9fc626267de..2f85951508f 100644
--- a/gcc/config/riscv/sync.md
+++ b/gcc/config/riscv/sync.md
@@ -36,88 +36,55 @@
 (define_expand "mem_thread_fence"
   [(match_operand:SI 0 "const_int_operand" "")] ;; model
   ""
-{
-  if (INTVAL (operands[0]) != MEMMODEL_RELAXED)
-    {
-      rtx mem = gen_rtx_MEM (BLKmode, gen_rtx_SCRATCH (Pmode));
-      MEM_VOLATILE_P (mem) = 1;
-      emit_insn (gen_mem_thread_fence_1 (mem, operands[0]));
-    }
-  DONE;
-})
-
-(define_insn "mem_thread_fence_1"
-  [(set (match_operand:BLK 0 "" "")
-	(unspec:BLK [(match_dup 0)] UNSPEC_MEMORY_BARRIER))
-   (match_operand:SI 1 "const_int_operand" "")] ;; model
-  ""
   {
-    enum memmodel model = (enum memmodel) INTVAL (operands[1]);
-    model = memmodel_base (model);
-    if (model == MEMMODEL_SEQ_CST)
-	return "fence\trw,rw";
-    else if (model == MEMMODEL_ACQ_REL)
-	return "fence.tso";
-    else if (model == MEMMODEL_ACQUIRE)
-	return "fence\tr,rw";
-    else if (model == MEMMODEL_RELEASE)
-	return "fence\trw,w";
-    else
-	gcc_unreachable ();
-  }
-  [(set (attr "length") (const_int 4))])
+    enum memmodel model = memmodel_base (INTVAL (operands[0]));
+
+    if (TARGET_ZTSO && model == MEMMODEL_SEQ_CST)
+      {
+	rtx mem = gen_rtx_MEM (BLKmode, gen_rtx_SCRATCH (Pmode));
+	MEM_VOLATILE_P (mem) = 1;
+	emit_insn (gen_mem_thread_fence_ztso (mem, operands[0]));
+      }
+    else if (!TARGET_ZTSO && model != MEMMODEL_RELAXED)
+      {
+	rtx mem = gen_rtx_MEM (BLKmode, gen_rtx_SCRATCH (Pmode));
+	MEM_VOLATILE_P (mem) = 1;
+	emit_insn (gen_mem_thread_fence_rvwmo (mem, operands[0]));
+      }
+    DONE;
+  })
 
 ;; Atomic memory operations.
 
-(define_insn "atomic_load<mode>"
-  [(set (match_operand:GPR 0 "register_operand" "=r")
-    (unspec_volatile:GPR
-      [(match_operand:GPR 1 "memory_operand" "A")
-       (match_operand:SI 2 "const_int_operand")]      ;; model
-      UNSPEC_ATOMIC_LOAD))]
+(define_expand "atomic_load<mode>"
+  [(match_operand:GPR 0 "register_operand")
+   (match_operand:GPR 1 "memory_operand")
+   (match_operand:SI 2 "const_int_operand")] ;; model
   "TARGET_ATOMIC"
   {
-    enum memmodel model = (enum memmodel) INTVAL (operands[2]);
-    model = memmodel_base (model);
-
-    if (model == MEMMODEL_SEQ_CST)
-      return "fence\trw,rw\;"
-	     "l<amo>\t%0,%1\;"
-	     "fence\tr,rw";
-    if (model == MEMMODEL_ACQUIRE)
-      return "l<amo>\t%0,%1\;"
-	     "fence\tr,rw";
+    if (TARGET_ZTSO)
+      emit_insn (gen_atomic_load_ztso<mode> (operands[0], operands[1],
+					     operands[2]));
     else
-      return "l<amo>\t%0,%1";
-  }
-  [(set_attr "type" "atomic")
-   (set (attr "length") (const_int 12))])
-
-;; Implement atomic stores with conservative fences.
-;; This allows us to be compatible with the ISA manual Table A.6 and Table A.7.
-(define_insn "atomic_store<mode>"
-  [(set (match_operand:GPR 0 "memory_operand" "=A")
-    (unspec_volatile:GPR
-      [(match_operand:GPR 1 "reg_or_0_operand" "rJ")
-       (match_operand:SI 2 "const_int_operand")]      ;; model
-      UNSPEC_ATOMIC_STORE))]
+      emit_insn (gen_atomic_load_rvwmo<mode> (operands[0], operands[1],
+					      operands[2]));
+    DONE;
+  })
+
+(define_expand "atomic_store<mode>"
+  [(match_operand:GPR 0 "memory_operand")
+   (match_operand:GPR 1 "reg_or_0_operand")
+   (match_operand:SI 2 "const_int_operand")] ;; model
   "TARGET_ATOMIC"
   {
-    enum memmodel model = (enum memmodel) INTVAL (operands[2]);
-    model = memmodel_base (model);
-
-    if (model == MEMMODEL_SEQ_CST)
-      return "fence\trw,w\;"
-	     "s<amo>\t%z1,%0\;"
-	     "fence\trw,rw";
-    if (model == MEMMODEL_RELEASE)
-      return "fence\trw,w\;"
-	     "s<amo>\t%z1,%0";
+    if (TARGET_ZTSO)
+      emit_insn (gen_atomic_store_ztso<mode> (operands[0], operands[1],
+					      operands[2]));
     else
-      return "s<amo>\t%z1,%0";
-  }
-  [(set_attr "type" "atomic")
-   (set (attr "length") (const_int 12))])
+      emit_insn (gen_atomic_store_rvwmo<mode> (operands[0], operands[1],
+					       operands[2]));
+    DONE;
+  })
 
 (define_insn "atomic_<atomic_optab><mode>"
   [(set (match_operand:GPR 0 "memory_operand" "+A")
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-ztso-amo-add-1.c b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-amo-add-1.c
new file mode 100644
index 00000000000..a7097e9aab9
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-amo-add-1.c
@@ -0,0 +1,15 @@
+/* { dg-do compile } */
+/* Verify that atomic op mappings match the Ztso suggested mapping.  */
+/* { dg-options "-march=rv64id_ztso -O3" } */
+/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */
+/* { dg-final { check-function-bodies "**" "" } } */
+
+/*
+** foo:
+**	amoadd\.w\tzero,a1,0\(a0\)
+**	ret
+*/
+void foo (int* bar, int* baz)
+{
+  __atomic_add_fetch(bar, baz, __ATOMIC_RELAXED);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-ztso-amo-add-2.c b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-amo-add-2.c
new file mode 100644
index 00000000000..8e993903439
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-amo-add-2.c
@@ -0,0 +1,15 @@
+/* { dg-do compile } */
+/* Verify that atomic op mappings the Ztso suggested mapping.  */
+/* { dg-options "-march=rv64id_ztso -O3" } */
+/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */
+/* { dg-final { check-function-bodies "**" "" } } */
+
+/*
+** foo:
+**	amoadd\.w\tzero,a1,0\(a0\)
+**	ret
+*/
+void foo (int* bar, int* baz)
+{
+  __atomic_add_fetch(bar, baz, __ATOMIC_ACQUIRE);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-ztso-amo-add-3.c b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-amo-add-3.c
new file mode 100644
index 00000000000..e1cd209fcb2
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-amo-add-3.c
@@ -0,0 +1,15 @@
+/* { dg-do compile } */
+/* Verify that atomic op mappings match the Ztso suggested mapping.  */
+/* { dg-options "-march=rv64id_ztso -O3" } */
+/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */
+/* { dg-final { check-function-bodies "**" "" } } */
+
+/*
+** foo:
+**	amoadd\.w\tzero,a1,0\(a0\)
+**	ret
+*/
+void foo (int* bar, int* baz)
+{
+  __atomic_add_fetch(bar, baz, __ATOMIC_RELEASE);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-ztso-amo-add-4.c b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-amo-add-4.c
new file mode 100644
index 00000000000..484d11eb100
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-amo-add-4.c
@@ -0,0 +1,15 @@
+/* { dg-do compile } */
+/* Verify that atomic op mappings match the Ztso suggested mapping.  */
+/* { dg-options "-march=rv64id_ztso -O3" } */
+/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */
+/* { dg-final { check-function-bodies "**" "" } } */
+
+/*
+** foo:
+**	amoadd\.w\tzero,a1,0\(a0\)
+**	ret
+*/
+void foo (int* bar, int* baz)
+{
+  __atomic_add_fetch(bar, baz, __ATOMIC_ACQ_REL);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-ztso-amo-add-5.c b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-amo-add-5.c
new file mode 100644
index 00000000000..2117283fd0d
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-amo-add-5.c
@@ -0,0 +1,15 @@
+/* { dg-do compile } */
+/* Verify that atomic op mappings match the Ztso suggested mapping.  */
+/* { dg-options "-march=rv64id_ztso -O3" } */
+/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */
+/* { dg-final { check-function-bodies "**" "" } } */
+
+/*
+** foo:
+**	amoadd\.w\tzero,a1,0\(a0\)
+**	ret
+*/
+void foo (int* bar, int* baz)
+{
+  __atomic_add_fetch(bar, baz, __ATOMIC_SEQ_CST);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-ztso-compare-exchange-1.c b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-compare-exchange-1.c
new file mode 100644
index 00000000000..03247c632fd
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-compare-exchange-1.c
@@ -0,0 +1,10 @@
+/* { dg-do compile } */
+/* Verify that compare exchange mappings match the Ztso suggested mapping.  */
+/* { dg-options "-march=rv64id_ztso" } */
+/* { dg-final { scan-assembler-times "lr.w\t" 1 } } */
+/* { dg-final { scan-assembler-times "sc.w\t" 1 } } */
+
+void foo (int bar, int baz, int qux)
+{
+  __atomic_compare_exchange_n(&bar, &baz, qux, 1, __ATOMIC_RELAXED, __ATOMIC_RELAXED);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-ztso-compare-exchange-2.c b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-compare-exchange-2.c
new file mode 100644
index 00000000000..46356747067
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-compare-exchange-2.c
@@ -0,0 +1,10 @@
+/* { dg-do compile } */
+/* Verify that compare exchange mappings match the Ztso suggested mapping.  */
+/* { dg-options "-march=rv64id_ztso" } */
+/* { dg-final { scan-assembler-times "lr.w\t" 1 } } */
+/* { dg-final { scan-assembler-times "sc.w\t" 1 } } */
+
+void foo (int bar, int baz, int qux)
+{
+  __atomic_compare_exchange_n(&bar, &baz, qux, 1, __ATOMIC_CONSUME, __ATOMIC_CONSUME);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-ztso-compare-exchange-3.c b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-compare-exchange-3.c
new file mode 100644
index 00000000000..cf252dba6ec
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-compare-exchange-3.c
@@ -0,0 +1,10 @@
+/* { dg-do compile } */
+/* Verify that compare exchange mappings match the Ztso suggested mapping.  */
+/* { dg-options "-march=rv64id_ztso" } */
+/* { dg-final { scan-assembler-times "lr.w\t" 1 } } */
+/* { dg-final { scan-assembler-times "sc.w\t" 1 } } */
+
+void foo (int bar, int baz, int qux)
+{
+  __atomic_compare_exchange_n(&bar, &baz, qux, 1, __ATOMIC_ACQUIRE, __ATOMIC_ACQUIRE);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-ztso-compare-exchange-4.c b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-compare-exchange-4.c
new file mode 100644
index 00000000000..9a0c86569b5
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-compare-exchange-4.c
@@ -0,0 +1,10 @@
+/* { dg-do compile } */
+/* Verify that compare exchange mappings match the Ztso suggested mapping.  */
+/* { dg-options "-march=rv64id_ztso" } */
+/* { dg-final { scan-assembler-times "lr.w\t" 1 } } */
+/* { dg-final { scan-assembler-times "sc.w\t" 1 } } */
+
+void foo (int bar, int baz, int qux)
+{
+  __atomic_compare_exchange_n(&bar, &baz, qux, 1, __ATOMIC_RELEASE, __ATOMIC_RELAXED);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-ztso-compare-exchange-5.c b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-compare-exchange-5.c
new file mode 100644
index 00000000000..750ba89cd74
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-compare-exchange-5.c
@@ -0,0 +1,10 @@
+/* { dg-do compile } */
+/* Verify that compare exchange mappings match the Ztso suggested mapping.  */
+/* { dg-options "-march=rv64id_ztso" } */
+/* { dg-final { scan-assembler-times "lr.w.aqrl\t" 1 } } */
+/* { dg-final { scan-assembler-times "sc.w.rl\t" 1 } } */
+
+void foo (int bar, int baz, int qux)
+{
+  __atomic_compare_exchange_n(&bar, &baz, qux, 1, __ATOMIC_SEQ_CST, __ATOMIC_SEQ_CST);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-ztso-compare-exchange-6.c b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-compare-exchange-6.c
new file mode 100644
index 00000000000..0d0bf26d95a
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-compare-exchange-6.c
@@ -0,0 +1,10 @@
+/* { dg-do compile } */
+/* Verify that compare exchange mappings match the Ztso suggested mapping.  */
+/* { dg-options "-march=rv64id_ztso" } */
+/* { dg-final { scan-assembler-times "lr.w\t" 1 } } */
+/* { dg-final { scan-assembler-times "sc.w\t" 1 } } */
+
+void foo (int bar, int baz, int qux)
+{
+  __atomic_compare_exchange_n(&bar, &baz, qux, 1, __ATOMIC_RELEASE, __ATOMIC_ACQUIRE);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-ztso-compare-exchange-7.c b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-compare-exchange-7.c
new file mode 100644
index 00000000000..838568ed588
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-compare-exchange-7.c
@@ -0,0 +1,10 @@
+/* { dg-do compile } */
+/* Verify that compare exchange mappings match the Ztso suggested mapping.  */
+/* { dg-options "-march=rv64id_ztso" } */
+/* { dg-final { scan-assembler-times "lr.w.aqrl\t" 1 } } */
+/* { dg-final { scan-assembler-times "sc.w.rl\t" 1 } } */
+
+void foo (int bar, int baz, int qux)
+{
+  __atomic_compare_exchange_n(&bar, &baz, qux, 1, __ATOMIC_SEQ_CST, __ATOMIC_RELAXED);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-ztso-fence-1.c b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-fence-1.c
new file mode 100644
index 00000000000..a9ba4a252e7
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-fence-1.c
@@ -0,0 +1,14 @@
+/* { dg-do compile } */
+/* Verify that fence mappings match the Ztso suggested mapping.  */
+/* { dg-options "-march=rv64id_ztso -O3" } */
+/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */
+/* { dg-final { check-function-bodies "**" "" } } */
+
+/*
+** foo:
+**	ret
+*/
+void foo()
+{
+  __atomic_thread_fence(__ATOMIC_RELAXED);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-ztso-fence-2.c b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-fence-2.c
new file mode 100644
index 00000000000..9b083431d1a
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-fence-2.c
@@ -0,0 +1,14 @@
+/* { dg-do compile } */
+/* Verify that fence mappings match the Ztso suggested mapping.  */
+/* { dg-options "-march=rv64id_ztso -O3" } */
+/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */
+/* { dg-final { check-function-bodies "**" "" } } */
+
+/*
+** foo:
+**	ret
+*/
+void foo()
+{
+  __atomic_thread_fence(__ATOMIC_ACQUIRE);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-ztso-fence-3.c b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-fence-3.c
new file mode 100644
index 00000000000..db21ee6f031
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-fence-3.c
@@ -0,0 +1,14 @@
+/* { dg-do compile } */
+/* Verify that fence mappings match the Ztso suggested mapping.  */
+/* { dg-options "-march=rv64id_ztso -O3" } */
+/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */
+/* { dg-final { check-function-bodies "**" "" } } */
+
+/*
+** foo:
+**	ret
+*/
+void foo()
+{
+  __atomic_thread_fence(__ATOMIC_RELEASE);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-ztso-fence-4.c b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-fence-4.c
new file mode 100644
index 00000000000..d5bc61b57e8
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-fence-4.c
@@ -0,0 +1,14 @@
+/* { dg-do compile } */
+/* Verify that fence mappings match the Ztso suggested mapping.  */
+/* { dg-options "-march=rv64id_ztso -O3" } */
+/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */
+/* { dg-final { check-function-bodies "**" "" } } */
+
+/*
+** foo:
+**	ret
+*/
+void foo()
+{
+  __atomic_thread_fence(__ATOMIC_ACQ_REL);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-ztso-fence-5.c b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-fence-5.c
new file mode 100644
index 00000000000..e5e5f4e5012
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-fence-5.c
@@ -0,0 +1,15 @@
+/* { dg-do compile } */
+/* Verify that fence mappings match the Ztso suggested mapping.  */
+/* { dg-options "-march=rv64id_ztso -O3" } */
+/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */
+/* { dg-final { check-function-bodies "**" "" } } */
+
+/*
+** foo:
+**	fence\trw,rw
+**	ret
+*/
+void foo()
+{
+  __atomic_thread_fence(__ATOMIC_SEQ_CST);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-ztso-load-1.c b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-load-1.c
new file mode 100644
index 00000000000..be5f8d310a4
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-load-1.c
@@ -0,0 +1,16 @@
+/* { dg-do compile } */
+/* Verify that load mappings match the Ztso suggested mapping.  */
+/* { dg-options "-march=rv64id_ztso -O3" } */
+/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */
+/* { dg-final { check-function-bodies "**" "" } } */
+
+/*
+** foo:
+**	lw\ta[0-9]+,0\(a0\)
+**	sw\ta[0-9]+,0\(a1\)
+**	ret
+*/
+void foo (int* bar, int* baz)
+{
+  __atomic_load(bar, baz, __ATOMIC_RELAXED);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-ztso-load-2.c b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-load-2.c
new file mode 100644
index 00000000000..30583841648
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-load-2.c
@@ -0,0 +1,16 @@
+/* { dg-do compile } */
+/* Verify that load mappings match the Ztso suggested mapping.  */
+/* { dg-options "-march=rv64id_ztso -O3" } */
+/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */
+/* { dg-final { check-function-bodies "**" "" } } */
+
+/*
+** foo:
+**	lw\ta[0-9]+,0\(a0\)
+**	sw\ta[0-9]+,0\(a1\)
+**	ret
+*/
+void foo (int* bar, int* baz)
+{
+  __atomic_load(bar, baz, __ATOMIC_ACQUIRE);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-ztso-load-3.c b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-load-3.c
new file mode 100644
index 00000000000..c1c3b57bfe5
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-load-3.c
@@ -0,0 +1,17 @@
+/* { dg-do compile } */
+/* Verify that load mappings match the Ztso suggested mapping.  */
+/* { dg-options "-march=rv64id_ztso -O3" } */
+/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */
+/* { dg-final { check-function-bodies "**" "" } } */
+
+/*
+** foo:
+**	fence\trw,rw
+**	lw\ta[0-9]+,0\(a0\)
+**	sw\ta[0-9]+,0\(a1\)
+**	ret
+*/
+void foo (int* bar, int* baz)
+{
+  __atomic_load(bar, baz, __ATOMIC_SEQ_CST);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-ztso-store-1.c b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-store-1.c
new file mode 100644
index 00000000000..f5519db22f7
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-store-1.c
@@ -0,0 +1,16 @@
+/* { dg-do compile } */
+/* Verify that store mappings match the Ztso suggested mapping.  */
+/* { dg-options "-march=rv64id_ztso -O3" } */
+/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */
+/* { dg-final { check-function-bodies "**" "" } } */
+
+/*
+** foo:
+**	lw\ta[0-9]+,0\(a1\)
+**	sw\ta[0-9]+,0\(a0\)
+**	ret
+*/
+void foo (int* bar, int* baz)
+{
+  __atomic_store(bar, baz, __ATOMIC_RELAXED);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-ztso-store-2.c b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-store-2.c
new file mode 100644
index 00000000000..440f407e66a
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-store-2.c
@@ -0,0 +1,16 @@
+/* { dg-do compile } */
+/* Verify that store mappings match the Ztso suggested mapping.  */
+/* { dg-options "-march=rv64id_ztso -O3" } */
+/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */
+/* { dg-final { check-function-bodies "**" "" } } */
+
+/*
+** foo:
+**	lw\ta[0-9]+,0\(a1\)
+**	sw\ta[0-9]+,0\(a0\)
+**	ret
+*/
+void foo (int* bar, int* baz)
+{
+  __atomic_store(bar, baz, __ATOMIC_RELEASE);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-ztso-store-3.c b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-store-3.c
new file mode 100644
index 00000000000..a92d4d1ce9a
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-store-3.c
@@ -0,0 +1,17 @@
+/* { dg-do compile } */
+/* Verify that store mappings match the Ztso suggested mapping.  */
+/* { dg-options "-march=rv64id_ztso -O3" } */
+/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */
+/* { dg-final { check-function-bodies "**" "" } } */
+
+/*
+** foo:
+**	lw\ta[0-9]+,0\(a1\)
+**	sw\ta[0-9]+,0\(a0\)
+**	fence\trw,rw
+**	ret
+*/
+void foo (int* bar, int* baz)
+{
+  __atomic_store(bar, baz, __ATOMIC_SEQ_CST);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-ztso-subword-amo-add-1.c b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-subword-amo-add-1.c
new file mode 100644
index 00000000000..e187a6b7335
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-subword-amo-add-1.c
@@ -0,0 +1,10 @@
+/* { dg-do compile } */
+/* Verify that subword atomic op mappings match the Ztso suggested mapping.  */
+/* { dg-options "-march=rv64id_ztso" } */
+/* { dg-final { scan-assembler-times "lr.w\t" 1 } } */
+/* { dg-final { scan-assembler-times "sc.w\t" 1 } } */
+
+void foo (short* bar, short* baz)
+{
+  __atomic_add_fetch(bar, baz, __ATOMIC_RELAXED);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-ztso-subword-amo-add-2.c b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-subword-amo-add-2.c
new file mode 100644
index 00000000000..1e0867624ba
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-subword-amo-add-2.c
@@ -0,0 +1,10 @@
+/* { dg-do compile } */
+/* Verify that subword atomic op mappings match the Ztso suggested mapping.  */
+/* { dg-options "-march=rv64id_ztso" } */
+/* { dg-final { scan-assembler-times "lr.w\t" 1 } } */
+/* { dg-final { scan-assembler-times "sc.w\t" 1 } } */
+
+void foo (short* bar, short* baz)
+{
+  __atomic_add_fetch(bar, baz, __ATOMIC_ACQUIRE);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-ztso-subword-amo-add-3.c b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-subword-amo-add-3.c
new file mode 100644
index 00000000000..4b2ef967375
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-subword-amo-add-3.c
@@ -0,0 +1,10 @@
+/* { dg-do compile } */
+/* Verify that subword atomic op mappings match the Ztso suggested mapping.  */
+/* { dg-options "-march=rv64id_ztso" } */
+/* { dg-final { scan-assembler-times "lr.w\t" 1 } } */
+/* { dg-final { scan-assembler-times "sc.w\t" 1 } } */
+
+void foo (short* bar, short* baz)
+{
+  __atomic_add_fetch(bar, baz, __ATOMIC_RELEASE);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-ztso-subword-amo-add-4.c b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-subword-amo-add-4.c
new file mode 100644
index 00000000000..87e5356f0a7
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-subword-amo-add-4.c
@@ -0,0 +1,10 @@
+/* { dg-do compile } */
+/* Verify that subword atomic op mappings match the Ztso suggested mapping.  */
+/* { dg-options "-march=rv64id_ztso" } */
+/* { dg-final { scan-assembler-times "lr.w\t" 1 } } */
+/* { dg-final { scan-assembler-times "sc.w\t" 1 } } */
+
+void foo (short* bar, short* baz)
+{
+  __atomic_add_fetch(bar, baz, __ATOMIC_ACQ_REL);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-ztso-subword-amo-add-5.c b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-subword-amo-add-5.c
new file mode 100644
index 00000000000..fd59c8880ec
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-subword-amo-add-5.c
@@ -0,0 +1,10 @@
+/* { dg-do compile } */
+/* Verify that subword atomic op mappings match the Ztso suggested mapping.  */
+/* { dg-options "-march=rv64id_ztso" } */
+/* { dg-final { scan-assembler-times "lr.w.aqrl\t" 1 } } */
+/* { dg-final { scan-assembler-times "sc.w.rl\t" 1 } } */
+
+void foo (short* bar, short* baz)
+{
+  __atomic_add_fetch(bar, baz, __ATOMIC_SEQ_CST);
+}
-- 
2.34.1