Re: [PATCH v3] RISC-V: Add Ztso atomic mappings

[Palmer Dabbelt <palmer@rivosinc.com>]

On Tue, 08 Aug 2023 14:52:14 PDT (-0700), Patrick O'Neill wrote:
> The RISC-V Ztso extension currently has no effect on generated code.
> With the additional ordering constraints guarenteed by Ztso, we can emit
> more optimized atomic mappings than the RVWMO mappings.
>
> This PR implements the Ztso psABI mappings[1].
>
> [1] https://github.com/riscv-non-isa/riscv-elf-psabi-doc/pull/391

Which was just merged this morning, we talked some in the GCC patchwork 
sync.  IIUC that was the last blocker for getting the implementations 
merged, so

Reviewed-by: Palmer Dabbelt <palmer@rivosinc.com>

Thanks!

> 2023-08-08 Patrick O'Neill <patrick@rivosinc.com>
>
> gcc/ChangeLog:
>
> 	* common/config/riscv/riscv-common.cc: Add Ztso and mark Ztso as
> 	dependent on 'a' extension.
> 	* config/riscv/riscv-opts.h (MASK_ZTSO): New mask.
> 	(TARGET_ZTSO): New target.
> 	* config/riscv/riscv.cc (riscv_memmodel_needs_amo_acquire): Add
> 	Ztso case.
> 	(riscv_memmodel_needs_amo_release): Add Ztso case.
> 	(riscv_print_operand): Add Ztso case for LR/SC annotations.
> 	* config/riscv/riscv.md: Import sync-rvwmo.md and sync-ztso.md.
> 	* config/riscv/riscv.opt: Add Ztso target variable.
> 	* config/riscv/sync.md (mem_thread_fence_1): Expand to RVWMO or
> 	Ztso specific insn.
> 	(atomic_load<mode>): Expand to RVWMO or Ztso specific insn.
> 	(atomic_store<mode>): Expand to RVWMO or Ztso specific insn.
> 	* config/riscv/sync-rvwmo.md: New file. Seperate out RVWMO
> 	specific load/store/fence mappings.
> 	* config/riscv/sync-ztso.md: New file. Seperate out Ztso
> 	specific load/store/fence mappings.
>
> gcc/testsuite/ChangeLog:
>
> 	* gcc.target/riscv/amo-table-ztso-amo-add-1.c: New test.
> 	* gcc.target/riscv/amo-table-ztso-amo-add-2.c: New test.
> 	* gcc.target/riscv/amo-table-ztso-amo-add-3.c: New test.
> 	* gcc.target/riscv/amo-table-ztso-amo-add-4.c: New test.
> 	* gcc.target/riscv/amo-table-ztso-amo-add-5.c: New test.
> 	* gcc.target/riscv/amo-table-ztso-compare-exchange-1.c: New test.
> 	* gcc.target/riscv/amo-table-ztso-compare-exchange-2.c: New test.
> 	* gcc.target/riscv/amo-table-ztso-compare-exchange-3.c: New test.
> 	* gcc.target/riscv/amo-table-ztso-compare-exchange-4.c: New test.
> 	* gcc.target/riscv/amo-table-ztso-compare-exchange-5.c: New test.
> 	* gcc.target/riscv/amo-table-ztso-compare-exchange-6.c: New test.
> 	* gcc.target/riscv/amo-table-ztso-compare-exchange-7.c: New test.
> 	* gcc.target/riscv/amo-table-ztso-fence-1.c: New test.
> 	* gcc.target/riscv/amo-table-ztso-fence-2.c: New test.
> 	* gcc.target/riscv/amo-table-ztso-fence-3.c: New test.
> 	* gcc.target/riscv/amo-table-ztso-fence-4.c: New test.
> 	* gcc.target/riscv/amo-table-ztso-fence-5.c: New test.
> 	* gcc.target/riscv/amo-table-ztso-load-1.c: New test.
> 	* gcc.target/riscv/amo-table-ztso-load-2.c: New test.
> 	* gcc.target/riscv/amo-table-ztso-load-3.c: New test.
> 	* gcc.target/riscv/amo-table-ztso-store-1.c: New test.
> 	* gcc.target/riscv/amo-table-ztso-store-2.c: New test.
> 	* gcc.target/riscv/amo-table-ztso-store-3.c: New test.
> 	* gcc.target/riscv/amo-table-ztso-subword-amo-add-1.c: New test.
> 	* gcc.target/riscv/amo-table-ztso-subword-amo-add-2.c: New test.
> 	* gcc.target/riscv/amo-table-ztso-subword-amo-add-3.c: New test.
> 	* gcc.target/riscv/amo-table-ztso-subword-amo-add-4.c: New test.
> 	* gcc.target/riscv/amo-table-ztso-subword-amo-add-5.c: New test.
>
> Signed-off-by: Patrick O'Neill <patrick@rivosinc.com>
> ---
> V3:
> - Added missing Ztso extension version in riscv-common.cc
> - Reformatted patterns/expanders
> - Fix minor formatting issues
> ---
>  gcc/common/config/riscv/riscv-common.cc       |   6 +
>  gcc/config/riscv/riscv-opts.h                 |   4 +
>  gcc/config/riscv/riscv.cc                     |  20 +++-
>  gcc/config/riscv/riscv.md                     |   2 +
>  gcc/config/riscv/riscv.opt                    |   3 +
>  gcc/config/riscv/sync-rvwmo.md                |  96 +++++++++++++++
>  gcc/config/riscv/sync-ztso.md                 |  80 +++++++++++++
>  gcc/config/riscv/sync.md                      | 111 ++++++------------
>  .../riscv/amo-table-ztso-amo-add-1.c          |  15 +++
>  .../riscv/amo-table-ztso-amo-add-2.c          |  15 +++
>  .../riscv/amo-table-ztso-amo-add-3.c          |  15 +++
>  .../riscv/amo-table-ztso-amo-add-4.c          |  15 +++
>  .../riscv/amo-table-ztso-amo-add-5.c          |  15 +++
>  .../riscv/amo-table-ztso-compare-exchange-1.c |  10 ++
>  .../riscv/amo-table-ztso-compare-exchange-2.c |  10 ++
>  .../riscv/amo-table-ztso-compare-exchange-3.c |  10 ++
>  .../riscv/amo-table-ztso-compare-exchange-4.c |  10 ++
>  .../riscv/amo-table-ztso-compare-exchange-5.c |  10 ++
>  .../riscv/amo-table-ztso-compare-exchange-6.c |  10 ++
>  .../riscv/amo-table-ztso-compare-exchange-7.c |  10 ++
>  .../gcc.target/riscv/amo-table-ztso-fence-1.c |  14 +++
>  .../gcc.target/riscv/amo-table-ztso-fence-2.c |  14 +++
>  .../gcc.target/riscv/amo-table-ztso-fence-3.c |  14 +++
>  .../gcc.target/riscv/amo-table-ztso-fence-4.c |  14 +++
>  .../gcc.target/riscv/amo-table-ztso-fence-5.c |  15 +++
>  .../gcc.target/riscv/amo-table-ztso-load-1.c  |  16 +++
>  .../gcc.target/riscv/amo-table-ztso-load-2.c  |  16 +++
>  .../gcc.target/riscv/amo-table-ztso-load-3.c  |  17 +++
>  .../gcc.target/riscv/amo-table-ztso-store-1.c |  16 +++
>  .../gcc.target/riscv/amo-table-ztso-store-2.c |  16 +++
>  .../gcc.target/riscv/amo-table-ztso-store-3.c |  17 +++
>  .../riscv/amo-table-ztso-subword-amo-add-1.c  |  10 ++
>  .../riscv/amo-table-ztso-subword-amo-add-2.c  |  10 ++
>  .../riscv/amo-table-ztso-subword-amo-add-3.c  |  10 ++
>  .../riscv/amo-table-ztso-subword-amo-add-4.c  |  10 ++
>  .../riscv/amo-table-ztso-subword-amo-add-5.c  |  10 ++
>  36 files changed, 612 insertions(+), 74 deletions(-)
>  create mode 100644 gcc/config/riscv/sync-rvwmo.md
>  create mode 100644 gcc/config/riscv/sync-ztso.md
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-ztso-amo-add-1.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-ztso-amo-add-2.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-ztso-amo-add-3.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-ztso-amo-add-4.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-ztso-amo-add-5.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-ztso-compare-exchange-1.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-ztso-compare-exchange-2.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-ztso-compare-exchange-3.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-ztso-compare-exchange-4.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-ztso-compare-exchange-5.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-ztso-compare-exchange-6.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-ztso-compare-exchange-7.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-ztso-fence-1.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-ztso-fence-2.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-ztso-fence-3.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-ztso-fence-4.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-ztso-fence-5.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-ztso-load-1.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-ztso-load-2.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-ztso-load-3.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-ztso-store-1.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-ztso-store-2.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-ztso-store-3.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-ztso-subword-amo-add-1.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-ztso-subword-amo-add-2.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-ztso-subword-amo-add-3.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-ztso-subword-amo-add-4.c
>  create mode 100644 gcc/testsuite/gcc.target/riscv/amo-table-ztso-subword-amo-add-5.c
>
> diff --git a/gcc/common/config/riscv/riscv-common.cc b/gcc/common/config/riscv/riscv-common.cc
> index 2eb8c7cadff..025dc4e0d92 100644
> --- a/gcc/common/config/riscv/riscv-common.cc
> +++ b/gcc/common/config/riscv/riscv-common.cc
> @@ -71,6 +71,8 @@ static const riscv_implied_info_t riscv_implied_info[] =
>    {"zks", "zksed"},
>    {"zks", "zksh"},
>
> +  {"ztso", "a"},
> +
>    {"v", "zvl128b"},
>    {"v", "zve64d"},
>
> @@ -218,6 +220,8 @@ static const struct riscv_ext_version riscv_ext_version_table[] =
>    {"zkn",   ISA_SPEC_CLASS_NONE, 1, 0},
>    {"zks",   ISA_SPEC_CLASS_NONE, 1, 0},
>
> +  {"ztso",  ISA_SPEC_CLASS_NONE, 1, 0},
> +
>    {"zve32x", ISA_SPEC_CLASS_NONE, 1, 0},
>    {"zve32f", ISA_SPEC_CLASS_NONE, 1, 0},
>    {"zve32d", ISA_SPEC_CLASS_NONE, 1, 0},
> @@ -1407,6 +1411,8 @@ static const riscv_ext_flag_table_t riscv_ext_flag_table[] =
>    {"svinval", &gcc_options::x_riscv_sv_subext, MASK_SVINVAL},
>    {"svnapot", &gcc_options::x_riscv_sv_subext, MASK_SVNAPOT},
>
> +  {"ztso", &gcc_options::x_riscv_ztso_subext, MASK_ZTSO},
> +
>    {"xtheadba",      &gcc_options::x_riscv_xthead_subext, MASK_XTHEADBA},
>    {"xtheadbb",      &gcc_options::x_riscv_xthead_subext, MASK_XTHEADBB},
>    {"xtheadbs",      &gcc_options::x_riscv_xthead_subext, MASK_XTHEADBS},
> diff --git a/gcc/config/riscv/riscv-opts.h b/gcc/config/riscv/riscv-opts.h
> index 28d9b81bd80..d6d785d0075 100644
> --- a/gcc/config/riscv/riscv-opts.h
> +++ b/gcc/config/riscv/riscv-opts.h
> @@ -152,6 +152,10 @@ enum riscv_entity
>  #define TARGET_ZKSH   ((riscv_zk_subext & MASK_ZKSH) != 0)
>  #define TARGET_ZKT    ((riscv_zk_subext & MASK_ZKT) != 0)
>
> +#define MASK_ZTSO   (1 << 0)
> +
> +#define TARGET_ZTSO ((riscv_ztso_subext & MASK_ZTSO) != 0)
> +
>  #define MASK_VECTOR_ELEN_32    (1 << 0)
>  #define MASK_VECTOR_ELEN_64    (1 << 1)
>  #define MASK_VECTOR_ELEN_FP_32 (1 << 2)
> diff --git a/gcc/config/riscv/riscv.cc b/gcc/config/riscv/riscv.cc
> index f9b53d21d1b..0b48a8e618e 100644
> --- a/gcc/config/riscv/riscv.cc
> +++ b/gcc/config/riscv/riscv.cc
> @@ -4765,6 +4765,10 @@ riscv_union_memmodels (enum memmodel model1, enum memmodel model2)
>  static bool
>  riscv_memmodel_needs_amo_acquire (enum memmodel model)
>  {
> +  /* ZTSO amo mappings require no annotations.  */
> +  if (TARGET_ZTSO)
> +    return false;
> +
>    switch (model)
>      {
>        case MEMMODEL_ACQ_REL:
> @@ -4788,6 +4792,10 @@ riscv_memmodel_needs_amo_acquire (enum memmodel model)
>  static bool
>  riscv_memmodel_needs_amo_release (enum memmodel model)
>  {
> +  /* ZTSO amo mappings require no annotations.  */
> +  if (TARGET_ZTSO)
> +    return false;
> +
>    switch (model)
>      {
>        case MEMMODEL_ACQ_REL:
> @@ -5007,7 +5015,10 @@ riscv_print_operand (FILE *file, rtx op, int letter)
>
>      case 'I': {
>        const enum memmodel model = memmodel_base (INTVAL (op));
> -      if (model == MEMMODEL_SEQ_CST)
> +      if (TARGET_ZTSO && model != MEMMODEL_SEQ_CST)
> +	/* LR ops only have an annotation for SEQ_CST in the Ztso mapping.  */
> +	break;
> +      else if (model == MEMMODEL_SEQ_CST)
>  	fputs (".aqrl", file);
>        else if (riscv_memmodel_needs_amo_acquire (model))
>  	fputs (".aq", file);
> @@ -5016,7 +5027,12 @@ riscv_print_operand (FILE *file, rtx op, int letter)
>
>      case 'J': {
>        const enum memmodel model = memmodel_base (INTVAL (op));
> -      if (riscv_memmodel_needs_amo_release (model))
> +      if (TARGET_ZTSO && model == MEMMODEL_SEQ_CST)
> +	/* SC ops only have an annotation for SEQ_CST in the Ztso mapping.  */
> +	fputs (".rl", file);
> +      else if (TARGET_ZTSO)
> +	break;
> +      else if (riscv_memmodel_needs_amo_release (model))
>  	fputs (".rl", file);
>        break;
>      }
> diff --git a/gcc/config/riscv/riscv.md b/gcc/config/riscv/riscv.md
> index 688fd697255..b456fa6abb3 100644
> --- a/gcc/config/riscv/riscv.md
> +++ b/gcc/config/riscv/riscv.md
> @@ -3321,6 +3321,8 @@
>  (include "bitmanip.md")
>  (include "crypto.md")
>  (include "sync.md")
> +(include "sync-rvwmo.md")
> +(include "sync-ztso.md")
>  (include "peephole.md")
>  (include "pic.md")
>  (include "generic.md")
> diff --git a/gcc/config/riscv/riscv.opt b/gcc/config/riscv/riscv.opt
> index 4dfd8f78ad5..717b51b5305 100644
> --- a/gcc/config/riscv/riscv.opt
> +++ b/gcc/config/riscv/riscv.opt
> @@ -245,6 +245,9 @@ int riscv_zm_subext
>  TargetVariable
>  int riscv_sv_subext
>
> +TargetVariable
> +int riscv_ztso_subext
> +
>  TargetVariable
>  int riscv_xthead_subext
>
> diff --git a/gcc/config/riscv/sync-rvwmo.md b/gcc/config/riscv/sync-rvwmo.md
> new file mode 100644
> index 00000000000..1fc7cf16b5b
> --- /dev/null
> +++ b/gcc/config/riscv/sync-rvwmo.md
> @@ -0,0 +1,96 @@
> +;; Machine description for RISC-V atomic operations.
> +;; Copyright (C) 2011-2023 Free Software Foundation, Inc.
> +;; Contributed by Andrew Waterman (andrew@sifive.com).
> +;; Based on MIPS target for GNU compiler.
> +
> +;; This file is part of GCC.
> +
> +;; GCC is free software; you can redistribute it and/or modify
> +;; it under the terms of the GNU General Public License as published by
> +;; the Free Software Foundation; either version 3, or (at your option)
> +;; any later version.
> +
> +;; GCC is distributed in the hope that it will be useful,
> +;; but WITHOUT ANY WARRANTY; without even the implied warranty of
> +;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +;; GNU General Public License for more details.
> +
> +;; You should have received a copy of the GNU General Public License
> +;; along with GCC; see the file COPYING3.  If not see
> +;; <http://www.gnu.org/licenses/>.
> +
> +;; Memory barrier.
> +
> +(define_insn "mem_thread_fence_rvwmo"
> +  [(set (match_operand:BLK 0 "" "")
> +	(unspec:BLK [(match_dup 0)] UNSPEC_MEMORY_BARRIER))
> +   (match_operand:SI 1 "const_int_operand" "")]  ;; model
> +  "!TARGET_ZTSO"
> +  {
> +    enum memmodel model = (enum memmodel) INTVAL (operands[1]);
> +    model = memmodel_base (model);
> +
> +    if (model == MEMMODEL_SEQ_CST)
> +	return "fence\trw,rw";
> +    else if (model == MEMMODEL_ACQ_REL)
> +	return "fence.tso";
> +    else if (model == MEMMODEL_ACQUIRE)
> +	return "fence\tr,rw";
> +    else if (model == MEMMODEL_RELEASE)
> +	return "fence\trw,w";
> +    else
> +	gcc_unreachable ();
> +  }
> +  [(set (attr "length") (const_int 4))])
> +
> +;; Atomic memory operations.
> +
> +(define_insn "atomic_load_rvwmo<mode>"
> +  [(set (match_operand:GPR 0 "register_operand" "=r")
> +	(unspec_volatile:GPR
> +	    [(match_operand:GPR 1 "memory_operand" "A")
> +	     (match_operand:SI 2 "const_int_operand")]  ;; model
> +	 UNSPEC_ATOMIC_LOAD))]
> +  "TARGET_ATOMIC && !TARGET_ZTSO"
> +  {
> +    enum memmodel model = (enum memmodel) INTVAL (operands[2]);
> +    model = memmodel_base (model);
> +
> +    if (model == MEMMODEL_SEQ_CST)
> +      return "fence\trw,rw\;"
> +	     "l<amo>\t%0,%1\;"
> +	     "fence\tr,rw";
> +    if (model == MEMMODEL_ACQUIRE)
> +      return "l<amo>\t%0,%1\;"
> +	     "fence\tr,rw";
> +    else
> +      return "l<amo>\t%0,%1";
> +  }
> +  [(set_attr "type" "atomic")
> +   (set (attr "length") (const_int 12))])
> +
> +;; Implement atomic stores with conservative fences.
> +;; This allows us to be compatible with the ISA manual Table A.6 and Table A.7.
> +(define_insn "atomic_store_rvwmo<mode>"
> +  [(set (match_operand:GPR 0 "memory_operand" "=A")
> +	(unspec_volatile:GPR
> +	    [(match_operand:GPR 1 "reg_or_0_operand" "rJ")
> +	     (match_operand:SI 2 "const_int_operand")]  ;; model
> +	 UNSPEC_ATOMIC_STORE))]
> +  "TARGET_ATOMIC && !TARGET_ZTSO"
> +  {
> +    enum memmodel model = (enum memmodel) INTVAL (operands[2]);
> +    model = memmodel_base (model);
> +
> +    if (model == MEMMODEL_SEQ_CST)
> +      return "fence\trw,w\;"
> +	     "s<amo>\t%z1,%0\;"
> +	     "fence\trw,rw";
> +    if (model == MEMMODEL_RELEASE)
> +      return "fence\trw,w\;"
> +	     "s<amo>\t%z1,%0";
> +    else
> +      return "s<amo>\t%z1,%0";
> +  }
> +  [(set_attr "type" "atomic")
> +   (set (attr "length") (const_int 12))])
> diff --git a/gcc/config/riscv/sync-ztso.md b/gcc/config/riscv/sync-ztso.md
> new file mode 100644
> index 00000000000..91c2a48c069
> --- /dev/null
> +++ b/gcc/config/riscv/sync-ztso.md
> @@ -0,0 +1,80 @@
> +;; Machine description for RISC-V atomic operations.
> +;; Copyright (C) 2011-2023 Free Software Foundation, Inc.
> +;; Contributed by Andrew Waterman (andrew@sifive.com).
> +;; Based on MIPS target for GNU compiler.
> +
> +;; This file is part of GCC.
> +
> +;; GCC is free software; you can redistribute it and/or modify
> +;; it under the terms of the GNU General Public License as published by
> +;; the Free Software Foundation; either version 3, or (at your option)
> +;; any later version.
> +
> +;; GCC is distributed in the hope that it will be useful,
> +;; but WITHOUT ANY WARRANTY; without even the implied warranty of
> +;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +;; GNU General Public License for more details.
> +
> +;; You should have received a copy of the GNU General Public License
> +;; along with GCC; see the file COPYING3.  If not see
> +;; <http://www.gnu.org/licenses/>.
> +
> +;; Memory barriers.
> +
> +(define_insn "mem_thread_fence_ztso"
> +  [(set (match_operand:BLK 0 "" "")
> +	(unspec:BLK [(match_dup 0)] UNSPEC_MEMORY_BARRIER))
> +   (match_operand:SI 1 "const_int_operand" "")]  ;; model
> +  "TARGET_ZTSO"
> +  {
> +    enum memmodel model = (enum memmodel) INTVAL (operands[1]);
> +    model = memmodel_base (model);
> +
> +    if (model == MEMMODEL_SEQ_CST)
> +	return "fence\trw,rw";
> +    else
> +	gcc_unreachable ();
> +  }
> +  [(set (attr "length") (const_int 4))])
> +
> +;; Atomic memory operations.
> +
> +(define_insn "atomic_load_ztso<mode>"
> +  [(set (match_operand:GPR 0 "register_operand" "=r")
> +	(unspec_volatile:GPR
> +	    [(match_operand:GPR 1 "memory_operand" "A")
> +	     (match_operand:SI 2 "const_int_operand")]  ;; model
> +	 UNSPEC_ATOMIC_LOAD))]
> +  "TARGET_ATOMIC && TARGET_ZTSO"
> +  {
> +    enum memmodel model = (enum memmodel) INTVAL (operands[2]);
> +    model = memmodel_base (model);
> +
> +    if (model == MEMMODEL_SEQ_CST)
> +      return "fence\trw,rw\;"
> +	     "l<amo>\t%0,%1\;";
> +    else
> +      return "l<amo>\t%0,%1";
> +  }
> +  [(set_attr "type" "atomic")
> +   (set (attr "length") (const_int 12))])
> +
> +(define_insn "atomic_store_ztso<mode>"
> +  [(set (match_operand:GPR 0 "memory_operand" "=A")
> +	(unspec_volatile:GPR
> +	    [(match_operand:GPR 1 "reg_or_0_operand" "rJ")
> +	     (match_operand:SI 2 "const_int_operand")]  ;; model
> +	 UNSPEC_ATOMIC_STORE))]
> +  "TARGET_ATOMIC && TARGET_ZTSO"
> +  {
> +    enum memmodel model = (enum memmodel) INTVAL (operands[2]);
> +    model = memmodel_base (model);
> +
> +    if (model == MEMMODEL_SEQ_CST)
> +      return "s<amo>\t%z1,%0\;"
> +	     "fence\trw,rw";
> +    else
> +      return "s<amo>\t%z1,%0";
> +  }
> +  [(set_attr "type" "atomic")
> +   (set (attr "length") (const_int 8))])
> \ No newline at end of file
> diff --git a/gcc/config/riscv/sync.md b/gcc/config/riscv/sync.md
> index 9fc626267de..2f85951508f 100644
> --- a/gcc/config/riscv/sync.md
> +++ b/gcc/config/riscv/sync.md
> @@ -36,88 +36,55 @@
>  (define_expand "mem_thread_fence"
>    [(match_operand:SI 0 "const_int_operand" "")] ;; model
>    ""
> -{
> -  if (INTVAL (operands[0]) != MEMMODEL_RELAXED)
> -    {
> -      rtx mem = gen_rtx_MEM (BLKmode, gen_rtx_SCRATCH (Pmode));
> -      MEM_VOLATILE_P (mem) = 1;
> -      emit_insn (gen_mem_thread_fence_1 (mem, operands[0]));
> -    }
> -  DONE;
> -})
> -
> -(define_insn "mem_thread_fence_1"
> -  [(set (match_operand:BLK 0 "" "")
> -	(unspec:BLK [(match_dup 0)] UNSPEC_MEMORY_BARRIER))
> -   (match_operand:SI 1 "const_int_operand" "")] ;; model
> -  ""
>    {
> -    enum memmodel model = (enum memmodel) INTVAL (operands[1]);
> -    model = memmodel_base (model);
> -    if (model == MEMMODEL_SEQ_CST)
> -	return "fence\trw,rw";
> -    else if (model == MEMMODEL_ACQ_REL)
> -	return "fence.tso";
> -    else if (model == MEMMODEL_ACQUIRE)
> -	return "fence\tr,rw";
> -    else if (model == MEMMODEL_RELEASE)
> -	return "fence\trw,w";
> -    else
> -	gcc_unreachable ();
> -  }
> -  [(set (attr "length") (const_int 4))])
> +    enum memmodel model = memmodel_base (INTVAL (operands[0]));
> +
> +    if (TARGET_ZTSO && model == MEMMODEL_SEQ_CST)
> +      {
> +	rtx mem = gen_rtx_MEM (BLKmode, gen_rtx_SCRATCH (Pmode));
> +	MEM_VOLATILE_P (mem) = 1;
> +	emit_insn (gen_mem_thread_fence_ztso (mem, operands[0]));
> +      }
> +    else if (!TARGET_ZTSO && model != MEMMODEL_RELAXED)
> +      {
> +	rtx mem = gen_rtx_MEM (BLKmode, gen_rtx_SCRATCH (Pmode));
> +	MEM_VOLATILE_P (mem) = 1;
> +	emit_insn (gen_mem_thread_fence_rvwmo (mem, operands[0]));
> +      }
> +    DONE;
> +  })
>
>  ;; Atomic memory operations.
>
> -(define_insn "atomic_load<mode>"
> -  [(set (match_operand:GPR 0 "register_operand" "=r")
> -    (unspec_volatile:GPR
> -      [(match_operand:GPR 1 "memory_operand" "A")
> -       (match_operand:SI 2 "const_int_operand")]      ;; model
> -      UNSPEC_ATOMIC_LOAD))]
> +(define_expand "atomic_load<mode>"
> +  [(match_operand:GPR 0 "register_operand")
> +   (match_operand:GPR 1 "memory_operand")
> +   (match_operand:SI 2 "const_int_operand")] ;; model
>    "TARGET_ATOMIC"
>    {
> -    enum memmodel model = (enum memmodel) INTVAL (operands[2]);
> -    model = memmodel_base (model);
> -
> -    if (model == MEMMODEL_SEQ_CST)
> -      return "fence\trw,rw\;"
> -	     "l<amo>\t%0,%1\;"
> -	     "fence\tr,rw";
> -    if (model == MEMMODEL_ACQUIRE)
> -      return "l<amo>\t%0,%1\;"
> -	     "fence\tr,rw";
> +    if (TARGET_ZTSO)
> +      emit_insn (gen_atomic_load_ztso<mode> (operands[0], operands[1],
> +					     operands[2]));
>      else
> -      return "l<amo>\t%0,%1";
> -  }
> -  [(set_attr "type" "atomic")
> -   (set (attr "length") (const_int 12))])
> -
> -;; Implement atomic stores with conservative fences.
> -;; This allows us to be compatible with the ISA manual Table A.6 and Table A.7.
> -(define_insn "atomic_store<mode>"
> -  [(set (match_operand:GPR 0 "memory_operand" "=A")
> -    (unspec_volatile:GPR
> -      [(match_operand:GPR 1 "reg_or_0_operand" "rJ")
> -       (match_operand:SI 2 "const_int_operand")]      ;; model
> -      UNSPEC_ATOMIC_STORE))]
> +      emit_insn (gen_atomic_load_rvwmo<mode> (operands[0], operands[1],
> +					      operands[2]));
> +    DONE;
> +  })
> +
> +(define_expand "atomic_store<mode>"
> +  [(match_operand:GPR 0 "memory_operand")
> +   (match_operand:GPR 1 "reg_or_0_operand")
> +   (match_operand:SI 2 "const_int_operand")] ;; model
>    "TARGET_ATOMIC"
>    {
> -    enum memmodel model = (enum memmodel) INTVAL (operands[2]);
> -    model = memmodel_base (model);
> -
> -    if (model == MEMMODEL_SEQ_CST)
> -      return "fence\trw,w\;"
> -	     "s<amo>\t%z1,%0\;"
> -	     "fence\trw,rw";
> -    if (model == MEMMODEL_RELEASE)
> -      return "fence\trw,w\;"
> -	     "s<amo>\t%z1,%0";
> +    if (TARGET_ZTSO)
> +      emit_insn (gen_atomic_store_ztso<mode> (operands[0], operands[1],
> +					      operands[2]));
>      else
> -      return "s<amo>\t%z1,%0";
> -  }
> -  [(set_attr "type" "atomic")
> -   (set (attr "length") (const_int 12))])
> +      emit_insn (gen_atomic_store_rvwmo<mode> (operands[0], operands[1],
> +					       operands[2]));
> +    DONE;
> +  })
>
>  (define_insn "atomic_<atomic_optab><mode>"
>    [(set (match_operand:GPR 0 "memory_operand" "+A")
> diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-ztso-amo-add-1.c b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-amo-add-1.c
> new file mode 100644
> index 00000000000..a7097e9aab9
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-amo-add-1.c
> @@ -0,0 +1,15 @@
> +/* { dg-do compile } */
> +/* Verify that atomic op mappings match the Ztso suggested mapping.  */
> +/* { dg-options "-march=rv64id_ztso -O3" } */
> +/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */
> +/* { dg-final { check-function-bodies "**" "" } } */
> +
> +/*
> +** foo:
> +**	amoadd\.w\tzero,a1,0\(a0\)
> +**	ret
> +*/
> +void foo (int* bar, int* baz)
> +{
> +  __atomic_add_fetch(bar, baz, __ATOMIC_RELAXED);
> +}
> diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-ztso-amo-add-2.c b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-amo-add-2.c
> new file mode 100644
> index 00000000000..8e993903439
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-amo-add-2.c
> @@ -0,0 +1,15 @@
> +/* { dg-do compile } */
> +/* Verify that atomic op mappings the Ztso suggested mapping.  */
> +/* { dg-options "-march=rv64id_ztso -O3" } */
> +/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */
> +/* { dg-final { check-function-bodies "**" "" } } */
> +
> +/*
> +** foo:
> +**	amoadd\.w\tzero,a1,0\(a0\)
> +**	ret
> +*/
> +void foo (int* bar, int* baz)
> +{
> +  __atomic_add_fetch(bar, baz, __ATOMIC_ACQUIRE);
> +}
> diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-ztso-amo-add-3.c b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-amo-add-3.c
> new file mode 100644
> index 00000000000..e1cd209fcb2
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-amo-add-3.c
> @@ -0,0 +1,15 @@
> +/* { dg-do compile } */
> +/* Verify that atomic op mappings match the Ztso suggested mapping.  */
> +/* { dg-options "-march=rv64id_ztso -O3" } */
> +/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */
> +/* { dg-final { check-function-bodies "**" "" } } */
> +
> +/*
> +** foo:
> +**	amoadd\.w\tzero,a1,0\(a0\)
> +**	ret
> +*/
> +void foo (int* bar, int* baz)
> +{
> +  __atomic_add_fetch(bar, baz, __ATOMIC_RELEASE);
> +}
> diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-ztso-amo-add-4.c b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-amo-add-4.c
> new file mode 100644
> index 00000000000..484d11eb100
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-amo-add-4.c
> @@ -0,0 +1,15 @@
> +/* { dg-do compile } */
> +/* Verify that atomic op mappings match the Ztso suggested mapping.  */
> +/* { dg-options "-march=rv64id_ztso -O3" } */
> +/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */
> +/* { dg-final { check-function-bodies "**" "" } } */
> +
> +/*
> +** foo:
> +**	amoadd\.w\tzero,a1,0\(a0\)
> +**	ret
> +*/
> +void foo (int* bar, int* baz)
> +{
> +  __atomic_add_fetch(bar, baz, __ATOMIC_ACQ_REL);
> +}
> diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-ztso-amo-add-5.c b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-amo-add-5.c
> new file mode 100644
> index 00000000000..2117283fd0d
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-amo-add-5.c
> @@ -0,0 +1,15 @@
> +/* { dg-do compile } */
> +/* Verify that atomic op mappings match the Ztso suggested mapping.  */
> +/* { dg-options "-march=rv64id_ztso -O3" } */
> +/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */
> +/* { dg-final { check-function-bodies "**" "" } } */
> +
> +/*
> +** foo:
> +**	amoadd\.w\tzero,a1,0\(a0\)
> +**	ret
> +*/
> +void foo (int* bar, int* baz)
> +{
> +  __atomic_add_fetch(bar, baz, __ATOMIC_SEQ_CST);
> +}
> diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-ztso-compare-exchange-1.c b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-compare-exchange-1.c
> new file mode 100644
> index 00000000000..03247c632fd
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-compare-exchange-1.c
> @@ -0,0 +1,10 @@
> +/* { dg-do compile } */
> +/* Verify that compare exchange mappings match the Ztso suggested mapping.  */
> +/* { dg-options "-march=rv64id_ztso" } */
> +/* { dg-final { scan-assembler-times "lr.w\t" 1 } } */
> +/* { dg-final { scan-assembler-times "sc.w\t" 1 } } */
> +
> +void foo (int bar, int baz, int qux)
> +{
> +  __atomic_compare_exchange_n(&bar, &baz, qux, 1, __ATOMIC_RELAXED, __ATOMIC_RELAXED);
> +}
> diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-ztso-compare-exchange-2.c b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-compare-exchange-2.c
> new file mode 100644
> index 00000000000..46356747067
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-compare-exchange-2.c
> @@ -0,0 +1,10 @@
> +/* { dg-do compile } */
> +/* Verify that compare exchange mappings match the Ztso suggested mapping.  */
> +/* { dg-options "-march=rv64id_ztso" } */
> +/* { dg-final { scan-assembler-times "lr.w\t" 1 } } */
> +/* { dg-final { scan-assembler-times "sc.w\t" 1 } } */
> +
> +void foo (int bar, int baz, int qux)
> +{
> +  __atomic_compare_exchange_n(&bar, &baz, qux, 1, __ATOMIC_CONSUME, __ATOMIC_CONSUME);
> +}
> diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-ztso-compare-exchange-3.c b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-compare-exchange-3.c
> new file mode 100644
> index 00000000000..cf252dba6ec
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-compare-exchange-3.c
> @@ -0,0 +1,10 @@
> +/* { dg-do compile } */
> +/* Verify that compare exchange mappings match the Ztso suggested mapping.  */
> +/* { dg-options "-march=rv64id_ztso" } */
> +/* { dg-final { scan-assembler-times "lr.w\t" 1 } } */
> +/* { dg-final { scan-assembler-times "sc.w\t" 1 } } */
> +
> +void foo (int bar, int baz, int qux)
> +{
> +  __atomic_compare_exchange_n(&bar, &baz, qux, 1, __ATOMIC_ACQUIRE, __ATOMIC_ACQUIRE);
> +}
> diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-ztso-compare-exchange-4.c b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-compare-exchange-4.c
> new file mode 100644
> index 00000000000..9a0c86569b5
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-compare-exchange-4.c
> @@ -0,0 +1,10 @@
> +/* { dg-do compile } */
> +/* Verify that compare exchange mappings match the Ztso suggested mapping.  */
> +/* { dg-options "-march=rv64id_ztso" } */
> +/* { dg-final { scan-assembler-times "lr.w\t" 1 } } */
> +/* { dg-final { scan-assembler-times "sc.w\t" 1 } } */
> +
> +void foo (int bar, int baz, int qux)
> +{
> +  __atomic_compare_exchange_n(&bar, &baz, qux, 1, __ATOMIC_RELEASE, __ATOMIC_RELAXED);
> +}
> diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-ztso-compare-exchange-5.c b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-compare-exchange-5.c
> new file mode 100644
> index 00000000000..750ba89cd74
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-compare-exchange-5.c
> @@ -0,0 +1,10 @@
> +/* { dg-do compile } */
> +/* Verify that compare exchange mappings match the Ztso suggested mapping.  */
> +/* { dg-options "-march=rv64id_ztso" } */
> +/* { dg-final { scan-assembler-times "lr.w.aqrl\t" 1 } } */
> +/* { dg-final { scan-assembler-times "sc.w.rl\t" 1 } } */
> +
> +void foo (int bar, int baz, int qux)
> +{
> +  __atomic_compare_exchange_n(&bar, &baz, qux, 1, __ATOMIC_SEQ_CST, __ATOMIC_SEQ_CST);
> +}
> diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-ztso-compare-exchange-6.c b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-compare-exchange-6.c
> new file mode 100644
> index 00000000000..0d0bf26d95a
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-compare-exchange-6.c
> @@ -0,0 +1,10 @@
> +/* { dg-do compile } */
> +/* Verify that compare exchange mappings match the Ztso suggested mapping.  */
> +/* { dg-options "-march=rv64id_ztso" } */
> +/* { dg-final { scan-assembler-times "lr.w\t" 1 } } */
> +/* { dg-final { scan-assembler-times "sc.w\t" 1 } } */
> +
> +void foo (int bar, int baz, int qux)
> +{
> +  __atomic_compare_exchange_n(&bar, &baz, qux, 1, __ATOMIC_RELEASE, __ATOMIC_ACQUIRE);
> +}
> diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-ztso-compare-exchange-7.c b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-compare-exchange-7.c
> new file mode 100644
> index 00000000000..838568ed588
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-compare-exchange-7.c
> @@ -0,0 +1,10 @@
> +/* { dg-do compile } */
> +/* Verify that compare exchange mappings match the Ztso suggested mapping.  */
> +/* { dg-options "-march=rv64id_ztso" } */
> +/* { dg-final { scan-assembler-times "lr.w.aqrl\t" 1 } } */
> +/* { dg-final { scan-assembler-times "sc.w.rl\t" 1 } } */
> +
> +void foo (int bar, int baz, int qux)
> +{
> +  __atomic_compare_exchange_n(&bar, &baz, qux, 1, __ATOMIC_SEQ_CST, __ATOMIC_RELAXED);
> +}
> diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-ztso-fence-1.c b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-fence-1.c
> new file mode 100644
> index 00000000000..a9ba4a252e7
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-fence-1.c
> @@ -0,0 +1,14 @@
> +/* { dg-do compile } */
> +/* Verify that fence mappings match the Ztso suggested mapping.  */
> +/* { dg-options "-march=rv64id_ztso -O3" } */
> +/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */
> +/* { dg-final { check-function-bodies "**" "" } } */
> +
> +/*
> +** foo:
> +**	ret
> +*/
> +void foo()
> +{
> +  __atomic_thread_fence(__ATOMIC_RELAXED);
> +}
> diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-ztso-fence-2.c b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-fence-2.c
> new file mode 100644
> index 00000000000..9b083431d1a
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-fence-2.c
> @@ -0,0 +1,14 @@
> +/* { dg-do compile } */
> +/* Verify that fence mappings match the Ztso suggested mapping.  */
> +/* { dg-options "-march=rv64id_ztso -O3" } */
> +/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */
> +/* { dg-final { check-function-bodies "**" "" } } */
> +
> +/*
> +** foo:
> +**	ret
> +*/
> +void foo()
> +{
> +  __atomic_thread_fence(__ATOMIC_ACQUIRE);
> +}
> diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-ztso-fence-3.c b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-fence-3.c
> new file mode 100644
> index 00000000000..db21ee6f031
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-fence-3.c
> @@ -0,0 +1,14 @@
> +/* { dg-do compile } */
> +/* Verify that fence mappings match the Ztso suggested mapping.  */
> +/* { dg-options "-march=rv64id_ztso -O3" } */
> +/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */
> +/* { dg-final { check-function-bodies "**" "" } } */
> +
> +/*
> +** foo:
> +**	ret
> +*/
> +void foo()
> +{
> +  __atomic_thread_fence(__ATOMIC_RELEASE);
> +}
> diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-ztso-fence-4.c b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-fence-4.c
> new file mode 100644
> index 00000000000..d5bc61b57e8
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-fence-4.c
> @@ -0,0 +1,14 @@
> +/* { dg-do compile } */
> +/* Verify that fence mappings match the Ztso suggested mapping.  */
> +/* { dg-options "-march=rv64id_ztso -O3" } */
> +/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */
> +/* { dg-final { check-function-bodies "**" "" } } */
> +
> +/*
> +** foo:
> +**	ret
> +*/
> +void foo()
> +{
> +  __atomic_thread_fence(__ATOMIC_ACQ_REL);
> +}
> diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-ztso-fence-5.c b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-fence-5.c
> new file mode 100644
> index 00000000000..e5e5f4e5012
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-fence-5.c
> @@ -0,0 +1,15 @@
> +/* { dg-do compile } */
> +/* Verify that fence mappings match the Ztso suggested mapping.  */
> +/* { dg-options "-march=rv64id_ztso -O3" } */
> +/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */
> +/* { dg-final { check-function-bodies "**" "" } } */
> +
> +/*
> +** foo:
> +**	fence\trw,rw
> +**	ret
> +*/
> +void foo()
> +{
> +  __atomic_thread_fence(__ATOMIC_SEQ_CST);
> +}
> diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-ztso-load-1.c b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-load-1.c
> new file mode 100644
> index 00000000000..be5f8d310a4
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-load-1.c
> @@ -0,0 +1,16 @@
> +/* { dg-do compile } */
> +/* Verify that load mappings match the Ztso suggested mapping.  */
> +/* { dg-options "-march=rv64id_ztso -O3" } */
> +/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */
> +/* { dg-final { check-function-bodies "**" "" } } */
> +
> +/*
> +** foo:
> +**	lw\ta[0-9]+,0\(a0\)
> +**	sw\ta[0-9]+,0\(a1\)
> +**	ret
> +*/
> +void foo (int* bar, int* baz)
> +{
> +  __atomic_load(bar, baz, __ATOMIC_RELAXED);
> +}
> diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-ztso-load-2.c b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-load-2.c
> new file mode 100644
> index 00000000000..30583841648
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-load-2.c
> @@ -0,0 +1,16 @@
> +/* { dg-do compile } */
> +/* Verify that load mappings match the Ztso suggested mapping.  */
> +/* { dg-options "-march=rv64id_ztso -O3" } */
> +/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */
> +/* { dg-final { check-function-bodies "**" "" } } */
> +
> +/*
> +** foo:
> +**	lw\ta[0-9]+,0\(a0\)
> +**	sw\ta[0-9]+,0\(a1\)
> +**	ret
> +*/
> +void foo (int* bar, int* baz)
> +{
> +  __atomic_load(bar, baz, __ATOMIC_ACQUIRE);
> +}
> diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-ztso-load-3.c b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-load-3.c
> new file mode 100644
> index 00000000000..c1c3b57bfe5
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-load-3.c
> @@ -0,0 +1,17 @@
> +/* { dg-do compile } */
> +/* Verify that load mappings match the Ztso suggested mapping.  */
> +/* { dg-options "-march=rv64id_ztso -O3" } */
> +/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */
> +/* { dg-final { check-function-bodies "**" "" } } */
> +
> +/*
> +** foo:
> +**	fence\trw,rw
> +**	lw\ta[0-9]+,0\(a0\)
> +**	sw\ta[0-9]+,0\(a1\)
> +**	ret
> +*/
> +void foo (int* bar, int* baz)
> +{
> +  __atomic_load(bar, baz, __ATOMIC_SEQ_CST);
> +}
> diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-ztso-store-1.c b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-store-1.c
> new file mode 100644
> index 00000000000..f5519db22f7
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-store-1.c
> @@ -0,0 +1,16 @@
> +/* { dg-do compile } */
> +/* Verify that store mappings match the Ztso suggested mapping.  */
> +/* { dg-options "-march=rv64id_ztso -O3" } */
> +/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */
> +/* { dg-final { check-function-bodies "**" "" } } */
> +
> +/*
> +** foo:
> +**	lw\ta[0-9]+,0\(a1\)
> +**	sw\ta[0-9]+,0\(a0\)
> +**	ret
> +*/
> +void foo (int* bar, int* baz)
> +{
> +  __atomic_store(bar, baz, __ATOMIC_RELAXED);
> +}
> diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-ztso-store-2.c b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-store-2.c
> new file mode 100644
> index 00000000000..440f407e66a
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-store-2.c
> @@ -0,0 +1,16 @@
> +/* { dg-do compile } */
> +/* Verify that store mappings match the Ztso suggested mapping.  */
> +/* { dg-options "-march=rv64id_ztso -O3" } */
> +/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */
> +/* { dg-final { check-function-bodies "**" "" } } */
> +
> +/*
> +** foo:
> +**	lw\ta[0-9]+,0\(a1\)
> +**	sw\ta[0-9]+,0\(a0\)
> +**	ret
> +*/
> +void foo (int* bar, int* baz)
> +{
> +  __atomic_store(bar, baz, __ATOMIC_RELEASE);
> +}
> diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-ztso-store-3.c b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-store-3.c
> new file mode 100644
> index 00000000000..a92d4d1ce9a
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-store-3.c
> @@ -0,0 +1,17 @@
> +/* { dg-do compile } */
> +/* Verify that store mappings match the Ztso suggested mapping.  */
> +/* { dg-options "-march=rv64id_ztso -O3" } */
> +/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */
> +/* { dg-final { check-function-bodies "**" "" } } */
> +
> +/*
> +** foo:
> +**	lw\ta[0-9]+,0\(a1\)
> +**	sw\ta[0-9]+,0\(a0\)
> +**	fence\trw,rw
> +**	ret
> +*/
> +void foo (int* bar, int* baz)
> +{
> +  __atomic_store(bar, baz, __ATOMIC_SEQ_CST);
> +}
> diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-ztso-subword-amo-add-1.c b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-subword-amo-add-1.c
> new file mode 100644
> index 00000000000..e187a6b7335
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-subword-amo-add-1.c
> @@ -0,0 +1,10 @@
> +/* { dg-do compile } */
> +/* Verify that subword atomic op mappings match the Ztso suggested mapping.  */
> +/* { dg-options "-march=rv64id_ztso" } */
> +/* { dg-final { scan-assembler-times "lr.w\t" 1 } } */
> +/* { dg-final { scan-assembler-times "sc.w\t" 1 } } */
> +
> +void foo (short* bar, short* baz)
> +{
> +  __atomic_add_fetch(bar, baz, __ATOMIC_RELAXED);
> +}
> diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-ztso-subword-amo-add-2.c b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-subword-amo-add-2.c
> new file mode 100644
> index 00000000000..1e0867624ba
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-subword-amo-add-2.c
> @@ -0,0 +1,10 @@
> +/* { dg-do compile } */
> +/* Verify that subword atomic op mappings match the Ztso suggested mapping.  */
> +/* { dg-options "-march=rv64id_ztso" } */
> +/* { dg-final { scan-assembler-times "lr.w\t" 1 } } */
> +/* { dg-final { scan-assembler-times "sc.w\t" 1 } } */
> +
> +void foo (short* bar, short* baz)
> +{
> +  __atomic_add_fetch(bar, baz, __ATOMIC_ACQUIRE);
> +}
> diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-ztso-subword-amo-add-3.c b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-subword-amo-add-3.c
> new file mode 100644
> index 00000000000..4b2ef967375
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-subword-amo-add-3.c
> @@ -0,0 +1,10 @@
> +/* { dg-do compile } */
> +/* Verify that subword atomic op mappings match the Ztso suggested mapping.  */
> +/* { dg-options "-march=rv64id_ztso" } */
> +/* { dg-final { scan-assembler-times "lr.w\t" 1 } } */
> +/* { dg-final { scan-assembler-times "sc.w\t" 1 } } */
> +
> +void foo (short* bar, short* baz)
> +{
> +  __atomic_add_fetch(bar, baz, __ATOMIC_RELEASE);
> +}
> diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-ztso-subword-amo-add-4.c b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-subword-amo-add-4.c
> new file mode 100644
> index 00000000000..87e5356f0a7
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-subword-amo-add-4.c
> @@ -0,0 +1,10 @@
> +/* { dg-do compile } */
> +/* Verify that subword atomic op mappings match the Ztso suggested mapping.  */
> +/* { dg-options "-march=rv64id_ztso" } */
> +/* { dg-final { scan-assembler-times "lr.w\t" 1 } } */
> +/* { dg-final { scan-assembler-times "sc.w\t" 1 } } */
> +
> +void foo (short* bar, short* baz)
> +{
> +  __atomic_add_fetch(bar, baz, __ATOMIC_ACQ_REL);
> +}
> diff --git a/gcc/testsuite/gcc.target/riscv/amo-table-ztso-subword-amo-add-5.c b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-subword-amo-add-5.c
> new file mode 100644
> index 00000000000..fd59c8880ec
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/riscv/amo-table-ztso-subword-amo-add-5.c
> @@ -0,0 +1,10 @@
> +/* { dg-do compile } */
> +/* Verify that subword atomic op mappings match the Ztso suggested mapping.  */
> +/* { dg-options "-march=rv64id_ztso" } */
> +/* { dg-final { scan-assembler-times "lr.w.aqrl\t" 1 } } */
> +/* { dg-final { scan-assembler-times "sc.w.rl\t" 1 } } */
> +
> +void foo (short* bar, short* baz)
> +{
> +  __atomic_add_fetch(bar, baz, __ATOMIC_SEQ_CST);
> +}