From: Patrick O'Neill <patrick@rivosinc.com>
To: gcc-patches@gcc.gnu.org
Cc: jeffreyalaw@gmail.com, palmer@dabbelt.com,
gnu-toolchain@rivosinc.com,
Patrick O'Neill <patrick@rivosinc.com>
Subject: [PATCH v2 3/3] RISC-V: Add Zalrsc amo-op patterns
Date: Mon, 3 Jun 2024 14:53:29 -0700 [thread overview]
Message-ID: <20240603215329.2664893-4-patrick@rivosinc.com> (raw)
In-Reply-To: <20240603215329.2664893-1-patrick@rivosinc.com>
All amo<op> patterns can be represented with lrsc sequences.
Add these patterns as a fallback when Zaamo is not enabled.
gcc/ChangeLog:
* config/riscv/sync.md (atomic_<atomic_optab><mode>): New expand pattern.
(amo_atomic_<atomic_optab><mode>): Rename amo pattern.
(atomic_fetch_<atomic_optab><mode>): New lrsc sequence pattern.
(lrsc_atomic_<atomic_optab><mode>): New expand pattern.
(amo_atomic_fetch_<atomic_optab><mode>): Rename amo pattern.
(lrsc_atomic_fetch_<atomic_optab><mode>): New lrsc sequence pattern.
(atomic_exchange<mode>): New expand pattern.
(amo_atomic_exchange<mode>): Rename amo pattern.
(lrsc_atomic_exchange<mode>): New lrsc sequence pattern.
gcc/testsuite/ChangeLog:
* gcc.target/riscv/amo-zaamo-preferred-over-zalrsc.c: New test.
* gcc.target/riscv/amo-zalrsc-amo-add-1.c: New test.
* gcc.target/riscv/amo-zalrsc-amo-add-2.c: New test.
* gcc.target/riscv/amo-zalrsc-amo-add-3.c: New test.
* gcc.target/riscv/amo-zalrsc-amo-add-4.c: New test.
* gcc.target/riscv/amo-zalrsc-amo-add-5.c: New test.
Signed-off-by: Patrick O'Neill <patrick@rivosinc.com>
------
rv64imfdc_zalrsc has the same testsuite results as rv64imafdc after this
patch is applied.
---
AFAIK there isn't a way to subtract an extension similar to dg-add-options.
As a result I needed to specify a -march string for
amo-zaamo-preferred-over-zalrsc.c instead of using testsuite infra.
---
gcc/config/riscv/sync.md | 124 +++++++++++++++++-
.../riscv/amo-zaamo-preferred-over-zalrsc.c | 17 +++
.../gcc.target/riscv/amo-zalrsc-amo-add-1.c | 19 +++
.../gcc.target/riscv/amo-zalrsc-amo-add-2.c | 19 +++
.../gcc.target/riscv/amo-zalrsc-amo-add-3.c | 19 +++
.../gcc.target/riscv/amo-zalrsc-amo-add-4.c | 19 +++
.../gcc.target/riscv/amo-zalrsc-amo-add-5.c | 19 +++
7 files changed, 231 insertions(+), 5 deletions(-)
create mode 100644 gcc/testsuite/gcc.target/riscv/amo-zaamo-preferred-over-zalrsc.c
create mode 100644 gcc/testsuite/gcc.target/riscv/amo-zalrsc-amo-add-1.c
create mode 100644 gcc/testsuite/gcc.target/riscv/amo-zalrsc-amo-add-2.c
create mode 100644 gcc/testsuite/gcc.target/riscv/amo-zalrsc-amo-add-3.c
create mode 100644 gcc/testsuite/gcc.target/riscv/amo-zalrsc-amo-add-4.c
create mode 100644 gcc/testsuite/gcc.target/riscv/amo-zalrsc-amo-add-5.c
diff --git a/gcc/config/riscv/sync.md b/gcc/config/riscv/sync.md
index c9544176ead..4df9d0b5a5f 100644
--- a/gcc/config/riscv/sync.md
+++ b/gcc/config/riscv/sync.md
@@ -86,7 +86,24 @@
DONE;
})
-(define_insn "atomic_<atomic_optab><mode>"
+;; AMO ops
+
+(define_expand "atomic_<atomic_optab><mode>"
+ [(any_atomic:GPR (match_operand:GPR 0 "memory_operand") ;; mem location
+ (match_operand:GPR 1 "reg_or_0_operand")) ;; value for op
+ (match_operand:SI 2 "const_int_operand")] ;; model
+ "TARGET_ZAAMO || TARGET_ZALRSC"
+{
+ if (TARGET_ZAAMO)
+ emit_insn (gen_amo_atomic_<atomic_optab><mode> (operands[0], operands[1],
+ operands[2]));
+ else
+ emit_insn (gen_lrsc_atomic_<atomic_optab><mode> (operands[0], operands[1],
+ operands[2]));
+ DONE;
+})
+
+(define_insn "amo_atomic_<atomic_optab><mode>"
[(set (match_operand:GPR 0 "memory_operand" "+A")
(unspec_volatile:GPR
[(any_atomic:GPR (match_dup 0)
@@ -98,7 +115,44 @@
[(set_attr "type" "atomic")
(set (attr "length") (const_int 4))])
-(define_insn "atomic_fetch_<atomic_optab><mode>"
+(define_insn "lrsc_atomic_<atomic_optab><mode>"
+ [(set (match_operand:GPR 0 "memory_operand" "+A")
+ (unspec_volatile:GPR
+ [(any_atomic:GPR (match_dup 0)
+ (match_operand:GPR 1 "reg_or_0_operand" "rJ"))
+ (match_operand:SI 2 "const_int_operand")] ;; model
+ UNSPEC_SYNC_OLD_OP))
+ (clobber (match_scratch:GPR 3 "=&r"))] ;; tmp_1
+ "!TARGET_ZAAMO && TARGET_ZALRSC"
+ {
+ return "1:\;"
+ "lr.<amo>%I2\t%3, %0\;"
+ "<insn>\t%3, %3, %1\;"
+ "sc.<amo>%J2\t%3, %3, %0\;"
+ "bnez\t%3, 1b";
+ }
+ [(set_attr "type" "atomic")
+ (set (attr "length") (const_int 16))])
+
+;; AMO fetch ops
+
+(define_expand "atomic_fetch_<atomic_optab><mode>"
+ [(match_operand:GPR 0 "register_operand") ;; old value at mem
+ (any_atomic:GPR (match_operand:GPR 1 "memory_operand") ;; mem location
+ (match_operand:GPR 2 "reg_or_0_operand")) ;; value for op
+ (match_operand:SI 3 "const_int_operand")] ;; model
+ "TARGET_ZAAMO || TARGET_ZALRSC"
+ {
+ if (TARGET_ZAAMO)
+ emit_insn (gen_amo_atomic_fetch_<atomic_optab><mode> (operands[0], operands[1],
+ operands[2], operands[3]));
+ else
+ emit_insn (gen_lrsc_atomic_fetch_<atomic_optab><mode> (operands[0], operands[1],
+ operands[2], operands[3]));
+ DONE;
+ })
+
+(define_insn "amo_atomic_fetch_<atomic_optab><mode>"
[(set (match_operand:GPR 0 "register_operand" "=&r")
(match_operand:GPR 1 "memory_operand" "+A"))
(set (match_dup 1)
@@ -112,6 +166,27 @@
[(set_attr "type" "atomic")
(set (attr "length") (const_int 4))])
+(define_insn "lrsc_atomic_fetch_<atomic_optab><mode>"
+ [(set (match_operand:GPR 0 "register_operand" "=&r")
+ (match_operand:GPR 1 "memory_operand" "+A"))
+ (set (match_dup 1)
+ (unspec_volatile:GPR
+ [(any_atomic:GPR (match_dup 1)
+ (match_operand:GPR 2 "reg_or_0_operand" "rJ"))
+ (match_operand:SI 3 "const_int_operand")] ;; model
+ UNSPEC_SYNC_OLD_OP))
+ (clobber (match_scratch:GPR 4 "=&r"))] ;; tmp_1
+ "!TARGET_ZAAMO && TARGET_ZALRSC"
+ {
+ return "1:\;"
+ "lr.<amo>%I3\t%0, %1\;"
+ "<insn>\t%4, %0, %2\;"
+ "sc.<amo>%J3\t%4, %4, %1\;"
+ "bnez\t%4, 1b";
+ }
+ [(set_attr "type" "atomic")
+ (set (attr "length") (const_int 20))])
+
(define_insn "subword_atomic_fetch_strong_<atomic_optab>"
[(set (match_operand:SI 0 "register_operand" "=&r") ;; old value at mem
(match_operand:SI 1 "memory_operand" "+A")) ;; mem location
@@ -248,7 +323,23 @@
DONE;
})
-(define_insn "atomic_exchange<mode>"
+(define_expand "atomic_exchange<mode>"
+ [(match_operand:GPR 0 "register_operand") ;; old value at mem
+ (match_operand:GPR 1 "memory_operand") ;; mem location
+ (match_operand:GPR 2 "register_operand") ;; value for op
+ (match_operand:SI 3 "const_int_operand")] ;; model
+ "TARGET_ZAAMO || TARGET_ZALRSC"
+ {
+ if (TARGET_ZAAMO)
+ emit_insn (gen_amo_atomic_exchange<mode> (operands[0], operands[1],
+ operands[2], operands[3]));
+ else
+ emit_insn (gen_lrsc_atomic_exchange<mode> (operands[0], operands[1],
+ operands[2], operands[3]));
+ DONE;
+ })
+
+(define_insn "amo_atomic_exchange<mode>"
[(set (match_operand:GPR 0 "register_operand" "=&r")
(unspec_volatile:GPR
[(match_operand:GPR 1 "memory_operand" "+A")
@@ -261,6 +352,26 @@
[(set_attr "type" "atomic")
(set (attr "length") (const_int 4))])
+(define_insn "lrsc_atomic_exchange<mode>"
+ [(set (match_operand:GPR 0 "register_operand" "=&r")
+ (unspec_volatile:GPR
+ [(match_operand:GPR 1 "memory_operand" "+A")
+ (match_operand:SI 3 "const_int_operand")] ;; model
+ UNSPEC_SYNC_EXCHANGE))
+ (set (match_dup 1)
+ (match_operand:GPR 2 "register_operand" "0"))
+ (clobber (match_scratch:GPR 4 "=&r"))] ;; tmp_1
+ "!TARGET_ZAAMO && TARGET_ZALRSC"
+ {
+ return "1:\;"
+ "lr.<amo>%I3\t%4, %1\;"
+ "sc.<amo>%J3\t%0, %0, %1\;"
+ "bnez\t%0, 1b\;"
+ "mv\t%0, %4";
+ }
+ [(set_attr "type" "atomic")
+ (set (attr "length") (const_int 20))])
+
(define_expand "atomic_exchange<mode>"
[(match_operand:SHORT 0 "register_operand") ;; old value at mem
(match_operand:SHORT 1 "memory_operand") ;; mem location
@@ -516,7 +627,7 @@
[(match_operand:QI 0 "register_operand" "") ;; bool output
(match_operand:QI 1 "memory_operand" "+A") ;; memory
(match_operand:SI 2 "const_int_operand" "")] ;; model
- "TARGET_ZALRSC"
+ "TARGET_ZAAMO || TARGET_ZALRSC"
{
/* We have no QImode atomics, so use the address LSBs to form a mask,
then use an aligned SImode atomic. */
@@ -537,7 +648,10 @@
rtx shifted_set = gen_reg_rtx (SImode);
riscv_lshift_subword (QImode, set, shift, &shifted_set);
- emit_insn (gen_atomic_fetch_orsi (old, aligned_mem, shifted_set, model));
+ if (TARGET_ZAAMO)
+ emit_insn (gen_amo_atomic_fetch_orsi (old, aligned_mem, shifted_set, model));
+ else if (TARGET_ZALRSC)
+ emit_insn (gen_lrsc_atomic_fetch_orsi (old, aligned_mem, shifted_set, model));
emit_move_insn (old, gen_rtx_ASHIFTRT (SImode, old,
gen_lowpart (QImode, shift)));
diff --git a/gcc/testsuite/gcc.target/riscv/amo-zaamo-preferred-over-zalrsc.c b/gcc/testsuite/gcc.target/riscv/amo-zaamo-preferred-over-zalrsc.c
new file mode 100644
index 00000000000..1c124c2b8b1
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-zaamo-preferred-over-zalrsc.c
@@ -0,0 +1,17 @@
+/* { dg-do compile } */
+/* Ensure that AMO ops are emitted when both zalrsc and zaamo are enabled. */
+/* { dg-options "-O3" } */
+/* { dg-add-options riscv_zalrsc } */
+/* { dg-add-options riscv_zaamo } */
+/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */
+/* { dg-final { check-function-bodies "**" "" } } */
+
+/*
+** foo:
+** amoadd\.w\tzero,a1,0\(a0\)
+** ret
+*/
+void foo (int* bar, int* baz)
+{
+ __atomic_add_fetch(bar, baz, __ATOMIC_RELAXED);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-zalrsc-amo-add-1.c b/gcc/testsuite/gcc.target/riscv/amo-zalrsc-amo-add-1.c
new file mode 100644
index 00000000000..3cd6ce04830
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-zalrsc-amo-add-1.c
@@ -0,0 +1,19 @@
+/* { dg-do compile } */
+/* Verify that lrsc atomic op mappings match Table A.6's recommended mapping. */
+/* { dg-options "-O3 -march=rv64id_zalrsc" } */
+/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */
+/* { dg-final { check-function-bodies "**" "" } } */
+
+/*
+** foo:
+** 1:
+** lr.w\ta5, 0\(a0\)
+** add\ta5, a5, a1
+** sc.w\ta5, a5, 0\(a0\)
+** bnez\ta5, 1b
+** ret
+*/
+void foo (int* bar, int* baz)
+{
+ __atomic_add_fetch(bar, baz, __ATOMIC_RELAXED);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-zalrsc-amo-add-2.c b/gcc/testsuite/gcc.target/riscv/amo-zalrsc-amo-add-2.c
new file mode 100644
index 00000000000..d7371dac301
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-zalrsc-amo-add-2.c
@@ -0,0 +1,19 @@
+/* { dg-do compile } */
+/* Verify that lrsc atomic op mappings match Table A.6's recommended mapping. */
+/* { dg-options "-O3 -march=rv64id_zalrsc" } */
+/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */
+/* { dg-final { check-function-bodies "**" "" } } */
+
+/*
+** foo:
+** 1:
+** lr.w.aq\ta5, 0\(a0\)
+** add\ta5, a5, a1
+** sc.w\ta5, a5, 0\(a0\)
+** bnez\ta5, 1b
+** ret
+*/
+void foo (int* bar, int* baz)
+{
+ __atomic_add_fetch(bar, baz, __ATOMIC_ACQUIRE);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-zalrsc-amo-add-3.c b/gcc/testsuite/gcc.target/riscv/amo-zalrsc-amo-add-3.c
new file mode 100644
index 00000000000..25060b0894f
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-zalrsc-amo-add-3.c
@@ -0,0 +1,19 @@
+/* { dg-do compile } */
+/* Verify that lrsc atomic op mappings match Table A.6's recommended mapping. */
+/* { dg-options "-O3 -march=rv64id_zalrsc" } */
+/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */
+/* { dg-final { check-function-bodies "**" "" } } */
+
+/*
+** foo:
+** 1:
+** lr.w\ta5, 0\(a0\)
+** add\ta5, a5, a1
+** sc.w.rl\ta5, a5, 0\(a0\)
+** bnez\ta5, 1b
+** ret
+*/
+void foo (int* bar, int* baz)
+{
+ __atomic_add_fetch(bar, baz, __ATOMIC_RELEASE);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-zalrsc-amo-add-4.c b/gcc/testsuite/gcc.target/riscv/amo-zalrsc-amo-add-4.c
new file mode 100644
index 00000000000..7cf9357fdbc
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-zalrsc-amo-add-4.c
@@ -0,0 +1,19 @@
+/* { dg-do compile } */
+/* Verify that lrsc atomic op mappings match Table A.6's recommended mapping. */
+/* { dg-options "-O3 -march=rv64id_zalrsc" } */
+/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */
+/* { dg-final { check-function-bodies "**" "" } } */
+
+/*
+** foo:
+** 1:
+** lr.w.aq\ta5, 0\(a0\)
+** add\ta5, a5, a1
+** sc.w.rl\ta5, a5, 0\(a0\)
+** bnez\ta5, 1b
+** ret
+*/
+void foo (int* bar, int* baz)
+{
+ __atomic_add_fetch(bar, baz, __ATOMIC_ACQ_REL);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/amo-zalrsc-amo-add-5.c b/gcc/testsuite/gcc.target/riscv/amo-zalrsc-amo-add-5.c
new file mode 100644
index 00000000000..df761892493
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/amo-zalrsc-amo-add-5.c
@@ -0,0 +1,19 @@
+/* { dg-do compile } */
+/* Verify that lrsc atomic op mappings match Table A.6's recommended mapping. */
+/* { dg-options "-O3 -march=rv64id_zalrsc" } */
+/* { dg-skip-if "" { *-*-* } { "-g" "-flto"} } */
+/* { dg-final { check-function-bodies "**" "" } } */
+
+/*
+** foo:
+** 1:
+** lr.w.aqrl\ta5, 0\(a0\)
+** add\ta5, a5, a1
+** sc.w.rl\ta5, a5, 0\(a0\)
+** bnez\ta5, 1b
+** ret
+*/
+void foo (int* bar, int* baz)
+{
+ __atomic_add_fetch(bar, baz, __ATOMIC_SEQ_CST);
+}
--
2.34.1
next prev parent reply other threads:[~2024-06-03 21:54 UTC|newest]
Thread overview: 13+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-06-03 21:53 [PATCH v2 0/3] RISC-V: Add basic Zaamo and Zalrsc support Patrick O'Neill
2024-06-03 21:53 ` [PATCH v2 1/3] " Patrick O'Neill
2024-06-04 3:00 ` Kito Cheng
2024-06-04 17:30 ` Patrick O'Neill
2024-06-04 21:27 ` Andrew Waterman
2024-06-07 22:35 ` Jeff Law
2024-06-03 21:53 ` [PATCH v2 2/3] RISC-V: Add Zalrsc and Zaamo testsuite support Patrick O'Neill
2024-06-07 23:04 ` Jeff Law
2024-06-10 16:39 ` Patrick O'Neill
2024-06-11 18:21 ` Patrick O'Neill
2024-06-12 18:14 ` Jeff Law
2024-06-03 21:53 ` Patrick O'Neill [this message]
2024-06-07 23:11 ` [PATCH v2 3/3] RISC-V: Add Zalrsc amo-op patterns Jeff Law
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20240603215329.2664893-4-patrick@rivosinc.com \
--to=patrick@rivosinc.com \
--cc=gcc-patches@gcc.gnu.org \
--cc=gnu-toolchain@rivosinc.com \
--cc=jeffreyalaw@gmail.com \
--cc=palmer@dabbelt.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).