public inbox for gcc-cvs@sourceware.org help / color / mirror / Atom feed
From: Jeff Law <law@gcc.gnu.org> To: gcc-cvs@gcc.gnu.org Subject: [gcc(refs/vendors/riscv/heads/gcc-13-with-riscv-opts)] RISC-V: Add RVV FNMA auto-vectorization support Date: Fri, 14 Jul 2023 02:40:28 +0000 (GMT) [thread overview] Message-ID: <20230714024028.9C9F73858298@sourceware.org> (raw) https://gcc.gnu.org/g:303972b5c12cc85aa517171bacc085028dde0f73 commit 303972b5c12cc85aa517171bacc085028dde0f73 Author: Juzhe-Zhong <juzhe.zhong@rivai.ai> Date: Tue May 30 10:13:43 2023 +0800 RISC-V: Add RVV FNMA auto-vectorization support Like FMA, Add FNMA (VNMSAC or VNMSUB) auto-vectorization support. Signed-off-by: Juzhe-Zhong <juzhe.zhong@rivai.ai> gcc/ChangeLog: * config/riscv/autovec.md (fnma<mode>4): New pattern. (*fnma<mode>): Ditto. gcc/testsuite/ChangeLog: * gcc.target/riscv/rvv/autovec/ternop/ternop-4.c: New test. * gcc.target/riscv/rvv/autovec/ternop/ternop-5.c: New test. * gcc.target/riscv/rvv/autovec/ternop/ternop-6.c: New test. * gcc.target/riscv/rvv/autovec/ternop/ternop_run-4.c: New test. * gcc.target/riscv/rvv/autovec/ternop/ternop_run-5.c: New test. * gcc.target/riscv/rvv/autovec/ternop/ternop_run-6.c: New test. Diff: --- gcc/config/riscv/autovec.md | 45 +++++++++ .../gcc.target/riscv/rvv/autovec/ternop/ternop-4.c | 28 ++++++ .../gcc.target/riscv/rvv/autovec/ternop/ternop-5.c | 34 +++++++ .../gcc.target/riscv/rvv/autovec/ternop/ternop-6.c | 33 +++++++ .../riscv/rvv/autovec/ternop/ternop_run-4.c | 84 +++++++++++++++++ .../riscv/rvv/autovec/ternop/ternop_run-5.c | 104 +++++++++++++++++++++ .../riscv/rvv/autovec/ternop/ternop_run-6.c | 104 +++++++++++++++++++++ 7 files changed, 432 insertions(+) diff --git a/gcc/config/riscv/autovec.md b/gcc/config/riscv/autovec.md index 0314e7587d0..d9b15714047 100644 --- a/gcc/config/riscv/autovec.md +++ b/gcc/config/riscv/autovec.md @@ -583,3 +583,48 @@ } [(set_attr "type" "vimuladd") (set_attr "mode" "<MODE>")]) + +;; ------------------------------------------------------------------------- +;; ---- [INT] VNMSAC and VNMSUB +;; ------------------------------------------------------------------------- +;; Includes: +;; - vnmsac +;; - vnmsub +;; ------------------------------------------------------------------------- + +(define_expand "fnma<mode>4" + [(parallel + [(set (match_operand:VI 0 "register_operand" "=vr") + (minus:VI + (match_operand:VI 3 "register_operand" " vr") + (mult:VI + (match_operand:VI 1 "register_operand" " vr") + (match_operand:VI 2 "register_operand" " vr")))) + (clobber (match_scratch:SI 4))])] + "TARGET_VECTOR" + {}) + +(define_insn_and_split "*fnma<mode>" + [(set (match_operand:VI 0 "register_operand" "=vr, vr, ?&vr") + (minus:VI + (match_operand:VI 3 "register_operand" " vr, 0, vr") + (mult:VI + (match_operand:VI 1 "register_operand" " %0, vr, vr") + (match_operand:VI 2 "register_operand" " vr, vr, vr")))) + (clobber (match_scratch:SI 4 "=r,r,r"))] + "TARGET_VECTOR" + "#" + "&& reload_completed" + [(const_int 0)] + { + PUT_MODE (operands[4], Pmode); + riscv_vector::emit_vlmax_vsetvl (<MODE>mode, operands[4]); + if (which_alternative == 2) + emit_insn (gen_rtx_SET (operands[0], operands[3])); + rtx ops[] = {operands[0], operands[1], operands[2], operands[3], operands[0]}; + riscv_vector::emit_vlmax_ternary_insn (code_for_pred_minus_mul (<MODE>mode), + riscv_vector::RVV_TERNOP, ops, operands[4]); + DONE; + } + [(set_attr "type" "vimuladd") + (set_attr "mode" "<MODE>")]) diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/ternop/ternop-4.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/ternop/ternop-4.c new file mode 100644 index 00000000000..1b8b934149e --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/ternop/ternop-4.c @@ -0,0 +1,28 @@ +/* { dg-do compile } */ +/* { dg-additional-options "-march=rv32gcv -mabi=ilp32d --param=riscv-autovec-preference=scalable" } */ + +#include <stdint-gcc.h> + +#define TEST_TYPE(TYPE) \ + __attribute__ ((noipa)) void ternop_##TYPE (TYPE *__restrict dst, \ + TYPE *__restrict a, \ + TYPE *__restrict b, int n) \ + { \ + for (int i = 0; i < n; i++) \ + dst[i] += -(a[i] * b[i]); \ + } + +#define TEST_ALL() \ + TEST_TYPE (int8_t) \ + TEST_TYPE (uint8_t) \ + TEST_TYPE (int16_t) \ + TEST_TYPE (uint16_t) \ + TEST_TYPE (int32_t) \ + TEST_TYPE (uint32_t) \ + TEST_TYPE (int64_t) \ + TEST_TYPE (uint64_t) + +TEST_ALL () + +/* { dg-final { scan-assembler-times {\tvnmsub\.vv} 8 } } */ +/* { dg-final { scan-assembler-not {\tvmv} } } */ diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/ternop/ternop-5.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/ternop/ternop-5.c new file mode 100644 index 00000000000..a9a7198feb4 --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/ternop/ternop-5.c @@ -0,0 +1,34 @@ +/* { dg-do compile } */ +/* { dg-additional-options "-march=rv32gcv -mabi=ilp32d --param=riscv-autovec-preference=scalable" } */ + +#include <stdint-gcc.h> + +#define TEST_TYPE(TYPE) \ + __attribute__ ((noipa)) void ternop_##TYPE (TYPE *__restrict dest1, \ + TYPE *__restrict dest2, \ + TYPE *__restrict dest3, \ + TYPE *__restrict src1, \ + TYPE *__restrict src2, int n) \ + { \ + for (int i = 0; i < n; ++i) \ + { \ + dest1[i] += -(src1[i] * src2[i]); \ + dest2[i] += src1[i] * dest1[i]; \ + dest3[i] += src2[i] * dest2[i]; \ + } \ + } + +#define TEST_ALL() \ + TEST_TYPE (int8_t) \ + TEST_TYPE (uint8_t) \ + TEST_TYPE (int16_t) \ + TEST_TYPE (uint16_t) \ + TEST_TYPE (int32_t) \ + TEST_TYPE (uint32_t) \ + TEST_TYPE (int64_t) \ + TEST_TYPE (uint64_t) + +TEST_ALL () + +/* { dg-final { scan-assembler-times {\tvnmsac\.vv} 8 } } */ +/* { dg-final { scan-assembler-not {\tvmv} } } */ diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/ternop/ternop-6.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/ternop/ternop-6.c new file mode 100644 index 00000000000..f38f303574f --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/ternop/ternop-6.c @@ -0,0 +1,33 @@ +/* { dg-do compile } */ +/* { dg-additional-options "-march=rv32gcv -mabi=ilp32d --param=riscv-autovec-preference=scalable" } */ + +#include <stdint-gcc.h> + +#define TEST_TYPE(TYPE) \ + __attribute__ ((noipa)) void ternop_##TYPE (TYPE *__restrict dest1, \ + TYPE *__restrict dest2, \ + TYPE *__restrict dest3, \ + TYPE *__restrict src1, \ + TYPE *__restrict src2, int n) \ + { \ + for (int i = 0; i < n; ++i) \ + { \ + dest1[i] = -(src1[i] * src2[i]) + dest2[i]; \ + dest2[i] += src1[i] * dest1[i]; \ + dest3[i] += src2[i] * dest2[i]; \ + } \ + } + +#define TEST_ALL() \ + TEST_TYPE (int8_t) \ + TEST_TYPE (uint8_t) \ + TEST_TYPE (int16_t) \ + TEST_TYPE (uint16_t) \ + TEST_TYPE (int32_t) \ + TEST_TYPE (uint32_t) \ + TEST_TYPE (int64_t) \ + TEST_TYPE (uint64_t) + +TEST_ALL () + +/* { dg-final { scan-assembler-times {\tvmv} 8 } } */ diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/ternop/ternop_run-4.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/ternop/ternop_run-4.c new file mode 100644 index 00000000000..c6f1fe591f3 --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/ternop/ternop_run-4.c @@ -0,0 +1,84 @@ +/* { dg-do run { target { riscv_vector } } } */ +/* { dg-additional-options "--param=riscv-autovec-preference=scalable" } */ + +#include "ternop-4.c" + +#define TEST_LOOP(TYPE, NUM) \ + { \ + TYPE array1_##NUM[NUM] = {}; \ + TYPE array2_##NUM[NUM] = {}; \ + TYPE array3_##NUM[NUM] = {}; \ + TYPE array4_##NUM[NUM] = {}; \ + for (int i = 0; i < NUM; ++i) \ + { \ + array1_##NUM[i] = (i & 1) + 5; \ + array2_##NUM[i] = i - NUM / 3; \ + array3_##NUM[i] = NUM - NUM / 3 - i; \ + array4_##NUM[i] = NUM - NUM / 3 - i; \ + asm volatile("" ::: "memory"); \ + } \ + ternop_##TYPE (array3_##NUM, array1_##NUM, array2_##NUM, NUM); \ + for (int i = 0; i < NUM; i++) \ + if (array3_##NUM[i] \ + != (TYPE) (-(array1_##NUM[i] * array2_##NUM[i]) + array4_##NUM[i])) \ + __builtin_abort (); \ + } + +int __attribute__ ((optimize (0))) main () +{ + TEST_LOOP (int8_t, 7) + TEST_LOOP (uint8_t, 7) + TEST_LOOP (int16_t, 7) + TEST_LOOP (uint16_t, 7) + TEST_LOOP (int32_t, 7) + TEST_LOOP (uint32_t, 7) + TEST_LOOP (int64_t, 7) + TEST_LOOP (uint64_t, 7) + + TEST_LOOP (int8_t, 16) + TEST_LOOP (uint8_t, 16) + TEST_LOOP (int16_t, 16) + TEST_LOOP (uint16_t, 16) + TEST_LOOP (int32_t, 16) + TEST_LOOP (uint32_t, 16) + TEST_LOOP (int64_t, 16) + TEST_LOOP (uint64_t, 16) + + TEST_LOOP (int8_t, 77) + TEST_LOOP (uint8_t, 77) + TEST_LOOP (int16_t, 77) + TEST_LOOP (uint16_t, 77) + TEST_LOOP (int32_t, 77) + TEST_LOOP (uint32_t, 77) + TEST_LOOP (int64_t, 77) + TEST_LOOP (uint64_t, 77) + + TEST_LOOP (int8_t, 128) + TEST_LOOP (uint8_t, 128) + TEST_LOOP (int16_t, 128) + TEST_LOOP (uint16_t, 128) + TEST_LOOP (int32_t, 128) + TEST_LOOP (uint32_t, 128) + TEST_LOOP (int64_t, 128) + TEST_LOOP (uint64_t, 128) + + TEST_LOOP (int8_t, 15641) + TEST_LOOP (uint8_t, 15641) + TEST_LOOP (int16_t, 15641) + TEST_LOOP (uint16_t, 15641) + TEST_LOOP (int32_t, 15641) + TEST_LOOP (uint32_t, 15641) + TEST_LOOP (int64_t, 15641) + TEST_LOOP (uint64_t, 15641) + + TEST_LOOP (int8_t, 795) + TEST_LOOP (uint8_t, 795) + TEST_LOOP (int16_t, 795) + TEST_LOOP (uint16_t, 795) + TEST_LOOP (int32_t, 795) + TEST_LOOP (uint32_t, 795) + TEST_LOOP (int64_t, 795) + TEST_LOOP (uint64_t, 795) + + return 0; +} diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/ternop/ternop_run-5.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/ternop/ternop_run-5.c new file mode 100644 index 00000000000..81af4b672a5 --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/ternop/ternop_run-5.c @@ -0,0 +1,104 @@ +/* { dg-do run { target { riscv_vector } } } */ +/* { dg-additional-options "--param=riscv-autovec-preference=scalable" } */ + +#include "ternop-5.c" + +#define TEST_LOOP(TYPE, NUM) \ + { \ + TYPE array1_##NUM[NUM] = {}; \ + TYPE array2_##NUM[NUM] = {}; \ + TYPE array3_##NUM[NUM] = {}; \ + TYPE array4_##NUM[NUM] = {}; \ + TYPE array5_##NUM[NUM] = {}; \ + TYPE array6_##NUM[NUM] = {}; \ + TYPE array7_##NUM[NUM] = {}; \ + TYPE array8_##NUM[NUM] = {}; \ + for (int i = 0; i < NUM; ++i) \ + { \ + array1_##NUM[i] = (i & 1) + 5; \ + array2_##NUM[i] = i - NUM / 3; \ + array3_##NUM[i] = NUM - NUM / 3 - i; \ + array6_##NUM[i] = NUM - NUM / 3 - i; \ + array4_##NUM[i] = NUM - NUM / 2 + i; \ + array7_##NUM[i] = NUM - NUM / 2 + i; \ + array5_##NUM[i] = NUM + i * 7; \ + array8_##NUM[i] = NUM + i * 7; \ + asm volatile("" ::: "memory"); \ + } \ + ternop_##TYPE (array3_##NUM, array4_##NUM, array5_##NUM, array1_##NUM, \ + array2_##NUM, NUM); \ + for (int i = 0; i < NUM; i++) \ + { \ + array6_##NUM[i] \ + = (TYPE) (-(array1_##NUM[i] * array2_##NUM[i]) + array6_##NUM[i]); \ + if (array3_##NUM[i] != array6_##NUM[i]) \ + __builtin_abort (); \ + array7_##NUM[i] \ + = (TYPE) (array1_##NUM[i] * array6_##NUM[i] + array7_##NUM[i]); \ + if (array4_##NUM[i] != array7_##NUM[i]) \ + __builtin_abort (); \ + array8_##NUM[i] \ + = (TYPE) (array2_##NUM[i] * array7_##NUM[i] + array8_##NUM[i]); \ + if (array5_##NUM[i] != array8_##NUM[i]) \ + __builtin_abort (); \ + } \ + } + +int __attribute__ ((optimize (0))) main () +{ + TEST_LOOP (int8_t, 7) + TEST_LOOP (uint8_t, 7) + TEST_LOOP (int16_t, 7) + TEST_LOOP (uint16_t, 7) + TEST_LOOP (int32_t, 7) + TEST_LOOP (uint32_t, 7) + TEST_LOOP (int64_t, 7) + TEST_LOOP (uint64_t, 7) + + TEST_LOOP (int8_t, 16) + TEST_LOOP (uint8_t, 16) + TEST_LOOP (int16_t, 16) + TEST_LOOP (uint16_t, 16) + TEST_LOOP (int32_t, 16) + TEST_LOOP (uint32_t, 16) + TEST_LOOP (int64_t, 16) + TEST_LOOP (uint64_t, 16) + + TEST_LOOP (int8_t, 77) + TEST_LOOP (uint8_t, 77) + TEST_LOOP (int16_t, 77) + TEST_LOOP (uint16_t, 77) + TEST_LOOP (int32_t, 77) + TEST_LOOP (uint32_t, 77) + TEST_LOOP (int64_t, 77) + TEST_LOOP (uint64_t, 77) + + TEST_LOOP (int8_t, 128) + TEST_LOOP (uint8_t, 128) + TEST_LOOP (int16_t, 128) + TEST_LOOP (uint16_t, 128) + TEST_LOOP (int32_t, 128) + TEST_LOOP (uint32_t, 128) + TEST_LOOP (int64_t, 128) + TEST_LOOP (uint64_t, 128) + + TEST_LOOP (int8_t, 15641) + TEST_LOOP (uint8_t, 15641) + TEST_LOOP (int16_t, 15641) + TEST_LOOP (uint16_t, 15641) + TEST_LOOP (int32_t, 15641) + TEST_LOOP (uint32_t, 15641) + TEST_LOOP (int64_t, 15641) + TEST_LOOP (uint64_t, 15641) + + TEST_LOOP (int8_t, 795) + TEST_LOOP (uint8_t, 795) + TEST_LOOP (int16_t, 795) + TEST_LOOP (uint16_t, 795) + TEST_LOOP (int32_t, 795) + TEST_LOOP (uint32_t, 795) + TEST_LOOP (int64_t, 795) + TEST_LOOP (uint64_t, 795) + + return 0; +} diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/ternop/ternop_run-6.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/ternop/ternop_run-6.c new file mode 100644 index 00000000000..b5e579ef55a --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/ternop/ternop_run-6.c @@ -0,0 +1,104 @@ +/* { dg-do run { target { riscv_vector } } } */ +/* { dg-additional-options "--param=riscv-autovec-preference=scalable" } */ + +#include "ternop-6.c" + +#define TEST_LOOP(TYPE, NUM) \ + { \ + TYPE array1_##NUM[NUM] = {}; \ + TYPE array2_##NUM[NUM] = {}; \ + TYPE array3_##NUM[NUM] = {}; \ + TYPE array4_##NUM[NUM] = {}; \ + TYPE array5_##NUM[NUM] = {}; \ + TYPE array6_##NUM[NUM] = {}; \ + TYPE array7_##NUM[NUM] = {}; \ + TYPE array8_##NUM[NUM] = {}; \ + for (int i = 0; i < NUM; ++i) \ + { \ + array1_##NUM[i] = (i & 1) + 5; \ + array2_##NUM[i] = i - NUM / 3; \ + array3_##NUM[i] = NUM - NUM / 3 - i; \ + array6_##NUM[i] = NUM - NUM / 3 - i; \ + array4_##NUM[i] = NUM - NUM / 2 + i; \ + array7_##NUM[i] = NUM - NUM / 2 + i; \ + array5_##NUM[i] = NUM + i * 7; \ + array8_##NUM[i] = NUM + i * 7; \ + asm volatile("" ::: "memory"); \ + } \ + ternop_##TYPE (array3_##NUM, array4_##NUM, array5_##NUM, array1_##NUM, \ + array2_##NUM, NUM); \ + for (int i = 0; i < NUM; i++) \ + { \ + array6_##NUM[i] \ + = (TYPE) (-(array1_##NUM[i] * array2_##NUM[i]) + array7_##NUM[i]); \ + if (array3_##NUM[i] != array6_##NUM[i]) \ + __builtin_abort (); \ + array7_##NUM[i] \ + = (TYPE) (array1_##NUM[i] * array6_##NUM[i] + array7_##NUM[i]); \ + if (array4_##NUM[i] != array7_##NUM[i]) \ + __builtin_abort (); \ + array8_##NUM[i] \ + = (TYPE) (array2_##NUM[i] * array7_##NUM[i] + array8_##NUM[i]); \ + if (array5_##NUM[i] != array8_##NUM[i]) \ + __builtin_abort (); \ + } \ + } + +int __attribute__ ((optimize (0))) main () +{ + TEST_LOOP (int8_t, 7) + TEST_LOOP (uint8_t, 7) + TEST_LOOP (int16_t, 7) + TEST_LOOP (uint16_t, 7) + TEST_LOOP (int32_t, 7) + TEST_LOOP (uint32_t, 7) + TEST_LOOP (int64_t, 7) + TEST_LOOP (uint64_t, 7) + + TEST_LOOP (int8_t, 16) + TEST_LOOP (uint8_t, 16) + TEST_LOOP (int16_t, 16) + TEST_LOOP (uint16_t, 16) + TEST_LOOP (int32_t, 16) + TEST_LOOP (uint32_t, 16) + TEST_LOOP (int64_t, 16) + TEST_LOOP (uint64_t, 16) + + TEST_LOOP (int8_t, 77) + TEST_LOOP (uint8_t, 77) + TEST_LOOP (int16_t, 77) + TEST_LOOP (uint16_t, 77) + TEST_LOOP (int32_t, 77) + TEST_LOOP (uint32_t, 77) + TEST_LOOP (int64_t, 77) + TEST_LOOP (uint64_t, 77) + + TEST_LOOP (int8_t, 128) + TEST_LOOP (uint8_t, 128) + TEST_LOOP (int16_t, 128) + TEST_LOOP (uint16_t, 128) + TEST_LOOP (int32_t, 128) + TEST_LOOP (uint32_t, 128) + TEST_LOOP (int64_t, 128) + TEST_LOOP (uint64_t, 128) + + TEST_LOOP (int8_t, 15641) + TEST_LOOP (uint8_t, 15641) + TEST_LOOP (int16_t, 15641) + TEST_LOOP (uint16_t, 15641) + TEST_LOOP (int32_t, 15641) + TEST_LOOP (uint32_t, 15641) + TEST_LOOP (int64_t, 15641) + TEST_LOOP (uint64_t, 15641) + + TEST_LOOP (int8_t, 795) + TEST_LOOP (uint8_t, 795) + TEST_LOOP (int16_t, 795) + TEST_LOOP (uint16_t, 795) + TEST_LOOP (int32_t, 795) + TEST_LOOP (uint32_t, 795) + TEST_LOOP (int64_t, 795) + TEST_LOOP (uint64_t, 795) + + return 0; +}
next reply other threads:[~2023-07-14 2:40 UTC|newest] Thread overview: 2+ messages / expand[flat|nested] mbox.gz Atom feed top 2023-07-14 2:40 Jeff Law [this message] -- strict thread matches above, loose matches on Subject: below -- 2023-05-30 13:57 Jeff Law
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20230714024028.9C9F73858298@sourceware.org \ --to=law@gcc.gnu.org \ --cc=gcc-cvs@gcc.gnu.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).