* [PATCH] RISC-V: Support integer mult highpart auto-vectorization
@ 2023-07-12 8:39 juzhe.zhong
2023-07-12 9:17 ` Kito Cheng
0 siblings, 1 reply; 3+ messages in thread
From: juzhe.zhong @ 2023-07-12 8:39 UTC (permalink / raw)
To: gcc-patches; +Cc: kito.cheng, kito.cheng, jeffreyalaw, rdapp.gcc, Ju-Zhe Zhong
From: Ju-Zhe Zhong <juzhe.zhong@rivai.ai>
This patch is adding an obvious missing mult_high auto-vectorization pattern.
Consider this following case:
#define DEF_LOOP(TYPE) \
void __attribute__ ((noipa)) \
mod_##TYPE (TYPE *__restrict dst, TYPE *__restrict src, int count) \
{ \
for (int i = 0; i < count; ++i) \
dst[i] = src[i] / 17; \
}
#define TEST_ALL(T) \
T (int32_t) \
TEST_ALL (DEF_LOOP)
Before this patch:
mod_int32_t:
ble a2,zero,.L5
li a5,17
vsetvli a3,zero,e32,m1,ta,ma
vmv.v.x v2,a5
.L3:
vsetvli a5,a2,e8,mf4,ta,ma
vle32.v v1,0(a1)
vsetvli a3,zero,e32,m1,ta,ma
slli a4,a5,2
vdiv.vv v1,v1,v2
sub a2,a2,a5
vsetvli zero,a5,e32,m1,ta,ma
vse32.v v1,0(a0)
add a1,a1,a4
add a0,a0,a4
bne a2,zero,.L3
.L5:
ret
After this patch:
mod_int32_t:
ble a2,zero,.L5
li a5,2021163008
addiw a5,a5,-1927
vsetvli a3,zero,e32,m1,ta,ma
vmv.v.x v3,a5
.L3:
vsetvli a5,a2,e8,mf4,ta,ma
vle32.v v2,0(a1)
vsetvli a3,zero,e32,m1,ta,ma
slli a4,a5,2
vmulh.vv v1,v2,v3
sub a2,a2,a5
vsra.vi v2,v2,31
vsra.vi v1,v1,3
vsub.vv v1,v1,v2
vsetvli zero,a5,e32,m1,ta,ma
vse32.v v1,0(a0)
add a1,a1,a4
add a0,a0,a4
bne a2,zero,.L3
.L5:
ret
Even though a single "vdiv" is lower into "1 vmulh + 2 vsra + 1 vsub",
4 more instructions are generated, we belive it's much better than before
since division is very slow in the hardward.
gcc/ChangeLog:
* config/riscv/autovec.md (smul<mode>3_highpart): New pattern.
(umul<mode>3_highpart): Ditto.
gcc/testsuite/ChangeLog:
* gcc.target/riscv/rvv/autovec/binop/mulh-1.c: New test.
* gcc.target/riscv/rvv/autovec/binop/mulh-2.c: New test.
* gcc.target/riscv/rvv/autovec/binop/mulh_run-1.c: New test.
* gcc.target/riscv/rvv/autovec/binop/mulh_run-2.c: New test.
---
gcc/config/riscv/autovec.md | 30 +++++++++++++++++++
.../riscv/rvv/autovec/binop/mulh-1.c | 26 ++++++++++++++++
.../riscv/rvv/autovec/binop/mulh-2.c | 27 +++++++++++++++++
.../riscv/rvv/autovec/binop/mulh_run-1.c | 29 ++++++++++++++++++
.../riscv/rvv/autovec/binop/mulh_run-2.c | 29 ++++++++++++++++++
5 files changed, 141 insertions(+)
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/mulh-1.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/mulh-2.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/mulh_run-1.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/mulh_run-2.c
diff --git a/gcc/config/riscv/autovec.md b/gcc/config/riscv/autovec.md
index 9e61b2e41d8..d98a63c285e 100644
--- a/gcc/config/riscv/autovec.md
+++ b/gcc/config/riscv/autovec.md
@@ -1178,3 +1178,33 @@
riscv_vector::RVV_BINOP, operands);
DONE;
})
+
+;; -------------------------------------------------------------------------
+;; ---- [INT] Highpart multiplication
+;; -------------------------------------------------------------------------
+;; Includes:
+;; - vmulh.vv
+;; - vmulhu.vv
+;; -------------------------------------------------------------------------
+
+(define_expand "smul<mode>3_highpart"
+ [(match_operand:VFULLI 0 "register_operand")
+ (match_operand:VFULLI 1 "register_operand")
+ (match_operand:VFULLI 2 "register_operand")]
+ "TARGET_VECTOR"
+{
+ insn_code icode = code_for_pred_mulh (UNSPEC_VMULHS, <MODE>mode);
+ riscv_vector::emit_vlmax_insn (icode, riscv_vector::RVV_BINOP, operands);
+ DONE;
+})
+
+(define_expand "umul<mode>3_highpart"
+ [(match_operand:VFULLI 0 "register_operand")
+ (match_operand:VFULLI 1 "register_operand")
+ (match_operand:VFULLI 2 "register_operand")]
+ "TARGET_VECTOR"
+{
+ insn_code icode = code_for_pred_mulh (UNSPEC_VMULHU, <MODE>mode);
+ riscv_vector::emit_vlmax_insn (icode, riscv_vector::RVV_BINOP, operands);
+ DONE;
+})
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/mulh-1.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/mulh-1.c
new file mode 100644
index 00000000000..265a332712a
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/mulh-1.c
@@ -0,0 +1,26 @@
+/* { dg-do compile } */
+/* { dg-additional-options "-march=rv32gcv -mabi=ilp32d --param=riscv-autovec-preference=scalable -fno-vect-cost-model" } */
+
+#include <stdint-gcc.h>
+
+#define DEF_LOOP(TYPE) \
+ void __attribute__ ((noipa)) mod_##TYPE (TYPE *dst, TYPE *src, int count) \
+ { \
+ for (int i = 0; i < count; ++i) \
+ dst[i] = src[i] % 19; \
+ }
+
+#define TEST_ALL(T) \
+ T (int8_t) \
+ T (uint8_t) \
+ T (int16_t) \
+ T (uint16_t) \
+ T (int32_t) \
+ T (uint32_t) \
+ T (int64_t) \
+ T (uint64_t)
+
+TEST_ALL (DEF_LOOP)
+
+/* { dg-final { scan-assembler-times {\tvmulh\.vv} 4 } } */
+/* { dg-final { scan-assembler-times {\tvmulhu\.vv} 4 } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/mulh-2.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/mulh-2.c
new file mode 100644
index 00000000000..18faaadd68c
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/mulh-2.c
@@ -0,0 +1,27 @@
+/* { dg-do compile } */
+/* { dg-additional-options "-march=rv32gcv -mabi=ilp32d --param=riscv-autovec-preference=scalable -fno-vect-cost-model" } */
+
+#include <stdint-gcc.h>
+
+#define DEF_LOOP(TYPE) \
+void __attribute__ ((noipa)) \
+mod_##TYPE (TYPE *dst, TYPE *src, int count) \
+{ \
+ for (int i = 0; i < count; ++i) \
+ dst[i] = src[i] / 17; \
+}
+
+#define TEST_ALL(T) \
+ T (int8_t) \
+ T (uint8_t) \
+ T (int16_t) \
+ T (uint16_t) \
+ T (int32_t) \
+ T (uint32_t) \
+ T (int64_t) \
+ T (uint64_t)
+
+TEST_ALL (DEF_LOOP)
+
+/* { dg-final { scan-assembler-times {\tvmulh\.vv} 4 } } */
+/* { dg-final { scan-assembler-times {\tvmulhu\.vv} 4 } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/mulh_run-1.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/mulh_run-1.c
new file mode 100644
index 00000000000..7a47e11a3a0
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/mulh_run-1.c
@@ -0,0 +1,29 @@
+/* { dg-do run { target { riscv_vector } } } */
+/* { dg-additional-options "--param=riscv-autovec-preference=scalable" } */
+
+#include "mulh-1.c"
+
+#define N 79
+
+#define TEST_LOOP(TYPE) \
+ { \
+ TYPE dst[N], src[N]; \
+ for (int i = 0; i < N; ++i) \
+ { \
+ src[i] = i * 7 + i % 3; \
+ if (i % 11 > 7) \
+ src[i] = -src[i]; \
+ asm volatile ("" ::: "memory"); \
+ } \
+ mod_##TYPE (dst, src, N); \
+ for (int i = 0; i < N; ++i) \
+ if (dst[i] != src[i] % 19) \
+ __builtin_abort (); \
+ }
+
+int
+main (void)
+{
+ TEST_ALL (TEST_LOOP);
+ return 0;
+}
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/mulh_run-2.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/mulh_run-2.c
new file mode 100644
index 00000000000..72c72b0f4eb
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/mulh_run-2.c
@@ -0,0 +1,29 @@
+/* { dg-do run { target { riscv_vector } } } */
+/* { dg-additional-options "--param=riscv-autovec-preference=scalable" } */
+
+#include "mulh-2.c"
+
+#define N 79
+
+#define TEST_LOOP(TYPE) \
+ { \
+ TYPE dst[N], src[N]; \
+ for (int i = 0; i < N; ++i) \
+ { \
+ src[i] = i * 7 + i % 3; \
+ if (i % 11 > 7) \
+ src[i] = -src[i]; \
+ asm volatile ("" ::: "memory"); \
+ } \
+ mod_##TYPE (dst, src, N); \
+ for (int i = 0; i < N; ++i) \
+ if (dst[i] != src[i] / 17) \
+ __builtin_abort (); \
+ }
+
+int
+main (void)
+{
+ TEST_ALL (TEST_LOOP);
+ return 0;
+}
--
2.36.1
^ permalink raw reply [flat|nested] 3+ messages in thread* Re: [PATCH] RISC-V: Support integer mult highpart auto-vectorization
2023-07-12 8:39 [PATCH] RISC-V: Support integer mult highpart auto-vectorization juzhe.zhong
@ 2023-07-12 9:17 ` Kito Cheng
2023-07-12 10:17 ` Li, Pan2
0 siblings, 1 reply; 3+ messages in thread
From: Kito Cheng @ 2023-07-12 9:17 UTC (permalink / raw)
To: 钟居哲; +Cc: GCC Patches, Kito Cheng, Jeff Law, Robin Dapp
[-- Attachment #1: Type: text/plain, Size: 9881 bytes --]
LGTM, thanks:)
<juzhe.zhong@rivai.ai> 於 2023年7月12日 週三 16:40 寫道：
> From: Ju-Zhe Zhong <juzhe.zhong@rivai.ai>
>
> This patch is adding an obvious missing mult_high auto-vectorization
> pattern.
>
> Consider this following case:
> #define DEF_LOOP(TYPE) \
> void __attribute__ ((noipa)) \
> mod_##TYPE (TYPE *__restrict dst, TYPE *__restrict src, int count) \
> { \
> for (int i = 0; i < count; ++i) \
> dst[i] = src[i] / 17; \
> }
>
> #define TEST_ALL(T) \
> T (int32_t) \
>
> TEST_ALL (DEF_LOOP)
>
> Before this patch:
> mod_int32_t:
> ble a2,zero,.L5
> li a5,17
> vsetvli a3,zero,e32,m1,ta,ma
> vmv.v.x v2,a5
> .L3:
> vsetvli a5,a2,e8,mf4,ta,ma
> vle32.v v1,0(a1)
> vsetvli a3,zero,e32,m1,ta,ma
> slli a4,a5,2
> vdiv.vv v1,v1,v2
> sub a2,a2,a5
> vsetvli zero,a5,e32,m1,ta,ma
> vse32.v v1,0(a0)
> add a1,a1,a4
> add a0,a0,a4
> bne a2,zero,.L3
> .L5:
> ret
>
> After this patch:
> mod_int32_t:
> ble a2,zero,.L5
> li a5,2021163008
> addiw a5,a5,-1927
> vsetvli a3,zero,e32,m1,ta,ma
> vmv.v.x v3,a5
> .L3:
> vsetvli a5,a2,e8,mf4,ta,ma
> vle32.v v2,0(a1)
> vsetvli a3,zero,e32,m1,ta,ma
> slli a4,a5,2
> vmulh.vv v1,v2,v3
> sub a2,a2,a5
> vsra.vi v2,v2,31
> vsra.vi v1,v1,3
> vsub.vv v1,v1,v2
> vsetvli zero,a5,e32,m1,ta,ma
> vse32.v v1,0(a0)
> add a1,a1,a4
> add a0,a0,a4
> bne a2,zero,.L3
> .L5:
> ret
>
> Even though a single "vdiv" is lower into "1 vmulh + 2 vsra + 1 vsub",
> 4 more instructions are generated, we belive it's much better than before
> since division is very slow in the hardward.
>
> gcc/ChangeLog:
>
> * config/riscv/autovec.md (smul<mode>3_highpart): New pattern.
> (umul<mode>3_highpart): Ditto.
>
> gcc/testsuite/ChangeLog:
>
> * gcc.target/riscv/rvv/autovec/binop/mulh-1.c: New test.
> * gcc.target/riscv/rvv/autovec/binop/mulh-2.c: New test.
> * gcc.target/riscv/rvv/autovec/binop/mulh_run-1.c: New test.
> * gcc.target/riscv/rvv/autovec/binop/mulh_run-2.c: New test.
>
> ---
> gcc/config/riscv/autovec.md | 30 +++++++++++++++++++
> .../riscv/rvv/autovec/binop/mulh-1.c | 26 ++++++++++++++++
> .../riscv/rvv/autovec/binop/mulh-2.c | 27 +++++++++++++++++
> .../riscv/rvv/autovec/binop/mulh_run-1.c | 29 ++++++++++++++++++
> .../riscv/rvv/autovec/binop/mulh_run-2.c | 29 ++++++++++++++++++
> 5 files changed, 141 insertions(+)
> create mode 100644
> gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/mulh-1.c
> create mode 100644
> gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/mulh-2.c
> create mode 100644
> gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/mulh_run-1.c
> create mode 100644
> gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/mulh_run-2.c
>
> diff --git a/gcc/config/riscv/autovec.md b/gcc/config/riscv/autovec.md
> index 9e61b2e41d8..d98a63c285e 100644
> --- a/gcc/config/riscv/autovec.md
> +++ b/gcc/config/riscv/autovec.md
> @@ -1178,3 +1178,33 @@
> riscv_vector::RVV_BINOP, operands);
> DONE;
> })
> +
> +;;
> -------------------------------------------------------------------------
> +;; ---- [INT] Highpart multiplication
> +;;
> -------------------------------------------------------------------------
> +;; Includes:
> +;; - vmulh.vv
> +;; - vmulhu.vv
> +;;
> -------------------------------------------------------------------------
> +
> +(define_expand "smul<mode>3_highpart"
> + [(match_operand:VFULLI 0 "register_operand")
> + (match_operand:VFULLI 1 "register_operand")
> + (match_operand:VFULLI 2 "register_operand")]
> + "TARGET_VECTOR"
> +{
> + insn_code icode = code_for_pred_mulh (UNSPEC_VMULHS, <MODE>mode);
> + riscv_vector::emit_vlmax_insn (icode, riscv_vector::RVV_BINOP,
> operands);
> + DONE;
> +})
> +
> +(define_expand "umul<mode>3_highpart"
> + [(match_operand:VFULLI 0 "register_operand")
> + (match_operand:VFULLI 1 "register_operand")
> + (match_operand:VFULLI 2 "register_operand")]
> + "TARGET_VECTOR"
> +{
> + insn_code icode = code_for_pred_mulh (UNSPEC_VMULHU, <MODE>mode);
> + riscv_vector::emit_vlmax_insn (icode, riscv_vector::RVV_BINOP,
> operands);
> + DONE;
> +})
> diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/mulh-1.c
> b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/mulh-1.c
> new file mode 100644
> index 00000000000..265a332712a
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/mulh-1.c
> @@ -0,0 +1,26 @@
> +/* { dg-do compile } */
> +/* { dg-additional-options "-march=rv32gcv -mabi=ilp32d
> --param=riscv-autovec-preference=scalable -fno-vect-cost-model" } */
> +
> +#include <stdint-gcc.h>
> +
> +#define DEF_LOOP(TYPE)
> \
> + void __attribute__ ((noipa)) mod_##TYPE (TYPE *dst, TYPE *src, int
> count) \
> + {
> \
> + for (int i = 0; i < count; ++i)
> \
> + dst[i] = src[i] % 19;
> \
> + }
> +
> +#define TEST_ALL(T)
> \
> + T (int8_t)
> \
> + T (uint8_t)
> \
> + T (int16_t)
> \
> + T (uint16_t)
> \
> + T (int32_t)
> \
> + T (uint32_t)
> \
> + T (int64_t)
> \
> + T (uint64_t)
> +
> +TEST_ALL (DEF_LOOP)
> +
> +/* { dg-final { scan-assembler-times {\tvmulh\.vv} 4 } } */
> +/* { dg-final { scan-assembler-times {\tvmulhu\.vv} 4 } } */
> diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/mulh-2.c
> b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/mulh-2.c
> new file mode 100644
> index 00000000000..18faaadd68c
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/mulh-2.c
> @@ -0,0 +1,27 @@
> +/* { dg-do compile } */
> +/* { dg-additional-options "-march=rv32gcv -mabi=ilp32d
> --param=riscv-autovec-preference=scalable -fno-vect-cost-model" } */
> +
> +#include <stdint-gcc.h>
> +
> +#define DEF_LOOP(TYPE) \
> +void __attribute__ ((noipa)) \
> +mod_##TYPE (TYPE *dst, TYPE *src, int count) \
> +{ \
> + for (int i = 0; i < count; ++i) \
> + dst[i] = src[i] / 17; \
> +}
> +
> +#define TEST_ALL(T) \
> + T (int8_t) \
> + T (uint8_t) \
> + T (int16_t) \
> + T (uint16_t) \
> + T (int32_t) \
> + T (uint32_t) \
> + T (int64_t) \
> + T (uint64_t)
> +
> +TEST_ALL (DEF_LOOP)
> +
> +/* { dg-final { scan-assembler-times {\tvmulh\.vv} 4 } } */
> +/* { dg-final { scan-assembler-times {\tvmulhu\.vv} 4 } } */
> diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/mulh_run-1.c
> b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/mulh_run-1.c
> new file mode 100644
> index 00000000000..7a47e11a3a0
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/mulh_run-1.c
> @@ -0,0 +1,29 @@
> +/* { dg-do run { target { riscv_vector } } } */
> +/* { dg-additional-options "--param=riscv-autovec-preference=scalable" }
> */
> +
> +#include "mulh-1.c"
> +
> +#define N 79
> +
> +#define TEST_LOOP(TYPE) \
> + { \
> + TYPE dst[N], src[N]; \
> + for (int i = 0; i < N; ++i) \
> + { \
> + src[i] = i * 7 + i % 3; \
> + if (i % 11 > 7) \
> + src[i] = -src[i]; \
> + asm volatile ("" ::: "memory"); \
> + } \
> + mod_##TYPE (dst, src, N); \
> + for (int i = 0; i < N; ++i) \
> + if (dst[i] != src[i] % 19) \
> + __builtin_abort (); \
> + }
> +
> +int
> +main (void)
> +{
> + TEST_ALL (TEST_LOOP);
> + return 0;
> +}
> diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/mulh_run-2.c
> b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/mulh_run-2.c
> new file mode 100644
> index 00000000000..72c72b0f4eb
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/mulh_run-2.c
> @@ -0,0 +1,29 @@
> +/* { dg-do run { target { riscv_vector } } } */
> +/* { dg-additional-options "--param=riscv-autovec-preference=scalable" }
> */
> +
> +#include "mulh-2.c"
> +
> +#define N 79
> +
> +#define TEST_LOOP(TYPE) \
> + { \
> + TYPE dst[N], src[N]; \
> + for (int i = 0; i < N; ++i) \
> + { \
> + src[i] = i * 7 + i % 3; \
> + if (i % 11 > 7) \
> + src[i] = -src[i]; \
> + asm volatile ("" ::: "memory"); \
> + } \
> + mod_##TYPE (dst, src, N); \
> + for (int i = 0; i < N; ++i) \
> + if (dst[i] != src[i] / 17) \
> + __builtin_abort (); \
> + }
> +
> +int
> +main (void)
> +{
> + TEST_ALL (TEST_LOOP);
> + return 0;
> +}
> --
> 2.36.1
>
>
^ permalink raw reply [flat|nested] 3+ messages in thread* RE: [PATCH] RISC-V: Support integer mult highpart auto-vectorization
2023-07-12 9:17 ` Kito Cheng
@ 2023-07-12 10:17 ` Li, Pan2
0 siblings, 0 replies; 3+ messages in thread
From: Li, Pan2 @ 2023-07-12 10:17 UTC (permalink / raw)
To: Kito Cheng, 钟居哲
Cc: GCC Patches, Kito Cheng, Jeff Law, Robin Dapp
Committed, thanks Kito.
Pan
-----Original Message-----
From: Gcc-patches <gcc-patches-bounces+pan2.li=intel.com@gcc.gnu.org> On Behalf Of Kito Cheng via Gcc-patches
Sent: Wednesday, July 12, 2023 5:17 PM
To: 钟居哲 <juzhe.zhong@rivai.ai>
Cc: GCC Patches <gcc-patches@gcc.gnu.org>; Kito Cheng <kito.cheng@sifive.com>; Jeff Law <jeffreyalaw@gmail.com>; Robin Dapp <rdapp.gcc@gmail.com>
Subject: Re: [PATCH] RISC-V: Support integer mult highpart auto-vectorization
LGTM, thanks:)
<juzhe.zhong@rivai.ai> 於 2023年7月12日 週三 16:40 寫道：
> From: Ju-Zhe Zhong <juzhe.zhong@rivai.ai>
>
> This patch is adding an obvious missing mult_high auto-vectorization
> pattern.
>
> Consider this following case:
> #define DEF_LOOP(TYPE) \
> void __attribute__ ((noipa)) \
> mod_##TYPE (TYPE *__restrict dst, TYPE *__restrict src, int count) \
> { \
> for (int i = 0; i < count; ++i) \
> dst[i] = src[i] / 17; \
> }
>
> #define TEST_ALL(T) \
> T (int32_t) \
>
> TEST_ALL (DEF_LOOP)
>
> Before this patch:
> mod_int32_t:
> ble a2,zero,.L5
> li a5,17
> vsetvli a3,zero,e32,m1,ta,ma
> vmv.v.x v2,a5
> .L3:
> vsetvli a5,a2,e8,mf4,ta,ma
> vle32.v v1,0(a1)
> vsetvli a3,zero,e32,m1,ta,ma
> slli a4,a5,2
> vdiv.vv v1,v1,v2
> sub a2,a2,a5
> vsetvli zero,a5,e32,m1,ta,ma
> vse32.v v1,0(a0)
> add a1,a1,a4
> add a0,a0,a4
> bne a2,zero,.L3
> .L5:
> ret
>
> After this patch:
> mod_int32_t:
> ble a2,zero,.L5
> li a5,2021163008
> addiw a5,a5,-1927
> vsetvli a3,zero,e32,m1,ta,ma
> vmv.v.x v3,a5
> .L3:
> vsetvli a5,a2,e8,mf4,ta,ma
> vle32.v v2,0(a1)
> vsetvli a3,zero,e32,m1,ta,ma
> slli a4,a5,2
> vmulh.vv v1,v2,v3
> sub a2,a2,a5
> vsra.vi v2,v2,31
> vsra.vi v1,v1,3
> vsub.vv v1,v1,v2
> vsetvli zero,a5,e32,m1,ta,ma
> vse32.v v1,0(a0)
> add a1,a1,a4
> add a0,a0,a4
> bne a2,zero,.L3
> .L5:
> ret
>
> Even though a single "vdiv" is lower into "1 vmulh + 2 vsra + 1 vsub",
> 4 more instructions are generated, we belive it's much better than before
> since division is very slow in the hardward.
>
> gcc/ChangeLog:
>
> * config/riscv/autovec.md (smul<mode>3_highpart): New pattern.
> (umul<mode>3_highpart): Ditto.
>
> gcc/testsuite/ChangeLog:
>
> * gcc.target/riscv/rvv/autovec/binop/mulh-1.c: New test.
> * gcc.target/riscv/rvv/autovec/binop/mulh-2.c: New test.
> * gcc.target/riscv/rvv/autovec/binop/mulh_run-1.c: New test.
> * gcc.target/riscv/rvv/autovec/binop/mulh_run-2.c: New test.
>
> ---
> gcc/config/riscv/autovec.md | 30 +++++++++++++++++++
> .../riscv/rvv/autovec/binop/mulh-1.c | 26 ++++++++++++++++
> .../riscv/rvv/autovec/binop/mulh-2.c | 27 +++++++++++++++++
> .../riscv/rvv/autovec/binop/mulh_run-1.c | 29 ++++++++++++++++++
> .../riscv/rvv/autovec/binop/mulh_run-2.c | 29 ++++++++++++++++++
> 5 files changed, 141 insertions(+)
> create mode 100644
> gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/mulh-1.c
> create mode 100644
> gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/mulh-2.c
> create mode 100644
> gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/mulh_run-1.c
> create mode 100644
> gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/mulh_run-2.c
>
> diff --git a/gcc/config/riscv/autovec.md b/gcc/config/riscv/autovec.md
> index 9e61b2e41d8..d98a63c285e 100644
> --- a/gcc/config/riscv/autovec.md
> +++ b/gcc/config/riscv/autovec.md
> @@ -1178,3 +1178,33 @@
> riscv_vector::RVV_BINOP, operands);
> DONE;
> })
> +
> +;;
> -------------------------------------------------------------------------
> +;; ---- [INT] Highpart multiplication
> +;;
> -------------------------------------------------------------------------
> +;; Includes:
> +;; - vmulh.vv
> +;; - vmulhu.vv
> +;;
> -------------------------------------------------------------------------
> +
> +(define_expand "smul<mode>3_highpart"
> + [(match_operand:VFULLI 0 "register_operand")
> + (match_operand:VFULLI 1 "register_operand")
> + (match_operand:VFULLI 2 "register_operand")]
> + "TARGET_VECTOR"
> +{
> + insn_code icode = code_for_pred_mulh (UNSPEC_VMULHS, <MODE>mode);
> + riscv_vector::emit_vlmax_insn (icode, riscv_vector::RVV_BINOP,
> operands);
> + DONE;
> +})
> +
> +(define_expand "umul<mode>3_highpart"
> + [(match_operand:VFULLI 0 "register_operand")
> + (match_operand:VFULLI 1 "register_operand")
> + (match_operand:VFULLI 2 "register_operand")]
> + "TARGET_VECTOR"
> +{
> + insn_code icode = code_for_pred_mulh (UNSPEC_VMULHU, <MODE>mode);
> + riscv_vector::emit_vlmax_insn (icode, riscv_vector::RVV_BINOP,
> operands);
> + DONE;
> +})
> diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/mulh-1.c
> b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/mulh-1.c
> new file mode 100644
> index 00000000000..265a332712a
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/mulh-1.c
> @@ -0,0 +1,26 @@
> +/* { dg-do compile } */
> +/* { dg-additional-options "-march=rv32gcv -mabi=ilp32d
> --param=riscv-autovec-preference=scalable -fno-vect-cost-model" } */
> +
> +#include <stdint-gcc.h>
> +
> +#define DEF_LOOP(TYPE)
> \
> + void __attribute__ ((noipa)) mod_##TYPE (TYPE *dst, TYPE *src, int
> count) \
> + {
> \
> + for (int i = 0; i < count; ++i)
> \
> + dst[i] = src[i] % 19;
> \
> + }
> +
> +#define TEST_ALL(T)
> \
> + T (int8_t)
> \
> + T (uint8_t)
> \
> + T (int16_t)
> \
> + T (uint16_t)
> \
> + T (int32_t)
> \
> + T (uint32_t)
> \
> + T (int64_t)
> \
> + T (uint64_t)
> +
> +TEST_ALL (DEF_LOOP)
> +
> +/* { dg-final { scan-assembler-times {\tvmulh\.vv} 4 } } */
> +/* { dg-final { scan-assembler-times {\tvmulhu\.vv} 4 } } */
> diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/mulh-2.c
> b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/mulh-2.c
> new file mode 100644
> index 00000000000..18faaadd68c
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/mulh-2.c
> @@ -0,0 +1,27 @@
> +/* { dg-do compile } */
> +/* { dg-additional-options "-march=rv32gcv -mabi=ilp32d
> --param=riscv-autovec-preference=scalable -fno-vect-cost-model" } */
> +
> +#include <stdint-gcc.h>
> +
> +#define DEF_LOOP(TYPE) \
> +void __attribute__ ((noipa)) \
> +mod_##TYPE (TYPE *dst, TYPE *src, int count) \
> +{ \
> + for (int i = 0; i < count; ++i) \
> + dst[i] = src[i] / 17; \
> +}
> +
> +#define TEST_ALL(T) \
> + T (int8_t) \
> + T (uint8_t) \
> + T (int16_t) \
> + T (uint16_t) \
> + T (int32_t) \
> + T (uint32_t) \
> + T (int64_t) \
> + T (uint64_t)
> +
> +TEST_ALL (DEF_LOOP)
> +
> +/* { dg-final { scan-assembler-times {\tvmulh\.vv} 4 } } */
> +/* { dg-final { scan-assembler-times {\tvmulhu\.vv} 4 } } */
> diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/mulh_run-1.c
> b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/mulh_run-1.c
> new file mode 100644
> index 00000000000..7a47e11a3a0
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/mulh_run-1.c
> @@ -0,0 +1,29 @@
> +/* { dg-do run { target { riscv_vector } } } */
> +/* { dg-additional-options "--param=riscv-autovec-preference=scalable" }
> */
> +
> +#include "mulh-1.c"
> +
> +#define N 79
> +
> +#define TEST_LOOP(TYPE) \
> + { \
> + TYPE dst[N], src[N]; \
> + for (int i = 0; i < N; ++i) \
> + { \
> + src[i] = i * 7 + i % 3; \
> + if (i % 11 > 7) \
> + src[i] = -src[i]; \
> + asm volatile ("" ::: "memory"); \
> + } \
> + mod_##TYPE (dst, src, N); \
> + for (int i = 0; i < N; ++i) \
> + if (dst[i] != src[i] % 19) \
> + __builtin_abort (); \
> + }
> +
> +int
> +main (void)
> +{
> + TEST_ALL (TEST_LOOP);
> + return 0;
> +}
> diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/mulh_run-2.c
> b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/mulh_run-2.c
> new file mode 100644
> index 00000000000..72c72b0f4eb
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/mulh_run-2.c
> @@ -0,0 +1,29 @@
> +/* { dg-do run { target { riscv_vector } } } */
> +/* { dg-additional-options "--param=riscv-autovec-preference=scalable" }
> */
> +
> +#include "mulh-2.c"
> +
> +#define N 79
> +
> +#define TEST_LOOP(TYPE) \
> + { \
> + TYPE dst[N], src[N]; \
> + for (int i = 0; i < N; ++i) \
> + { \
> + src[i] = i * 7 + i % 3; \
> + if (i % 11 > 7) \
> + src[i] = -src[i]; \
> + asm volatile ("" ::: "memory"); \
> + } \
> + mod_##TYPE (dst, src, N); \
> + for (int i = 0; i < N; ++i) \
> + if (dst[i] != src[i] / 17) \
> + __builtin_abort (); \
> + }
> +
> +int
> +main (void)
> +{
> + TEST_ALL (TEST_LOOP);
> + return 0;
> +}
> --
> 2.36.1
>
>
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2023-07-12 10:17 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-07-12 8:39 [PATCH] RISC-V: Support integer mult highpart auto-vectorization juzhe.zhong
2023-07-12 9:17 ` Kito Cheng
2023-07-12 10:17 ` Li, Pan2
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).