public inbox for gcc-cvs@sourceware.org
help / color / mirror / Atom feed
* [gcc(refs/vendors/riscv/heads/gcc-13-with-riscv-opts)] RISC-V: Fix incorrect index(offset) of gather/scatter
@ 2023-10-12 22:00 Jeff Law
0 siblings, 0 replies; only message in thread
From: Jeff Law @ 2023-10-12 22:00 UTC (permalink / raw)
To: gcc-cvs
https://gcc.gnu.org/g:b23512e68a588455b664908a8523762714f3314d
commit b23512e68a588455b664908a8523762714f3314d
Author: Juzhe-Zhong <juzhe.zhong@rivai.ai>
Date: Wed Oct 11 17:54:44 2023 +0800
RISC-V: Fix incorrect index(offset) of gather/scatter
I suddenly discovered I made a mistake that was lucky un-exposed.
https://godbolt.org/z/c3jzrh7or
GCC is using 32 bit index offset:
vsll.vi v1,v1,2
vsetvli zero,a5,e32,m1,ta,ma
vluxei32.v v1,(a1),v1
This is wrong since v1 may overflow 32bit after vsll.vi.
After this patch:
vsext.vf2 v8,v4
vsll.vi v8,v8,2
vluxei64.v v8,(a1),v8
Same as Clang.
Regression passed. Ok for trunk ?
gcc/ChangeLog:
* config/riscv/autovec.md: Fix index bug.
* config/riscv/riscv-protos.h (gather_scatter_valid_offset_mode_p): New function.
* config/riscv/riscv-v.cc (expand_gather_scatter): Fix index bug.
(gather_scatter_valid_offset_mode_p): New function.
gcc/testsuite/ChangeLog:
* gcc.target/riscv/rvv/autovec/gather-scatter/offset_extend-1.c: New test.
(cherry picked from commit f6c5e247b2913f0a7b3be986fe96afbed29c1d5e)
Diff:
---
gcc/config/riscv/autovec.md | 28 +++++++++++-----------
gcc/config/riscv/riscv-protos.h | 1 +
gcc/config/riscv/riscv-v.cc | 13 +++++++---
.../rvv/autovec/gather-scatter/offset_extend-1.c | 14 +++++++++++
4 files changed, 39 insertions(+), 17 deletions(-)
diff --git a/gcc/config/riscv/autovec.md b/gcc/config/riscv/autovec.md
index 75948f58f58..6d51ce36e78 100644
--- a/gcc/config/riscv/autovec.md
+++ b/gcc/config/riscv/autovec.md
@@ -59,7 +59,7 @@
(match_operand:<RATIO64:VM> 5 "vector_mask_operand")
(match_operand 6 "autovec_length_operand")
(match_operand 7 "const_0_operand")]
- "TARGET_VECTOR"
+ "TARGET_VECTOR && riscv_vector::gather_scatter_valid_offset_mode_p (<RATIO64:MODE>mode)"
{
riscv_vector::expand_gather_scatter (operands, true);
DONE;
@@ -74,7 +74,7 @@
(match_operand:<RATIO32:VM> 5 "vector_mask_operand")
(match_operand 6 "autovec_length_operand")
(match_operand 7 "const_0_operand")]
- "TARGET_VECTOR"
+ "TARGET_VECTOR && riscv_vector::gather_scatter_valid_offset_mode_p (<RATIO32:MODE>mode)"
{
riscv_vector::expand_gather_scatter (operands, true);
DONE;
@@ -89,7 +89,7 @@
(match_operand:<RATIO16:VM> 5 "vector_mask_operand")
(match_operand 6 "autovec_length_operand")
(match_operand 7 "const_0_operand")]
- "TARGET_VECTOR"
+ "TARGET_VECTOR && riscv_vector::gather_scatter_valid_offset_mode_p (<RATIO16:MODE>mode)"
{
riscv_vector::expand_gather_scatter (operands, true);
DONE;
@@ -104,7 +104,7 @@
(match_operand:<RATIO8:VM> 5 "vector_mask_operand")
(match_operand 6 "autovec_length_operand")
(match_operand 7 "const_0_operand")]
- "TARGET_VECTOR"
+ "TARGET_VECTOR && riscv_vector::gather_scatter_valid_offset_mode_p (<RATIO8:MODE>mode)"
{
riscv_vector::expand_gather_scatter (operands, true);
DONE;
@@ -119,7 +119,7 @@
(match_operand:<RATIO4:VM> 5 "vector_mask_operand")
(match_operand 6 "autovec_length_operand")
(match_operand 7 "const_0_operand")]
- "TARGET_VECTOR"
+ "TARGET_VECTOR && riscv_vector::gather_scatter_valid_offset_mode_p (<RATIO4:MODE>mode)"
{
riscv_vector::expand_gather_scatter (operands, true);
DONE;
@@ -134,7 +134,7 @@
(match_operand:<RATIO2:VM> 5 "vector_mask_operand")
(match_operand 6 "autovec_length_operand")
(match_operand 7 "const_0_operand")]
- "TARGET_VECTOR"
+ "TARGET_VECTOR && riscv_vector::gather_scatter_valid_offset_mode_p (<RATIO2:MODE>mode)"
{
riscv_vector::expand_gather_scatter (operands, true);
DONE;
@@ -153,7 +153,7 @@
(match_operand:<RATIO1:VM> 5 "vector_mask_operand")
(match_operand 6 "autovec_length_operand")
(match_operand 7 "const_0_operand")]
- "TARGET_VECTOR"
+ "TARGET_VECTOR && riscv_vector::gather_scatter_valid_offset_mode_p (<RATIO1:MODE>mode)"
{
riscv_vector::expand_gather_scatter (operands, true);
DONE;
@@ -172,7 +172,7 @@
(match_operand:<RATIO64:VM> 5 "vector_mask_operand")
(match_operand 6 "autovec_length_operand")
(match_operand 7 "const_0_operand")]
- "TARGET_VECTOR"
+ "TARGET_VECTOR && riscv_vector::gather_scatter_valid_offset_mode_p (<RATIO64:MODE>mode)"
{
riscv_vector::expand_gather_scatter (operands, false);
DONE;
@@ -187,7 +187,7 @@
(match_operand:<RATIO32:VM> 5 "vector_mask_operand")
(match_operand 6 "autovec_length_operand")
(match_operand 7 "const_0_operand")]
- "TARGET_VECTOR"
+ "TARGET_VECTOR && riscv_vector::gather_scatter_valid_offset_mode_p (<RATIO32:MODE>mode)"
{
riscv_vector::expand_gather_scatter (operands, false);
DONE;
@@ -202,7 +202,7 @@
(match_operand:<RATIO16:VM> 5 "vector_mask_operand")
(match_operand 6 "autovec_length_operand")
(match_operand 7 "const_0_operand")]
- "TARGET_VECTOR"
+ "TARGET_VECTOR && riscv_vector::gather_scatter_valid_offset_mode_p (<RATIO16:MODE>mode)"
{
riscv_vector::expand_gather_scatter (operands, false);
DONE;
@@ -217,7 +217,7 @@
(match_operand:<RATIO8:VM> 5 "vector_mask_operand")
(match_operand 6 "autovec_length_operand")
(match_operand 7 "const_0_operand")]
- "TARGET_VECTOR"
+ "TARGET_VECTOR && riscv_vector::gather_scatter_valid_offset_mode_p (<RATIO8:MODE>mode)"
{
riscv_vector::expand_gather_scatter (operands, false);
DONE;
@@ -232,7 +232,7 @@
(match_operand:<RATIO4:VM> 5 "vector_mask_operand")
(match_operand 6 "autovec_length_operand")
(match_operand 7 "const_0_operand")]
- "TARGET_VECTOR"
+ "TARGET_VECTOR && riscv_vector::gather_scatter_valid_offset_mode_p (<RATIO4:MODE>mode)"
{
riscv_vector::expand_gather_scatter (operands, false);
DONE;
@@ -247,7 +247,7 @@
(match_operand:<RATIO2:VM> 5 "vector_mask_operand")
(match_operand 6 "autovec_length_operand")
(match_operand 7 "const_0_operand")]
- "TARGET_VECTOR"
+ "TARGET_VECTOR && riscv_vector::gather_scatter_valid_offset_mode_p (<RATIO2:MODE>mode)"
{
riscv_vector::expand_gather_scatter (operands, false);
DONE;
@@ -266,7 +266,7 @@
(match_operand:<RATIO1:VM> 5 "vector_mask_operand")
(match_operand 6 "autovec_length_operand")
(match_operand 7 "const_0_operand")]
- "TARGET_VECTOR"
+ "TARGET_VECTOR && riscv_vector::gather_scatter_valid_offset_mode_p (<RATIO1:MODE>mode)"
{
riscv_vector::expand_gather_scatter (operands, false);
DONE;
diff --git a/gcc/config/riscv/riscv-protos.h b/gcc/config/riscv/riscv-protos.h
index f6bd15b47b0..8c9f7e0ab11 100644
--- a/gcc/config/riscv/riscv-protos.h
+++ b/gcc/config/riscv/riscv-protos.h
@@ -544,6 +544,7 @@ opt_machine_mode vectorize_related_mode (machine_mode, scalar_mode,
unsigned int autovectorize_vector_modes (vec<machine_mode> *, bool);
bool cmp_lmul_le_one (machine_mode);
bool cmp_lmul_gt_one (machine_mode);
+bool gather_scatter_valid_offset_mode_p (machine_mode);
}
/* We classify builtin types into two classes:
diff --git a/gcc/config/riscv/riscv-v.cc b/gcc/config/riscv/riscv-v.cc
index 64f99d85d91..a75eb59eb43 100644
--- a/gcc/config/riscv/riscv-v.cc
+++ b/gcc/config/riscv/riscv-v.cc
@@ -3491,15 +3491,14 @@ expand_gather_scatter (rtx *ops, bool is_load)
machine_mode vec_mode = GET_MODE (vec_reg);
machine_mode idx_mode = GET_MODE (vec_offset);
- scalar_mode inner_vec_mode = GET_MODE_INNER (vec_mode);
scalar_mode inner_idx_mode = GET_MODE_INNER (idx_mode);
- unsigned inner_vsize = GET_MODE_BITSIZE (inner_vec_mode);
unsigned inner_offsize = GET_MODE_BITSIZE (inner_idx_mode);
poly_int64 nunits = GET_MODE_NUNITS (vec_mode);
poly_int64 value;
bool is_vlmax = poly_int_rtx_p (len, &value) && known_eq (value, nunits);
- if (inner_offsize < inner_vsize)
+ /* Extend the offset element to address width. */
+ if (inner_offsize < BITS_PER_WORD)
{
/* 7.2. Vector Load/Store Addressing Modes.
If the vector offset elements are narrower than XLEN, they are
@@ -3796,6 +3795,14 @@ cmp_lmul_gt_one (machine_mode mode)
return false;
}
+/* Return true if the gather/scatter offset mode is valid. */
+bool
+gather_scatter_valid_offset_mode_p (machine_mode mode)
+{
+ machine_mode new_mode;
+ return get_vector_mode (Pmode, GET_MODE_NUNITS (mode)).exists (&new_mode);
+}
+
/* We don't have to convert the floating point to integer when the
mantissa is zero. Thus, ther will be a limitation for both the
single and double precision floating point. There will be no
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/offset_extend-1.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/offset_extend-1.c
new file mode 100644
index 00000000000..b7936eb749a
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/offset_extend-1.c
@@ -0,0 +1,14 @@
+/* { dg-do compile } */
+/* { dg-additional-options "-march=rv64gcv -mabi=lp64d" } */
+
+void
+f (int *restrict y, int *restrict x, int *restrict indices, int n)
+{
+ for (int i = 0; i < n; ++i)
+ y[i] = x[indices[i]] + 1;
+}
+
+/* { dg-final { scan-assembler {vluxei64\.v} } } */
+/* { dg-final { scan-assembler {vsll\.vi} } } */
+/* { dg-final { scan-assembler {vsext\.vf2} } } */
+/* { dg-final { scan-assembler-not {vluxei32\.v} } } */
^ permalink raw reply [flat|nested] only message in thread
only message in thread, other threads:[~2023-10-12 22:00 UTC | newest]
Thread overview: (only message) (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-10-12 22:00 [gcc(refs/vendors/riscv/heads/gcc-13-with-riscv-opts)] RISC-V: Fix incorrect index(offset) of gather/scatter Jeff Law
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).