* [PATCH V7] RISC-V: Enable basic RVV auto-vectorization support.
@ 2023-05-06 3:43 juzhe.zhong
2023-05-06 13:10 ` Kito Cheng
2023-05-06 15:58 ` Jeff Law
0 siblings, 2 replies; 3+ messages in thread
From: juzhe.zhong @ 2023-05-06 3:43 UTC (permalink / raw)
To: gcc-patches; +Cc: kito.cheng, palmer, jeffreyalaw, rdapp.gcc, Juzhe-Zhong
From: Juzhe-Zhong <juzhe.zhong@rivai.ai>
gcc/ChangeLog:
* config/riscv/riscv-protos.h (preferred_simd_mode): New function.
* config/riscv/riscv-v.cc (autovec_use_vlmax_p): Ditto.
(preferred_simd_mode): Ditto.
* config/riscv/riscv.cc (riscv_get_arg_info): Handle RVV type in function arg.
(riscv_convert_vector_bits): Adjust for RVV auto-vectorization.
(riscv_preferred_simd_mode): New function.
(TARGET_VECTORIZE_PREFERRED_SIMD_MODE): New target hook support.
* config/riscv/vector.md: Add autovec.md.
* config/riscv/autovec.md: New file.
gcc/testsuite/ChangeLog:
* gcc.target/riscv/rvv/rvv.exp: Add testcases for RVV auto-vectorization.
* gcc.target/riscv/rvv/autovec/fixed-vlmax-1.c: New test.
* gcc.target/riscv/rvv/autovec/partial/single_rgroup-1.c: New test.
* gcc.target/riscv/rvv/autovec/partial/single_rgroup-1.h: New test.
* gcc.target/riscv/rvv/autovec/partial/single_rgroup_run-1.c: New test.
* gcc.target/riscv/rvv/autovec/scalable-1.c: New test.
* gcc.target/riscv/rvv/autovec/template-1.h: New test.
* gcc.target/riscv/rvv/autovec/v-1.c: New test.
* gcc.target/riscv/rvv/autovec/v-2.c: New test.
* gcc.target/riscv/rvv/autovec/zve32f-1.c: New test.
* gcc.target/riscv/rvv/autovec/zve32f-2.c: New test.
* gcc.target/riscv/rvv/autovec/zve32f-3.c: New test.
* gcc.target/riscv/rvv/autovec/zve32f_zvl128b-1.c: New test.
* gcc.target/riscv/rvv/autovec/zve32f_zvl128b-2.c: New test.
* gcc.target/riscv/rvv/autovec/zve32x-1.c: New test.
* gcc.target/riscv/rvv/autovec/zve32x-2.c: New test.
* gcc.target/riscv/rvv/autovec/zve32x-3.c: New test.
* gcc.target/riscv/rvv/autovec/zve32x_zvl128b-1.c: New test.
* gcc.target/riscv/rvv/autovec/zve32x_zvl128b-2.c: New test.
* gcc.target/riscv/rvv/autovec/zve64d-1.c: New test.
* gcc.target/riscv/rvv/autovec/zve64d-2.c: New test.
* gcc.target/riscv/rvv/autovec/zve64d-3.c: New test.
* gcc.target/riscv/rvv/autovec/zve64d_zvl128b-1.c: New test.
* gcc.target/riscv/rvv/autovec/zve64d_zvl128b-2.c: New test.
* gcc.target/riscv/rvv/autovec/zve64f-1.c: New test.
* gcc.target/riscv/rvv/autovec/zve64f-2.c: New test.
* gcc.target/riscv/rvv/autovec/zve64f-3.c: New test.
* gcc.target/riscv/rvv/autovec/zve64f_zvl128b-1.c: New test.
* gcc.target/riscv/rvv/autovec/zve64f_zvl128b-2.c: New test.
* gcc.target/riscv/rvv/autovec/zve64x-1.c: New test.
* gcc.target/riscv/rvv/autovec/zve64x-2.c: New test.
* gcc.target/riscv/rvv/autovec/zve64x-3.c: New test.
* gcc.target/riscv/rvv/autovec/zve64x_zvl128b-1.c: New test.
* gcc.target/riscv/rvv/autovec/zve64x_zvl128b-2.c: New test.
---
gcc/config/riscv/autovec.md | 49 ++++++++
gcc/config/riscv/riscv-protos.h | 1 +
gcc/config/riscv/riscv-v.cc | 51 +++++++++
gcc/config/riscv/riscv.cc | 31 ++++-
gcc/config/riscv/vector.md | 4 +-
.../riscv/rvv/autovec/fixed-vlmax-1.c | 24 ++++
.../rvv/autovec/partial/single_rgroup-1.c | 8 ++
.../rvv/autovec/partial/single_rgroup-1.h | 106 ++++++++++++++++++
.../rvv/autovec/partial/single_rgroup_run-1.c | 19 ++++
.../gcc.target/riscv/rvv/autovec/scalable-1.c | 17 +++
.../gcc.target/riscv/rvv/autovec/template-1.h | 68 +++++++++++
.../gcc.target/riscv/rvv/autovec/v-1.c | 11 ++
.../gcc.target/riscv/rvv/autovec/v-2.c | 6 +
.../gcc.target/riscv/rvv/autovec/zve32f-1.c | 6 +
.../gcc.target/riscv/rvv/autovec/zve32f-2.c | 6 +
.../gcc.target/riscv/rvv/autovec/zve32f-3.c | 6 +
.../riscv/rvv/autovec/zve32f_zvl128b-1.c | 6 +
.../riscv/rvv/autovec/zve32f_zvl128b-2.c | 6 +
.../gcc.target/riscv/rvv/autovec/zve32x-1.c | 6 +
.../gcc.target/riscv/rvv/autovec/zve32x-2.c | 6 +
.../gcc.target/riscv/rvv/autovec/zve32x-3.c | 6 +
.../riscv/rvv/autovec/zve32x_zvl128b-1.c | 6 +
.../riscv/rvv/autovec/zve32x_zvl128b-2.c | 6 +
.../gcc.target/riscv/rvv/autovec/zve64d-1.c | 6 +
.../gcc.target/riscv/rvv/autovec/zve64d-2.c | 6 +
.../gcc.target/riscv/rvv/autovec/zve64d-3.c | 6 +
.../riscv/rvv/autovec/zve64d_zvl128b-1.c | 6 +
.../riscv/rvv/autovec/zve64d_zvl128b-2.c | 6 +
.../gcc.target/riscv/rvv/autovec/zve64f-1.c | 6 +
.../gcc.target/riscv/rvv/autovec/zve64f-2.c | 6 +
.../gcc.target/riscv/rvv/autovec/zve64f-3.c | 6 +
.../riscv/rvv/autovec/zve64f_zvl128b-1.c | 6 +
.../riscv/rvv/autovec/zve64f_zvl128b-2.c | 6 +
.../gcc.target/riscv/rvv/autovec/zve64x-1.c | 6 +
.../gcc.target/riscv/rvv/autovec/zve64x-2.c | 6 +
.../gcc.target/riscv/rvv/autovec/zve64x-3.c | 6 +
.../riscv/rvv/autovec/zve64x_zvl128b-1.c | 6 +
.../riscv/rvv/autovec/zve64x_zvl128b-2.c | 6 +
gcc/testsuite/gcc.target/riscv/rvv/rvv.exp | 16 +++
39 files changed, 559 insertions(+), 2 deletions(-)
create mode 100644 gcc/config/riscv/autovec.md
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/fixed-vlmax-1.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/partial/single_rgroup-1.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/partial/single_rgroup-1.h
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/partial/single_rgroup_run-1.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/scalable-1.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/template-1.h
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/v-1.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/v-2.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32f-1.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32f-2.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32f-3.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32f_zvl128b-1.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32f_zvl128b-2.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32x-1.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32x-2.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32x-3.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32x_zvl128b-1.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32x_zvl128b-2.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64d-1.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64d-2.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64d-3.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64d_zvl128b-1.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64d_zvl128b-2.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64f-1.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64f-2.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64f-3.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64f_zvl128b-1.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64f_zvl128b-2.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64x-1.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64x-2.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64x-3.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64x_zvl128b-1.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64x_zvl128b-2.c
diff --git a/gcc/config/riscv/autovec.md b/gcc/config/riscv/autovec.md
new file mode 100644
index 00000000000..b5d46ff57ab
--- /dev/null
+++ b/gcc/config/riscv/autovec.md
@@ -0,0 +1,49 @@
+;; Machine description for auto-vectorization using RVV for GNU compiler.
+;; Copyright (C) 2023 Free Software Foundation, Inc.
+;; Contributed by Juzhe Zhong (juzhe.zhong@rivai.ai), RiVAI Technologies Ltd.
+
+;; This file is part of GCC.
+
+;; GCC is free software; you can redistribute it and/or modify
+;; it under the terms of the GNU General Public License as published by
+;; the Free Software Foundation; either version 3, or (at your option)
+;; any later version.
+
+;; GCC is distributed in the hope that it will be useful,
+;; but WITHOUT ANY WARRANTY; without even the implied warranty of
+;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+;; GNU General Public License for more details.
+
+;; You should have received a copy of the GNU General Public License
+;; along with GCC; see the file COPYING3. If not see
+;; <http://www.gnu.org/licenses/>.
+
+;; =========================================================================
+;; == Loads/Stores
+;; =========================================================================
+
+;; len_load/len_store is a sub-optimal pattern for RVV auto-vectorization support.
+;; We will replace them when len_maskload/len_maskstore is supported in loop vectorizer.
+(define_expand "len_load_<mode>"
+ [(match_operand:V 0 "register_operand")
+ (match_operand:V 1 "memory_operand")
+ (match_operand 2 "vector_length_operand")
+ (match_operand 3 "const_0_operand")]
+ "TARGET_VECTOR"
+{
+ riscv_vector::emit_nonvlmax_op (code_for_pred_mov (<MODE>mode), operands[0],
+ operands[1], operands[2], <VM>mode);
+ DONE;
+})
+
+(define_expand "len_store_<mode>"
+ [(match_operand:V 0 "memory_operand")
+ (match_operand:V 1 "register_operand")
+ (match_operand 2 "vector_length_operand")
+ (match_operand 3 "const_0_operand")]
+ "TARGET_VECTOR"
+{
+ riscv_vector::emit_nonvlmax_op (code_for_pred_mov (<MODE>mode), operands[0],
+ operands[1], operands[2], <VM>mode);
+ DONE;
+})
diff --git a/gcc/config/riscv/riscv-protos.h b/gcc/config/riscv/riscv-protos.h
index 7760a9cac8d..6086306a924 100644
--- a/gcc/config/riscv/riscv-protos.h
+++ b/gcc/config/riscv/riscv-protos.h
@@ -217,6 +217,7 @@ bool slide1_sew64_helper (int, machine_mode, machine_mode,
machine_mode, rtx *);
rtx gen_avl_for_scalar_move (rtx);
void expand_tuple_move (machine_mode, rtx *);
+machine_mode preferred_simd_mode (scalar_mode);
}
/* We classify builtin types into two classes:
diff --git a/gcc/config/riscv/riscv-v.cc b/gcc/config/riscv/riscv-v.cc
index 3950aa80338..1f887f7e747 100644
--- a/gcc/config/riscv/riscv-v.cc
+++ b/gcc/config/riscv/riscv-v.cc
@@ -43,6 +43,7 @@
#include "optabs.h"
#include "tm-constrs.h"
#include "rtx-vector-builder.h"
+#include "targhooks.h"
using namespace riscv_vector;
@@ -924,4 +925,54 @@ expand_tuple_move (machine_mode mask_mode, rtx *ops)
}
}
+/* SCALABLE means that the vector-length is agnostic (run-time invariant and
+ compile-time unknown). FIXED meands that the vector-length is specific
+ (compile-time known). Both RVV_SCALABLE and RVV_FIXED_VLMAX are doing
+ auto-vectorization using VLMAX vsetvl configuration. */
+static bool
+autovec_use_vlmax_p (void)
+{
+ return riscv_autovec_preference == RVV_SCALABLE
+ || riscv_autovec_preference == RVV_FIXED_VLMAX;
+}
+
+/* Return the vectorization machine mode for RVV according to LMUL. */
+machine_mode
+preferred_simd_mode (scalar_mode mode)
+{
+ /* We will disable auto-vectorization when TARGET_MIN_VLEN < 128 &&
+ riscv_autovec_lmul < RVV_M2. Since GCC loop vectorizer report ICE when we
+ enable -march=rv64gc_zve32* and -march=rv32gc_zve64*. in the
+ 'can_duplicate_and_interleave_p' of tree-vect-slp.cc. Since we have
+ VNx1SImode in -march=*zve32* and VNx1DImode in -march=*zve64*, they are
+ enabled in targetm. vector_mode_supported_p and SLP vectorizer will try to
+ use them. Currently, we can support auto-vectorization in
+ -march=rv32_zve32x_zvl128b. Wheras, -march=rv32_zve32x_zvl32b or
+ -march=rv32_zve32x_zvl64b are disabled.
+ */
+ if (autovec_use_vlmax_p ())
+ {
+ /* If TARGET_MIN_VLEN < 128, we don't allow LMUL < 2
+ auto-vectorization since Loop Vectorizer may use VNx1SImode or
+ VNx1DImode to vectorize which will create ICE in the
+ 'can_duplicate_and_interleave_p' of tree-vect-slp.cc. */
+ if (TARGET_MIN_VLEN < 128 && riscv_autovec_lmul < RVV_M2)
+ return word_mode;
+ /* We use LMUL = 1 as base bytesize which is BYTES_PER_RISCV_VECTOR and
+ riscv_autovec_lmul as multiply factor to calculate the the NUNITS to
+ get the auto-vectorization mode. */
+ poly_uint64 nunits;
+ poly_uint64 vector_size
+ = BYTES_PER_RISCV_VECTOR * ((int) riscv_autovec_lmul);
+ poly_uint64 scalar_size = GET_MODE_SIZE (mode);
+ gcc_assert (multiple_p (vector_size, scalar_size, &nunits));
+ machine_mode rvv_mode;
+ if (get_vector_mode (mode, nunits).exists (&rvv_mode))
+ return rvv_mode;
+ }
+ /* TODO: We will support minimum length VLS auto-vectorization in the future.
+ */
+ return word_mode;
+}
+
} // namespace riscv_vector
diff --git a/gcc/config/riscv/riscv.cc b/gcc/config/riscv/riscv.cc
index 45a63cab9c9..aa985c2f456 100644
--- a/gcc/config/riscv/riscv.cc
+++ b/gcc/config/riscv/riscv.cc
@@ -3791,6 +3791,13 @@ riscv_get_arg_info (struct riscv_arg_info *info, const CUMULATIVE_ARGS *cum,
info->gpr_offset = cum->num_gprs;
info->fpr_offset = cum->num_fprs;
+ /* TODO: Currently, it will cause an ICE for --param
+ riscv-autovec-preference=fixed-vlmax. So, we just return NULL_RTX here
+ let GCC generate loads/stores. Ideally, we should either warn the user not
+ to use an RVV vector type as function argument or support the calling
+ convention directly. */
+ if (riscv_v_ext_mode_p (mode))
+ return NULL_RTX;
if (named)
{
riscv_aggregate_field fields[2];
@@ -6334,7 +6341,15 @@ riscv_convert_vector_bits (void)
to set RVV mode size. The RVV machine modes size are run-time constant if
TARGET_VECTOR is enabled. The RVV machine modes size remains default
compile-time constant if TARGET_VECTOR is disabled. */
- return TARGET_VECTOR ? poly_uint16 (1, 1) : 1;
+ if (TARGET_VECTOR)
+ {
+ if (riscv_autovec_preference == RVV_FIXED_VLMAX)
+ return (int) TARGET_MIN_VLEN / (riscv_bytes_per_vector_chunk * 8);
+ else
+ return poly_uint16 (1, 1);
+ }
+ else
+ return 1;
}
/* Implement TARGET_OPTION_OVERRIDE. */
@@ -7345,6 +7360,17 @@ riscv_use_divmod_expander (void)
return tune_param->use_divmod_expansion;
}
+/* Implement TARGET_VECTORIZE_PREFERRED_SIMD_MODE. */
+
+static machine_mode
+riscv_preferred_simd_mode (scalar_mode mode)
+{
+ if (TARGET_VECTOR)
+ return riscv_vector::preferred_simd_mode (mode);
+
+ return word_mode;
+}
+
/* Initialize the GCC target structure. */
#undef TARGET_ASM_ALIGNED_HI_OP
#define TARGET_ASM_ALIGNED_HI_OP "\t.half\t"
@@ -7602,6 +7628,9 @@ riscv_use_divmod_expander (void)
#undef TARGET_ARRAY_MODE
#define TARGET_ARRAY_MODE riscv_array_mode
+#undef TARGET_VECTORIZE_PREFERRED_SIMD_MODE
+#define TARGET_VECTORIZE_PREFERRED_SIMD_MODE riscv_preferred_simd_mode
+
struct gcc_target targetm = TARGET_INITIALIZER;
#include "gt-riscv.h"
diff --git a/gcc/config/riscv/vector.md b/gcc/config/riscv/vector.md
index 92115e3935f..1ab0172d804 100644
--- a/gcc/config/riscv/vector.md
+++ b/gcc/config/riscv/vector.md
@@ -23,7 +23,7 @@
;; This file include :
;;
;; - Intrinsics (https://github.com/riscv/rvv-intrinsic-doc)
-;; - Auto-vectorization (TBD)
+;; - Auto-vectorization (autovec.md)
;; - Combine optimization (TBD)
(include "vector-iterators.md")
@@ -8187,3 +8187,5 @@
DONE;
}
)
+
+(include "autovec.md")
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/fixed-vlmax-1.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/fixed-vlmax-1.c
new file mode 100644
index 00000000000..09e8396936e
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/fixed-vlmax-1.c
@@ -0,0 +1,24 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcv -mabi=ilp32 -mpreferred-stack-boundary=3 -fno-schedule-insns -fno-schedule-insns2 -O3 --param riscv-autovec-preference=fixed-vlmax" } */
+
+#include "riscv_vector.h"
+
+void f (char*);
+
+void stack_check_alloca_1 (vuint8m1_t data, uint8_t *base, int y, ...)
+{
+ vuint8m8_t v0, v8, v16, v24;
+ asm volatile ("nop"
+ : "=vr" (v0), "=vr" (v8), "=vr" (v16), "=vr" (v24)
+ :
+ :);
+ asm volatile ("nop"
+ :
+ : "vr" (v0), "vr" (v8), "vr" (v16), "vr" (v24)
+ :);
+ *(vuint8m1_t *)base = data;
+ char* pStr = (char*)__builtin_alloca(y);
+ f(pStr);
+}
+
+/* Compiler should not cause an ICE in this testcase. */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/partial/single_rgroup-1.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/partial/single_rgroup-1.c
new file mode 100644
index 00000000000..10cc698a7cd
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/partial/single_rgroup-1.c
@@ -0,0 +1,8 @@
+/* { dg-do compile } */
+/* { dg-additional-options "-march=rv32gcv -mabi=ilp32d --param riscv-autovec-preference=scalable -fno-vect-cost-model -fno-tree-loop-distribute-patterns -fdump-tree-vect-details" } */
+
+#include "single_rgroup-1.h"
+
+TEST_ALL (test_1)
+
+/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 10 "vect" } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/partial/single_rgroup-1.h b/gcc/testsuite/gcc.target/riscv/rvv/autovec/partial/single_rgroup-1.h
new file mode 100644
index 00000000000..be6b4c641cb
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/partial/single_rgroup-1.h
@@ -0,0 +1,106 @@
+#include <stddef.h>
+#include <stdint.h>
+
+#define N 777
+
+#define test_1(TYPE) \
+ TYPE a_##TYPE[N]; \
+ TYPE b_##TYPE[N]; \
+ void __attribute__ ((noinline, noclone)) test_1_##TYPE (unsigned int n) \
+ { \
+ unsigned int i = 0; \
+ for (i = 0; i < n; i++) \
+ b_##TYPE[i] = a_##TYPE[i]; \
+ }
+
+#define run_1(TYPE) \
+ for (unsigned int i = 0; i < N; i++) \
+ a_##TYPE[i] = i * 2 * 33 + 1 + 109; \
+ test_1_##TYPE (5); \
+ for (unsigned int i = 0; i < 5; i++) \
+ if (b_##TYPE[i] != a_##TYPE[i]) \
+ __builtin_abort ();
+
+#define run_2(TYPE) \
+ for (unsigned int i = 0; i < N; i++) \
+ a_##TYPE[i] = i * 2 * 57 + 1 + 999; \
+ test_1_##TYPE (17); \
+ for (unsigned int i = 0; i < 17; i++) \
+ if (b_##TYPE[i] != a_##TYPE[i]) \
+ __builtin_abort ();
+
+#define run_3(TYPE) \
+ for (unsigned int i = 0; i < N; i++) \
+ a_##TYPE[i] = i * 2 * 77 + 1 + 3; \
+ test_1_##TYPE (32); \
+ for (unsigned int i = 0; i < 32; i++) \
+ if (b_##TYPE[i] != a_##TYPE[i]) \
+ __builtin_abort ();
+
+#define run_4(TYPE) \
+ for (unsigned int i = 0; i < N; i++) \
+ a_##TYPE[i] = i * 2 * 45 + 1 + 11; \
+ test_1_##TYPE (128); \
+ for (unsigned int i = 0; i < 128; i++) \
+ if (b_##TYPE[i] != a_##TYPE[i]) \
+ __builtin_abort ();
+
+#define run_5(TYPE) \
+ for (unsigned int i = 0; i < N; i++) \
+ a_##TYPE[i] = i * 2 * 199 + 1 + 79; \
+ test_1_##TYPE (177); \
+ for (unsigned int i = 0; i < 177; i++) \
+ if (b_##TYPE[i] != a_##TYPE[i]) \
+ __builtin_abort ();
+
+#define run_6(TYPE) \
+ for (unsigned int i = 0; i < N; i++) \
+ a_##TYPE[i] = i * 2 * 377 + 1 + 73; \
+ test_1_##TYPE (255); \
+ for (unsigned int i = 0; i < 255; i++) \
+ if (b_##TYPE[i] != a_##TYPE[i]) \
+ __builtin_abort ();
+
+#define run_7(TYPE) \
+ for (unsigned int i = 0; i < N; i++) \
+ a_##TYPE[i] = i * 2 * 98 + 1 + 66; \
+ test_1_##TYPE (333); \
+ for (unsigned int i = 0; i < 333; i++) \
+ if (b_##TYPE[i] != a_##TYPE[i]) \
+ __builtin_abort ();
+
+#define run_8(TYPE) \
+ for (unsigned int i = 0; i < N; i++) \
+ a_##TYPE[i] = i * 2 * 7 + 1 * 7; \
+ test_1_##TYPE (512); \
+ for (unsigned int i = 0; i < 512; i++) \
+ if (b_##TYPE[i] != a_##TYPE[i]) \
+ __builtin_abort ();
+
+#define run_9(TYPE) \
+ for (unsigned int i = 0; i < N; i++) \
+ a_##TYPE[i] = i * 2 + 1 + 88; \
+ test_1_##TYPE (637); \
+ for (unsigned int i = 0; i < 637; i++) \
+ if (b_##TYPE[i] != a_##TYPE[i]) \
+ __builtin_abort ();
+
+#define run_10(TYPE) \
+ for (unsigned int i = 0; i < N; i++) \
+ a_##TYPE[i] = i * 2 * 331 + 1 + 547; \
+ test_1_##TYPE (777); \
+ for (unsigned int i = 0; i < 777; i++) \
+ if (b_##TYPE[i] != a_##TYPE[i]) \
+ __builtin_abort ();
+
+#define TEST_ALL(T) \
+ T (int8_t) \
+ T (uint8_t) \
+ T (int16_t) \
+ T (uint16_t) \
+ T (int32_t) \
+ T (uint32_t) \
+ T (int64_t) \
+ T (uint64_t) \
+ T (float) \
+ T (double)
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/partial/single_rgroup_run-1.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/partial/single_rgroup_run-1.c
new file mode 100644
index 00000000000..4af2f18de8a
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/partial/single_rgroup_run-1.c
@@ -0,0 +1,19 @@
+/* { dg-do run { target { riscv_vector } } } */
+/* { dg-additional-options "-fno-vect-cost-model -fno-tree-loop-distribute-patterns --param riscv-autovec-preference=scalable" } */
+
+#include "single_rgroup-1.c"
+
+int main (void)
+{
+ TEST_ALL (run_1)
+ TEST_ALL (run_2)
+ TEST_ALL (run_3)
+ TEST_ALL (run_4)
+ TEST_ALL (run_5)
+ TEST_ALL (run_6)
+ TEST_ALL (run_7)
+ TEST_ALL (run_8)
+ TEST_ALL (run_9)
+ TEST_ALL (run_10)
+ return 0;
+}
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/scalable-1.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/scalable-1.c
new file mode 100644
index 00000000000..500b0adce66
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/scalable-1.c
@@ -0,0 +1,17 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gc_zve64d -mabi=ilp32 -O3 -fno-vect-cost-model --param=riscv-autovec-preference=scalable -fdump-tree-vect-details" } */
+
+#include "riscv_vector.h"
+
+void
+f (int32_t *__restrict f, int32_t *__restrict d, int n)
+{
+ for (int i = 0; i < n; ++i)
+ {
+ f[i * 2 + 0] = 1;
+ f[i * 2 + 1] = 2;
+ d[i] = 3;
+ }
+}
+
+/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 0 "vect" } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/template-1.h b/gcc/testsuite/gcc.target/riscv/rvv/autovec/template-1.h
new file mode 100644
index 00000000000..799e2d7d754
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/template-1.h
@@ -0,0 +1,68 @@
+#include <stddef.h>
+#include <stdint.h>
+
+void
+foo0 (int8_t *__restrict f, int16_t *__restrict d, int n)
+{
+ for (int i = 0; i < n; ++i)
+ {
+ f[i * 2 + 0] = 1;
+ f[i * 2 + 1] = 2;
+ d[i] = 3;
+ }
+}
+
+void
+foo1 (int16_t *__restrict f, int32_t *__restrict d, int n)
+{
+ for (int i = 0; i < n; ++i)
+ {
+ f[i * 2 + 0] = 1;
+ f[i * 2 + 1] = 2;
+ d[i] = 3;
+ }
+}
+
+void
+foo2 (int32_t *__restrict f, int64_t *__restrict d, int n)
+{
+ for (int i = 0; i < n; ++i)
+ {
+ f[i * 2 + 0] = 1;
+ f[i * 2 + 1] = 2;
+ d[i] = 3;
+ }
+}
+
+void
+foo3 (int16_t *__restrict f, float *__restrict d, int n)
+{
+ for (int i = 0; i < n; ++i)
+ {
+ f[i * 2 + 0] = 1;
+ f[i * 2 + 1] = 2;
+ d[i] = 3;
+ }
+}
+
+void
+foo4 (int32_t *__restrict f, float *__restrict d, int n)
+{
+ for (int i = 0; i < n; ++i)
+ {
+ f[i * 2 + 0] = 1;
+ f[i * 2 + 1] = 2;
+ d[i] = 3;
+ }
+}
+
+void
+foo5 (float *__restrict f, double *__restrict d, int n)
+{
+ for (int i = 0; i < n; ++i)
+ {
+ f[i * 2 + 0] = 1;
+ f[i * 2 + 1] = 2;
+ d[i] = 3;
+ }
+}
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/v-1.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/v-1.c
new file mode 100644
index 00000000000..383c82a3b7c
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/v-1.c
@@ -0,0 +1,11 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcv -mabi=ilp32d --param riscv-autovec-preference=scalable -fdump-tree-vect-details" } */
+
+#include "template-1.h"
+
+/* Currently, we don't support SLP auto-vectorization for VLA. But it's
+ necessary that we add this testcase here to make sure such unsupported SLP
+ auto-vectorization will not cause an ICE. We will enable "vect" checking when
+ we support SLP auto-vectorization for VLA in the future. */
+
+/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 0 "vect" } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/v-2.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/v-2.c
new file mode 100644
index 00000000000..3d086e30081
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/v-2.c
@@ -0,0 +1,6 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcv -mabi=ilp32d --param riscv-autovec-preference=fixed-vlmax -fdump-tree-vect-details" } */
+
+#include "template-1.h"
+
+/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 5 "vect" } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32f-1.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32f-1.c
new file mode 100644
index 00000000000..ab57e89b1cd
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32f-1.c
@@ -0,0 +1,6 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gc_zve32f -mabi=ilp32d --param riscv-autovec-preference=scalable -fdump-tree-vect-details" } */
+
+#include "template-1.h"
+
+/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 2 "vect" } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32f-2.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32f-2.c
new file mode 100644
index 00000000000..d6199665126
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32f-2.c
@@ -0,0 +1,6 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gc_zve32f -mabi=ilp32d --param riscv-autovec-preference=fixed-vlmax -fdump-tree-vect-details" } */
+
+#include "template-1.h"
+
+/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 1 "vect" } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32f-3.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32f-3.c
new file mode 100644
index 00000000000..d5109c72045
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32f-3.c
@@ -0,0 +1,6 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gc_zve32f -mabi=ilp32d --param riscv-autovec-preference=fixed-vlmax --param riscv-autovec-lmul=m2 -fdump-tree-vect-details" } */
+
+#include "template-1.h"
+
+/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 4 "vect" } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32f_zvl128b-1.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32f_zvl128b-1.c
new file mode 100644
index 00000000000..23cc1c8651f
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32f_zvl128b-1.c
@@ -0,0 +1,6 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gc_zve32f_zvl128b -mabi=ilp32d --param riscv-autovec-preference=scalable -fdump-tree-vect-details" } */
+
+#include "template-1.h"
+
+/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 0 "vect" } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32f_zvl128b-2.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32f_zvl128b-2.c
new file mode 100644
index 00000000000..c53975e9b01
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32f_zvl128b-2.c
@@ -0,0 +1,6 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gc_zve32f_zvl128b -mabi=ilp32d --param riscv-autovec-preference=fixed-vlmax -fno-vect-cost-model -fdump-tree-vect-details" } */
+
+#include "template-1.h"
+
+/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 4 "vect" } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32x-1.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32x-1.c
new file mode 100644
index 00000000000..d23de3e4c3b
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32x-1.c
@@ -0,0 +1,6 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gc_zve32x -mabi=ilp32d --param riscv-autovec-preference=scalable -fdump-tree-vect-details" } */
+
+#include "template-1.h"
+
+/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 2 "vect" } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32x-2.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32x-2.c
new file mode 100644
index 00000000000..575e9479f94
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32x-2.c
@@ -0,0 +1,6 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gc_zve32x -mabi=ilp32d --param riscv-autovec-preference=fixed-vlmax -fdump-tree-vect-details" } */
+
+#include "template-1.h"
+
+/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 1 "vect" } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32x-3.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32x-3.c
new file mode 100644
index 00000000000..50e8963033b
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32x-3.c
@@ -0,0 +1,6 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gc_zve32x -mabi=ilp32d --param riscv-autovec-preference=fixed-vlmax --param riscv-autovec-lmul=m2 -fdump-tree-vect-details" } */
+
+#include "template-1.h"
+
+/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 3 "vect" } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32x_zvl128b-1.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32x_zvl128b-1.c
new file mode 100644
index 00000000000..4f130f02f67
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32x_zvl128b-1.c
@@ -0,0 +1,6 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gc_zve32x_zvl128b -mabi=ilp32d --param riscv-autovec-preference=scalable -fdump-tree-vect-details" } */
+
+#include "template-1.h"
+
+/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 0 "vect" } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32x_zvl128b-2.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32x_zvl128b-2.c
new file mode 100644
index 00000000000..80a4796fd38
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32x_zvl128b-2.c
@@ -0,0 +1,6 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gc_zve32x_zvl128b -mabi=ilp32d --param riscv-autovec-preference=fixed-vlmax -fdump-tree-vect-details" } */
+
+#include "template-1.h"
+
+/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 2 "vect" } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64d-1.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64d-1.c
new file mode 100644
index 00000000000..823d51a03cb
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64d-1.c
@@ -0,0 +1,6 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gc_zve64d -mabi=ilp32d --param riscv-autovec-preference=scalable -fdump-tree-vect-details" } */
+
+#include "template-1.h"
+
+/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 2 "vect" } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64d-2.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64d-2.c
new file mode 100644
index 00000000000..0d6ebc53d1a
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64d-2.c
@@ -0,0 +1,6 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gc_zve64d -mabi=ilp32d --param riscv-autovec-preference=fixed-vlmax -fdump-tree-vect-details" } */
+
+#include "template-1.h"
+
+/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 1 "vect" } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64d-3.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64d-3.c
new file mode 100644
index 00000000000..2a72030c3bc
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64d-3.c
@@ -0,0 +1,6 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gc_zve64d -mabi=ilp32d --param riscv-autovec-preference=fixed-vlmax --param riscv-autovec-lmul=m2 -fdump-tree-vect-details" } */
+
+#include "template-1.h"
+
+/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 5 "vect" } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64d_zvl128b-1.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64d_zvl128b-1.c
new file mode 100644
index 00000000000..5ead22746d3
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64d_zvl128b-1.c
@@ -0,0 +1,6 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gc_zve64d_zvl128b -mabi=ilp32d --param riscv-autovec-preference=scalable -fdump-tree-vect-details" } */
+
+#include "template-1.h"
+
+/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 0 "vect" } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64d_zvl128b-2.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64d_zvl128b-2.c
new file mode 100644
index 00000000000..9e236e0af1b
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64d_zvl128b-2.c
@@ -0,0 +1,6 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gc_zve64d_zvl128b -mabi=ilp32d --param riscv-autovec-preference=fixed-vlmax -fdump-tree-vect-details" } */
+
+#include "template-1.h"
+
+/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 5 "vect" } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64f-1.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64f-1.c
new file mode 100644
index 00000000000..e03d1b44ca6
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64f-1.c
@@ -0,0 +1,6 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gc_zve64f -mabi=ilp32d --param riscv-autovec-preference=scalable -fdump-tree-vect-details" } */
+
+#include "template-1.h"
+
+/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 2 "vect" } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64f-2.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64f-2.c
new file mode 100644
index 00000000000..6bb6b919c77
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64f-2.c
@@ -0,0 +1,6 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gc_zve64f -mabi=ilp32d --param riscv-autovec-preference=fixed-vlmax -fdump-tree-vect-details" } */
+
+#include "template-1.h"
+
+/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 1 "vect" } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64f-3.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64f-3.c
new file mode 100644
index 00000000000..43eca9d0727
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64f-3.c
@@ -0,0 +1,6 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gc_zve64f -mabi=ilp32d --param riscv-autovec-preference=fixed-vlmax --param riscv-autovec-lmul=m2 -fdump-tree-vect-details" } */
+
+#include "template-1.h"
+
+/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 4 "vect" } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64f_zvl128b-1.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64f_zvl128b-1.c
new file mode 100644
index 00000000000..5bb2d9d96fa
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64f_zvl128b-1.c
@@ -0,0 +1,6 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gc_zve64f_zvl128b -mabi=ilp32d --param riscv-autovec-preference=scalable -fdump-tree-vect-details" } */
+
+#include "template-1.h"
+
+/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 0 "vect" } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64f_zvl128b-2.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64f_zvl128b-2.c
new file mode 100644
index 00000000000..f1500074370
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64f_zvl128b-2.c
@@ -0,0 +1,6 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gc_zve64f_zvl128b -mabi=ilp32d --param riscv-autovec-preference=fixed-vlmax -fdump-tree-vect-details" } */
+
+#include "template-1.h"
+
+/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 4 "vect" } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64x-1.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64x-1.c
new file mode 100644
index 00000000000..8fbfa8a8b68
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64x-1.c
@@ -0,0 +1,6 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gc_zve64x -mabi=ilp32d --param riscv-autovec-preference=scalable -fdump-tree-vect-details" } */
+
+#include "template-1.h"
+
+/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 2 "vect" } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64x-2.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64x-2.c
new file mode 100644
index 00000000000..ede28e88dac
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64x-2.c
@@ -0,0 +1,6 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gc_zve64x -mabi=ilp32d --param riscv-autovec-preference=fixed-vlmax -fdump-tree-vect-details" } */
+
+#include "template-1.h"
+
+/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 1 "vect" } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64x-3.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64x-3.c
new file mode 100644
index 00000000000..d59f7362b43
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64x-3.c
@@ -0,0 +1,6 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gc_zve64x -mabi=ilp32d --param riscv-autovec-preference=fixed-vlmax --param riscv-autovec-lmul=m2 -fdump-tree-vect-details" } */
+
+#include "template-1.h"
+
+/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 3 "vect" } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64x_zvl128b-1.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64x_zvl128b-1.c
new file mode 100644
index 00000000000..71820ece4b2
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64x_zvl128b-1.c
@@ -0,0 +1,6 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gc_zve64x_zvl128b -mabi=ilp32d --param riscv-autovec-preference=scalable -fdump-tree-vect-details" } */
+
+#include "template-1.h"
+
+/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 0 "vect" } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64x_zvl128b-2.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64x_zvl128b-2.c
new file mode 100644
index 00000000000..90398db52a0
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64x_zvl128b-2.c
@@ -0,0 +1,6 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gc_zve64x_zvl128b -mabi=ilp32d --param riscv-autovec-preference=fixed-vlmax -fdump-tree-vect-details" } */
+
+#include "template-1.h"
+
+/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 3 "vect" } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/rvv.exp b/gcc/testsuite/gcc.target/riscv/rvv/rvv.exp
index 4b5509db385..0eef068058a 100644
--- a/gcc/testsuite/gcc.target/riscv/rvv/rvv.exp
+++ b/gcc/testsuite/gcc.target/riscv/rvv/rvv.exp
@@ -46,6 +46,22 @@ dg-runtest [lsort [glob -nocomplain $srcdir/$subdir/base/*.\[cS\]]] \
"" $CFLAGS
gcc-dg-runtest [lsort [glob -nocomplain $srcdir/$subdir/vsetvl/*.\[cS\]]] \
"" $CFLAGS
+dg-runtest [lsort [glob -nocomplain $srcdir/$subdir/autovec/*.\[cS\]]] \
+ "-O3 -ftree-vectorize" $CFLAGS
+
+set AUTOVEC_TEST_OPTS [list \
+ {-ftree-vectorize -O3 --param riscv-autovec-lmul=m1} \
+ {-ftree-vectorize -O3 --param riscv-autovec-lmul=m2} \
+ {-ftree-vectorize -O3 --param riscv-autovec-lmul=m4} \
+ {-ftree-vectorize -O3 --param riscv-autovec-lmul=m8} \
+ {-ftree-vectorize -O2 --param riscv-autovec-lmul=m1} \
+ {-ftree-vectorize -O2 --param riscv-autovec-lmul=m2} \
+ {-ftree-vectorize -O2 --param riscv-autovec-lmul=m4} \
+ {-ftree-vectorize -O2 --param riscv-autovec-lmul=m8} ]
+foreach op $AUTOVEC_TEST_OPTS {
+ dg-runtest [lsort [glob -nocomplain $srcdir/$subdir/autovec/partial/*.\[cS\]]] \
+ "" "$op"
+}
# All done.
dg-finish
--
2.36.3
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [PATCH V7] RISC-V: Enable basic RVV auto-vectorization support.
2023-05-06 3:43 [PATCH V7] RISC-V: Enable basic RVV auto-vectorization support juzhe.zhong
@ 2023-05-06 13:10 ` Kito Cheng
2023-05-06 15:58 ` Jeff Law
1 sibling, 0 replies; 3+ messages in thread
From: Kito Cheng @ 2023-05-06 13:10 UTC (permalink / raw)
To: juzhe.zhong; +Cc: gcc-patches, jeffreyalaw, palmer, rdapp.gcc
[-- Attachment #1: Type: text/plain, Size: 44231 bytes --]
LGTM, thanks :)
<juzhe.zhong@rivai.ai>於 2023年5月6日 週六，11:44寫道：
> From: Juzhe-Zhong <juzhe.zhong@rivai.ai>
>
> gcc/ChangeLog:
>
> * config/riscv/riscv-protos.h (preferred_simd_mode): New function.
> * config/riscv/riscv-v.cc (autovec_use_vlmax_p): Ditto.
> (preferred_simd_mode): Ditto.
> * config/riscv/riscv.cc (riscv_get_arg_info): Handle RVV type in
> function arg.
> (riscv_convert_vector_bits): Adjust for RVV auto-vectorization.
> (riscv_preferred_simd_mode): New function.
> (TARGET_VECTORIZE_PREFERRED_SIMD_MODE): New target hook support.
> * config/riscv/vector.md: Add autovec.md.
> * config/riscv/autovec.md: New file.
>
> gcc/testsuite/ChangeLog:
>
> * gcc.target/riscv/rvv/rvv.exp: Add testcases for RVV
> auto-vectorization.
> * gcc.target/riscv/rvv/autovec/fixed-vlmax-1.c: New test.
> * gcc.target/riscv/rvv/autovec/partial/single_rgroup-1.c: New test.
> * gcc.target/riscv/rvv/autovec/partial/single_rgroup-1.h: New test.
> * gcc.target/riscv/rvv/autovec/partial/single_rgroup_run-1.c: New
> test.
> * gcc.target/riscv/rvv/autovec/scalable-1.c: New test.
> * gcc.target/riscv/rvv/autovec/template-1.h: New test.
> * gcc.target/riscv/rvv/autovec/v-1.c: New test.
> * gcc.target/riscv/rvv/autovec/v-2.c: New test.
> * gcc.target/riscv/rvv/autovec/zve32f-1.c: New test.
> * gcc.target/riscv/rvv/autovec/zve32f-2.c: New test.
> * gcc.target/riscv/rvv/autovec/zve32f-3.c: New test.
> * gcc.target/riscv/rvv/autovec/zve32f_zvl128b-1.c: New test.
> * gcc.target/riscv/rvv/autovec/zve32f_zvl128b-2.c: New test.
> * gcc.target/riscv/rvv/autovec/zve32x-1.c: New test.
> * gcc.target/riscv/rvv/autovec/zve32x-2.c: New test.
> * gcc.target/riscv/rvv/autovec/zve32x-3.c: New test.
> * gcc.target/riscv/rvv/autovec/zve32x_zvl128b-1.c: New test.
> * gcc.target/riscv/rvv/autovec/zve32x_zvl128b-2.c: New test.
> * gcc.target/riscv/rvv/autovec/zve64d-1.c: New test.
> * gcc.target/riscv/rvv/autovec/zve64d-2.c: New test.
> * gcc.target/riscv/rvv/autovec/zve64d-3.c: New test.
> * gcc.target/riscv/rvv/autovec/zve64d_zvl128b-1.c: New test.
> * gcc.target/riscv/rvv/autovec/zve64d_zvl128b-2.c: New test.
> * gcc.target/riscv/rvv/autovec/zve64f-1.c: New test.
> * gcc.target/riscv/rvv/autovec/zve64f-2.c: New test.
> * gcc.target/riscv/rvv/autovec/zve64f-3.c: New test.
> * gcc.target/riscv/rvv/autovec/zve64f_zvl128b-1.c: New test.
> * gcc.target/riscv/rvv/autovec/zve64f_zvl128b-2.c: New test.
> * gcc.target/riscv/rvv/autovec/zve64x-1.c: New test.
> * gcc.target/riscv/rvv/autovec/zve64x-2.c: New test.
> * gcc.target/riscv/rvv/autovec/zve64x-3.c: New test.
> * gcc.target/riscv/rvv/autovec/zve64x_zvl128b-1.c: New test.
> * gcc.target/riscv/rvv/autovec/zve64x_zvl128b-2.c: New test.
>
> ---
> gcc/config/riscv/autovec.md | 49 ++++++++
> gcc/config/riscv/riscv-protos.h | 1 +
> gcc/config/riscv/riscv-v.cc | 51 +++++++++
> gcc/config/riscv/riscv.cc | 31 ++++-
> gcc/config/riscv/vector.md | 4 +-
> .../riscv/rvv/autovec/fixed-vlmax-1.c | 24 ++++
> .../rvv/autovec/partial/single_rgroup-1.c | 8 ++
> .../rvv/autovec/partial/single_rgroup-1.h | 106 ++++++++++++++++++
> .../rvv/autovec/partial/single_rgroup_run-1.c | 19 ++++
> .../gcc.target/riscv/rvv/autovec/scalable-1.c | 17 +++
> .../gcc.target/riscv/rvv/autovec/template-1.h | 68 +++++++++++
> .../gcc.target/riscv/rvv/autovec/v-1.c | 11 ++
> .../gcc.target/riscv/rvv/autovec/v-2.c | 6 +
> .../gcc.target/riscv/rvv/autovec/zve32f-1.c | 6 +
> .../gcc.target/riscv/rvv/autovec/zve32f-2.c | 6 +
> .../gcc.target/riscv/rvv/autovec/zve32f-3.c | 6 +
> .../riscv/rvv/autovec/zve32f_zvl128b-1.c | 6 +
> .../riscv/rvv/autovec/zve32f_zvl128b-2.c | 6 +
> .../gcc.target/riscv/rvv/autovec/zve32x-1.c | 6 +
> .../gcc.target/riscv/rvv/autovec/zve32x-2.c | 6 +
> .../gcc.target/riscv/rvv/autovec/zve32x-3.c | 6 +
> .../riscv/rvv/autovec/zve32x_zvl128b-1.c | 6 +
> .../riscv/rvv/autovec/zve32x_zvl128b-2.c | 6 +
> .../gcc.target/riscv/rvv/autovec/zve64d-1.c | 6 +
> .../gcc.target/riscv/rvv/autovec/zve64d-2.c | 6 +
> .../gcc.target/riscv/rvv/autovec/zve64d-3.c | 6 +
> .../riscv/rvv/autovec/zve64d_zvl128b-1.c | 6 +
> .../riscv/rvv/autovec/zve64d_zvl128b-2.c | 6 +
> .../gcc.target/riscv/rvv/autovec/zve64f-1.c | 6 +
> .../gcc.target/riscv/rvv/autovec/zve64f-2.c | 6 +
> .../gcc.target/riscv/rvv/autovec/zve64f-3.c | 6 +
> .../riscv/rvv/autovec/zve64f_zvl128b-1.c | 6 +
> .../riscv/rvv/autovec/zve64f_zvl128b-2.c | 6 +
> .../gcc.target/riscv/rvv/autovec/zve64x-1.c | 6 +
> .../gcc.target/riscv/rvv/autovec/zve64x-2.c | 6 +
> .../gcc.target/riscv/rvv/autovec/zve64x-3.c | 6 +
> .../riscv/rvv/autovec/zve64x_zvl128b-1.c | 6 +
> .../riscv/rvv/autovec/zve64x_zvl128b-2.c | 6 +
> gcc/testsuite/gcc.target/riscv/rvv/rvv.exp | 16 +++
> 39 files changed, 559 insertions(+), 2 deletions(-)
> create mode 100644 gcc/config/riscv/autovec.md
> create mode 100644
> gcc/testsuite/gcc.target/riscv/rvv/autovec/fixed-vlmax-1.c
> create mode 100644
> gcc/testsuite/gcc.target/riscv/rvv/autovec/partial/single_rgroup-1.c
> create mode 100644
> gcc/testsuite/gcc.target/riscv/rvv/autovec/partial/single_rgroup-1.h
> create mode 100644
> gcc/testsuite/gcc.target/riscv/rvv/autovec/partial/single_rgroup_run-1.c
> create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/scalable-1.c
> create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/template-1.h
> create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/v-1.c
> create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/v-2.c
> create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32f-1.c
> create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32f-2.c
> create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32f-3.c
> create mode 100644
> gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32f_zvl128b-1.c
> create mode 100644
> gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32f_zvl128b-2.c
> create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32x-1.c
> create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32x-2.c
> create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32x-3.c
> create mode 100644
> gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32x_zvl128b-1.c
> create mode 100644
> gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32x_zvl128b-2.c
> create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64d-1.c
> create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64d-2.c
> create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64d-3.c
> create mode 100644
> gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64d_zvl128b-1.c
> create mode 100644
> gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64d_zvl128b-2.c
> create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64f-1.c
> create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64f-2.c
> create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64f-3.c
> create mode 100644
> gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64f_zvl128b-1.c
> create mode 100644
> gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64f_zvl128b-2.c
> create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64x-1.c
> create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64x-2.c
> create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64x-3.c
> create mode 100644
> gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64x_zvl128b-1.c
> create mode 100644
> gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64x_zvl128b-2.c
>
> diff --git a/gcc/config/riscv/autovec.md b/gcc/config/riscv/autovec.md
> new file mode 100644
> index 00000000000..b5d46ff57ab
> --- /dev/null
> +++ b/gcc/config/riscv/autovec.md
> @@ -0,0 +1,49 @@
> +;; Machine description for auto-vectorization using RVV for GNU compiler.
> +;; Copyright (C) 2023 Free Software Foundation, Inc.
> +;; Contributed by Juzhe Zhong (juzhe.zhong@rivai.ai), RiVAI Technologies
> Ltd.
> +
> +;; This file is part of GCC.
> +
> +;; GCC is free software; you can redistribute it and/or modify
> +;; it under the terms of the GNU General Public License as published by
> +;; the Free Software Foundation; either version 3, or (at your option)
> +;; any later version.
> +
> +;; GCC is distributed in the hope that it will be useful,
> +;; but WITHOUT ANY WARRANTY; without even the implied warranty of
> +;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> +;; GNU General Public License for more details.
> +
> +;; You should have received a copy of the GNU General Public License
> +;; along with GCC; see the file COPYING3. If not see
> +;; <http://www.gnu.org/licenses/>.
> +
> +;;
> =========================================================================
> +;; == Loads/Stores
> +;;
> =========================================================================
> +
> +;; len_load/len_store is a sub-optimal pattern for RVV auto-vectorization
> support.
> +;; We will replace them when len_maskload/len_maskstore is supported in
> loop vectorizer.
> +(define_expand "len_load_<mode>"
> + [(match_operand:V 0 "register_operand")
> + (match_operand:V 1 "memory_operand")
> + (match_operand 2 "vector_length_operand")
> + (match_operand 3 "const_0_operand")]
> + "TARGET_VECTOR"
> +{
> + riscv_vector::emit_nonvlmax_op (code_for_pred_mov (<MODE>mode),
> operands[0],
> + operands[1], operands[2], <VM>mode);
> + DONE;
> +})
> +
> +(define_expand "len_store_<mode>"
> + [(match_operand:V 0 "memory_operand")
> + (match_operand:V 1 "register_operand")
> + (match_operand 2 "vector_length_operand")
> + (match_operand 3 "const_0_operand")]
> + "TARGET_VECTOR"
> +{
> + riscv_vector::emit_nonvlmax_op (code_for_pred_mov (<MODE>mode),
> operands[0],
> + operands[1], operands[2], <VM>mode);
> + DONE;
> +})
> diff --git a/gcc/config/riscv/riscv-protos.h
> b/gcc/config/riscv/riscv-protos.h
> index 7760a9cac8d..6086306a924 100644
> --- a/gcc/config/riscv/riscv-protos.h
> +++ b/gcc/config/riscv/riscv-protos.h
> @@ -217,6 +217,7 @@ bool slide1_sew64_helper (int, machine_mode,
> machine_mode,
> machine_mode, rtx *);
> rtx gen_avl_for_scalar_move (rtx);
> void expand_tuple_move (machine_mode, rtx *);
> +machine_mode preferred_simd_mode (scalar_mode);
> }
>
> /* We classify builtin types into two classes:
> diff --git a/gcc/config/riscv/riscv-v.cc b/gcc/config/riscv/riscv-v.cc
> index 3950aa80338..1f887f7e747 100644
> --- a/gcc/config/riscv/riscv-v.cc
> +++ b/gcc/config/riscv/riscv-v.cc
> @@ -43,6 +43,7 @@
> #include "optabs.h"
> #include "tm-constrs.h"
> #include "rtx-vector-builder.h"
> +#include "targhooks.h"
>
> using namespace riscv_vector;
>
> @@ -924,4 +925,54 @@ expand_tuple_move (machine_mode mask_mode, rtx *ops)
> }
> }
>
> +/* SCALABLE means that the vector-length is agnostic (run-time invariant
> and
> + compile-time unknown). FIXED meands that the vector-length is specific
> + (compile-time known). Both RVV_SCALABLE and RVV_FIXED_VLMAX are doing
> + auto-vectorization using VLMAX vsetvl configuration. */
> +static bool
> +autovec_use_vlmax_p (void)
> +{
> + return riscv_autovec_preference == RVV_SCALABLE
> + || riscv_autovec_preference == RVV_FIXED_VLMAX;
> +}
> +
> +/* Return the vectorization machine mode for RVV according to LMUL. */
> +machine_mode
> +preferred_simd_mode (scalar_mode mode)
> +{
> + /* We will disable auto-vectorization when TARGET_MIN_VLEN < 128 &&
> + riscv_autovec_lmul < RVV_M2. Since GCC loop vectorizer report ICE
> when we
> + enable -march=rv64gc_zve32* and -march=rv32gc_zve64*. in the
> + 'can_duplicate_and_interleave_p' of tree-vect-slp.cc. Since we have
> + VNx1SImode in -march=*zve32* and VNx1DImode in -march=*zve64*, they
> are
> + enabled in targetm. vector_mode_supported_p and SLP vectorizer will
> try to
> + use them. Currently, we can support auto-vectorization in
> + -march=rv32_zve32x_zvl128b. Wheras, -march=rv32_zve32x_zvl32b or
> + -march=rv32_zve32x_zvl64b are disabled.
> + */
> + if (autovec_use_vlmax_p ())
> + {
> + /* If TARGET_MIN_VLEN < 128, we don't allow LMUL < 2
> + auto-vectorization since Loop Vectorizer may use VNx1SImode or
> + VNx1DImode to vectorize which will create ICE in the
> + 'can_duplicate_and_interleave_p' of tree-vect-slp.cc. */
> + if (TARGET_MIN_VLEN < 128 && riscv_autovec_lmul < RVV_M2)
> + return word_mode;
> + /* We use LMUL = 1 as base bytesize which is BYTES_PER_RISCV_VECTOR
> and
> + riscv_autovec_lmul as multiply factor to calculate the the NUNITS
> to
> + get the auto-vectorization mode. */
> + poly_uint64 nunits;
> + poly_uint64 vector_size
> + = BYTES_PER_RISCV_VECTOR * ((int) riscv_autovec_lmul);
> + poly_uint64 scalar_size = GET_MODE_SIZE (mode);
> + gcc_assert (multiple_p (vector_size, scalar_size, &nunits));
> + machine_mode rvv_mode;
> + if (get_vector_mode (mode, nunits).exists (&rvv_mode))
> + return rvv_mode;
> + }
> + /* TODO: We will support minimum length VLS auto-vectorization in the
> future.
> + */
> + return word_mode;
> +}
> +
> } // namespace riscv_vector
> diff --git a/gcc/config/riscv/riscv.cc b/gcc/config/riscv/riscv.cc
> index 45a63cab9c9..aa985c2f456 100644
> --- a/gcc/config/riscv/riscv.cc
> +++ b/gcc/config/riscv/riscv.cc
> @@ -3791,6 +3791,13 @@ riscv_get_arg_info (struct riscv_arg_info *info,
> const CUMULATIVE_ARGS *cum,
> info->gpr_offset = cum->num_gprs;
> info->fpr_offset = cum->num_fprs;
>
> + /* TODO: Currently, it will cause an ICE for --param
> + riscv-autovec-preference=fixed-vlmax. So, we just return NULL_RTX
> here
> + let GCC generate loads/stores. Ideally, we should either warn the
> user not
> + to use an RVV vector type as function argument or support the calling
> + convention directly. */
> + if (riscv_v_ext_mode_p (mode))
> + return NULL_RTX;
> if (named)
> {
> riscv_aggregate_field fields[2];
> @@ -6334,7 +6341,15 @@ riscv_convert_vector_bits (void)
> to set RVV mode size. The RVV machine modes size are run-time
> constant if
> TARGET_VECTOR is enabled. The RVV machine modes size remains default
> compile-time constant if TARGET_VECTOR is disabled. */
> - return TARGET_VECTOR ? poly_uint16 (1, 1) : 1;
> + if (TARGET_VECTOR)
> + {
> + if (riscv_autovec_preference == RVV_FIXED_VLMAX)
> + return (int) TARGET_MIN_VLEN / (riscv_bytes_per_vector_chunk * 8);
> + else
> + return poly_uint16 (1, 1);
> + }
> + else
> + return 1;
> }
>
> /* Implement TARGET_OPTION_OVERRIDE. */
> @@ -7345,6 +7360,17 @@ riscv_use_divmod_expander (void)
> return tune_param->use_divmod_expansion;
> }
>
> +/* Implement TARGET_VECTORIZE_PREFERRED_SIMD_MODE. */
> +
> +static machine_mode
> +riscv_preferred_simd_mode (scalar_mode mode)
> +{
> + if (TARGET_VECTOR)
> + return riscv_vector::preferred_simd_mode (mode);
> +
> + return word_mode;
> +}
> +
> /* Initialize the GCC target structure. */
> #undef TARGET_ASM_ALIGNED_HI_OP
> #define TARGET_ASM_ALIGNED_HI_OP "\t.half\t"
> @@ -7602,6 +7628,9 @@ riscv_use_divmod_expander (void)
> #undef TARGET_ARRAY_MODE
> #define TARGET_ARRAY_MODE riscv_array_mode
>
> +#undef TARGET_VECTORIZE_PREFERRED_SIMD_MODE
> +#define TARGET_VECTORIZE_PREFERRED_SIMD_MODE riscv_preferred_simd_mode
> +
> struct gcc_target targetm = TARGET_INITIALIZER;
>
> #include "gt-riscv.h"
> diff --git a/gcc/config/riscv/vector.md b/gcc/config/riscv/vector.md
> index 92115e3935f..1ab0172d804 100644
> --- a/gcc/config/riscv/vector.md
> +++ b/gcc/config/riscv/vector.md
> @@ -23,7 +23,7 @@
> ;; This file include :
> ;;
> ;; - Intrinsics (https://github.com/riscv/rvv-intrinsic-doc)
> -;; - Auto-vectorization (TBD)
> +;; - Auto-vectorization (autovec.md)
> ;; - Combine optimization (TBD)
>
> (include "vector-iterators.md")
> @@ -8187,3 +8187,5 @@
> DONE;
> }
> )
> +
> +(include "autovec.md")
> diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/fixed-vlmax-1.c
> b/gcc/testsuite/gcc.target/riscv/rvv/autovec/fixed-vlmax-1.c
> new file mode 100644
> index 00000000000..09e8396936e
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/fixed-vlmax-1.c
> @@ -0,0 +1,24 @@
> +/* { dg-do compile } */
> +/* { dg-options "-march=rv32gcv -mabi=ilp32 -mpreferred-stack-boundary=3
> -fno-schedule-insns -fno-schedule-insns2 -O3 --param
> riscv-autovec-preference=fixed-vlmax" } */
> +
> +#include "riscv_vector.h"
> +
> +void f (char*);
> +
> +void stack_check_alloca_1 (vuint8m1_t data, uint8_t *base, int y, ...)
> +{
> + vuint8m8_t v0, v8, v16, v24;
> + asm volatile ("nop"
> + : "=vr" (v0), "=vr" (v8), "=vr" (v16), "=vr" (v24)
> + :
> + :);
> + asm volatile ("nop"
> + :
> + : "vr" (v0), "vr" (v8), "vr" (v16), "vr" (v24)
> + :);
> + *(vuint8m1_t *)base = data;
> + char* pStr = (char*)__builtin_alloca(y);
> + f(pStr);
> +}
> +
> +/* Compiler should not cause an ICE in this testcase. */
> diff --git
> a/gcc/testsuite/gcc.target/riscv/rvv/autovec/partial/single_rgroup-1.c
> b/gcc/testsuite/gcc.target/riscv/rvv/autovec/partial/single_rgroup-1.c
> new file mode 100644
> index 00000000000..10cc698a7cd
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/partial/single_rgroup-1.c
> @@ -0,0 +1,8 @@
> +/* { dg-do compile } */
> +/* { dg-additional-options "-march=rv32gcv -mabi=ilp32d --param
> riscv-autovec-preference=scalable -fno-vect-cost-model
> -fno-tree-loop-distribute-patterns -fdump-tree-vect-details" } */
> +
> +#include "single_rgroup-1.h"
> +
> +TEST_ALL (test_1)
> +
> +/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 10
> "vect" } } */
> diff --git
> a/gcc/testsuite/gcc.target/riscv/rvv/autovec/partial/single_rgroup-1.h
> b/gcc/testsuite/gcc.target/riscv/rvv/autovec/partial/single_rgroup-1.h
> new file mode 100644
> index 00000000000..be6b4c641cb
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/partial/single_rgroup-1.h
> @@ -0,0 +1,106 @@
> +#include <stddef.h>
> +#include <stdint.h>
> +
> +#define N 777
> +
> +#define test_1(TYPE)
> \
> + TYPE a_##TYPE[N];
> \
> + TYPE b_##TYPE[N];
> \
> + void __attribute__ ((noinline, noclone)) test_1_##TYPE (unsigned int
> n) \
> + {
> \
> + unsigned int i = 0;
> \
> + for (i = 0; i < n; i++)
> \
> + b_##TYPE[i] = a_##TYPE[i];
> \
> + }
> +
> +#define run_1(TYPE)
> \
> + for (unsigned int i = 0; i < N; i++)
> \
> + a_##TYPE[i] = i * 2 * 33 + 1 + 109;
> \
> + test_1_##TYPE (5);
> \
> + for (unsigned int i = 0; i < 5; i++)
> \
> + if (b_##TYPE[i] != a_##TYPE[i])
> \
> + __builtin_abort ();
> +
> +#define run_2(TYPE)
> \
> + for (unsigned int i = 0; i < N; i++)
> \
> + a_##TYPE[i] = i * 2 * 57 + 1 + 999;
> \
> + test_1_##TYPE (17);
> \
> + for (unsigned int i = 0; i < 17; i++)
> \
> + if (b_##TYPE[i] != a_##TYPE[i])
> \
> + __builtin_abort ();
> +
> +#define run_3(TYPE)
> \
> + for (unsigned int i = 0; i < N; i++)
> \
> + a_##TYPE[i] = i * 2 * 77 + 1 + 3;
> \
> + test_1_##TYPE (32);
> \
> + for (unsigned int i = 0; i < 32; i++)
> \
> + if (b_##TYPE[i] != a_##TYPE[i])
> \
> + __builtin_abort ();
> +
> +#define run_4(TYPE)
> \
> + for (unsigned int i = 0; i < N; i++)
> \
> + a_##TYPE[i] = i * 2 * 45 + 1 + 11;
> \
> + test_1_##TYPE (128);
> \
> + for (unsigned int i = 0; i < 128; i++)
> \
> + if (b_##TYPE[i] != a_##TYPE[i])
> \
> + __builtin_abort ();
> +
> +#define run_5(TYPE)
> \
> + for (unsigned int i = 0; i < N; i++)
> \
> + a_##TYPE[i] = i * 2 * 199 + 1 + 79;
> \
> + test_1_##TYPE (177);
> \
> + for (unsigned int i = 0; i < 177; i++)
> \
> + if (b_##TYPE[i] != a_##TYPE[i])
> \
> + __builtin_abort ();
> +
> +#define run_6(TYPE)
> \
> + for (unsigned int i = 0; i < N; i++)
> \
> + a_##TYPE[i] = i * 2 * 377 + 1 + 73;
> \
> + test_1_##TYPE (255);
> \
> + for (unsigned int i = 0; i < 255; i++)
> \
> + if (b_##TYPE[i] != a_##TYPE[i])
> \
> + __builtin_abort ();
> +
> +#define run_7(TYPE)
> \
> + for (unsigned int i = 0; i < N; i++)
> \
> + a_##TYPE[i] = i * 2 * 98 + 1 + 66;
> \
> + test_1_##TYPE (333);
> \
> + for (unsigned int i = 0; i < 333; i++)
> \
> + if (b_##TYPE[i] != a_##TYPE[i])
> \
> + __builtin_abort ();
> +
> +#define run_8(TYPE)
> \
> + for (unsigned int i = 0; i < N; i++)
> \
> + a_##TYPE[i] = i * 2 * 7 + 1 * 7;
> \
> + test_1_##TYPE (512);
> \
> + for (unsigned int i = 0; i < 512; i++)
> \
> + if (b_##TYPE[i] != a_##TYPE[i])
> \
> + __builtin_abort ();
> +
> +#define run_9(TYPE)
> \
> + for (unsigned int i = 0; i < N; i++)
> \
> + a_##TYPE[i] = i * 2 + 1 + 88;
> \
> + test_1_##TYPE (637);
> \
> + for (unsigned int i = 0; i < 637; i++)
> \
> + if (b_##TYPE[i] != a_##TYPE[i])
> \
> + __builtin_abort ();
> +
> +#define run_10(TYPE)
> \
> + for (unsigned int i = 0; i < N; i++)
> \
> + a_##TYPE[i] = i * 2 * 331 + 1 + 547;
> \
> + test_1_##TYPE (777);
> \
> + for (unsigned int i = 0; i < 777; i++)
> \
> + if (b_##TYPE[i] != a_##TYPE[i])
> \
> + __builtin_abort ();
> +
> +#define TEST_ALL(T)
> \
> + T (int8_t)
> \
> + T (uint8_t)
> \
> + T (int16_t)
> \
> + T (uint16_t)
> \
> + T (int32_t)
> \
> + T (uint32_t)
> \
> + T (int64_t)
> \
> + T (uint64_t)
> \
> + T (float)
> \
> + T (double)
> diff --git
> a/gcc/testsuite/gcc.target/riscv/rvv/autovec/partial/single_rgroup_run-1.c
> b/gcc/testsuite/gcc.target/riscv/rvv/autovec/partial/single_rgroup_run-1.c
> new file mode 100644
> index 00000000000..4af2f18de8a
> --- /dev/null
> +++
> b/gcc/testsuite/gcc.target/riscv/rvv/autovec/partial/single_rgroup_run-1.c
> @@ -0,0 +1,19 @@
> +/* { dg-do run { target { riscv_vector } } } */
> +/* { dg-additional-options "-fno-vect-cost-model
> -fno-tree-loop-distribute-patterns --param
> riscv-autovec-preference=scalable" } */
> +
> +#include "single_rgroup-1.c"
> +
> +int main (void)
> +{
> + TEST_ALL (run_1)
> + TEST_ALL (run_2)
> + TEST_ALL (run_3)
> + TEST_ALL (run_4)
> + TEST_ALL (run_5)
> + TEST_ALL (run_6)
> + TEST_ALL (run_7)
> + TEST_ALL (run_8)
> + TEST_ALL (run_9)
> + TEST_ALL (run_10)
> + return 0;
> +}
> diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/scalable-1.c
> b/gcc/testsuite/gcc.target/riscv/rvv/autovec/scalable-1.c
> new file mode 100644
> index 00000000000..500b0adce66
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/scalable-1.c
> @@ -0,0 +1,17 @@
> +/* { dg-do compile } */
> +/* { dg-options "-march=rv32gc_zve64d -mabi=ilp32 -O3
> -fno-vect-cost-model --param=riscv-autovec-preference=scalable
> -fdump-tree-vect-details" } */
> +
> +#include "riscv_vector.h"
> +
> +void
> +f (int32_t *__restrict f, int32_t *__restrict d, int n)
> +{
> + for (int i = 0; i < n; ++i)
> + {
> + f[i * 2 + 0] = 1;
> + f[i * 2 + 1] = 2;
> + d[i] = 3;
> + }
> +}
> +
> +/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 0
> "vect" } } */
> diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/template-1.h
> b/gcc/testsuite/gcc.target/riscv/rvv/autovec/template-1.h
> new file mode 100644
> index 00000000000..799e2d7d754
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/template-1.h
> @@ -0,0 +1,68 @@
> +#include <stddef.h>
> +#include <stdint.h>
> +
> +void
> +foo0 (int8_t *__restrict f, int16_t *__restrict d, int n)
> +{
> + for (int i = 0; i < n; ++i)
> + {
> + f[i * 2 + 0] = 1;
> + f[i * 2 + 1] = 2;
> + d[i] = 3;
> + }
> +}
> +
> +void
> +foo1 (int16_t *__restrict f, int32_t *__restrict d, int n)
> +{
> + for (int i = 0; i < n; ++i)
> + {
> + f[i * 2 + 0] = 1;
> + f[i * 2 + 1] = 2;
> + d[i] = 3;
> + }
> +}
> +
> +void
> +foo2 (int32_t *__restrict f, int64_t *__restrict d, int n)
> +{
> + for (int i = 0; i < n; ++i)
> + {
> + f[i * 2 + 0] = 1;
> + f[i * 2 + 1] = 2;
> + d[i] = 3;
> + }
> +}
> +
> +void
> +foo3 (int16_t *__restrict f, float *__restrict d, int n)
> +{
> + for (int i = 0; i < n; ++i)
> + {
> + f[i * 2 + 0] = 1;
> + f[i * 2 + 1] = 2;
> + d[i] = 3;
> + }
> +}
> +
> +void
> +foo4 (int32_t *__restrict f, float *__restrict d, int n)
> +{
> + for (int i = 0; i < n; ++i)
> + {
> + f[i * 2 + 0] = 1;
> + f[i * 2 + 1] = 2;
> + d[i] = 3;
> + }
> +}
> +
> +void
> +foo5 (float *__restrict f, double *__restrict d, int n)
> +{
> + for (int i = 0; i < n; ++i)
> + {
> + f[i * 2 + 0] = 1;
> + f[i * 2 + 1] = 2;
> + d[i] = 3;
> + }
> +}
> diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/v-1.c
> b/gcc/testsuite/gcc.target/riscv/rvv/autovec/v-1.c
> new file mode 100644
> index 00000000000..383c82a3b7c
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/v-1.c
> @@ -0,0 +1,11 @@
> +/* { dg-do compile } */
> +/* { dg-options "-march=rv32gcv -mabi=ilp32d --param
> riscv-autovec-preference=scalable -fdump-tree-vect-details" } */
> +
> +#include "template-1.h"
> +
> +/* Currently, we don't support SLP auto-vectorization for VLA. But it's
> + necessary that we add this testcase here to make sure such unsupported
> SLP
> + auto-vectorization will not cause an ICE. We will enable "vect"
> checking when
> + we support SLP auto-vectorization for VLA in the future. */
> +
> +/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 0
> "vect" } } */
> diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/v-2.c
> b/gcc/testsuite/gcc.target/riscv/rvv/autovec/v-2.c
> new file mode 100644
> index 00000000000..3d086e30081
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/v-2.c
> @@ -0,0 +1,6 @@
> +/* { dg-do compile } */
> +/* { dg-options "-march=rv32gcv -mabi=ilp32d --param
> riscv-autovec-preference=fixed-vlmax -fdump-tree-vect-details" } */
> +
> +#include "template-1.h"
> +
> +/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 5
> "vect" } } */
> diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32f-1.c
> b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32f-1.c
> new file mode 100644
> index 00000000000..ab57e89b1cd
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32f-1.c
> @@ -0,0 +1,6 @@
> +/* { dg-do compile } */
> +/* { dg-options "-march=rv32gc_zve32f -mabi=ilp32d --param
> riscv-autovec-preference=scalable -fdump-tree-vect-details" } */
> +
> +#include "template-1.h"
> +
> +/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 2
> "vect" } } */
> diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32f-2.c
> b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32f-2.c
> new file mode 100644
> index 00000000000..d6199665126
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32f-2.c
> @@ -0,0 +1,6 @@
> +/* { dg-do compile } */
> +/* { dg-options "-march=rv32gc_zve32f -mabi=ilp32d --param
> riscv-autovec-preference=fixed-vlmax -fdump-tree-vect-details" } */
> +
> +#include "template-1.h"
> +
> +/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 1
> "vect" } } */
> diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32f-3.c
> b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32f-3.c
> new file mode 100644
> index 00000000000..d5109c72045
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32f-3.c
> @@ -0,0 +1,6 @@
> +/* { dg-do compile } */
> +/* { dg-options "-march=rv32gc_zve32f -mabi=ilp32d --param
> riscv-autovec-preference=fixed-vlmax --param riscv-autovec-lmul=m2
> -fdump-tree-vect-details" } */
> +
> +#include "template-1.h"
> +
> +/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 4
> "vect" } } */
> diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32f_zvl128b-1.c
> b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32f_zvl128b-1.c
> new file mode 100644
> index 00000000000..23cc1c8651f
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32f_zvl128b-1.c
> @@ -0,0 +1,6 @@
> +/* { dg-do compile } */
> +/* { dg-options "-march=rv32gc_zve32f_zvl128b -mabi=ilp32d --param
> riscv-autovec-preference=scalable -fdump-tree-vect-details" } */
> +
> +#include "template-1.h"
> +
> +/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 0
> "vect" } } */
> diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32f_zvl128b-2.c
> b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32f_zvl128b-2.c
> new file mode 100644
> index 00000000000..c53975e9b01
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32f_zvl128b-2.c
> @@ -0,0 +1,6 @@
> +/* { dg-do compile } */
> +/* { dg-options "-march=rv32gc_zve32f_zvl128b -mabi=ilp32d --param
> riscv-autovec-preference=fixed-vlmax -fno-vect-cost-model
> -fdump-tree-vect-details" } */
> +
> +#include "template-1.h"
> +
> +/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 4
> "vect" } } */
> diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32x-1.c
> b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32x-1.c
> new file mode 100644
> index 00000000000..d23de3e4c3b
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32x-1.c
> @@ -0,0 +1,6 @@
> +/* { dg-do compile } */
> +/* { dg-options "-march=rv32gc_zve32x -mabi=ilp32d --param
> riscv-autovec-preference=scalable -fdump-tree-vect-details" } */
> +
> +#include "template-1.h"
> +
> +/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 2
> "vect" } } */
> diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32x-2.c
> b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32x-2.c
> new file mode 100644
> index 00000000000..575e9479f94
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32x-2.c
> @@ -0,0 +1,6 @@
> +/* { dg-do compile } */
> +/* { dg-options "-march=rv32gc_zve32x -mabi=ilp32d --param
> riscv-autovec-preference=fixed-vlmax -fdump-tree-vect-details" } */
> +
> +#include "template-1.h"
> +
> +/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 1
> "vect" } } */
> diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32x-3.c
> b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32x-3.c
> new file mode 100644
> index 00000000000..50e8963033b
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32x-3.c
> @@ -0,0 +1,6 @@
> +/* { dg-do compile } */
> +/* { dg-options "-march=rv32gc_zve32x -mabi=ilp32d --param
> riscv-autovec-preference=fixed-vlmax --param riscv-autovec-lmul=m2
> -fdump-tree-vect-details" } */
> +
> +#include "template-1.h"
> +
> +/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 3
> "vect" } } */
> diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32x_zvl128b-1.c
> b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32x_zvl128b-1.c
> new file mode 100644
> index 00000000000..4f130f02f67
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32x_zvl128b-1.c
> @@ -0,0 +1,6 @@
> +/* { dg-do compile } */
> +/* { dg-options "-march=rv32gc_zve32x_zvl128b -mabi=ilp32d --param
> riscv-autovec-preference=scalable -fdump-tree-vect-details" } */
> +
> +#include "template-1.h"
> +
> +/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 0
> "vect" } } */
> diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32x_zvl128b-2.c
> b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32x_zvl128b-2.c
> new file mode 100644
> index 00000000000..80a4796fd38
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve32x_zvl128b-2.c
> @@ -0,0 +1,6 @@
> +/* { dg-do compile } */
> +/* { dg-options "-march=rv32gc_zve32x_zvl128b -mabi=ilp32d --param
> riscv-autovec-preference=fixed-vlmax -fdump-tree-vect-details" } */
> +
> +#include "template-1.h"
> +
> +/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 2
> "vect" } } */
> diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64d-1.c
> b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64d-1.c
> new file mode 100644
> index 00000000000..823d51a03cb
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64d-1.c
> @@ -0,0 +1,6 @@
> +/* { dg-do compile } */
> +/* { dg-options "-march=rv32gc_zve64d -mabi=ilp32d --param
> riscv-autovec-preference=scalable -fdump-tree-vect-details" } */
> +
> +#include "template-1.h"
> +
> +/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 2
> "vect" } } */
> diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64d-2.c
> b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64d-2.c
> new file mode 100644
> index 00000000000..0d6ebc53d1a
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64d-2.c
> @@ -0,0 +1,6 @@
> +/* { dg-do compile } */
> +/* { dg-options "-march=rv32gc_zve64d -mabi=ilp32d --param
> riscv-autovec-preference=fixed-vlmax -fdump-tree-vect-details" } */
> +
> +#include "template-1.h"
> +
> +/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 1
> "vect" } } */
> diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64d-3.c
> b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64d-3.c
> new file mode 100644
> index 00000000000..2a72030c3bc
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64d-3.c
> @@ -0,0 +1,6 @@
> +/* { dg-do compile } */
> +/* { dg-options "-march=rv32gc_zve64d -mabi=ilp32d --param
> riscv-autovec-preference=fixed-vlmax --param riscv-autovec-lmul=m2
> -fdump-tree-vect-details" } */
> +
> +#include "template-1.h"
> +
> +/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 5
> "vect" } } */
> diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64d_zvl128b-1.c
> b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64d_zvl128b-1.c
> new file mode 100644
> index 00000000000..5ead22746d3
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64d_zvl128b-1.c
> @@ -0,0 +1,6 @@
> +/* { dg-do compile } */
> +/* { dg-options "-march=rv32gc_zve64d_zvl128b -mabi=ilp32d --param
> riscv-autovec-preference=scalable -fdump-tree-vect-details" } */
> +
> +#include "template-1.h"
> +
> +/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 0
> "vect" } } */
> diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64d_zvl128b-2.c
> b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64d_zvl128b-2.c
> new file mode 100644
> index 00000000000..9e236e0af1b
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64d_zvl128b-2.c
> @@ -0,0 +1,6 @@
> +/* { dg-do compile } */
> +/* { dg-options "-march=rv32gc_zve64d_zvl128b -mabi=ilp32d --param
> riscv-autovec-preference=fixed-vlmax -fdump-tree-vect-details" } */
> +
> +#include "template-1.h"
> +
> +/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 5
> "vect" } } */
> diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64f-1.c
> b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64f-1.c
> new file mode 100644
> index 00000000000..e03d1b44ca6
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64f-1.c
> @@ -0,0 +1,6 @@
> +/* { dg-do compile } */
> +/* { dg-options "-march=rv32gc_zve64f -mabi=ilp32d --param
> riscv-autovec-preference=scalable -fdump-tree-vect-details" } */
> +
> +#include "template-1.h"
> +
> +/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 2
> "vect" } } */
> diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64f-2.c
> b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64f-2.c
> new file mode 100644
> index 00000000000..6bb6b919c77
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64f-2.c
> @@ -0,0 +1,6 @@
> +/* { dg-do compile } */
> +/* { dg-options "-march=rv32gc_zve64f -mabi=ilp32d --param
> riscv-autovec-preference=fixed-vlmax -fdump-tree-vect-details" } */
> +
> +#include "template-1.h"
> +
> +/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 1
> "vect" } } */
> diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64f-3.c
> b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64f-3.c
> new file mode 100644
> index 00000000000..43eca9d0727
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64f-3.c
> @@ -0,0 +1,6 @@
> +/* { dg-do compile } */
> +/* { dg-options "-march=rv32gc_zve64f -mabi=ilp32d --param
> riscv-autovec-preference=fixed-vlmax --param riscv-autovec-lmul=m2
> -fdump-tree-vect-details" } */
> +
> +#include "template-1.h"
> +
> +/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 4
> "vect" } } */
> diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64f_zvl128b-1.c
> b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64f_zvl128b-1.c
> new file mode 100644
> index 00000000000..5bb2d9d96fa
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64f_zvl128b-1.c
> @@ -0,0 +1,6 @@
> +/* { dg-do compile } */
> +/* { dg-options "-march=rv32gc_zve64f_zvl128b -mabi=ilp32d --param
> riscv-autovec-preference=scalable -fdump-tree-vect-details" } */
> +
> +#include "template-1.h"
> +
> +/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 0
> "vect" } } */
> diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64f_zvl128b-2.c
> b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64f_zvl128b-2.c
> new file mode 100644
> index 00000000000..f1500074370
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64f_zvl128b-2.c
> @@ -0,0 +1,6 @@
> +/* { dg-do compile } */
> +/* { dg-options "-march=rv32gc_zve64f_zvl128b -mabi=ilp32d --param
> riscv-autovec-preference=fixed-vlmax -fdump-tree-vect-details" } */
> +
> +#include "template-1.h"
> +
> +/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 4
> "vect" } } */
> diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64x-1.c
> b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64x-1.c
> new file mode 100644
> index 00000000000..8fbfa8a8b68
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64x-1.c
> @@ -0,0 +1,6 @@
> +/* { dg-do compile } */
> +/* { dg-options "-march=rv32gc_zve64x -mabi=ilp32d --param
> riscv-autovec-preference=scalable -fdump-tree-vect-details" } */
> +
> +#include "template-1.h"
> +
> +/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 2
> "vect" } } */
> diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64x-2.c
> b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64x-2.c
> new file mode 100644
> index 00000000000..ede28e88dac
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64x-2.c
> @@ -0,0 +1,6 @@
> +/* { dg-do compile } */
> +/* { dg-options "-march=rv32gc_zve64x -mabi=ilp32d --param
> riscv-autovec-preference=fixed-vlmax -fdump-tree-vect-details" } */
> +
> +#include "template-1.h"
> +
> +/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 1
> "vect" } } */
> diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64x-3.c
> b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64x-3.c
> new file mode 100644
> index 00000000000..d59f7362b43
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64x-3.c
> @@ -0,0 +1,6 @@
> +/* { dg-do compile } */
> +/* { dg-options "-march=rv32gc_zve64x -mabi=ilp32d --param
> riscv-autovec-preference=fixed-vlmax --param riscv-autovec-lmul=m2
> -fdump-tree-vect-details" } */
> +
> +#include "template-1.h"
> +
> +/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 3
> "vect" } } */
> diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64x_zvl128b-1.c
> b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64x_zvl128b-1.c
> new file mode 100644
> index 00000000000..71820ece4b2
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64x_zvl128b-1.c
> @@ -0,0 +1,6 @@
> +/* { dg-do compile } */
> +/* { dg-options "-march=rv32gc_zve64x_zvl128b -mabi=ilp32d --param
> riscv-autovec-preference=scalable -fdump-tree-vect-details" } */
> +
> +#include "template-1.h"
> +
> +/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 0
> "vect" } } */
> diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64x_zvl128b-2.c
> b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64x_zvl128b-2.c
> new file mode 100644
> index 00000000000..90398db52a0
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/zve64x_zvl128b-2.c
> @@ -0,0 +1,6 @@
> +/* { dg-do compile } */
> +/* { dg-options "-march=rv32gc_zve64x_zvl128b -mabi=ilp32d --param
> riscv-autovec-preference=fixed-vlmax -fdump-tree-vect-details" } */
> +
> +#include "template-1.h"
> +
> +/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 3
> "vect" } } */
> diff --git a/gcc/testsuite/gcc.target/riscv/rvv/rvv.exp
> b/gcc/testsuite/gcc.target/riscv/rvv/rvv.exp
> index 4b5509db385..0eef068058a 100644
> --- a/gcc/testsuite/gcc.target/riscv/rvv/rvv.exp
> +++ b/gcc/testsuite/gcc.target/riscv/rvv/rvv.exp
> @@ -46,6 +46,22 @@ dg-runtest [lsort [glob -nocomplain
> $srcdir/$subdir/base/*.\[cS\]]] \
> "" $CFLAGS
> gcc-dg-runtest [lsort [glob -nocomplain $srcdir/$subdir/vsetvl/*.\[cS\]]]
> \
> "" $CFLAGS
> +dg-runtest [lsort [glob -nocomplain $srcdir/$subdir/autovec/*.\[cS\]]] \
> + "-O3 -ftree-vectorize" $CFLAGS
> +
> +set AUTOVEC_TEST_OPTS [list \
> + {-ftree-vectorize -O3 --param riscv-autovec-lmul=m1} \
> + {-ftree-vectorize -O3 --param riscv-autovec-lmul=m2} \
> + {-ftree-vectorize -O3 --param riscv-autovec-lmul=m4} \
> + {-ftree-vectorize -O3 --param riscv-autovec-lmul=m8} \
> + {-ftree-vectorize -O2 --param riscv-autovec-lmul=m1} \
> + {-ftree-vectorize -O2 --param riscv-autovec-lmul=m2} \
> + {-ftree-vectorize -O2 --param riscv-autovec-lmul=m4} \
> + {-ftree-vectorize -O2 --param riscv-autovec-lmul=m8} ]
> +foreach op $AUTOVEC_TEST_OPTS {
> + dg-runtest [lsort [glob -nocomplain
> $srcdir/$subdir/autovec/partial/*.\[cS\]]] \
> + "" "$op"
> +}
>
> # All done.
> dg-finish
> --
> 2.36.3
>
>
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [PATCH V7] RISC-V: Enable basic RVV auto-vectorization support.
2023-05-06 3:43 [PATCH V7] RISC-V: Enable basic RVV auto-vectorization support juzhe.zhong
2023-05-06 13:10 ` Kito Cheng
@ 2023-05-06 15:58 ` Jeff Law
1 sibling, 0 replies; 3+ messages in thread
From: Jeff Law @ 2023-05-06 15:58 UTC (permalink / raw)
To: juzhe.zhong, gcc-patches; +Cc: kito.cheng, palmer, rdapp.gcc
On 5/5/23 21:43, juzhe.zhong@rivai.ai wrote:
> From: Juzhe-Zhong <juzhe.zhong@rivai.ai>
>
> gcc/ChangeLog:
>
> * config/riscv/riscv-protos.h (preferred_simd_mode): New function.
> * config/riscv/riscv-v.cc (autovec_use_vlmax_p): Ditto.
> (preferred_simd_mode): Ditto.
> * config/riscv/riscv.cc (riscv_get_arg_info): Handle RVV type in function arg.
> (riscv_convert_vector_bits): Adjust for RVV auto-vectorization.
> (riscv_preferred_simd_mode): New function.
> (TARGET_VECTORIZE_PREFERRED_SIMD_MODE): New target hook support.
> * config/riscv/vector.md: Add autovec.md.
> * config/riscv/autovec.md: New file.
>
> gcc/testsuite/ChangeLog:
>
> * gcc.target/riscv/rvv/rvv.exp: Add testcases for RVV auto-vectorization.
> * gcc.target/riscv/rvv/autovec/fixed-vlmax-1.c: New test.
> * gcc.target/riscv/rvv/autovec/partial/single_rgroup-1.c: New test.
> * gcc.target/riscv/rvv/autovec/partial/single_rgroup-1.h: New test.
> * gcc.target/riscv/rvv/autovec/partial/single_rgroup_run-1.c: New test.
> * gcc.target/riscv/rvv/autovec/scalable-1.c: New test.
> * gcc.target/riscv/rvv/autovec/template-1.h: New test.
> * gcc.target/riscv/rvv/autovec/v-1.c: New test.
> * gcc.target/riscv/rvv/autovec/v-2.c: New test.
> * gcc.target/riscv/rvv/autovec/zve32f-1.c: New test.
> * gcc.target/riscv/rvv/autovec/zve32f-2.c: New test.
> * gcc.target/riscv/rvv/autovec/zve32f-3.c: New test.
> * gcc.target/riscv/rvv/autovec/zve32f_zvl128b-1.c: New test.
> * gcc.target/riscv/rvv/autovec/zve32f_zvl128b-2.c: New test.
> * gcc.target/riscv/rvv/autovec/zve32x-1.c: New test.
> * gcc.target/riscv/rvv/autovec/zve32x-2.c: New test.
> * gcc.target/riscv/rvv/autovec/zve32x-3.c: New test.
> * gcc.target/riscv/rvv/autovec/zve32x_zvl128b-1.c: New test.
> * gcc.target/riscv/rvv/autovec/zve32x_zvl128b-2.c: New test.
> * gcc.target/riscv/rvv/autovec/zve64d-1.c: New test.
> * gcc.target/riscv/rvv/autovec/zve64d-2.c: New test.
> * gcc.target/riscv/rvv/autovec/zve64d-3.c: New test.
> * gcc.target/riscv/rvv/autovec/zve64d_zvl128b-1.c: New test.
> * gcc.target/riscv/rvv/autovec/zve64d_zvl128b-2.c: New test.
> * gcc.target/riscv/rvv/autovec/zve64f-1.c: New test.
> * gcc.target/riscv/rvv/autovec/zve64f-2.c: New test.
> * gcc.target/riscv/rvv/autovec/zve64f-3.c: New test.
> * gcc.target/riscv/rvv/autovec/zve64f_zvl128b-1.c: New test.
> * gcc.target/riscv/rvv/autovec/zve64f_zvl128b-2.c: New test.
> * gcc.target/riscv/rvv/autovec/zve64x-1.c: New test.
> * gcc.target/riscv/rvv/autovec/zve64x-2.c: New test.
> * gcc.target/riscv/rvv/autovec/zve64x-3.c: New test.
> * gcc.target/riscv/rvv/autovec/zve64x_zvl128b-1.c: New test.
> * gcc.target/riscv/rvv/autovec/zve64x_zvl128b-2.c: New test.
I went ahead and committed this. Hopefully we can get things unblocked
for everyone this weekend :-)
jeff
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2023-05-06 15:58 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-05-06 3:43 [PATCH V7] RISC-V: Enable basic RVV auto-vectorization support juzhe.zhong
2023-05-06 13:10 ` Kito Cheng
2023-05-06 15:58 ` Jeff Law
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).