* Re：Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
[not found] <84F7577214BA46AD+202401101900269053910@rivai.ai>
@ 2024-01-10 11:06 ` joshua
2024-01-10 11:08 ` joshua
2024-01-10 11:09 ` Re：[PATCH " juzhe.zhong
0 siblings, 2 replies; 17+ messages in thread
From: joshua @ 2024-01-10 11:06 UTC (permalink / raw)
To: juzhe.zhong, gcc-patches
Cc: Jim Wilson, palmer, andrew, philipp.tomsich, jeffreyalaw,
christoph.muellner, jinma, cooper.qu
The key difference between vlb/vlh/vlw is not output type too.
Their difference is the range of datatype, not one specific type.
We have dived into the xtheadvector special intrinsics and are
sure about that.
------------------------------------------------------------------
发件人：juzhe.zhong@rivai.ai <juzhe.zhong@rivai.ai>
发送时间：2024年1月10日(星期三) 19:00
收件人："cooper.joshua"<cooper.joshua@linux.alibaba.com>; "gcc-patches"<gcc-patches@gcc.gnu.org>
抄 送：Jim Wilson<jim.wilson.gcc@gmail.com>; palmer<palmer@dabbelt.com>; andrew<andrew@sifive.com>; "philipp.tomsich"<philipp.tomsich@vrull.eu>; jeffreyalaw<jeffreyalaw@gmail.com>; "christoph.muellner"<christoph.muellner@vrull.eu>; jinma<jinma@linux.alibaba.com>; "cooper.qu"<cooper.qu@linux.alibaba.com>
主 题：Re: Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
instance.op_info->args[i].get_tree_type (instance.type.index) is output type.
You can use GDB debug it .
juzhe.zhong@rivai.ai
发件人： joshua
发送时间： 2024-01-10 18:57
收件人： juzhe.zhong@rivai.ai; gcc-patches
抄送： Jim Wilson; palmer; andrew; philipp.tomsich; jeffreyalaw; christoph.muellner; jinma; cooper.qu
主题： Re：Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
Hi Juzhe,
Perhaps things are not as simple as imagined.
The differences between vlb/vlh/vlw is not the same
as vle8/vle16/vle32. "8", "16" or "32" in vle8/vle16/vle32
can be appended from "vle" according to input type.
But vlb/vlh/vlw is different not in input type.
------------------------------------------------------------------
发件人：juzhe.zhong@rivai.ai <juzhe.zhong@rivai.ai>
发送时间：2024年1月10日(星期三) 18:03
收件人："cooper.joshua"<cooper.joshua@linux.alibaba.com>; "gcc-patches"<gcc-patches@gcc.gnu.org>
抄 送：Jim Wilson<jim.wilson.gcc@gmail.com>; palmer<palmer@dabbelt.com>; andrew<andrew@sifive.com>; "philipp.tomsich"<philipp.tomsich@vrull.eu>; jeffreyalaw<jeffreyalaw@gmail.com>; "christoph.muellner"<christoph.muellner@vrull.eu>; jinma<jinma@linux.alibaba.com>; "cooper.qu"<cooper.qu@linux.alibaba.com>
主 题：Re: Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
I mean change these:
+DEF_RVV_FUNCTION (th_vlb, th_loadstore_width, full_preds, i8_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (th_vlh, th_loadstore_width, full_preds, i16_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (th_vlw, th_loadstore_width, full_preds, i32_v_scalar_const_ptr_ops)
into a single:
+DEF_RVV_FUNCTION (th_vl, th_loadstore_width, full_preds, all_v_scalar_const_ptr_ops)
and append "h", "w", or"b" according to
TYPE_UNSIGNED and
GET_MODE_BITSIZE (GET_MODE_INNER (TYPE_MODE (instance.op_info->args[i].get_tree_type (instance.type.index))))
in th_loadstore_width.
It should definitely works, I allow this flexibility in design of the framework.
juzhe.zhong@rivai.ai
发件人： joshua
发送时间： 2024-01-10 17:55
收件人： juzhe.zhong@rivai.ai; gcc-patches
抄送： Jim Wilson; palmer; andrew; philipp.tomsich; jeffreyalaw; christoph.muellner; jinma; cooper.qu
主题： Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
And revise th_loadstore_width, append the name according TYPE_UNSIGNED and
GET_MODE_BITSIZE (GET_MODE_INNER (TYPE_MODE (instance.op_info->args[i].get_tree_type (instance.type.index))))
What do you mean by it? I'm a bit confused.
Changing i8_v_scalar_const_ptr_ops into all_v_scalar_const_ptr_ops
will expand the datatypes that can be used in th_vlb. Can we restrict
again in th_loadstore_width?
------------------------------------------------------------------
发件人：juzhe.zhong@rivai.ai <juzhe.zhong@rivai.ai>
发送时间：2024年1月10日(星期三) 17:35
收件人："cooper.joshua"<cooper.joshua@linux.alibaba.com>; "gcc-patches"<gcc-patches@gcc.gnu.org>
抄 送：Jim Wilson<jim.wilson.gcc@gmail.com>; palmer<palmer@dabbelt.com>; andrew<andrew@sifive.com>; "philipp.tomsich"<philipp.tomsich@vrull.eu>; jeffreyalaw<jeffreyalaw@gmail.com>; "christoph.muellner"<christoph.muellner@vrull.eu>; "cooper.joshua"<cooper.joshua@linux.alibaba.com>; jinma<jinma@linux.alibaba.com>; "cooper.qu"<cooper.qu@linux.alibaba.com>
主 题：Re: [PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
+DEF_RVV_FUNCTION (th_vlb, th_loadstore_width, full_preds, i8_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (th_vlh, th_loadstore_width, full_preds, i16_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (th_vlw, th_loadstore_width, full_preds, i32_v_scalar_const_ptr_ops)
I think we should remove those many data structure you added like: i8_v_scalar_const_ptr_ops
Instead, you should use all_v_scalar_const_ptr_ops
And revise th_loadstore_width, append the name according TYPE_UNSIGNED and
GET_MODE_BITSIZE (GET_MODE_INNER (TYPE_MODE (instance.op_info->args[i].get_tree_type (instance.type.index))))
juzhe.zhong@rivai.ai
From: Jun Sha (Joshua)
Date: 2024-01-10 17:27
To: gcc-patches
CC: jim.wilson.gcc; palmer; andrew; philipp.tomsich; jeffreyalaw; christoph.muellner; juzhe.zhong; Jun Sha (Joshua); Jin Ma; Xianmiao Qu
Subject: [PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
This patch only involves the generation of xtheadvector
special load/store instructions and vext instructions.
gcc/ChangeLog:
* config/riscv/riscv-vector-builtins-bases.cc
(class th_loadstore_width): Define new builtin bases.
(BASE): Define new builtin bases.
* config/riscv/riscv-vector-builtins-bases.h:
Define new builtin class.
* config/riscv/riscv-vector-builtins-functions.def (vlsegff):
Include thead-vector-builtins-functions.def.
* config/riscv/riscv-vector-builtins-shapes.cc
(struct th_loadstore_width_def): Define new builtin shapes.
(struct th_indexed_loadstore_width_def):
Define new builtin shapes.
(SHAPE): Define new builtin shapes.
* config/riscv/riscv-vector-builtins-shapes.h:
Define new builtin shapes.
* config/riscv/riscv-vector-builtins-types.def
(DEF_RVV_I8_OPS): Add datatypes for XTheadVector.
(DEF_RVV_I16_OPS): Add datatypes for XTheadVector.
(DEF_RVV_I32_OPS): Add datatypes for XTheadVector.
(DEF_RVV_U8_OPS): Add datatypes for XTheadVector.
(DEF_RVV_U16_OPS): Add datatypes for XTheadVector.
(DEF_RVV_U32_OPS): Add datatypes for XTheadVector.
(vint8m1_t): Add datatypes for XTheadVector.
(vint8m2_t): Likewise.
(vint8m4_t): Likewise.
(vint8m8_t): Likewise.
(vint16m1_t): Likewise.
(vint16m2_t): Likewise.
(vint16m4_t): Likewise.
(vint16m8_t): Likewise.
(vint32m1_t): Likewise.
(vint32m2_t): Likewise.
(vint32m4_t): Likewise.
(vint32m8_t): Likewise.
(vint64m1_t): Likewise.
(vint64m2_t): Likewise.
(vint64m4_t): Likewise.
(vint64m8_t): Likewise.
(vuint8m1_t): Likewise.
(vuint8m2_t): Likewise.
(vuint8m4_t): Likewise.
(vuint8m8_t): Likewise.
(vuint16m1_t): Likewise.
(vuint16m2_t): Likewise.
(vuint16m4_t): Likewise.
(vuint16m8_t): Likewise.
(vuint32m1_t): Likewise.
(vuint32m2_t): Likewise.
(vuint32m4_t): Likewise.
(vuint32m8_t): Likewise.
(vuint64m1_t): Likewise.
(vuint64m2_t): Likewise.
(vuint64m4_t): Likewise.
(vuint64m8_t): Likewise.
* config/riscv/riscv-vector-builtins.cc
(DEF_RVV_I8_OPS): Add datatypes for XTheadVector.
(DEF_RVV_I16_OPS): Add datatypes for XTheadVector.
(DEF_RVV_I32_OPS): Add datatypes for XTheadVector.
(DEF_RVV_U8_OPS): Add datatypes for XTheadVector.
(DEF_RVV_U16_OPS): Add datatypes for XTheadVector.
(DEF_RVV_U32_OPS): Add datatypes for XTheadVector.
* config/riscv/thead-vector-builtins-functions.def: New file.
* config/riscv/thead-vector.md: Add new patterns.
gcc/testsuite/ChangeLog:
* gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c: New test.
Co-authored-by: Jin Ma <jinma@linux.alibaba.com>
Co-authored-by: Xianmiao Qu <cooper.qu@linux.alibaba.com>
Co-authored-by: Christoph Müllner <christoph.muellner@vrull.eu>
---
.../riscv/riscv-vector-builtins-bases.cc | 139 ++++++++
.../riscv/riscv-vector-builtins-bases.h | 31 ++
.../riscv/riscv-vector-builtins-shapes.cc | 98 ++++++
.../riscv/riscv-vector-builtins-shapes.h | 3 +
.../riscv/riscv-vector-builtins-types.def | 120 +++++++
gcc/config/riscv/riscv-vector-builtins.cc | 311 ++++++++++++++++++
gcc/config/riscv/riscv-vector-builtins.h | 3 +
gcc/config/riscv/t-riscv | 1 +
.../riscv/thead-vector-builtins-functions.def | 39 +++
gcc/config/riscv/thead-vector.md | 253 ++++++++++++++
.../riscv/rvv/xtheadvector/vlb-vsb.c | 68 ++++
.../riscv/rvv/xtheadvector/vlbu-vsb.c | 68 ++++
.../riscv/rvv/xtheadvector/vlh-vsh.c | 68 ++++
.../riscv/rvv/xtheadvector/vlhu-vsh.c | 68 ++++
.../riscv/rvv/xtheadvector/vlw-vsw.c | 68 ++++
.../riscv/rvv/xtheadvector/vlwu-vsw.c | 68 ++++
16 files changed, 1406 insertions(+)
create mode 100644 gcc/config/riscv/thead-vector-builtins-functions.def
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c
diff --git a/gcc/config/riscv/riscv-vector-builtins-bases.cc b/gcc/config/riscv/riscv-vector-builtins-bases.cc
index 46f1a1da33e..5f44f31a12b 100644
--- a/gcc/config/riscv/riscv-vector-builtins-bases.cc
+++ b/gcc/config/riscv/riscv-vector-builtins-bases.cc
@@ -2125,6 +2125,83 @@ public:
}
};
+/* Implements
+ * th.vl(b/h/w)[u].v/th.vs(b/h/w)[u].v/th.vls(b/h/w)[u].v/th.vss(b/h/w)[u].v/
+ * th.vlx(b/h/w)[u].v/th.vs[u]x(b/h/w).v
+ * codegen. */
+template<bool STORE_P, lst_type LST_TYPE, int UNSPEC>
+class th_loadstore_width : public function_base
+{
+public:
+ bool apply_tail_policy_p () const override { return !STORE_P; }
+ bool apply_mask_policy_p () const override { return !STORE_P; }
+
+ unsigned int call_properties (const function_instance &) const override
+ {
+ if (STORE_P)
+ return CP_WRITE_MEMORY;
+ else
+ return CP_READ_MEMORY;
+ }
+
+ bool can_be_overloaded_p (enum predication_type_index pred) const override
+ {
+ if (STORE_P || LST_TYPE == LST_INDEXED)
+ return true;
+ return pred != PRED_TYPE_none;
+ }
+
+ rtx expand (function_expander &e) const override
+ {
+ gcc_assert (TARGET_XTHEADVECTOR);
+ if (LST_TYPE == LST_INDEXED)
+ {
+ if (STORE_P)
+ return e.use_exact_insn (
+ code_for_pred_indexed_store_width (UNSPEC, UNSPEC,
+ e.vector_mode ()));
+ else
+ return e.use_exact_insn (
+ code_for_pred_indexed_load_width (UNSPEC, e.vector_mode ()));
+ }
+ else if (LST_TYPE == LST_STRIDED)
+ {
+ if (STORE_P)
+ return e.use_contiguous_store_insn (
+ code_for_pred_strided_store_width (UNSPEC, e.vector_mode ()));
+ else
+ return e.use_contiguous_load_insn (
+ code_for_pred_strided_load_width (UNSPEC, e.vector_mode ()));
+ }
+ else
+ {
+ if (STORE_P)
+ return e.use_contiguous_store_insn (
+ code_for_pred_store_width (UNSPEC, e.vector_mode ()));
+ else
+ return e.use_contiguous_load_insn (
+ code_for_pred_mov_width (UNSPEC, e.vector_mode ()));
+ }
+ }
+};
+
+/* Implements vext.x.v. */
+class th_extract : public function_base
+{
+public:
+ bool apply_vl_p () const override { return false; }
+ bool apply_tail_policy_p () const override { return false; }
+ bool apply_mask_policy_p () const override { return false; }
+ bool use_mask_predication_p () const override { return false; }
+ bool has_merge_operand_p () const override { return false; }
+
+ rtx expand (function_expander &e) const override
+ {
+ gcc_assert (TARGET_XTHEADVECTOR);
+ return e.use_exact_insn (code_for_pred_th_extract (e.vector_mode ()));
+ }
+};
+
/* Below implements are vector crypto */
/* Implements vandn.[vv,vx] */
class vandn : public function_base
@@ -2587,6 +2664,37 @@ static CONSTEXPR const seg_indexed_load<UNSPEC_ORDERED> vloxseg_obj;
static CONSTEXPR const seg_indexed_store<UNSPEC_UNORDERED> vsuxseg_obj;
static CONSTEXPR const seg_indexed_store<UNSPEC_ORDERED> vsoxseg_obj;
static CONSTEXPR const vlsegff vlsegff_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLB> th_vlb_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLBU> th_vlbu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLH> th_vlh_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLHU> th_vlhu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLW> th_vlw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLWU> th_vlwu_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_UNIT_STRIDE, UNSPEC_TH_VLB> th_vsb_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_UNIT_STRIDE, UNSPEC_TH_VLH> th_vsh_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_UNIT_STRIDE, UNSPEC_TH_VLW> th_vsw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSB> th_vlsb_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSBU> th_vlsbu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSH> th_vlsh_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSHU> th_vlshu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSW> th_vlsw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSWU> th_vlswu_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_STRIDED, UNSPEC_TH_VLSB> th_vssb_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_STRIDED, UNSPEC_TH_VLSH> th_vssh_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_STRIDED, UNSPEC_TH_VLSW> th_vssw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXB> th_vlxb_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXBU> th_vlxbu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXH> th_vlxh_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXHU> th_vlxhu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXW> th_vlxw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXWU> th_vlxwu_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VLXB> th_vsxb_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VLXH> th_vsxh_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VLXW> th_vsxw_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VSUXB> th_vsuxb_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VSUXH> th_vsuxh_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VSUXW> th_vsuxw_obj;
+static CONSTEXPR const th_extract th_vext_x_v_obj;
/* Crypto Vector */
static CONSTEXPR const vandn vandn_obj;
@@ -2878,6 +2986,37 @@ BASE (vloxseg)
BASE (vsuxseg)
BASE (vsoxseg)
BASE (vlsegff)
+BASE (th_vlb)
+BASE (th_vlh)
+BASE (th_vlw)
+BASE (th_vlbu)
+BASE (th_vlhu)
+BASE (th_vlwu)
+BASE (th_vsb)
+BASE (th_vsh)
+BASE (th_vsw)
+BASE (th_vlsb)
+BASE (th_vlsh)
+BASE (th_vlsw)
+BASE (th_vlsbu)
+BASE (th_vlshu)
+BASE (th_vlswu)
+BASE (th_vssb)
+BASE (th_vssh)
+BASE (th_vssw)
+BASE (th_vlxb)
+BASE (th_vlxh)
+BASE (th_vlxw)
+BASE (th_vlxbu)
+BASE (th_vlxhu)
+BASE (th_vlxwu)
+BASE (th_vsxb)
+BASE (th_vsxh)
+BASE (th_vsxw)
+BASE (th_vsuxb)
+BASE (th_vsuxh)
+BASE (th_vsuxw)
+BASE (th_vext_x_v)
/* Crypto vector */
BASE (vandn)
BASE (vbrev)
diff --git a/gcc/config/riscv/riscv-vector-builtins-bases.h b/gcc/config/riscv/riscv-vector-builtins-bases.h
index 1122e3801a7..df43adf9a17 100644
--- a/gcc/config/riscv/riscv-vector-builtins-bases.h
+++ b/gcc/config/riscv/riscv-vector-builtins-bases.h
@@ -299,6 +299,37 @@ extern const function_base *const vloxseg;
extern const function_base *const vsuxseg;
extern const function_base *const vsoxseg;
extern const function_base *const vlsegff;
+extern const function_base *const th_vlb;
+extern const function_base *const th_vlh;
+extern const function_base *const th_vlw;
+extern const function_base *const th_vlbu;
+extern const function_base *const th_vlhu;
+extern const function_base *const th_vlwu;
+extern const function_base *const th_vsb;
+extern const function_base *const th_vsh;
+extern const function_base *const th_vsw;
+extern const function_base *const th_vlsb;
+extern const function_base *const th_vlsh;
+extern const function_base *const th_vlsw;
+extern const function_base *const th_vlsbu;
+extern const function_base *const th_vlshu;
+extern const function_base *const th_vlswu;
+extern const function_base *const th_vssb;
+extern const function_base *const th_vssh;
+extern const function_base *const th_vssw;
+extern const function_base *const th_vlxb;
+extern const function_base *const th_vlxh;
+extern const function_base *const th_vlxw;
+extern const function_base *const th_vlxbu;
+extern const function_base *const th_vlxhu;
+extern const function_base *const th_vlxwu;
+extern const function_base *const th_vsxb;
+extern const function_base *const th_vsxh;
+extern const function_base *const th_vsxw;
+extern const function_base *const th_vsuxb;
+extern const function_base *const th_vsuxh;
+extern const function_base *const th_vsuxw;
+extern const function_base *const th_vext_x_v;
/* Below function_base are Vectro Crypto*/
extern const function_base *const vandn;
extern const function_base *const vbrev;
diff --git a/gcc/config/riscv/riscv-vector-builtins-shapes.cc b/gcc/config/riscv/riscv-vector-builtins-shapes.cc
index 1e4f4d53de6..489a95cf684 100644
--- a/gcc/config/riscv/riscv-vector-builtins-shapes.cc
+++ b/gcc/config/riscv/riscv-vector-builtins-shapes.cc
@@ -211,6 +211,86 @@ struct indexed_loadstore_def : public function_shape
}
};
+/* th_loadstore_width_def class. */
+struct th_loadstore_width_def : public build_base
+{
+ char *get_name (function_builder &b, const function_instance &instance,
+ bool overloaded_p) const override
+ {
+ /* Return nullptr if it can not be overloaded. */
+ if (overloaded_p && !instance.base->can_be_overloaded_p (instance.pred))
+ return nullptr;
+
+ b.append_base_name (instance.base_name);
+
+ /* vop_v --> vop_v_<type>. */
+ if (!overloaded_p)
+ {
+ /* vop --> vop_v. */
+ b.append_name (operand_suffixes[instance.op_info->op]);
+ /* vop_v --> vop_v_<type>. */
+ b.append_name (type_suffixes[instance.type.index].vector);
+ }
+
+ /* According to rvv-intrinsic-doc, it does not add "_m" suffix
+ for vop_m C++ overloaded API. */
+ if (overloaded_p && instance.pred == PRED_TYPE_m)
+ return b.finish_name ();
+ b.append_name (predication_suffixes[instance.pred]);
+ return b.finish_name ();
+ }
+};
+
+
+/* th_indexed_loadstore_width_def class. */
+struct th_indexed_loadstore_width_def : public function_shape
+{
+ void build (function_builder &b,
+ const function_group_info &group) const override
+ {
+ for (unsigned int pred_idx = 0; group.preds[pred_idx] != NUM_PRED_TYPES;
+ ++pred_idx)
+ {
+ for (unsigned int vec_type_idx = 0;
+ group.ops_infos.types[vec_type_idx].index != NUM_VECTOR_TYPES;
+ ++vec_type_idx)
+ {
+ tree index_type = group.ops_infos.args[1].get_tree_type (
+ group.ops_infos.types[vec_type_idx].index);
+ if (!index_type)
+ continue;
+ build_one (b, group, pred_idx, vec_type_idx);
+ }
+ }
+ }
+
+ char *get_name (function_builder &b, const function_instance &instance,
+ bool overloaded_p) const override
+ {
+
+ /* Return nullptr if it can not be overloaded. */
+ if (overloaded_p && !instance.base->can_be_overloaded_p (instance.pred))
+ return nullptr;
+
+ b.append_base_name (instance.base_name);
+ /* vop_v --> vop_v_<type>. */
+ if (!overloaded_p)
+ {
+ /* vop --> vop_v. */
+ b.append_name (operand_suffixes[instance.op_info->op]);
+ /* vop_v --> vop_v_<type>. */
+ b.append_name (type_suffixes[instance.type.index].vector);
+ }
+
+ /* According to rvv-intrinsic-doc, it does not add "_m" suffix
+ for vop_m C++ overloaded API. */
+ if (overloaded_p && instance.pred == PRED_TYPE_m)
+ return b.finish_name ();
+ b.append_name (predication_suffixes[instance.pred]);
+ return b.finish_name ();
+ }
+};
+
/* alu_def class. */
struct alu_def : public build_base
{
@@ -632,6 +712,21 @@ struct reduc_alu_def : public build_base
}
};
+/* th_extract_def class. */
+struct th_extract_def : public build_base
+{
+ char *get_name (function_builder &b, const function_instance &instance,
+ bool overloaded_p) const override
+ {
+ b.append_base_name (instance.base_name);
+ if (overloaded_p)
+ return b.finish_name ();
+ b.append_name (type_suffixes[instance.type.index].vector);
+ b.append_name (type_suffixes[instance.type.index].scalar);
+ return b.finish_name ();
+ }
+};
+
/* scalar_move_def class. */
struct scalar_move_def : public build_base
{
@@ -1094,6 +1189,8 @@ SHAPE(vsetvl, vsetvl)
SHAPE(vsetvl, vsetvlmax)
SHAPE(loadstore, loadstore)
SHAPE(indexed_loadstore, indexed_loadstore)
+SHAPE(th_loadstore_width, th_loadstore_width)
+SHAPE(th_indexed_loadstore_width, th_indexed_loadstore_width)
SHAPE(alu, alu)
SHAPE(alu_frm, alu_frm)
SHAPE(widen_alu, widen_alu)
@@ -1106,6 +1203,7 @@ SHAPE(move, move)
SHAPE(mask_alu, mask_alu)
SHAPE(reduc_alu, reduc_alu)
SHAPE(reduc_alu_frm, reduc_alu_frm)
+SHAPE(th_extract, th_extract)
SHAPE(scalar_move, scalar_move)
SHAPE(vundefined, vundefined)
SHAPE(misc, misc)
diff --git a/gcc/config/riscv/riscv-vector-builtins-shapes.h b/gcc/config/riscv/riscv-vector-builtins-shapes.h
index ac2a28ce017..a7624d0fabd 100644
--- a/gcc/config/riscv/riscv-vector-builtins-shapes.h
+++ b/gcc/config/riscv/riscv-vector-builtins-shapes.h
@@ -28,6 +28,8 @@ extern const function_shape *const vsetvl;
extern const function_shape *const vsetvlmax;
extern const function_shape *const loadstore;
extern const function_shape *const indexed_loadstore;
+extern const function_shape *const th_loadstore_width;
+extern const function_shape *const th_indexed_loadstore_width;
extern const function_shape *const alu;
extern const function_shape *const alu_frm;
extern const function_shape *const widen_alu;
@@ -41,6 +43,7 @@ extern const function_shape *const mask_alu;
extern const function_shape *const reduc_alu;
extern const function_shape *const reduc_alu_frm;
extern const function_shape *const scalar_move;
+extern const function_shape *const th_extract;
extern const function_shape *const vundefined;
extern const function_shape *const misc;
extern const function_shape *const vset;
diff --git a/gcc/config/riscv/riscv-vector-builtins-types.def b/gcc/config/riscv/riscv-vector-builtins-types.def
index 61019a56844..abfeb4fcd9b 100644
--- a/gcc/config/riscv/riscv-vector-builtins-types.def
+++ b/gcc/config/riscv/riscv-vector-builtins-types.def
@@ -24,12 +24,48 @@ along with GCC; see the file COPYING3. If not see
#define DEF_RVV_I_OPS(TYPE, REQUIRE)
#endif
+/* Use "DEF_RVV_I8_OPS" macro include some signed integer (i8/i16/i32/i64)
+ which will be iterated and registered as intrinsic functions. */
+#ifndef DEF_RVV_I8_OPS
+#define DEF_RVV_I8_OPS(TYPE, REQUIRE)
+#endif
+
+/* Use "DEF_RVV_I16_OPS" macro include some signed integer (i16/i32/i64)
+ which will be iterated and registered as intrinsic functions. */
+#ifndef DEF_RVV_I16_OPS
+#define DEF_RVV_I16_OPS(TYPE, REQUIRE)
+#endif
+
+/* Use "DEF_RVV_I32_OPS" macro include some signed integer (i32/i64)
+ which will be iterated and registered as intrinsic functions. */
+#ifndef DEF_RVV_I32_OPS
+#define DEF_RVV_I32_OPS(TYPE, REQUIRE)
+#endif
+
/* Use "DEF_RVV_U_OPS" macro include all unsigned integer which will be
iterated and registered as intrinsic functions. */
#ifndef DEF_RVV_U_OPS
#define DEF_RVV_U_OPS(TYPE, REQUIRE)
#endif
+/* Use "DEF_RVV_U8_OPS" macro include some unsigned integer (i8/i16/i32/i64)
+ which will be iterated and registered as intrinsic functions. */
+#ifndef DEF_RVV_U8_OPS
+#define DEF_RVV_U8_OPS(TYPE, REQUIRE)
+#endif
+
+/* Use "DEF_RVV_U16_OPS" macro include some unsigned integer (i16/i32/i64)
+ which will be iterated and registered as intrinsic functions. */
+#ifndef DEF_RVV_U16_OPS
+#define DEF_RVV_U16_OPS(TYPE, REQUIRE)
+#endif
+
+/* Use "DEF_RVV_U32_OPS" macro include some unsigned integer (i32/i64)
+ which will be iterated and registered as intrinsic functions. */
+#ifndef DEF_RVV_U32_OPS
+#define DEF_RVV_U32_OPS(TYPE, REQUIRE)
+#endif
+
/* Use "DEF_RVV_F_OPS" macro include all floating-point which will be
iterated and registered as intrinsic functions. */
#ifndef DEF_RVV_F_OPS
@@ -374,6 +410,45 @@ DEF_RVV_I_OPS (vint64m2_t, RVV_REQUIRE_ELEN_64)
DEF_RVV_I_OPS (vint64m4_t, RVV_REQUIRE_ELEN_64)
DEF_RVV_I_OPS (vint64m8_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_I8_OPS (vint8m1_t, 0)
+DEF_RVV_I8_OPS (vint8m2_t, 0)
+DEF_RVV_I8_OPS (vint8m4_t, 0)
+DEF_RVV_I8_OPS (vint8m8_t, 0)
+DEF_RVV_I8_OPS (vint16m1_t, 0)
+DEF_RVV_I8_OPS (vint16m2_t, 0)
+DEF_RVV_I8_OPS (vint16m4_t, 0)
+DEF_RVV_I8_OPS (vint16m8_t, 0)
+DEF_RVV_I8_OPS (vint32m1_t, 0)
+DEF_RVV_I8_OPS (vint32m2_t, 0)
+DEF_RVV_I8_OPS (vint32m4_t, 0)
+DEF_RVV_I8_OPS (vint32m8_t, 0)
+DEF_RVV_I8_OPS (vint64m1_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_I8_OPS (vint64m2_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_I8_OPS (vint64m4_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_I8_OPS (vint64m8_t, RVV_REQUIRE_ELEN_64)
+
+DEF_RVV_I16_OPS (vint16m1_t, 0)
+DEF_RVV_I16_OPS (vint16m2_t, 0)
+DEF_RVV_I16_OPS (vint16m4_t, 0)
+DEF_RVV_I16_OPS (vint16m8_t, 0)
+DEF_RVV_I16_OPS (vint32m1_t, 0)
+DEF_RVV_I16_OPS (vint32m2_t, 0)
+DEF_RVV_I16_OPS (vint32m4_t, 0)
+DEF_RVV_I16_OPS (vint32m8_t, 0)
+DEF_RVV_I16_OPS (vint64m1_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_I16_OPS (vint64m2_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_I16_OPS (vint64m4_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_I16_OPS (vint64m8_t, RVV_REQUIRE_ELEN_64)
+
+DEF_RVV_I32_OPS (vint32m1_t, 0)
+DEF_RVV_I32_OPS (vint32m2_t, 0)
+DEF_RVV_I32_OPS (vint32m4_t, 0)
+DEF_RVV_I32_OPS (vint32m8_t, 0)
+DEF_RVV_I32_OPS (vint64m1_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_I32_OPS (vint64m2_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_I32_OPS (vint64m4_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_I32_OPS (vint64m8_t, RVV_REQUIRE_ELEN_64)
+
DEF_RVV_U_OPS (vuint8mf8_t, RVV_REQUIRE_MIN_VLEN_64)
DEF_RVV_U_OPS (vuint8mf4_t, 0)
DEF_RVV_U_OPS (vuint8mf2_t, 0)
@@ -397,6 +472,45 @@ DEF_RVV_U_OPS (vuint64m2_t, RVV_REQUIRE_ELEN_64)
DEF_RVV_U_OPS (vuint64m4_t, RVV_REQUIRE_ELEN_64)
DEF_RVV_U_OPS (vuint64m8_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_U8_OPS (vuint8m1_t, 0)
+DEF_RVV_U8_OPS (vuint8m2_t, 0)
+DEF_RVV_U8_OPS (vuint8m4_t, 0)
+DEF_RVV_U8_OPS (vuint8m8_t, 0)
+DEF_RVV_U8_OPS (vuint16m1_t, 0)
+DEF_RVV_U8_OPS (vuint16m2_t, 0)
+DEF_RVV_U8_OPS (vuint16m4_t, 0)
+DEF_RVV_U8_OPS (vuint16m8_t, 0)
+DEF_RVV_U8_OPS (vuint32m1_t, 0)
+DEF_RVV_U8_OPS (vuint32m2_t, 0)
+DEF_RVV_U8_OPS (vuint32m4_t, 0)
+DEF_RVV_U8_OPS (vuint32m8_t, 0)
+DEF_RVV_U8_OPS (vuint64m1_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_U8_OPS (vuint64m2_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_U8_OPS (vuint64m4_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_U8_OPS (vuint64m8_t, RVV_REQUIRE_ELEN_64)
+
+DEF_RVV_U16_OPS (vuint16m1_t, 0)
+DEF_RVV_U16_OPS (vuint16m2_t, 0)
+DEF_RVV_U16_OPS (vuint16m4_t, 0)
+DEF_RVV_U16_OPS (vuint16m8_t, 0)
+DEF_RVV_U16_OPS (vuint32m1_t, 0)
+DEF_RVV_U16_OPS (vuint32m2_t, 0)
+DEF_RVV_U16_OPS (vuint32m4_t, 0)
+DEF_RVV_U16_OPS (vuint32m8_t, 0)
+DEF_RVV_U16_OPS (vuint64m1_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_U16_OPS (vuint64m2_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_U16_OPS (vuint64m4_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_U16_OPS (vuint64m8_t, RVV_REQUIRE_ELEN_64)
+
+DEF_RVV_U32_OPS (vuint32m1_t, 0)
+DEF_RVV_U32_OPS (vuint32m2_t, 0)
+DEF_RVV_U32_OPS (vuint32m4_t, 0)
+DEF_RVV_U32_OPS (vuint32m8_t, 0)
+DEF_RVV_U32_OPS (vuint64m1_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_U32_OPS (vuint64m2_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_U32_OPS (vuint64m4_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_U32_OPS (vuint64m8_t, RVV_REQUIRE_ELEN_64)
+
DEF_RVV_F_OPS (vfloat16mf4_t, RVV_REQUIRE_ELEN_FP_16 | RVV_REQUIRE_MIN_VLEN_64)
DEF_RVV_F_OPS (vfloat16mf2_t, RVV_REQUIRE_ELEN_FP_16)
DEF_RVV_F_OPS (vfloat16m1_t, RVV_REQUIRE_ELEN_FP_16)
@@ -1379,7 +1493,13 @@ DEF_RVV_CRYPTO_SEW64_OPS (vuint64m4_t, RVV_REQUIRE_ELEN_64)
DEF_RVV_CRYPTO_SEW64_OPS (vuint64m8_t, RVV_REQUIRE_ELEN_64)
#undef DEF_RVV_I_OPS
+#undef DEF_RVV_I8_OPS
+#undef DEF_RVV_I16_OPS
+#undef DEF_RVV_I32_OPS
#undef DEF_RVV_U_OPS
+#undef DEF_RVV_U8_OPS
+#undef DEF_RVV_U16_OPS
+#undef DEF_RVV_U32_OPS
#undef DEF_RVV_F_OPS
#undef DEF_RVV_B_OPS
#undef DEF_RVV_WEXTI_OPS
diff --git a/gcc/config/riscv/riscv-vector-builtins.cc b/gcc/config/riscv/riscv-vector-builtins.cc
index 25e0b6e56de..f429f12dc18 100644
--- a/gcc/config/riscv/riscv-vector-builtins.cc
+++ b/gcc/config/riscv/riscv-vector-builtins.cc
@@ -246,6 +246,63 @@ static const rvv_type_info iu_ops[] = {
#include "riscv-vector-builtins-types.def"
{NUM_VECTOR_TYPES, 0}};
+/* A list of all integer will be registered for intrinsic functions. */
+static const rvv_type_info i8_ops[] = {
+#define DEF_RVV_I8_OPS(TYPE, REQUIRE) {VECTOR_TYPE_##TYPE, REQUIRE},
+#include "riscv-vector-builtins-types.def"
+ {NUM_VECTOR_TYPES, 0}};
+
+/* A list of all integer will be registered for intrinsic functions. */
+static const rvv_type_info i16_ops[] = {
+#define DEF_RVV_I16_OPS(TYPE, REQUIRE) {VECTOR_TYPE_##TYPE, REQUIRE},
+#include "riscv-vector-builtins-types.def"
+ {NUM_VECTOR_TYPES, 0}};
+
+/* A list of all integer will be registered for intrinsic functions. */
+static const rvv_type_info i32_ops[] = {
+#define DEF_RVV_I32_OPS(TYPE, REQUIRE) {VECTOR_TYPE_##TYPE, REQUIRE},
+#include "riscv-vector-builtins-types.def"
+ {NUM_VECTOR_TYPES, 0}};
+
+/* A list of all integer will be registered for intrinsic functions. */
+static const rvv_type_info u8_ops[] = {
+#define DEF_RVV_U8_OPS(TYPE, REQUIRE) {VECTOR_TYPE_##TYPE, REQUIRE},
+#include "riscv-vector-builtins-types.def"
+ {NUM_VECTOR_TYPES, 0}};
+
+/* A list of all integer will be registered for intrinsic functions. */
+static const rvv_type_info u16_ops[] = {
+#define DEF_RVV_U16_OPS(TYPE, REQUIRE) {VECTOR_TYPE_##TYPE, REQUIRE},
+#include "riscv-vector-builtins-types.def"
+ {NUM_VECTOR_TYPES, 0}};
+
+/* A list of all integer will be registered for intrinsic functions. */
+static const rvv_type_info u32_ops[] = {
+#define DEF_RVV_U32_OPS(TYPE, REQUIRE) {VECTOR_TYPE_##TYPE, REQUIRE},
+#include "riscv-vector-builtins-types.def"
+ {NUM_VECTOR_TYPES, 0}};
+
+/* A list of all integer will be registered for intrinsic functions. */
+static const rvv_type_info iu8_ops[] = {
+#define DEF_RVV_I8_OPS(TYPE, REQUIRE) {VECTOR_TYPE_##TYPE, REQUIRE},
+#define DEF_RVV_U8_OPS(TYPE, REQUIRE) {VECTOR_TYPE_##TYPE, REQUIRE},
+#include "riscv-vector-builtins-types.def"
+ {NUM_VECTOR_TYPES, 0}};
+
+/* A list of all integer will be registered for intrinsic functions. */
+static const rvv_type_info iu16_ops[] = {
+#define DEF_RVV_I16_OPS(TYPE, REQUIRE) {VECTOR_TYPE_##TYPE, REQUIRE},
+#define DEF_RVV_U16_OPS(TYPE, REQUIRE) {VECTOR_TYPE_##TYPE, REQUIRE},
+#include "riscv-vector-builtins-types.def"
+ {NUM_VECTOR_TYPES, 0}};
+
+/* A list of all integer will be registered for intrinsic functions. */
+static const rvv_type_info iu32_ops[] = {
+#define DEF_RVV_I32_OPS(TYPE, REQUIRE) {VECTOR_TYPE_##TYPE, REQUIRE},
+#define DEF_RVV_U32_OPS(TYPE, REQUIRE) {VECTOR_TYPE_##TYPE, REQUIRE},
+#include "riscv-vector-builtins-types.def"
+ {NUM_VECTOR_TYPES, 0}};
+
/* A list of all types will be registered for intrinsic functions. */
static const rvv_type_info all_ops[] = {
#define DEF_RVV_I_OPS(TYPE, REQUIRE) {VECTOR_TYPE_##TYPE, REQUIRE},
@@ -934,6 +991,32 @@ static CONSTEXPR const rvv_arg_type_info ext_vcreate_args[]
= {rvv_arg_type_info (RVV_BASE_vector),
rvv_arg_type_info_end};
+/* A list of args for vector_type func (const scalar_type *, size_t)
+ * function. */
+static CONSTEXPR const rvv_arg_type_info scalar_const_ptr_size_args[]
+ = {rvv_arg_type_info (RVV_BASE_scalar_const_ptr),
+ rvv_arg_type_info (RVV_BASE_size), rvv_arg_type_info_end};
+
+/* A list of args for vector_type func (const scalar_type *, eew8_index_type)
+ * function. */
+static CONSTEXPR const rvv_arg_type_info scalar_const_ptr_index_args[]
+ = {rvv_arg_type_info (RVV_BASE_scalar_const_ptr),
+ rvv_arg_type_info (RVV_BASE_unsigned_vector), rvv_arg_type_info_end};
+
+/* A list of args for void func (scalar_type *, eew8_index_type, vector_type)
+ * function. */
+static CONSTEXPR const rvv_arg_type_info scalar_ptr_index_args[]
+ = {rvv_arg_type_info (RVV_BASE_scalar_ptr),
+ rvv_arg_type_info (RVV_BASE_unsigned_vector),
+ rvv_arg_type_info (RVV_BASE_vector), rvv_arg_type_info_end};
+
+/* A list of args for void func (scalar_type *, size_t, vector_type)
+ * function. */
+static CONSTEXPR const rvv_arg_type_info scalar_ptr_size_args[]
+ = {rvv_arg_type_info (RVV_BASE_scalar_ptr),
+ rvv_arg_type_info (RVV_BASE_size), rvv_arg_type_info (RVV_BASE_vector),
+ rvv_arg_type_info_end};
+
/* A list of none preds that will be registered for intrinsic functions. */
static CONSTEXPR const predication_type_index none_preds[]
= {PRED_TYPE_none, NUM_PRED_TYPES};
@@ -1455,6 +1538,14 @@ static CONSTEXPR const rvv_op_info iu_shift_vvv_ops
rvv_arg_type_info (RVV_BASE_vector), /* Return type */
shift_vv_args /* Args */};
+/* A static operand information for scalar_type func (vector_type, size_t)
+ * function registration. */
+static CONSTEXPR const rvv_op_info iu_x_s_u_ops
+ = {iu_ops, /* Types */
+ OP_TYPE_vx, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_scalar), /* Return type */
+ v_size_args /* Args */};
+
/* A static operand information for vector_type func (vector_type, size_t)
* function registration. */
static CONSTEXPR const rvv_op_info iu_shift_vvx_ops
@@ -2638,6 +2729,222 @@ static CONSTEXPR const rvv_op_info all_v_vcreate_lmul4_x2_ops
rvv_arg_type_info (RVV_BASE_vlmul_ext_x2), /* Return type */
ext_vcreate_args /* Args */};
+/* A static operand information for vector_type func (const scalar_type *)
+ * function registration. */
+static CONSTEXPR const rvv_op_info i8_v_scalar_const_ptr_ops
+ = {i8_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *)
+ * function registration. */
+static CONSTEXPR const rvv_op_info i16_v_scalar_const_ptr_ops
+ = {i16_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *)
+ * function registration. */
+static CONSTEXPR const rvv_op_info i32_v_scalar_const_ptr_ops
+ = {i32_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *)
+ * function registration. */
+static CONSTEXPR const rvv_op_info u8_v_scalar_const_ptr_ops
+ = {u8_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *)
+ * function registration. */
+static CONSTEXPR const rvv_op_info u16_v_scalar_const_ptr_ops
+ = {u16_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *)
+ * function registration. */
+static CONSTEXPR const rvv_op_info u32_v_scalar_const_ptr_ops
+ = {u32_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * size_t) function registration. */
+static CONSTEXPR const rvv_op_info i8_v_scalar_const_ptr_size_ops
+ = {i8_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_size_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * size_t) function registration. */
+static CONSTEXPR const rvv_op_info i16_v_scalar_const_ptr_size_ops
+ = {i16_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_size_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * size_t) function registration. */
+static CONSTEXPR const rvv_op_info i32_v_scalar_const_ptr_size_ops
+ = {i32_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_size_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * size_t) function registration. */
+static CONSTEXPR const rvv_op_info u8_v_scalar_const_ptr_size_ops
+ = {u8_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_size_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * size_t) function registration. */
+static CONSTEXPR const rvv_op_info u16_v_scalar_const_ptr_size_ops
+ = {u16_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_size_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * size_t) function registration. */
+static CONSTEXPR const rvv_op_info u32_v_scalar_const_ptr_size_ops
+ = {u32_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_size_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * eew8_index_type) function registration. */
+static CONSTEXPR const rvv_op_info i8_v_scalar_const_ptr_index_ops
+ = {i8_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_index_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * eew8_index_type) function registration. */
+static CONSTEXPR const rvv_op_info u8_v_scalar_const_ptr_index_ops
+ = {u8_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_index_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * eew8_index_type) function registration. */
+static CONSTEXPR const rvv_op_info i16_v_scalar_const_ptr_index_ops
+ = {i16_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_index_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * eew8_index_type) function registration. */
+static CONSTEXPR const rvv_op_info u16_v_scalar_const_ptr_index_ops
+ = {u16_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_index_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * eew8_index_type) function registration. */
+static CONSTEXPR const rvv_op_info i32_v_scalar_const_ptr_index_ops
+ = {i32_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_index_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * eew8_index_type) function registration. */
+static CONSTEXPR const rvv_op_info u32_v_scalar_const_ptr_index_ops
+ = {u32_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_index_args /* Args */};
+
+/* A static operand information for void func (scalar_type *, eew8_index_type,
+ * vector_type) function registration. */
+static CONSTEXPR const rvv_op_info iu8_v_scalar_ptr_index_ops
+ = {iu8_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_void), /* Return type */
+ scalar_ptr_index_args /* Args */};
+
+/* A static operand information for void func (scalar_type *, eew16_index_type,
+ * vector_type) function registration. */
+static CONSTEXPR const rvv_op_info iu16_v_scalar_ptr_index_ops
+ = {iu16_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_void), /* Return type */
+ scalar_ptr_index_args /* Args */};
+
+/* A static operand information for void func (scalar_type *, eew32_index_type,
+ * vector_type) function registration. */
+static CONSTEXPR const rvv_op_info iu32_v_scalar_ptr_index_ops
+ = {iu32_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_void), /* Return type */
+ scalar_ptr_index_args /* Args */};
+
+/* A static operand information for void func (scalar_type *, vector_type,
+ * function registration. */
+static CONSTEXPR const rvv_op_info iu8_v_scalar_ptr_ops
+ = {iu8_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_void), /* Return type */
+ scalar_ptr_args /* Args */};
+
+/* A static operand information for void func (scalar_type *, vector_type)
+ * function registration. */
+static CONSTEXPR const rvv_op_info iu16_v_scalar_ptr_ops
+ = {iu16_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_void), /* Return type */
+ scalar_ptr_args /* Args */};
+
+/* A static operand information for void func (scalar_type *, vector_type)
+ * function registration. */
+static CONSTEXPR const rvv_op_info iu32_v_scalar_ptr_ops
+ = {iu32_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_void), /* Return type */
+ scalar_ptr_args /* Args */};
+
+/* A static operand information for void func (scalar_type *, size_t,
+ * vector_type) function registration. */
+static CONSTEXPR const rvv_op_info iu8_v_scalar_ptr_size_ops
+ = {iu8_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_void), /* Return type */
+ scalar_ptr_size_args /* Args */};
+
+/* A static operand information for void func (scalar_type *, size_t,
+ * vector_type) function registration. */
+static CONSTEXPR const rvv_op_info iu16_v_scalar_ptr_size_ops
+ = {iu16_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_void), /* Return type */
+ scalar_ptr_size_args /* Args */};
+
+/* A static operand information for void func (scalar_type *, size_t,
+ * vector_type) function registration. */
+static CONSTEXPR const rvv_op_info iu32_v_scalar_ptr_size_ops
+ = {iu32_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_void), /* Return type */
+ scalar_ptr_size_args /* Args */};
+
/* A static operand information for vector_type func (vector_type).
Some ins just supports SEW=32, such as crypto vectol Zvkg extension.
* function registration. */
@@ -2816,6 +3123,10 @@ static function_group_info function_groups[] = {
#define DEF_RVV_FUNCTION(NAME, SHAPE, PREDS, OPS_INFO) \
{#NAME, &bases::NAME, &shapes::SHAPE, PREDS, OPS_INFO, REQUIRED_EXTENSIONS},
#include "riscv-vector-builtins-functions.def"
+#undef DEF_RVV_FUNCTION
+#define DEF_RVV_FUNCTION(NAME, SHAPE, PREDS, OPS_INFO) \
+ {#NAME, &bases::NAME, &shapes::SHAPE, PREDS, OPS_INFO, REQUIRED_EXTENSIONS},
+#include "thead-vector-builtins-functions.def"
};
/* The RVV types, with their built-in
diff --git a/gcc/config/riscv/riscv-vector-builtins.h b/gcc/config/riscv/riscv-vector-builtins.h
index 54c8824ff92..a8ee39a3cb2 100644
--- a/gcc/config/riscv/riscv-vector-builtins.h
+++ b/gcc/config/riscv/riscv-vector-builtins.h
@@ -114,6 +114,7 @@ static const unsigned int CP_WRITE_CSR = 1U << 5;
enum required_ext
{
VECTOR_EXT, /* Vector extension */
+ XTHEADVECTOR_EXT, /* XTheadVector extension */
ZVBB_EXT, /* Cryto vector Zvbb sub-ext */
ZVBB_OR_ZVKB_EXT, /* Cryto vector Zvbb or zvkb sub-ext */
ZVBC_EXT, /* Crypto vector Zvbc sub-ext */
@@ -234,6 +235,8 @@ struct function_group_info
{
case VECTOR_EXT:
return TARGET_VECTOR;
+ case XTHEADVECTOR_EXT:
+ return TARGET_XTHEADVECTOR;
case ZVBB_EXT:
return TARGET_ZVBB;
case ZVBB_OR_ZVKB_EXT:
diff --git a/gcc/config/riscv/t-riscv b/gcc/config/riscv/t-riscv
index 32de6b851c1..38494320d8b 100644
--- a/gcc/config/riscv/t-riscv
+++ b/gcc/config/riscv/t-riscv
@@ -1,6 +1,7 @@
RISCV_BUILTINS_H = $(srcdir)/config/riscv/riscv-vector-builtins.h \
$(srcdir)/config/riscv/riscv-vector-builtins.def \
$(srcdir)/config/riscv/riscv-vector-builtins-functions.def \
+ $(srcdir)/config/riscv/thead-vector-builtins-functions.def \
riscv-vector-type-indexer.gen.def
riscv-builtins.o: $(srcdir)/config/riscv/riscv-builtins.cc $(CONFIG_H) \
diff --git a/gcc/config/riscv/thead-vector-builtins-functions.def b/gcc/config/riscv/thead-vector-builtins-functions.def
new file mode 100644
index 00000000000..667820d4c3e
--- /dev/null
+++ b/gcc/config/riscv/thead-vector-builtins-functions.def
@@ -0,0 +1,39 @@
+#ifndef DEF_RVV_FUNCTION
+#define DEF_RVV_FUNCTION(NAME, SHAPE, PREDS, OPS_INFO)
+#endif
+
+#define REQUIRED_EXTENSIONS XTHEADVECTOR_EXT
+DEF_RVV_FUNCTION (th_vlb, th_loadstore_width, full_preds, i8_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (th_vlh, th_loadstore_width, full_preds, i16_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (th_vlw, th_loadstore_width, full_preds, i32_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (th_vlbu, th_loadstore_width, full_preds, u8_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (th_vlhu, th_loadstore_width, full_preds, u16_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (th_vlwu, th_loadstore_width, full_preds, u32_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (th_vsb, th_loadstore_width, none_m_preds, iu8_v_scalar_ptr_ops)
+DEF_RVV_FUNCTION (th_vsh, th_loadstore_width, none_m_preds, iu16_v_scalar_ptr_ops)
+DEF_RVV_FUNCTION (th_vsw, th_loadstore_width, none_m_preds, iu32_v_scalar_ptr_ops)
+DEF_RVV_FUNCTION (th_vlsb, th_loadstore_width, full_preds, i8_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (th_vlsh, th_loadstore_width, full_preds, i16_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (th_vlsw, th_loadstore_width, full_preds, i32_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (th_vlsbu, th_loadstore_width, full_preds, u8_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (th_vlshu, th_loadstore_width, full_preds, u16_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (th_vlswu, th_loadstore_width, full_preds, u32_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (th_vssb, th_loadstore_width, none_m_preds, iu8_v_scalar_ptr_size_ops)
+DEF_RVV_FUNCTION (th_vssh, th_loadstore_width, none_m_preds, iu16_v_scalar_ptr_size_ops)
+DEF_RVV_FUNCTION (th_vssw, th_loadstore_width, none_m_preds, iu32_v_scalar_ptr_size_ops)
+DEF_RVV_FUNCTION (th_vlxb, th_indexed_loadstore_width, full_preds, i8_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (th_vlxh, th_indexed_loadstore_width, full_preds, i16_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (th_vlxw, th_indexed_loadstore_width, full_preds, i32_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (th_vlxbu, th_indexed_loadstore_width, full_preds, u8_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (th_vlxhu, th_indexed_loadstore_width, full_preds, u16_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (th_vlxwu, th_indexed_loadstore_width, full_preds, u32_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (th_vsxb, th_indexed_loadstore_width, none_m_preds, iu8_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (th_vsxh, th_indexed_loadstore_width, none_m_preds, iu16_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (th_vsxw, th_indexed_loadstore_width, none_m_preds, iu32_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (th_vsuxb, th_indexed_loadstore_width, none_m_preds, iu8_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (th_vsuxh, th_indexed_loadstore_width, none_m_preds, iu16_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (th_vsuxw, th_indexed_loadstore_width, none_m_preds, iu32_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (th_vext_x_v, th_extract, none_preds, iu_x_s_u_ops)
+#undef REQUIRED_EXTENSIONS
+
+#undef DEF_RVV_FUNCTION
diff --git a/gcc/config/riscv/thead-vector.md b/gcc/config/riscv/thead-vector.md
index 696b815252d..0f3700d9269 100644
--- a/gcc/config/riscv/thead-vector.md
+++ b/gcc/config/riscv/thead-vector.md
@@ -1,7 +1,95 @@
(define_c_enum "unspec" [
+ UNSPEC_TH_VLB
+ UNSPEC_TH_VLBU
+ UNSPEC_TH_VLH
+ UNSPEC_TH_VLHU
+ UNSPEC_TH_VLW
+ UNSPEC_TH_VLWU
+
+ UNSPEC_TH_VLSB
+ UNSPEC_TH_VLSBU
+ UNSPEC_TH_VLSH
+ UNSPEC_TH_VLSHU
+ UNSPEC_TH_VLSW
+ UNSPEC_TH_VLSWU
+
+ UNSPEC_TH_VLXB
+ UNSPEC_TH_VLXBU
+ UNSPEC_TH_VLXH
+ UNSPEC_TH_VLXHU
+ UNSPEC_TH_VLXW
+ UNSPEC_TH_VLXWU
+
+ UNSPEC_TH_VSUXB
+ UNSPEC_TH_VSUXH
+ UNSPEC_TH_VSUXW
+
UNSPEC_TH_VWLDST
])
+(define_int_iterator UNSPEC_TH_VLMEM_OP [
+ UNSPEC_TH_VLB UNSPEC_TH_VLBU
+ UNSPEC_TH_VLH UNSPEC_TH_VLHU
+ UNSPEC_TH_VLW UNSPEC_TH_VLWU
+])
+
+(define_int_iterator UNSPEC_TH_VLSMEM_OP [
+ UNSPEC_TH_VLSB UNSPEC_TH_VLSBU
+ UNSPEC_TH_VLSH UNSPEC_TH_VLSHU
+ UNSPEC_TH_VLSW UNSPEC_TH_VLSWU
+])
+
+(define_int_iterator UNSPEC_TH_VLXMEM_OP [
+ UNSPEC_TH_VLXB UNSPEC_TH_VLXBU
+ UNSPEC_TH_VLXH UNSPEC_TH_VLXHU
+ UNSPEC_TH_VLXW UNSPEC_TH_VLXWU
+])
+
+(define_int_attr vlmem_op_attr [
+ (UNSPEC_TH_VLB "b") (UNSPEC_TH_VLBU "bu")
+ (UNSPEC_TH_VLH "h") (UNSPEC_TH_VLHU "hu")
+ (UNSPEC_TH_VLW "w") (UNSPEC_TH_VLWU "wu")
+ (UNSPEC_TH_VLSB "b") (UNSPEC_TH_VLSBU "bu")
+ (UNSPEC_TH_VLSH "h") (UNSPEC_TH_VLSHU "hu")
+ (UNSPEC_TH_VLSW "w") (UNSPEC_TH_VLSWU "wu")
+ (UNSPEC_TH_VLXB "b") (UNSPEC_TH_VLXBU "bu")
+ (UNSPEC_TH_VLXH "h") (UNSPEC_TH_VLXHU "hu")
+ (UNSPEC_TH_VLXW "w") (UNSPEC_TH_VLXWU "wu")
+ (UNSPEC_TH_VSUXB "b")
+ (UNSPEC_TH_VSUXH "h")
+ (UNSPEC_TH_VSUXW "w")
+])
+
+(define_int_attr vlmem_order_attr [
+ (UNSPEC_TH_VLXB "")
+ (UNSPEC_TH_VLXH "")
+ (UNSPEC_TH_VLXW "")
+ (UNSPEC_TH_VSUXB "u")
+ (UNSPEC_TH_VSUXH "u")
+ (UNSPEC_TH_VSUXW "u")
+])
+
+(define_int_iterator UNSPEC_TH_VSMEM_OP [
+ UNSPEC_TH_VLB
+ UNSPEC_TH_VLH
+ UNSPEC_TH_VLW
+])
+
+(define_int_iterator UNSPEC_TH_VSSMEM_OP [
+ UNSPEC_TH_VLSB
+ UNSPEC_TH_VLSH
+ UNSPEC_TH_VLSW
+])
+
+(define_int_iterator UNSPEC_TH_VSXMEM_OP [
+ UNSPEC_TH_VLXB
+ UNSPEC_TH_VLXH
+ UNSPEC_TH_VLXW
+ UNSPEC_TH_VSUXB
+ UNSPEC_TH_VSUXH
+ UNSPEC_TH_VSUXW
+])
+
(define_mode_iterator V_VLS_VT [V VLS VT])
(define_mode_iterator V_VB_VLS_VT [V VB VLS VT])
@@ -100,3 +188,168 @@
}
[(set_attr "type" "vldm,vstm,vmalu,vmalu,vmalu")
(set_attr "mode" "<MODE>")])
+
+(define_expand "@pred_mov_width<vlmem_op_attr><mode>"
+ [(set (match_operand:V_VLS 0 "nonimmediate_operand")
+ (if_then_else:V_VLS
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand")
+ (match_operand 4 "vector_length_operand")
+ (match_operand 5 "const_int_operand")
+ (match_operand 6 "const_int_operand")
+ (match_operand 7 "const_int_operand")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLMEM_OP)
+ (match_operand:V_VLS 3 "vector_move_operand")
+ (match_operand:V_VLS 2 "vector_merge_operand")))]
+ "TARGET_XTHEADVECTOR"
+ {})
+
+(define_insn_and_split "*pred_mov_width<vlmem_op_attr><mode>"
+ [(set (match_operand:V_VLS 0 "nonimmediate_operand" "=vr, vr, vd, m, vr, vr")
+ (if_then_else:V_VLS
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1, Wc1, vm, vmWc1, Wc1, Wc1")
+ (match_operand 4 "vector_length_operand" " rK, rK, rK, rK, rK, rK")
+ (match_operand 5 "const_int_operand" " i, i, i, i, i, i")
+ (match_operand 6 "const_int_operand" " i, i, i, i, i, i")
+ (match_operand 7 "const_int_operand" " i, i, i, i, i, i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLMEM_OP)
+ (match_operand:V_VLS 3 "reg_or_mem_operand" " m, m, m, vr, vr, vr")
+ (match_operand:V_VLS 2 "vector_merge_operand" " 0, vu, vu, vu, vu, 0")))]
+ "(TARGET_XTHEADVECTOR
+ && (register_operand (operands[0], <MODE>mode)
+ || register_operand (operands[3], <MODE>mode)))"
+ "@
+ vl<vlmem_op_attr>.v\t%0,%3%p1
+ vl<vlmem_op_attr>.v\t%0,%3
+ vl<vlmem_op_attr>.v\t%0,%3,%1.t
+ vs<vlmem_op_attr>.v\t%3,%0%p1
+ vmv.v.v\t%0,%3
+ vmv.v.v\t%0,%3"
+ "&& register_operand (operands[0], <MODE>mode)
+ && register_operand (operands[3], <MODE>mode)
+ && satisfies_constraint_vu (operands[2])
+ && INTVAL (operands[7]) == riscv_vector::VLMAX"
+ [(set (match_dup 0) (match_dup 3))]
+ ""
+ [(set_attr "type" "vlde,vlde,vlde,vste,vimov,vimov")
+ (set_attr "mode" "<MODE>")])
+
+(define_insn "@pred_store_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "memory_operand" "+m")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1")
+ (match_operand 3 "vector_length_operand" " rK")
+ (match_operand 4 "const_int_operand" " i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VSMEM_OP)
+ (match_operand:VI 2 "register_operand" " vr")
+ (match_dup 0)))]
+ "TARGET_XTHEADVECTOR"
+ "vs<vlmem_op_attr>.v\t%2,%0%p1"
+ [(set_attr "type" "vste")
+ (set_attr "mode" "<MODE>")
+ (set (attr "avl_type_idx") (const_int 4))
+ (set_attr "vl_op_idx" "3")])
+
+(define_insn "@pred_strided_load_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "register_operand" "=vr, vr, vd")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1, Wc1, vm")
+ (match_operand 5 "vector_length_operand" " rK, rK, rK")
+ (match_operand 6 "const_int_operand" " i, i, i")
+ (match_operand 7 "const_int_operand" " i, i, i")
+ (match_operand 8 "const_int_operand" " i, i, i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLSMEM_OP)
+ (unspec:VI
+ [(match_operand:VI 3 "memory_operand" " m, m, m")
+ (match_operand 4 "pmode_reg_or_0_operand" " rJ, rJ, rJ")] UNSPEC_TH_VLSMEM_OP)
+ (match_operand:VI 2 "vector_merge_operand" " 0, vu, vu")))]
+ "TARGET_XTHEADVECTOR"
+ "vls<vlmem_op_attr>.v\t%0,%3,%z4%p1"
+ [(set_attr "type" "vlds")
+ (set_attr "mode" "<MODE>")])
+
+(define_insn "@pred_strided_store_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "memory_operand" "+m")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1")
+ (match_operand 4 "vector_length_operand" " rK")
+ (match_operand 5 "const_int_operand" " i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VSSMEM_OP)
+ (unspec:VI
+ [(match_operand 2 "pmode_reg_or_0_operand" " rJ")
+ (match_operand:VI 3 "register_operand" " vr")] UNSPEC_TH_VSSMEM_OP)
+ (match_dup 0)))]
+ "TARGET_XTHEADVECTOR"
+ "vss<vlmem_op_attr>.v\t%3,%0,%z2%p1"
+ [(set_attr "type" "vsts")
+ (set_attr "mode" "<MODE>")
+ (set (attr "avl_type_idx") (const_int 5))])
+
+(define_insn "@pred_indexed_load_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "register_operand" "=vd, vr,vd, vr")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" " vm,Wc1,vm,Wc1")
+ (match_operand 5 "vector_length_operand" " rK, rK,rK, rK")
+ (match_operand 6 "const_int_operand" " i, i, i, i")
+ (match_operand 7 "const_int_operand" " i, i, i, i")
+ (match_operand 8 "const_int_operand" " i, i, i, i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLXMEM_OP)
+ (unspec:VI
+ [(match_operand 3 "pmode_reg_or_0_operand" " rJ, rJ,rJ, rJ")
+ (mem:BLK (scratch))
+ (match_operand:VI 4 "register_operand" " vr, vr,vr, vr")] UNSPEC_TH_VLXMEM_OP)
+ (match_operand:VI 2 "vector_merge_operand" " vu, vu, 0, 0")))]
+ "TARGET_XTHEADVECTOR"
+ "vlx<vlmem_op_attr>.v\t%0,(%z3),%4%p1"
+ [(set_attr "type" "vldux")
+ (set_attr "mode" "<MODE>")])
+
+(define_insn "@pred_indexed_<vlmem_order_attr>store_width<vlmem_op_attr><mode>"
+ [(set (mem:BLK (scratch))
+ (unspec:BLK
+ [(unspec:<VM>
+ [(match_operand:<VM> 0 "vector_mask_operand" "vmWc1")
+ (match_operand 4 "vector_length_operand" " rK")
+ (match_operand 5 "const_int_operand" " i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VSXMEM_OP)
+ (match_operand 1 "pmode_reg_or_0_operand" " rJ")
+ (match_operand:VI 2 "register_operand" " vr")
+ (match_operand:VI 3 "register_operand" " vr")] UNSPEC_TH_VSXMEM_OP))]
+ "TARGET_XTHEADVECTOR"
+ "vs<vlmem_order_attr>x<vlmem_op_attr>.v\t%3,(%z1),%2%p0"
+ [(set_attr "type" "vstux")
+ (set_attr "mode" "<MODE>")])
+
+(define_expand "@pred_th_extract<mode>"
+ [(set (match_operand:<VEL> 0 "register_operand")
+ (unspec:<VEL>
+ [(vec_select:<VEL>
+ (match_operand:V_VLSI 1 "register_operand")
+ (parallel [(match_operand:DI 2 "register_operand" "r")]))
+ (reg:SI VTYPE_REGNUM)] UNSPEC_VPREDICATE))]
+ "TARGET_XTHEADVECTOR"
+{})
+
+(define_insn "*pred_th_extract<mode>"
+ [(set (match_operand:<VEL> 0 "register_operand" "=r")
+ (unspec:<VEL>
+ [(vec_select:<VEL>
+ (match_operand:V_VLSI 1 "register_operand" "vr")
+ (parallel [(match_operand:DI 2 "register_operand" "r")]))
+ (reg:SI VTYPE_REGNUM)] UNSPEC_VPREDICATE))]
+ "TARGET_XTHEADVECTOR"
+ "vext.x.v\t%0,%1,%2"
+ [(set_attr "type" "vimovvx")
+ (set_attr "mode" "<MODE>")])
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c
new file mode 100644
index 00000000000..4e192bbf025
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+
+** th.vadd\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+
+** th.vsb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out)
+{
+ vint32m1_t v = __riscv_th_vlb_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlb_v_i32m1_tu (v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vv_i32m1 (v2, v2, 4);
+ vint32m1_t v4 = __riscv_vadd_vv_i32m1_tu (v3, v2, v2, 4);
+ __riscv_th_vsb_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlb.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+
+** th.vadd\.vv\tv[1-9][0-9]?,\s*v[0-9]+,\s*v[0-9]+,\s*v0.t
+** th.vsb.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlb_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlb_v_i32m1_m (mask, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vv_i32m1 (v2, v2, 4);
+ vint32m1_t v4 = __riscv_vadd_vv_i32m1_m (mask, v3, v3, 4);
+ __riscv_th_vsb_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlb.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+
+** th.vadd\.vv\tv[1-9][0-9]?,\s*v[0-9]+,\s*v[0-9]+,\s*v0.t
+** th.vsb.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlb_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlb_v_i32m1_tumu (mask, v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vv_i32m1 (v2, v2, 4);
+ vint32m1_t v4 = __riscv_vadd_vv_i32m1_tumu (mask, v3, v2, v2, 4);
+ __riscv_th_vsb_v_i32m1 (out, v4, 4);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c
new file mode 100644
index 00000000000..1538afec68e
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlbu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlbu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vsb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, uint32_t x)
+{
+ vuint32m1_t v = __riscv_th_vlbu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlbu_v_u32m1_tu (v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tu (v3, v2, -16, 4);
+ __riscv_th_vsb_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlbu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsb.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlbu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlbu_v_u32m1_m (mask, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_m (mask, v3, -16, 4);
+ __riscv_th_vsb_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlbu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlbu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsb.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlbu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlbu_v_u32m1_tumu (mask, v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tumu (mask, v3, v2, -16, 4);
+ __riscv_th_vsb_v_u32m1 (out, v4, 4);
+}
\ No newline at end of file
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c
new file mode 100644
index 00000000000..bf4924a1d76
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vsh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, int32_t x)
+{
+ vint32m1_t v = __riscv_th_vlh_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlh_v_i32m1_tu (v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, -16, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_tu (v3, v2, -16, 4);
+ __riscv_th_vsh_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlh.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsh.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, int32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlh_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlh_v_i32m1_m (mask, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, -16, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_m (mask, v3, -16, 4);
+ __riscv_th_vsh_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlh.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsh.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, int32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlh_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlh_v_i32m1_tumu (mask, v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, -16, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_tumu (mask, v3, v2, -16, 4);
+ __riscv_th_vsh_v_i32m1 (out, v4, 4);
+}
\ No newline at end of file
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c
new file mode 100644
index 00000000000..8c451845175
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlhu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlhu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vsh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, uint32_t x)
+{
+ vuint32m1_t v = __riscv_th_vlhu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlhu_v_u32m1_tu (v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tu (v3, v2, -16, 4);
+ __riscv_th_vsh_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlhu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsh.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlhu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlhu_v_u32m1_m (mask, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_m (mask, v3, -16, 4);
+ __riscv_th_vsh_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlhu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlhu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsh.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlhu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlhu_v_u32m1_tumu (mask, v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tumu (mask, v3, v2, -16, 4);
+ __riscv_th_vsh_v_u32m1 (out, v4, 4);
+}
\ No newline at end of file
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c
new file mode 100644
index 00000000000..0f5b09684a5
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vx\tv[0-9]+,\s*v[0-9]+,\s*[a-x0-9]+
+** th.vadd\.vx\tv[0-9]+,\s*v[0-9]+,\s*[a-x0-9]+
+** th.vsw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, int32_t x)
+{
+ vint32m1_t v = __riscv_th_vlw_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlw_v_i32m1_tu (v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, x, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_tu (v3, v2, x, 4);
+ __riscv_th_vsw_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlw.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vx\tv[0-9]+,\s*v[0-9]+,\s*[a-x0-9]+
+** th.vadd\.vx\tv[1-9][0-9]?,\s*v[0-9]+,\s*[a-x0-9]+,\s*v0.t
+** th.vsw.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, int32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlw_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlw_v_i32m1_m (mask, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, x, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_m (mask, v3, x, 4);
+ __riscv_th_vsw_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlw.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vx\tv[0-9]+,\s*v[0-9]+,\s*[a-x0-9]+
+** th.vadd\.vx\tv[1-9][0-9]?,\s*v[0-9]+,\s*[a-x0-9]+,\s*v0.t
+** th.vsw.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, int32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlw_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlw_v_i32m1_tumu (mask, v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, x, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_tumu (mask, v3, v2, x, 4);
+ __riscv_th_vsw_v_i32m1 (out, v4, 4);
+}
\ No newline at end of file
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c
new file mode 100644
index 00000000000..aaa75be023d
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlwu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlwu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vsw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, uint32_t x)
+{
+ vuint32m1_t v = __riscv_th_vlwu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlwu_v_u32m1_tu (v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tu (v3, v2, -16, 4);
+ __riscv_th_vsw_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlwu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsw.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlwu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlwu_v_u32m1_m (mask, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_m (mask, v3, -16, 4);
+ __riscv_th_vsw_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlwu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlwu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsw.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlwu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlwu_v_u32m1_tumu (mask, v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tumu (mask, v3, v2, -16, 4);
+ __riscv_th_vsw_v_u32m1 (out, v4, 4);
+}
\ No newline at end of file
--
2.17.1
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re：Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
2024-01-10 11:06 ` Re：Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics joshua
@ 2024-01-10 11:08 ` joshua
2024-01-10 11:09 ` Re：[PATCH " juzhe.zhong
1 sibling, 0 replies; 17+ messages in thread
From: joshua @ 2024-01-10 11:08 UTC (permalink / raw)
To: joshua, juzhe.zhong, gcc-patches
Cc: Jim Wilson, palmer, andrew, philipp.tomsich, jeffreyalaw,
christoph.muellner, jinma, cooper.qu
Can you see the images that I sent to you in the last email？
If not, maybe you can refer to the last chapter in the thead spec.
------------------------------------------------------------------
发件人：joshua <cooper.joshua@linux.alibaba.com>
发送时间：2024年1月10日(星期三) 19:06
收件人："juzhe.zhong@rivai.ai"<juzhe.zhong@rivai.ai>; "gcc-patches"<gcc-patches@gcc.gnu.org>
抄 送：Jim Wilson<jim.wilson.gcc@gmail.com>; palmer<palmer@dabbelt.com>; andrew<andrew@sifive.com>; "philipp.tomsich"<philipp.tomsich@vrull.eu>; jeffreyalaw<jeffreyalaw@gmail.com>; "christoph.muellner"<christoph.muellner@vrull.eu>; jinma<jinma@linux.alibaba.com>; "cooper.qu"<cooper.qu@linux.alibaba.com>
主 题：Re：Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
The key difference between vlb/vlh/vlw is not output type too.
Their difference is the range of datatype, not one specific type.
We have dived into the xtheadvector special intrinsics and are
sure about that.
------------------------------------------------------------------
发件人：juzhe.zhong@rivai.ai <juzhe.zhong@rivai.ai>
发送时间：2024年1月10日(星期三) 19:00
收件人："cooper.joshua"<cooper.joshua@linux.alibaba.com>; "gcc-patches"<gcc-patches@gcc.gnu.org>
抄 送：Jim Wilson<jim.wilson.gcc@gmail.com>; palmer<palmer@dabbelt.com>; andrew<andrew@sifive.com>; "philipp.tomsich"<philipp.tomsich@vrull.eu>; jeffreyalaw<jeffreyalaw@gmail.com>; "christoph.muellner"<christoph.muellner@vrull.eu>; jinma<jinma@linux.alibaba.com>; "cooper.qu"<cooper.qu@linux.alibaba.com>
主 题：Re: Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
instance.op_info->args[i].get_tree_type (instance.type.index) is output type.
You can use GDB debug it .
juzhe.zhong@rivai.ai
发件人： joshua
发送时间： 2024-01-10 18:57
收件人： juzhe.zhong@rivai.ai; gcc-patches
抄送： Jim Wilson; palmer; andrew; philipp.tomsich; jeffreyalaw; christoph.muellner; jinma; cooper.qu
主题： Re：Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
Hi Juzhe,
Perhaps things are not as simple as imagined.
The differences between vlb/vlh/vlw is not the same
as vle8/vle16/vle32. "8", "16" or "32" in vle8/vle16/vle32
can be appended from "vle" according to input type.
But vlb/vlh/vlw is different not in input type.
------------------------------------------------------------------
发件人：juzhe.zhong@rivai.ai <juzhe.zhong@rivai.ai>
发送时间：2024年1月10日(星期三) 18:03
收件人："cooper.joshua"<cooper.joshua@linux.alibaba.com>; "gcc-patches"<gcc-patches@gcc.gnu.org>
抄 送：Jim Wilson<jim.wilson.gcc@gmail.com>; palmer<palmer@dabbelt.com>; andrew<andrew@sifive.com>; "philipp.tomsich"<philipp.tomsich@vrull.eu>; jeffreyalaw<jeffreyalaw@gmail.com>; "christoph.muellner"<christoph.muellner@vrull.eu>; jinma<jinma@linux.alibaba.com>; "cooper.qu"<cooper.qu@linux.alibaba.com>
主 题：Re: Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
I mean change these:
+DEF_RVV_FUNCTION (th_vlb, th_loadstore_width, full_preds, i8_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (th_vlh, th_loadstore_width, full_preds, i16_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (th_vlw, th_loadstore_width, full_preds, i32_v_scalar_const_ptr_ops)
into a single:
+DEF_RVV_FUNCTION (th_vl, th_loadstore_width, full_preds, all_v_scalar_const_ptr_ops)
and append "h", "w", or"b" according to
TYPE_UNSIGNED and
GET_MODE_BITSIZE (GET_MODE_INNER (TYPE_MODE (instance.op_info->args[i].get_tree_type (instance.type.index))))
in th_loadstore_width.
It should definitely works, I allow this flexibility in design of the framework.
juzhe.zhong@rivai.ai
发件人： joshua
发送时间： 2024-01-10 17:55
收件人： juzhe.zhong@rivai.ai; gcc-patches
抄送： Jim Wilson; palmer; andrew; philipp.tomsich; jeffreyalaw; christoph.muellner; jinma; cooper.qu
主题： Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
And revise th_loadstore_width, append the name according TYPE_UNSIGNED and
GET_MODE_BITSIZE (GET_MODE_INNER (TYPE_MODE (instance.op_info->args[i].get_tree_type (instance.type.index))))
What do you mean by it? I'm a bit confused.
Changing i8_v_scalar_const_ptr_ops into all_v_scalar_const_ptr_ops
will expand the datatypes that can be used in th_vlb. Can we restrict
again in th_loadstore_width?
------------------------------------------------------------------
发件人：juzhe.zhong@rivai.ai <juzhe.zhong@rivai.ai>
发送时间：2024年1月10日(星期三) 17:35
收件人："cooper.joshua"<cooper.joshua@linux.alibaba.com>; "gcc-patches"<gcc-patches@gcc.gnu.org>
抄 送：Jim Wilson<jim.wilson.gcc@gmail.com>; palmer<palmer@dabbelt.com>; andrew<andrew@sifive.com>; "philipp.tomsich"<philipp.tomsich@vrull.eu>; jeffreyalaw<jeffreyalaw@gmail.com>; "christoph.muellner"<christoph.muellner@vrull.eu>; "cooper.joshua"<cooper.joshua@linux.alibaba.com>; jinma<jinma@linux.alibaba.com>; "cooper.qu"<cooper.qu@linux.alibaba.com>
主 题：Re: [PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
+DEF_RVV_FUNCTION (th_vlb, th_loadstore_width, full_preds, i8_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (th_vlh, th_loadstore_width, full_preds, i16_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (th_vlw, th_loadstore_width, full_preds, i32_v_scalar_const_ptr_ops)
I think we should remove those many data structure you added like: i8_v_scalar_const_ptr_ops
Instead, you should use all_v_scalar_const_ptr_ops
And revise th_loadstore_width, append the name according TYPE_UNSIGNED and
GET_MODE_BITSIZE (GET_MODE_INNER (TYPE_MODE (instance.op_info->args[i].get_tree_type (instance.type.index))))
juzhe.zhong@rivai.ai
From: Jun Sha (Joshua)
Date: 2024-01-10 17:27
To: gcc-patches
CC: jim.wilson.gcc; palmer; andrew; philipp.tomsich; jeffreyalaw; christoph.muellner; juzhe.zhong; Jun Sha (Joshua); Jin Ma; Xianmiao Qu
Subject: [PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
This patch only involves the generation of xtheadvector
special load/store instructions and vext instructions.
gcc/ChangeLog:
* config/riscv/riscv-vector-builtins-bases.cc
(class th_loadstore_width): Define new builtin bases.
(BASE): Define new builtin bases.
* config/riscv/riscv-vector-builtins-bases.h:
Define new builtin class.
* config/riscv/riscv-vector-builtins-functions.def (vlsegff):
Include thead-vector-builtins-functions.def.
* config/riscv/riscv-vector-builtins-shapes.cc
(struct th_loadstore_width_def): Define new builtin shapes.
(struct th_indexed_loadstore_width_def):
Define new builtin shapes.
(SHAPE): Define new builtin shapes.
* config/riscv/riscv-vector-builtins-shapes.h:
Define new builtin shapes.
* config/riscv/riscv-vector-builtins-types.def
(DEF_RVV_I8_OPS): Add datatypes for XTheadVector.
(DEF_RVV_I16_OPS): Add datatypes for XTheadVector.
(DEF_RVV_I32_OPS): Add datatypes for XTheadVector.
(DEF_RVV_U8_OPS): Add datatypes for XTheadVector.
(DEF_RVV_U16_OPS): Add datatypes for XTheadVector.
(DEF_RVV_U32_OPS): Add datatypes for XTheadVector.
(vint8m1_t): Add datatypes for XTheadVector.
(vint8m2_t): Likewise.
(vint8m4_t): Likewise.
(vint8m8_t): Likewise.
(vint16m1_t): Likewise.
(vint16m2_t): Likewise.
(vint16m4_t): Likewise.
(vint16m8_t): Likewise.
(vint32m1_t): Likewise.
(vint32m2_t): Likewise.
(vint32m4_t): Likewise.
(vint32m8_t): Likewise.
(vint64m1_t): Likewise.
(vint64m2_t): Likewise.
(vint64m4_t): Likewise.
(vint64m8_t): Likewise.
(vuint8m1_t): Likewise.
(vuint8m2_t): Likewise.
(vuint8m4_t): Likewise.
(vuint8m8_t): Likewise.
(vuint16m1_t): Likewise.
(vuint16m2_t): Likewise.
(vuint16m4_t): Likewise.
(vuint16m8_t): Likewise.
(vuint32m1_t): Likewise.
(vuint32m2_t): Likewise.
(vuint32m4_t): Likewise.
(vuint32m8_t): Likewise.
(vuint64m1_t): Likewise.
(vuint64m2_t): Likewise.
(vuint64m4_t): Likewise.
(vuint64m8_t): Likewise.
* config/riscv/riscv-vector-builtins.cc
(DEF_RVV_I8_OPS): Add datatypes for XTheadVector.
(DEF_RVV_I16_OPS): Add datatypes for XTheadVector.
(DEF_RVV_I32_OPS): Add datatypes for XTheadVector.
(DEF_RVV_U8_OPS): Add datatypes for XTheadVector.
(DEF_RVV_U16_OPS): Add datatypes for XTheadVector.
(DEF_RVV_U32_OPS): Add datatypes for XTheadVector.
* config/riscv/thead-vector-builtins-functions.def: New file.
* config/riscv/thead-vector.md: Add new patterns.
gcc/testsuite/ChangeLog:
* gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c: New test.
Co-authored-by: Jin Ma <jinma@linux.alibaba.com>
Co-authored-by: Xianmiao Qu <cooper.qu@linux.alibaba.com>
Co-authored-by: Christoph Müllner <christoph.muellner@vrull.eu>
---
.../riscv/riscv-vector-builtins-bases.cc | 139 ++++++++
.../riscv/riscv-vector-builtins-bases.h | 31 ++
.../riscv/riscv-vector-builtins-shapes.cc | 98 ++++++
.../riscv/riscv-vector-builtins-shapes.h | 3 +
.../riscv/riscv-vector-builtins-types.def | 120 +++++++
gcc/config/riscv/riscv-vector-builtins.cc | 311 ++++++++++++++++++
gcc/config/riscv/riscv-vector-builtins.h | 3 +
gcc/config/riscv/t-riscv | 1 +
.../riscv/thead-vector-builtins-functions.def | 39 +++
gcc/config/riscv/thead-vector.md | 253 ++++++++++++++
.../riscv/rvv/xtheadvector/vlb-vsb.c | 68 ++++
.../riscv/rvv/xtheadvector/vlbu-vsb.c | 68 ++++
.../riscv/rvv/xtheadvector/vlh-vsh.c | 68 ++++
.../riscv/rvv/xtheadvector/vlhu-vsh.c | 68 ++++
.../riscv/rvv/xtheadvector/vlw-vsw.c | 68 ++++
.../riscv/rvv/xtheadvector/vlwu-vsw.c | 68 ++++
16 files changed, 1406 insertions(+)
create mode 100644 gcc/config/riscv/thead-vector-builtins-functions.def
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c
diff --git a/gcc/config/riscv/riscv-vector-builtins-bases.cc b/gcc/config/riscv/riscv-vector-builtins-bases.cc
index 46f1a1da33e..5f44f31a12b 100644
--- a/gcc/config/riscv/riscv-vector-builtins-bases.cc
+++ b/gcc/config/riscv/riscv-vector-builtins-bases.cc
@@ -2125,6 +2125,83 @@ public:
}
};
+/* Implements
+ * th.vl(b/h/w)[u].v/th.vs(b/h/w)[u].v/th.vls(b/h/w)[u].v/th.vss(b/h/w)[u].v/
+ * th.vlx(b/h/w)[u].v/th.vs[u]x(b/h/w).v
+ * codegen. */
+template<bool STORE_P, lst_type LST_TYPE, int UNSPEC>
+class th_loadstore_width : public function_base
+{
+public:
+ bool apply_tail_policy_p () const override { return !STORE_P; }
+ bool apply_mask_policy_p () const override { return !STORE_P; }
+
+ unsigned int call_properties (const function_instance &) const override
+ {
+ if (STORE_P)
+ return CP_WRITE_MEMORY;
+ else
+ return CP_READ_MEMORY;
+ }
+
+ bool can_be_overloaded_p (enum predication_type_index pred) const override
+ {
+ if (STORE_P || LST_TYPE == LST_INDEXED)
+ return true;
+ return pred != PRED_TYPE_none;
+ }
+
+ rtx expand (function_expander &e) const override
+ {
+ gcc_assert (TARGET_XTHEADVECTOR);
+ if (LST_TYPE == LST_INDEXED)
+ {
+ if (STORE_P)
+ return e.use_exact_insn (
+ code_for_pred_indexed_store_width (UNSPEC, UNSPEC,
+ e.vector_mode ()));
+ else
+ return e.use_exact_insn (
+ code_for_pred_indexed_load_width (UNSPEC, e.vector_mode ()));
+ }
+ else if (LST_TYPE == LST_STRIDED)
+ {
+ if (STORE_P)
+ return e.use_contiguous_store_insn (
+ code_for_pred_strided_store_width (UNSPEC, e.vector_mode ()));
+ else
+ return e.use_contiguous_load_insn (
+ code_for_pred_strided_load_width (UNSPEC, e.vector_mode ()));
+ }
+ else
+ {
+ if (STORE_P)
+ return e.use_contiguous_store_insn (
+ code_for_pred_store_width (UNSPEC, e.vector_mode ()));
+ else
+ return e.use_contiguous_load_insn (
+ code_for_pred_mov_width (UNSPEC, e.vector_mode ()));
+ }
+ }
+};
+
+/* Implements vext.x.v. */
+class th_extract : public function_base
+{
+public:
+ bool apply_vl_p () const override { return false; }
+ bool apply_tail_policy_p () const override { return false; }
+ bool apply_mask_policy_p () const override { return false; }
+ bool use_mask_predication_p () const override { return false; }
+ bool has_merge_operand_p () const override { return false; }
+
+ rtx expand (function_expander &e) const override
+ {
+ gcc_assert (TARGET_XTHEADVECTOR);
+ return e.use_exact_insn (code_for_pred_th_extract (e.vector_mode ()));
+ }
+};
+
/* Below implements are vector crypto */
/* Implements vandn.[vv,vx] */
class vandn : public function_base
@@ -2587,6 +2664,37 @@ static CONSTEXPR const seg_indexed_load<UNSPEC_ORDERED> vloxseg_obj;
static CONSTEXPR const seg_indexed_store<UNSPEC_UNORDERED> vsuxseg_obj;
static CONSTEXPR const seg_indexed_store<UNSPEC_ORDERED> vsoxseg_obj;
static CONSTEXPR const vlsegff vlsegff_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLB> th_vlb_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLBU> th_vlbu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLH> th_vlh_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLHU> th_vlhu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLW> th_vlw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLWU> th_vlwu_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_UNIT_STRIDE, UNSPEC_TH_VLB> th_vsb_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_UNIT_STRIDE, UNSPEC_TH_VLH> th_vsh_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_UNIT_STRIDE, UNSPEC_TH_VLW> th_vsw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSB> th_vlsb_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSBU> th_vlsbu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSH> th_vlsh_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSHU> th_vlshu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSW> th_vlsw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSWU> th_vlswu_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_STRIDED, UNSPEC_TH_VLSB> th_vssb_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_STRIDED, UNSPEC_TH_VLSH> th_vssh_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_STRIDED, UNSPEC_TH_VLSW> th_vssw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXB> th_vlxb_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXBU> th_vlxbu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXH> th_vlxh_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXHU> th_vlxhu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXW> th_vlxw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXWU> th_vlxwu_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VLXB> th_vsxb_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VLXH> th_vsxh_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VLXW> th_vsxw_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VSUXB> th_vsuxb_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VSUXH> th_vsuxh_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VSUXW> th_vsuxw_obj;
+static CONSTEXPR const th_extract th_vext_x_v_obj;
/* Crypto Vector */
static CONSTEXPR const vandn vandn_obj;
@@ -2878,6 +2986,37 @@ BASE (vloxseg)
BASE (vsuxseg)
BASE (vsoxseg)
BASE (vlsegff)
+BASE (th_vlb)
+BASE (th_vlh)
+BASE (th_vlw)
+BASE (th_vlbu)
+BASE (th_vlhu)
+BASE (th_vlwu)
+BASE (th_vsb)
+BASE (th_vsh)
+BASE (th_vsw)
+BASE (th_vlsb)
+BASE (th_vlsh)
+BASE (th_vlsw)
+BASE (th_vlsbu)
+BASE (th_vlshu)
+BASE (th_vlswu)
+BASE (th_vssb)
+BASE (th_vssh)
+BASE (th_vssw)
+BASE (th_vlxb)
+BASE (th_vlxh)
+BASE (th_vlxw)
+BASE (th_vlxbu)
+BASE (th_vlxhu)
+BASE (th_vlxwu)
+BASE (th_vsxb)
+BASE (th_vsxh)
+BASE (th_vsxw)
+BASE (th_vsuxb)
+BASE (th_vsuxh)
+BASE (th_vsuxw)
+BASE (th_vext_x_v)
/* Crypto vector */
BASE (vandn)
BASE (vbrev)
diff --git a/gcc/config/riscv/riscv-vector-builtins-bases.h b/gcc/config/riscv/riscv-vector-builtins-bases.h
index 1122e3801a7..df43adf9a17 100644
--- a/gcc/config/riscv/riscv-vector-builtins-bases.h
+++ b/gcc/config/riscv/riscv-vector-builtins-bases.h
@@ -299,6 +299,37 @@ extern const function_base *const vloxseg;
extern const function_base *const vsuxseg;
extern const function_base *const vsoxseg;
extern const function_base *const vlsegff;
+extern const function_base *const th_vlb;
+extern const function_base *const th_vlh;
+extern const function_base *const th_vlw;
+extern const function_base *const th_vlbu;
+extern const function_base *const th_vlhu;
+extern const function_base *const th_vlwu;
+extern const function_base *const th_vsb;
+extern const function_base *const th_vsh;
+extern const function_base *const th_vsw;
+extern const function_base *const th_vlsb;
+extern const function_base *const th_vlsh;
+extern const function_base *const th_vlsw;
+extern const function_base *const th_vlsbu;
+extern const function_base *const th_vlshu;
+extern const function_base *const th_vlswu;
+extern const function_base *const th_vssb;
+extern const function_base *const th_vssh;
+extern const function_base *const th_vssw;
+extern const function_base *const th_vlxb;
+extern const function_base *const th_vlxh;
+extern const function_base *const th_vlxw;
+extern const function_base *const th_vlxbu;
+extern const function_base *const th_vlxhu;
+extern const function_base *const th_vlxwu;
+extern const function_base *const th_vsxb;
+extern const function_base *const th_vsxh;
+extern const function_base *const th_vsxw;
+extern const function_base *const th_vsuxb;
+extern const function_base *const th_vsuxh;
+extern const function_base *const th_vsuxw;
+extern const function_base *const th_vext_x_v;
/* Below function_base are Vectro Crypto*/
extern const function_base *const vandn;
extern const function_base *const vbrev;
diff --git a/gcc/config/riscv/riscv-vector-builtins-shapes.cc b/gcc/config/riscv/riscv-vector-builtins-shapes.cc
index 1e4f4d53de6..489a95cf684 100644
--- a/gcc/config/riscv/riscv-vector-builtins-shapes.cc
+++ b/gcc/config/riscv/riscv-vector-builtins-shapes.cc
@@ -211,6 +211,86 @@ struct indexed_loadstore_def : public function_shape
}
};
+/* th_loadstore_width_def class. */
+struct th_loadstore_width_def : public build_base
+{
+ char *get_name (function_builder &b, const function_instance &instance,
+ bool overloaded_p) const override
+ {
+ /* Return nullptr if it can not be overloaded. */
+ if (overloaded_p && !instance.base->can_be_overloaded_p (instance.pred))
+ return nullptr;
+
+ b.append_base_name (instance.base_name);
+
+ /* vop_v --> vop_v_<type>. */
+ if (!overloaded_p)
+ {
+ /* vop --> vop_v. */
+ b.append_name (operand_suffixes[instance.op_info->op]);
+ /* vop_v --> vop_v_<type>. */
+ b.append_name (type_suffixes[instance.type.index].vector);
+ }
+
+ /* According to rvv-intrinsic-doc, it does not add "_m" suffix
+ for vop_m C++ overloaded API. */
+ if (overloaded_p && instance.pred == PRED_TYPE_m)
+ return b.finish_name ();
+ b.append_name (predication_suffixes[instance.pred]);
+ return b.finish_name ();
+ }
+};
+
+
+/* th_indexed_loadstore_width_def class. */
+struct th_indexed_loadstore_width_def : public function_shape
+{
+ void build (function_builder &b,
+ const function_group_info &group) const override
+ {
+ for (unsigned int pred_idx = 0; group.preds[pred_idx] != NUM_PRED_TYPES;
+ ++pred_idx)
+ {
+ for (unsigned int vec_type_idx = 0;
+ group.ops_infos.types[vec_type_idx].index != NUM_VECTOR_TYPES;
+ ++vec_type_idx)
+ {
+ tree index_type = group.ops_infos.args[1].get_tree_type (
+ group.ops_infos.types[vec_type_idx].index);
+ if (!index_type)
+ continue;
+ build_one (b, group, pred_idx, vec_type_idx);
+ }
+ }
+ }
+
+ char *get_name (function_builder &b, const function_instance &instance,
+ bool overloaded_p) const override
+ {
+
+ /* Return nullptr if it can not be overloaded. */
+ if (overloaded_p && !instance.base->can_be_overloaded_p (instance.pred))
+ return nullptr;
+
+ b.append_base_name (instance.base_name);
+ /* vop_v --> vop_v_<type>. */
+ if (!overloaded_p)
+ {
+ /* vop --> vop_v. */
+ b.append_name (operand_suffixes[instance.op_info->op]);
+ /* vop_v --> vop_v_<type>. */
+ b.append_name (type_suffixes[instance.type.index].vector);
+ }
+
+ /* According to rvv-intrinsic-doc, it does not add "_m" suffix
+ for vop_m C++ overloaded API. */
+ if (overloaded_p && instance.pred == PRED_TYPE_m)
+ return b.finish_name ();
+ b.append_name (predication_suffixes[instance.pred]);
+ return b.finish_name ();
+ }
+};
+
/* alu_def class. */
struct alu_def : public build_base
{
@@ -632,6 +712,21 @@ struct reduc_alu_def : public build_base
}
};
+/* th_extract_def class. */
+struct th_extract_def : public build_base
+{
+ char *get_name (function_builder &b, const function_instance &instance,
+ bool overloaded_p) const override
+ {
+ b.append_base_name (instance.base_name);
+ if (overloaded_p)
+ return b.finish_name ();
+ b.append_name (type_suffixes[instance.type.index].vector);
+ b.append_name (type_suffixes[instance.type.index].scalar);
+ return b.finish_name ();
+ }
+};
+
/* scalar_move_def class. */
struct scalar_move_def : public build_base
{
@@ -1094,6 +1189,8 @@ SHAPE(vsetvl, vsetvl)
SHAPE(vsetvl, vsetvlmax)
SHAPE(loadstore, loadstore)
SHAPE(indexed_loadstore, indexed_loadstore)
+SHAPE(th_loadstore_width, th_loadstore_width)
+SHAPE(th_indexed_loadstore_width, th_indexed_loadstore_width)
SHAPE(alu, alu)
SHAPE(alu_frm, alu_frm)
SHAPE(widen_alu, widen_alu)
@@ -1106,6 +1203,7 @@ SHAPE(move, move)
SHAPE(mask_alu, mask_alu)
SHAPE(reduc_alu, reduc_alu)
SHAPE(reduc_alu_frm, reduc_alu_frm)
+SHAPE(th_extract, th_extract)
SHAPE(scalar_move, scalar_move)
SHAPE(vundefined, vundefined)
SHAPE(misc, misc)
diff --git a/gcc/config/riscv/riscv-vector-builtins-shapes.h b/gcc/config/riscv/riscv-vector-builtins-shapes.h
index ac2a28ce017..a7624d0fabd 100644
--- a/gcc/config/riscv/riscv-vector-builtins-shapes.h
+++ b/gcc/config/riscv/riscv-vector-builtins-shapes.h
@@ -28,6 +28,8 @@ extern const function_shape *const vsetvl;
extern const function_shape *const vsetvlmax;
extern const function_shape *const loadstore;
extern const function_shape *const indexed_loadstore;
+extern const function_shape *const th_loadstore_width;
+extern const function_shape *const th_indexed_loadstore_width;
extern const function_shape *const alu;
extern const function_shape *const alu_frm;
extern const function_shape *const widen_alu;
@@ -41,6 +43,7 @@ extern const function_shape *const mask_alu;
extern const function_shape *const reduc_alu;
extern const function_shape *const reduc_alu_frm;
extern const function_shape *const scalar_move;
+extern const function_shape *const th_extract;
extern const function_shape *const vundefined;
extern const function_shape *const misc;
extern const function_shape *const vset;
diff --git a/gcc/config/riscv/riscv-vector-builtins-types.def b/gcc/config/riscv/riscv-vector-builtins-types.def
index 61019a56844..abfeb4fcd9b 100644
--- a/gcc/config/riscv/riscv-vector-builtins-types.def
+++ b/gcc/config/riscv/riscv-vector-builtins-types.def
@@ -24,12 +24,48 @@ along with GCC; see the file COPYING3. If not see
#define DEF_RVV_I_OPS(TYPE, REQUIRE)
#endif
+/* Use "DEF_RVV_I8_OPS" macro include some signed integer (i8/i16/i32/i64)
+ which will be iterated and registered as intrinsic functions. */
+#ifndef DEF_RVV_I8_OPS
+#define DEF_RVV_I8_OPS(TYPE, REQUIRE)
+#endif
+
+/* Use "DEF_RVV_I16_OPS" macro include some signed integer (i16/i32/i64)
+ which will be iterated and registered as intrinsic functions. */
+#ifndef DEF_RVV_I16_OPS
+#define DEF_RVV_I16_OPS(TYPE, REQUIRE)
+#endif
+
+/* Use "DEF_RVV_I32_OPS" macro include some signed integer (i32/i64)
+ which will be iterated and registered as intrinsic functions. */
+#ifndef DEF_RVV_I32_OPS
+#define DEF_RVV_I32_OPS(TYPE, REQUIRE)
+#endif
+
/* Use "DEF_RVV_U_OPS" macro include all unsigned integer which will be
iterated and registered as intrinsic functions. */
#ifndef DEF_RVV_U_OPS
#define DEF_RVV_U_OPS(TYPE, REQUIRE)
#endif
+/* Use "DEF_RVV_U8_OPS" macro include some unsigned integer (i8/i16/i32/i64)
+ which will be iterated and registered as intrinsic functions. */
+#ifndef DEF_RVV_U8_OPS
+#define DEF_RVV_U8_OPS(TYPE, REQUIRE)
+#endif
+
+/* Use "DEF_RVV_U16_OPS" macro include some unsigned integer (i16/i32/i64)
+ which will be iterated and registered as intrinsic functions. */
+#ifndef DEF_RVV_U16_OPS
+#define DEF_RVV_U16_OPS(TYPE, REQUIRE)
+#endif
+
+/* Use "DEF_RVV_U32_OPS" macro include some unsigned integer (i32/i64)
+ which will be iterated and registered as intrinsic functions. */
+#ifndef DEF_RVV_U32_OPS
+#define DEF_RVV_U32_OPS(TYPE, REQUIRE)
+#endif
+
/* Use "DEF_RVV_F_OPS" macro include all floating-point which will be
iterated and registered as intrinsic functions. */
#ifndef DEF_RVV_F_OPS
@@ -374,6 +410,45 @@ DEF_RVV_I_OPS (vint64m2_t, RVV_REQUIRE_ELEN_64)
DEF_RVV_I_OPS (vint64m4_t, RVV_REQUIRE_ELEN_64)
DEF_RVV_I_OPS (vint64m8_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_I8_OPS (vint8m1_t, 0)
+DEF_RVV_I8_OPS (vint8m2_t, 0)
+DEF_RVV_I8_OPS (vint8m4_t, 0)
+DEF_RVV_I8_OPS (vint8m8_t, 0)
+DEF_RVV_I8_OPS (vint16m1_t, 0)
+DEF_RVV_I8_OPS (vint16m2_t, 0)
+DEF_RVV_I8_OPS (vint16m4_t, 0)
+DEF_RVV_I8_OPS (vint16m8_t, 0)
+DEF_RVV_I8_OPS (vint32m1_t, 0)
+DEF_RVV_I8_OPS (vint32m2_t, 0)
+DEF_RVV_I8_OPS (vint32m4_t, 0)
+DEF_RVV_I8_OPS (vint32m8_t, 0)
+DEF_RVV_I8_OPS (vint64m1_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_I8_OPS (vint64m2_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_I8_OPS (vint64m4_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_I8_OPS (vint64m8_t, RVV_REQUIRE_ELEN_64)
+
+DEF_RVV_I16_OPS (vint16m1_t, 0)
+DEF_RVV_I16_OPS (vint16m2_t, 0)
+DEF_RVV_I16_OPS (vint16m4_t, 0)
+DEF_RVV_I16_OPS (vint16m8_t, 0)
+DEF_RVV_I16_OPS (vint32m1_t, 0)
+DEF_RVV_I16_OPS (vint32m2_t, 0)
+DEF_RVV_I16_OPS (vint32m4_t, 0)
+DEF_RVV_I16_OPS (vint32m8_t, 0)
+DEF_RVV_I16_OPS (vint64m1_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_I16_OPS (vint64m2_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_I16_OPS (vint64m4_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_I16_OPS (vint64m8_t, RVV_REQUIRE_ELEN_64)
+
+DEF_RVV_I32_OPS (vint32m1_t, 0)
+DEF_RVV_I32_OPS (vint32m2_t, 0)
+DEF_RVV_I32_OPS (vint32m4_t, 0)
+DEF_RVV_I32_OPS (vint32m8_t, 0)
+DEF_RVV_I32_OPS (vint64m1_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_I32_OPS (vint64m2_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_I32_OPS (vint64m4_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_I32_OPS (vint64m8_t, RVV_REQUIRE_ELEN_64)
+
DEF_RVV_U_OPS (vuint8mf8_t, RVV_REQUIRE_MIN_VLEN_64)
DEF_RVV_U_OPS (vuint8mf4_t, 0)
DEF_RVV_U_OPS (vuint8mf2_t, 0)
@@ -397,6 +472,45 @@ DEF_RVV_U_OPS (vuint64m2_t, RVV_REQUIRE_ELEN_64)
DEF_RVV_U_OPS (vuint64m4_t, RVV_REQUIRE_ELEN_64)
DEF_RVV_U_OPS (vuint64m8_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_U8_OPS (vuint8m1_t, 0)
+DEF_RVV_U8_OPS (vuint8m2_t, 0)
+DEF_RVV_U8_OPS (vuint8m4_t, 0)
+DEF_RVV_U8_OPS (vuint8m8_t, 0)
+DEF_RVV_U8_OPS (vuint16m1_t, 0)
+DEF_RVV_U8_OPS (vuint16m2_t, 0)
+DEF_RVV_U8_OPS (vuint16m4_t, 0)
+DEF_RVV_U8_OPS (vuint16m8_t, 0)
+DEF_RVV_U8_OPS (vuint32m1_t, 0)
+DEF_RVV_U8_OPS (vuint32m2_t, 0)
+DEF_RVV_U8_OPS (vuint32m4_t, 0)
+DEF_RVV_U8_OPS (vuint32m8_t, 0)
+DEF_RVV_U8_OPS (vuint64m1_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_U8_OPS (vuint64m2_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_U8_OPS (vuint64m4_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_U8_OPS (vuint64m8_t, RVV_REQUIRE_ELEN_64)
+
+DEF_RVV_U16_OPS (vuint16m1_t, 0)
+DEF_RVV_U16_OPS (vuint16m2_t, 0)
+DEF_RVV_U16_OPS (vuint16m4_t, 0)
+DEF_RVV_U16_OPS (vuint16m8_t, 0)
+DEF_RVV_U16_OPS (vuint32m1_t, 0)
+DEF_RVV_U16_OPS (vuint32m2_t, 0)
+DEF_RVV_U16_OPS (vuint32m4_t, 0)
+DEF_RVV_U16_OPS (vuint32m8_t, 0)
+DEF_RVV_U16_OPS (vuint64m1_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_U16_OPS (vuint64m2_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_U16_OPS (vuint64m4_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_U16_OPS (vuint64m8_t, RVV_REQUIRE_ELEN_64)
+
+DEF_RVV_U32_OPS (vuint32m1_t, 0)
+DEF_RVV_U32_OPS (vuint32m2_t, 0)
+DEF_RVV_U32_OPS (vuint32m4_t, 0)
+DEF_RVV_U32_OPS (vuint32m8_t, 0)
+DEF_RVV_U32_OPS (vuint64m1_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_U32_OPS (vuint64m2_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_U32_OPS (vuint64m4_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_U32_OPS (vuint64m8_t, RVV_REQUIRE_ELEN_64)
+
DEF_RVV_F_OPS (vfloat16mf4_t, RVV_REQUIRE_ELEN_FP_16 | RVV_REQUIRE_MIN_VLEN_64)
DEF_RVV_F_OPS (vfloat16mf2_t, RVV_REQUIRE_ELEN_FP_16)
DEF_RVV_F_OPS (vfloat16m1_t, RVV_REQUIRE_ELEN_FP_16)
@@ -1379,7 +1493,13 @@ DEF_RVV_CRYPTO_SEW64_OPS (vuint64m4_t, RVV_REQUIRE_ELEN_64)
DEF_RVV_CRYPTO_SEW64_OPS (vuint64m8_t, RVV_REQUIRE_ELEN_64)
#undef DEF_RVV_I_OPS
+#undef DEF_RVV_I8_OPS
+#undef DEF_RVV_I16_OPS
+#undef DEF_RVV_I32_OPS
#undef DEF_RVV_U_OPS
+#undef DEF_RVV_U8_OPS
+#undef DEF_RVV_U16_OPS
+#undef DEF_RVV_U32_OPS
#undef DEF_RVV_F_OPS
#undef DEF_RVV_B_OPS
#undef DEF_RVV_WEXTI_OPS
diff --git a/gcc/config/riscv/riscv-vector-builtins.cc b/gcc/config/riscv/riscv-vector-builtins.cc
index 25e0b6e56de..f429f12dc18 100644
--- a/gcc/config/riscv/riscv-vector-builtins.cc
+++ b/gcc/config/riscv/riscv-vector-builtins.cc
@@ -246,6 +246,63 @@ static const rvv_type_info iu_ops[] = {
#include "riscv-vector-builtins-types.def"
{NUM_VECTOR_TYPES, 0}};
+/* A list of all integer will be registered for intrinsic functions. */
+static const rvv_type_info i8_ops[] = {
+#define DEF_RVV_I8_OPS(TYPE, REQUIRE) {VECTOR_TYPE_##TYPE, REQUIRE},
+#include "riscv-vector-builtins-types.def"
+ {NUM_VECTOR_TYPES, 0}};
+
+/* A list of all integer will be registered for intrinsic functions. */
+static const rvv_type_info i16_ops[] = {
+#define DEF_RVV_I16_OPS(TYPE, REQUIRE) {VECTOR_TYPE_##TYPE, REQUIRE},
+#include "riscv-vector-builtins-types.def"
+ {NUM_VECTOR_TYPES, 0}};
+
+/* A list of all integer will be registered for intrinsic functions. */
+static const rvv_type_info i32_ops[] = {
+#define DEF_RVV_I32_OPS(TYPE, REQUIRE) {VECTOR_TYPE_##TYPE, REQUIRE},
+#include "riscv-vector-builtins-types.def"
+ {NUM_VECTOR_TYPES, 0}};
+
+/* A list of all integer will be registered for intrinsic functions. */
+static const rvv_type_info u8_ops[] = {
+#define DEF_RVV_U8_OPS(TYPE, REQUIRE) {VECTOR_TYPE_##TYPE, REQUIRE},
+#include "riscv-vector-builtins-types.def"
+ {NUM_VECTOR_TYPES, 0}};
+
+/* A list of all integer will be registered for intrinsic functions. */
+static const rvv_type_info u16_ops[] = {
+#define DEF_RVV_U16_OPS(TYPE, REQUIRE) {VECTOR_TYPE_##TYPE, REQUIRE},
+#include "riscv-vector-builtins-types.def"
+ {NUM_VECTOR_TYPES, 0}};
+
+/* A list of all integer will be registered for intrinsic functions. */
+static const rvv_type_info u32_ops[] = {
+#define DEF_RVV_U32_OPS(TYPE, REQUIRE) {VECTOR_TYPE_##TYPE, REQUIRE},
+#include "riscv-vector-builtins-types.def"
+ {NUM_VECTOR_TYPES, 0}};
+
+/* A list of all integer will be registered for intrinsic functions. */
+static const rvv_type_info iu8_ops[] = {
+#define DEF_RVV_I8_OPS(TYPE, REQUIRE) {VECTOR_TYPE_##TYPE, REQUIRE},
+#define DEF_RVV_U8_OPS(TYPE, REQUIRE) {VECTOR_TYPE_##TYPE, REQUIRE},
+#include "riscv-vector-builtins-types.def"
+ {NUM_VECTOR_TYPES, 0}};
+
+/* A list of all integer will be registered for intrinsic functions. */
+static const rvv_type_info iu16_ops[] = {
+#define DEF_RVV_I16_OPS(TYPE, REQUIRE) {VECTOR_TYPE_##TYPE, REQUIRE},
+#define DEF_RVV_U16_OPS(TYPE, REQUIRE) {VECTOR_TYPE_##TYPE, REQUIRE},
+#include "riscv-vector-builtins-types.def"
+ {NUM_VECTOR_TYPES, 0}};
+
+/* A list of all integer will be registered for intrinsic functions. */
+static const rvv_type_info iu32_ops[] = {
+#define DEF_RVV_I32_OPS(TYPE, REQUIRE) {VECTOR_TYPE_##TYPE, REQUIRE},
+#define DEF_RVV_U32_OPS(TYPE, REQUIRE) {VECTOR_TYPE_##TYPE, REQUIRE},
+#include "riscv-vector-builtins-types.def"
+ {NUM_VECTOR_TYPES, 0}};
+
/* A list of all types will be registered for intrinsic functions. */
static const rvv_type_info all_ops[] = {
#define DEF_RVV_I_OPS(TYPE, REQUIRE) {VECTOR_TYPE_##TYPE, REQUIRE},
@@ -934,6 +991,32 @@ static CONSTEXPR const rvv_arg_type_info ext_vcreate_args[]
= {rvv_arg_type_info (RVV_BASE_vector),
rvv_arg_type_info_end};
+/* A list of args for vector_type func (const scalar_type *, size_t)
+ * function. */
+static CONSTEXPR const rvv_arg_type_info scalar_const_ptr_size_args[]
+ = {rvv_arg_type_info (RVV_BASE_scalar_const_ptr),
+ rvv_arg_type_info (RVV_BASE_size), rvv_arg_type_info_end};
+
+/* A list of args for vector_type func (const scalar_type *, eew8_index_type)
+ * function. */
+static CONSTEXPR const rvv_arg_type_info scalar_const_ptr_index_args[]
+ = {rvv_arg_type_info (RVV_BASE_scalar_const_ptr),
+ rvv_arg_type_info (RVV_BASE_unsigned_vector), rvv_arg_type_info_end};
+
+/* A list of args for void func (scalar_type *, eew8_index_type, vector_type)
+ * function. */
+static CONSTEXPR const rvv_arg_type_info scalar_ptr_index_args[]
+ = {rvv_arg_type_info (RVV_BASE_scalar_ptr),
+ rvv_arg_type_info (RVV_BASE_unsigned_vector),
+ rvv_arg_type_info (RVV_BASE_vector), rvv_arg_type_info_end};
+
+/* A list of args for void func (scalar_type *, size_t, vector_type)
+ * function. */
+static CONSTEXPR const rvv_arg_type_info scalar_ptr_size_args[]
+ = {rvv_arg_type_info (RVV_BASE_scalar_ptr),
+ rvv_arg_type_info (RVV_BASE_size), rvv_arg_type_info (RVV_BASE_vector),
+ rvv_arg_type_info_end};
+
/* A list of none preds that will be registered for intrinsic functions. */
static CONSTEXPR const predication_type_index none_preds[]
= {PRED_TYPE_none, NUM_PRED_TYPES};
@@ -1455,6 +1538,14 @@ static CONSTEXPR const rvv_op_info iu_shift_vvv_ops
rvv_arg_type_info (RVV_BASE_vector), /* Return type */
shift_vv_args /* Args */};
+/* A static operand information for scalar_type func (vector_type, size_t)
+ * function registration. */
+static CONSTEXPR const rvv_op_info iu_x_s_u_ops
+ = {iu_ops, /* Types */
+ OP_TYPE_vx, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_scalar), /* Return type */
+ v_size_args /* Args */};
+
/* A static operand information for vector_type func (vector_type, size_t)
* function registration. */
static CONSTEXPR const rvv_op_info iu_shift_vvx_ops
@@ -2638,6 +2729,222 @@ static CONSTEXPR const rvv_op_info all_v_vcreate_lmul4_x2_ops
rvv_arg_type_info (RVV_BASE_vlmul_ext_x2), /* Return type */
ext_vcreate_args /* Args */};
+/* A static operand information for vector_type func (const scalar_type *)
+ * function registration. */
+static CONSTEXPR const rvv_op_info i8_v_scalar_const_ptr_ops
+ = {i8_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *)
+ * function registration. */
+static CONSTEXPR const rvv_op_info i16_v_scalar_const_ptr_ops
+ = {i16_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *)
+ * function registration. */
+static CONSTEXPR const rvv_op_info i32_v_scalar_const_ptr_ops
+ = {i32_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *)
+ * function registration. */
+static CONSTEXPR const rvv_op_info u8_v_scalar_const_ptr_ops
+ = {u8_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *)
+ * function registration. */
+static CONSTEXPR const rvv_op_info u16_v_scalar_const_ptr_ops
+ = {u16_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *)
+ * function registration. */
+static CONSTEXPR const rvv_op_info u32_v_scalar_const_ptr_ops
+ = {u32_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * size_t) function registration. */
+static CONSTEXPR const rvv_op_info i8_v_scalar_const_ptr_size_ops
+ = {i8_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_size_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * size_t) function registration. */
+static CONSTEXPR const rvv_op_info i16_v_scalar_const_ptr_size_ops
+ = {i16_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_size_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * size_t) function registration. */
+static CONSTEXPR const rvv_op_info i32_v_scalar_const_ptr_size_ops
+ = {i32_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_size_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * size_t) function registration. */
+static CONSTEXPR const rvv_op_info u8_v_scalar_const_ptr_size_ops
+ = {u8_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_size_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * size_t) function registration. */
+static CONSTEXPR const rvv_op_info u16_v_scalar_const_ptr_size_ops
+ = {u16_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_size_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * size_t) function registration. */
+static CONSTEXPR const rvv_op_info u32_v_scalar_const_ptr_size_ops
+ = {u32_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_size_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * eew8_index_type) function registration. */
+static CONSTEXPR const rvv_op_info i8_v_scalar_const_ptr_index_ops
+ = {i8_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_index_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * eew8_index_type) function registration. */
+static CONSTEXPR const rvv_op_info u8_v_scalar_const_ptr_index_ops
+ = {u8_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_index_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * eew8_index_type) function registration. */
+static CONSTEXPR const rvv_op_info i16_v_scalar_const_ptr_index_ops
+ = {i16_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_index_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * eew8_index_type) function registration. */
+static CONSTEXPR const rvv_op_info u16_v_scalar_const_ptr_index_ops
+ = {u16_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_index_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * eew8_index_type) function registration. */
+static CONSTEXPR const rvv_op_info i32_v_scalar_const_ptr_index_ops
+ = {i32_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_index_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * eew8_index_type) function registration. */
+static CONSTEXPR const rvv_op_info u32_v_scalar_const_ptr_index_ops
+ = {u32_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_index_args /* Args */};
+
+/* A static operand information for void func (scalar_type *, eew8_index_type,
+ * vector_type) function registration. */
+static CONSTEXPR const rvv_op_info iu8_v_scalar_ptr_index_ops
+ = {iu8_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_void), /* Return type */
+ scalar_ptr_index_args /* Args */};
+
+/* A static operand information for void func (scalar_type *, eew16_index_type,
+ * vector_type) function registration. */
+static CONSTEXPR const rvv_op_info iu16_v_scalar_ptr_index_ops
+ = {iu16_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_void), /* Return type */
+ scalar_ptr_index_args /* Args */};
+
+/* A static operand information for void func (scalar_type *, eew32_index_type,
+ * vector_type) function registration. */
+static CONSTEXPR const rvv_op_info iu32_v_scalar_ptr_index_ops
+ = {iu32_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_void), /* Return type */
+ scalar_ptr_index_args /* Args */};
+
+/* A static operand information for void func (scalar_type *, vector_type,
+ * function registration. */
+static CONSTEXPR const rvv_op_info iu8_v_scalar_ptr_ops
+ = {iu8_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_void), /* Return type */
+ scalar_ptr_args /* Args */};
+
+/* A static operand information for void func (scalar_type *, vector_type)
+ * function registration. */
+static CONSTEXPR const rvv_op_info iu16_v_scalar_ptr_ops
+ = {iu16_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_void), /* Return type */
+ scalar_ptr_args /* Args */};
+
+/* A static operand information for void func (scalar_type *, vector_type)
+ * function registration. */
+static CONSTEXPR const rvv_op_info iu32_v_scalar_ptr_ops
+ = {iu32_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_void), /* Return type */
+ scalar_ptr_args /* Args */};
+
+/* A static operand information for void func (scalar_type *, size_t,
+ * vector_type) function registration. */
+static CONSTEXPR const rvv_op_info iu8_v_scalar_ptr_size_ops
+ = {iu8_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_void), /* Return type */
+ scalar_ptr_size_args /* Args */};
+
+/* A static operand information for void func (scalar_type *, size_t,
+ * vector_type) function registration. */
+static CONSTEXPR const rvv_op_info iu16_v_scalar_ptr_size_ops
+ = {iu16_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_void), /* Return type */
+ scalar_ptr_size_args /* Args */};
+
+/* A static operand information for void func (scalar_type *, size_t,
+ * vector_type) function registration. */
+static CONSTEXPR const rvv_op_info iu32_v_scalar_ptr_size_ops
+ = {iu32_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_void), /* Return type */
+ scalar_ptr_size_args /* Args */};
+
/* A static operand information for vector_type func (vector_type).
Some ins just supports SEW=32, such as crypto vectol Zvkg extension.
* function registration. */
@@ -2816,6 +3123,10 @@ static function_group_info function_groups[] = {
#define DEF_RVV_FUNCTION(NAME, SHAPE, PREDS, OPS_INFO) \
{#NAME, &bases::NAME, &shapes::SHAPE, PREDS, OPS_INFO, REQUIRED_EXTENSIONS},
#include "riscv-vector-builtins-functions.def"
+#undef DEF_RVV_FUNCTION
+#define DEF_RVV_FUNCTION(NAME, SHAPE, PREDS, OPS_INFO) \
+ {#NAME, &bases::NAME, &shapes::SHAPE, PREDS, OPS_INFO, REQUIRED_EXTENSIONS},
+#include "thead-vector-builtins-functions.def"
};
/* The RVV types, with their built-in
diff --git a/gcc/config/riscv/riscv-vector-builtins.h b/gcc/config/riscv/riscv-vector-builtins.h
index 54c8824ff92..a8ee39a3cb2 100644
--- a/gcc/config/riscv/riscv-vector-builtins.h
+++ b/gcc/config/riscv/riscv-vector-builtins.h
@@ -114,6 +114,7 @@ static const unsigned int CP_WRITE_CSR = 1U << 5;
enum required_ext
{
VECTOR_EXT, /* Vector extension */
+ XTHEADVECTOR_EXT, /* XTheadVector extension */
ZVBB_EXT, /* Cryto vector Zvbb sub-ext */
ZVBB_OR_ZVKB_EXT, /* Cryto vector Zvbb or zvkb sub-ext */
ZVBC_EXT, /* Crypto vector Zvbc sub-ext */
@@ -234,6 +235,8 @@ struct function_group_info
{
case VECTOR_EXT:
return TARGET_VECTOR;
+ case XTHEADVECTOR_EXT:
+ return TARGET_XTHEADVECTOR;
case ZVBB_EXT:
return TARGET_ZVBB;
case ZVBB_OR_ZVKB_EXT:
diff --git a/gcc/config/riscv/t-riscv b/gcc/config/riscv/t-riscv
index 32de6b851c1..38494320d8b 100644
--- a/gcc/config/riscv/t-riscv
+++ b/gcc/config/riscv/t-riscv
@@ -1,6 +1,7 @@
RISCV_BUILTINS_H = $(srcdir)/config/riscv/riscv-vector-builtins.h \
$(srcdir)/config/riscv/riscv-vector-builtins.def \
$(srcdir)/config/riscv/riscv-vector-builtins-functions.def \
+ $(srcdir)/config/riscv/thead-vector-builtins-functions.def \
riscv-vector-type-indexer.gen.def
riscv-builtins.o: $(srcdir)/config/riscv/riscv-builtins.cc $(CONFIG_H) \
diff --git a/gcc/config/riscv/thead-vector-builtins-functions.def b/gcc/config/riscv/thead-vector-builtins-functions.def
new file mode 100644
index 00000000000..667820d4c3e
--- /dev/null
+++ b/gcc/config/riscv/thead-vector-builtins-functions.def
@@ -0,0 +1,39 @@
+#ifndef DEF_RVV_FUNCTION
+#define DEF_RVV_FUNCTION(NAME, SHAPE, PREDS, OPS_INFO)
+#endif
+
+#define REQUIRED_EXTENSIONS XTHEADVECTOR_EXT
+DEF_RVV_FUNCTION (th_vlb, th_loadstore_width, full_preds, i8_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (th_vlh, th_loadstore_width, full_preds, i16_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (th_vlw, th_loadstore_width, full_preds, i32_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (th_vlbu, th_loadstore_width, full_preds, u8_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (th_vlhu, th_loadstore_width, full_preds, u16_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (th_vlwu, th_loadstore_width, full_preds, u32_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (th_vsb, th_loadstore_width, none_m_preds, iu8_v_scalar_ptr_ops)
+DEF_RVV_FUNCTION (th_vsh, th_loadstore_width, none_m_preds, iu16_v_scalar_ptr_ops)
+DEF_RVV_FUNCTION (th_vsw, th_loadstore_width, none_m_preds, iu32_v_scalar_ptr_ops)
+DEF_RVV_FUNCTION (th_vlsb, th_loadstore_width, full_preds, i8_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (th_vlsh, th_loadstore_width, full_preds, i16_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (th_vlsw, th_loadstore_width, full_preds, i32_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (th_vlsbu, th_loadstore_width, full_preds, u8_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (th_vlshu, th_loadstore_width, full_preds, u16_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (th_vlswu, th_loadstore_width, full_preds, u32_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (th_vssb, th_loadstore_width, none_m_preds, iu8_v_scalar_ptr_size_ops)
+DEF_RVV_FUNCTION (th_vssh, th_loadstore_width, none_m_preds, iu16_v_scalar_ptr_size_ops)
+DEF_RVV_FUNCTION (th_vssw, th_loadstore_width, none_m_preds, iu32_v_scalar_ptr_size_ops)
+DEF_RVV_FUNCTION (th_vlxb, th_indexed_loadstore_width, full_preds, i8_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (th_vlxh, th_indexed_loadstore_width, full_preds, i16_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (th_vlxw, th_indexed_loadstore_width, full_preds, i32_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (th_vlxbu, th_indexed_loadstore_width, full_preds, u8_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (th_vlxhu, th_indexed_loadstore_width, full_preds, u16_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (th_vlxwu, th_indexed_loadstore_width, full_preds, u32_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (th_vsxb, th_indexed_loadstore_width, none_m_preds, iu8_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (th_vsxh, th_indexed_loadstore_width, none_m_preds, iu16_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (th_vsxw, th_indexed_loadstore_width, none_m_preds, iu32_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (th_vsuxb, th_indexed_loadstore_width, none_m_preds, iu8_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (th_vsuxh, th_indexed_loadstore_width, none_m_preds, iu16_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (th_vsuxw, th_indexed_loadstore_width, none_m_preds, iu32_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (th_vext_x_v, th_extract, none_preds, iu_x_s_u_ops)
+#undef REQUIRED_EXTENSIONS
+
+#undef DEF_RVV_FUNCTION
diff --git a/gcc/config/riscv/thead-vector.md b/gcc/config/riscv/thead-vector.md
index 696b815252d..0f3700d9269 100644
--- a/gcc/config/riscv/thead-vector.md
+++ b/gcc/config/riscv/thead-vector.md
@@ -1,7 +1,95 @@
(define_c_enum "unspec" [
+ UNSPEC_TH_VLB
+ UNSPEC_TH_VLBU
+ UNSPEC_TH_VLH
+ UNSPEC_TH_VLHU
+ UNSPEC_TH_VLW
+ UNSPEC_TH_VLWU
+
+ UNSPEC_TH_VLSB
+ UNSPEC_TH_VLSBU
+ UNSPEC_TH_VLSH
+ UNSPEC_TH_VLSHU
+ UNSPEC_TH_VLSW
+ UNSPEC_TH_VLSWU
+
+ UNSPEC_TH_VLXB
+ UNSPEC_TH_VLXBU
+ UNSPEC_TH_VLXH
+ UNSPEC_TH_VLXHU
+ UNSPEC_TH_VLXW
+ UNSPEC_TH_VLXWU
+
+ UNSPEC_TH_VSUXB
+ UNSPEC_TH_VSUXH
+ UNSPEC_TH_VSUXW
+
UNSPEC_TH_VWLDST
])
+(define_int_iterator UNSPEC_TH_VLMEM_OP [
+ UNSPEC_TH_VLB UNSPEC_TH_VLBU
+ UNSPEC_TH_VLH UNSPEC_TH_VLHU
+ UNSPEC_TH_VLW UNSPEC_TH_VLWU
+])
+
+(define_int_iterator UNSPEC_TH_VLSMEM_OP [
+ UNSPEC_TH_VLSB UNSPEC_TH_VLSBU
+ UNSPEC_TH_VLSH UNSPEC_TH_VLSHU
+ UNSPEC_TH_VLSW UNSPEC_TH_VLSWU
+])
+
+(define_int_iterator UNSPEC_TH_VLXMEM_OP [
+ UNSPEC_TH_VLXB UNSPEC_TH_VLXBU
+ UNSPEC_TH_VLXH UNSPEC_TH_VLXHU
+ UNSPEC_TH_VLXW UNSPEC_TH_VLXWU
+])
+
+(define_int_attr vlmem_op_attr [
+ (UNSPEC_TH_VLB "b") (UNSPEC_TH_VLBU "bu")
+ (UNSPEC_TH_VLH "h") (UNSPEC_TH_VLHU "hu")
+ (UNSPEC_TH_VLW "w") (UNSPEC_TH_VLWU "wu")
+ (UNSPEC_TH_VLSB "b") (UNSPEC_TH_VLSBU "bu")
+ (UNSPEC_TH_VLSH "h") (UNSPEC_TH_VLSHU "hu")
+ (UNSPEC_TH_VLSW "w") (UNSPEC_TH_VLSWU "wu")
+ (UNSPEC_TH_VLXB "b") (UNSPEC_TH_VLXBU "bu")
+ (UNSPEC_TH_VLXH "h") (UNSPEC_TH_VLXHU "hu")
+ (UNSPEC_TH_VLXW "w") (UNSPEC_TH_VLXWU "wu")
+ (UNSPEC_TH_VSUXB "b")
+ (UNSPEC_TH_VSUXH "h")
+ (UNSPEC_TH_VSUXW "w")
+])
+
+(define_int_attr vlmem_order_attr [
+ (UNSPEC_TH_VLXB "")
+ (UNSPEC_TH_VLXH "")
+ (UNSPEC_TH_VLXW "")
+ (UNSPEC_TH_VSUXB "u")
+ (UNSPEC_TH_VSUXH "u")
+ (UNSPEC_TH_VSUXW "u")
+])
+
+(define_int_iterator UNSPEC_TH_VSMEM_OP [
+ UNSPEC_TH_VLB
+ UNSPEC_TH_VLH
+ UNSPEC_TH_VLW
+])
+
+(define_int_iterator UNSPEC_TH_VSSMEM_OP [
+ UNSPEC_TH_VLSB
+ UNSPEC_TH_VLSH
+ UNSPEC_TH_VLSW
+])
+
+(define_int_iterator UNSPEC_TH_VSXMEM_OP [
+ UNSPEC_TH_VLXB
+ UNSPEC_TH_VLXH
+ UNSPEC_TH_VLXW
+ UNSPEC_TH_VSUXB
+ UNSPEC_TH_VSUXH
+ UNSPEC_TH_VSUXW
+])
+
(define_mode_iterator V_VLS_VT [V VLS VT])
(define_mode_iterator V_VB_VLS_VT [V VB VLS VT])
@@ -100,3 +188,168 @@
}
[(set_attr "type" "vldm,vstm,vmalu,vmalu,vmalu")
(set_attr "mode" "<MODE>")])
+
+(define_expand "@pred_mov_width<vlmem_op_attr><mode>"
+ [(set (match_operand:V_VLS 0 "nonimmediate_operand")
+ (if_then_else:V_VLS
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand")
+ (match_operand 4 "vector_length_operand")
+ (match_operand 5 "const_int_operand")
+ (match_operand 6 "const_int_operand")
+ (match_operand 7 "const_int_operand")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLMEM_OP)
+ (match_operand:V_VLS 3 "vector_move_operand")
+ (match_operand:V_VLS 2 "vector_merge_operand")))]
+ "TARGET_XTHEADVECTOR"
+ {})
+
+(define_insn_and_split "*pred_mov_width<vlmem_op_attr><mode>"
+ [(set (match_operand:V_VLS 0 "nonimmediate_operand" "=vr, vr, vd, m, vr, vr")
+ (if_then_else:V_VLS
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1, Wc1, vm, vmWc1, Wc1, Wc1")
+ (match_operand 4 "vector_length_operand" " rK, rK, rK, rK, rK, rK")
+ (match_operand 5 "const_int_operand" " i, i, i, i, i, i")
+ (match_operand 6 "const_int_operand" " i, i, i, i, i, i")
+ (match_operand 7 "const_int_operand" " i, i, i, i, i, i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLMEM_OP)
+ (match_operand:V_VLS 3 "reg_or_mem_operand" " m, m, m, vr, vr, vr")
+ (match_operand:V_VLS 2 "vector_merge_operand" " 0, vu, vu, vu, vu, 0")))]
+ "(TARGET_XTHEADVECTOR
+ && (register_operand (operands[0], <MODE>mode)
+ || register_operand (operands[3], <MODE>mode)))"
+ "@
+ vl<vlmem_op_attr>.v\t%0,%3%p1
+ vl<vlmem_op_attr>.v\t%0,%3
+ vl<vlmem_op_attr>.v\t%0,%3,%1.t
+ vs<vlmem_op_attr>.v\t%3,%0%p1
+ vmv.v.v\t%0,%3
+ vmv.v.v\t%0,%3"
+ "&& register_operand (operands[0], <MODE>mode)
+ && register_operand (operands[3], <MODE>mode)
+ && satisfies_constraint_vu (operands[2])
+ && INTVAL (operands[7]) == riscv_vector::VLMAX"
+ [(set (match_dup 0) (match_dup 3))]
+ ""
+ [(set_attr "type" "vlde,vlde,vlde,vste,vimov,vimov")
+ (set_attr "mode" "<MODE>")])
+
+(define_insn "@pred_store_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "memory_operand" "+m")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1")
+ (match_operand 3 "vector_length_operand" " rK")
+ (match_operand 4 "const_int_operand" " i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VSMEM_OP)
+ (match_operand:VI 2 "register_operand" " vr")
+ (match_dup 0)))]
+ "TARGET_XTHEADVECTOR"
+ "vs<vlmem_op_attr>.v\t%2,%0%p1"
+ [(set_attr "type" "vste")
+ (set_attr "mode" "<MODE>")
+ (set (attr "avl_type_idx") (const_int 4))
+ (set_attr "vl_op_idx" "3")])
+
+(define_insn "@pred_strided_load_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "register_operand" "=vr, vr, vd")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1, Wc1, vm")
+ (match_operand 5 "vector_length_operand" " rK, rK, rK")
+ (match_operand 6 "const_int_operand" " i, i, i")
+ (match_operand 7 "const_int_operand" " i, i, i")
+ (match_operand 8 "const_int_operand" " i, i, i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLSMEM_OP)
+ (unspec:VI
+ [(match_operand:VI 3 "memory_operand" " m, m, m")
+ (match_operand 4 "pmode_reg_or_0_operand" " rJ, rJ, rJ")] UNSPEC_TH_VLSMEM_OP)
+ (match_operand:VI 2 "vector_merge_operand" " 0, vu, vu")))]
+ "TARGET_XTHEADVECTOR"
+ "vls<vlmem_op_attr>.v\t%0,%3,%z4%p1"
+ [(set_attr "type" "vlds")
+ (set_attr "mode" "<MODE>")])
+
+(define_insn "@pred_strided_store_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "memory_operand" "+m")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1")
+ (match_operand 4 "vector_length_operand" " rK")
+ (match_operand 5 "const_int_operand" " i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VSSMEM_OP)
+ (unspec:VI
+ [(match_operand 2 "pmode_reg_or_0_operand" " rJ")
+ (match_operand:VI 3 "register_operand" " vr")] UNSPEC_TH_VSSMEM_OP)
+ (match_dup 0)))]
+ "TARGET_XTHEADVECTOR"
+ "vss<vlmem_op_attr>.v\t%3,%0,%z2%p1"
+ [(set_attr "type" "vsts")
+ (set_attr "mode" "<MODE>")
+ (set (attr "avl_type_idx") (const_int 5))])
+
+(define_insn "@pred_indexed_load_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "register_operand" "=vd, vr,vd, vr")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" " vm,Wc1,vm,Wc1")
+ (match_operand 5 "vector_length_operand" " rK, rK,rK, rK")
+ (match_operand 6 "const_int_operand" " i, i, i, i")
+ (match_operand 7 "const_int_operand" " i, i, i, i")
+ (match_operand 8 "const_int_operand" " i, i, i, i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLXMEM_OP)
+ (unspec:VI
+ [(match_operand 3 "pmode_reg_or_0_operand" " rJ, rJ,rJ, rJ")
+ (mem:BLK (scratch))
+ (match_operand:VI 4 "register_operand" " vr, vr,vr, vr")] UNSPEC_TH_VLXMEM_OP)
+ (match_operand:VI 2 "vector_merge_operand" " vu, vu, 0, 0")))]
+ "TARGET_XTHEADVECTOR"
+ "vlx<vlmem_op_attr>.v\t%0,(%z3),%4%p1"
+ [(set_attr "type" "vldux")
+ (set_attr "mode" "<MODE>")])
+
+(define_insn "@pred_indexed_<vlmem_order_attr>store_width<vlmem_op_attr><mode>"
+ [(set (mem:BLK (scratch))
+ (unspec:BLK
+ [(unspec:<VM>
+ [(match_operand:<VM> 0 "vector_mask_operand" "vmWc1")
+ (match_operand 4 "vector_length_operand" " rK")
+ (match_operand 5 "const_int_operand" " i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VSXMEM_OP)
+ (match_operand 1 "pmode_reg_or_0_operand" " rJ")
+ (match_operand:VI 2 "register_operand" " vr")
+ (match_operand:VI 3 "register_operand" " vr")] UNSPEC_TH_VSXMEM_OP))]
+ "TARGET_XTHEADVECTOR"
+ "vs<vlmem_order_attr>x<vlmem_op_attr>.v\t%3,(%z1),%2%p0"
+ [(set_attr "type" "vstux")
+ (set_attr "mode" "<MODE>")])
+
+(define_expand "@pred_th_extract<mode>"
+ [(set (match_operand:<VEL> 0 "register_operand")
+ (unspec:<VEL>
+ [(vec_select:<VEL>
+ (match_operand:V_VLSI 1 "register_operand")
+ (parallel [(match_operand:DI 2 "register_operand" "r")]))
+ (reg:SI VTYPE_REGNUM)] UNSPEC_VPREDICATE))]
+ "TARGET_XTHEADVECTOR"
+{})
+
+(define_insn "*pred_th_extract<mode>"
+ [(set (match_operand:<VEL> 0 "register_operand" "=r")
+ (unspec:<VEL>
+ [(vec_select:<VEL>
+ (match_operand:V_VLSI 1 "register_operand" "vr")
+ (parallel [(match_operand:DI 2 "register_operand" "r")]))
+ (reg:SI VTYPE_REGNUM)] UNSPEC_VPREDICATE))]
+ "TARGET_XTHEADVECTOR"
+ "vext.x.v\t%0,%1,%2"
+ [(set_attr "type" "vimovvx")
+ (set_attr "mode" "<MODE>")])
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c
new file mode 100644
index 00000000000..4e192bbf025
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+
+** th.vadd\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+
+** th.vsb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out)
+{
+ vint32m1_t v = __riscv_th_vlb_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlb_v_i32m1_tu (v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vv_i32m1 (v2, v2, 4);
+ vint32m1_t v4 = __riscv_vadd_vv_i32m1_tu (v3, v2, v2, 4);
+ __riscv_th_vsb_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlb.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+
+** th.vadd\.vv\tv[1-9][0-9]?,\s*v[0-9]+,\s*v[0-9]+,\s*v0.t
+** th.vsb.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlb_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlb_v_i32m1_m (mask, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vv_i32m1 (v2, v2, 4);
+ vint32m1_t v4 = __riscv_vadd_vv_i32m1_m (mask, v3, v3, 4);
+ __riscv_th_vsb_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlb.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+
+** th.vadd\.vv\tv[1-9][0-9]?,\s*v[0-9]+,\s*v[0-9]+,\s*v0.t
+** th.vsb.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlb_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlb_v_i32m1_tumu (mask, v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vv_i32m1 (v2, v2, 4);
+ vint32m1_t v4 = __riscv_vadd_vv_i32m1_tumu (mask, v3, v2, v2, 4);
+ __riscv_th_vsb_v_i32m1 (out, v4, 4);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c
new file mode 100644
index 00000000000..1538afec68e
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlbu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlbu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vsb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, uint32_t x)
+{
+ vuint32m1_t v = __riscv_th_vlbu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlbu_v_u32m1_tu (v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tu (v3, v2, -16, 4);
+ __riscv_th_vsb_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlbu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsb.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlbu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlbu_v_u32m1_m (mask, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_m (mask, v3, -16, 4);
+ __riscv_th_vsb_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlbu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlbu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsb.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlbu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlbu_v_u32m1_tumu (mask, v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tumu (mask, v3, v2, -16, 4);
+ __riscv_th_vsb_v_u32m1 (out, v4, 4);
+}
\ No newline at end of file
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c
new file mode 100644
index 00000000000..bf4924a1d76
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vsh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, int32_t x)
+{
+ vint32m1_t v = __riscv_th_vlh_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlh_v_i32m1_tu (v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, -16, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_tu (v3, v2, -16, 4);
+ __riscv_th_vsh_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlh.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsh.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, int32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlh_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlh_v_i32m1_m (mask, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, -16, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_m (mask, v3, -16, 4);
+ __riscv_th_vsh_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlh.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsh.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, int32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlh_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlh_v_i32m1_tumu (mask, v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, -16, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_tumu (mask, v3, v2, -16, 4);
+ __riscv_th_vsh_v_i32m1 (out, v4, 4);
+}
\ No newline at end of file
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c
new file mode 100644
index 00000000000..8c451845175
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlhu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlhu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vsh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, uint32_t x)
+{
+ vuint32m1_t v = __riscv_th_vlhu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlhu_v_u32m1_tu (v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tu (v3, v2, -16, 4);
+ __riscv_th_vsh_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlhu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsh.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlhu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlhu_v_u32m1_m (mask, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_m (mask, v3, -16, 4);
+ __riscv_th_vsh_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlhu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlhu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsh.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlhu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlhu_v_u32m1_tumu (mask, v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tumu (mask, v3, v2, -16, 4);
+ __riscv_th_vsh_v_u32m1 (out, v4, 4);
+}
\ No newline at end of file
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c
new file mode 100644
index 00000000000..0f5b09684a5
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vx\tv[0-9]+,\s*v[0-9]+,\s*[a-x0-9]+
+** th.vadd\.vx\tv[0-9]+,\s*v[0-9]+,\s*[a-x0-9]+
+** th.vsw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, int32_t x)
+{
+ vint32m1_t v = __riscv_th_vlw_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlw_v_i32m1_tu (v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, x, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_tu (v3, v2, x, 4);
+ __riscv_th_vsw_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlw.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vx\tv[0-9]+,\s*v[0-9]+,\s*[a-x0-9]+
+** th.vadd\.vx\tv[1-9][0-9]?,\s*v[0-9]+,\s*[a-x0-9]+,\s*v0.t
+** th.vsw.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, int32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlw_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlw_v_i32m1_m (mask, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, x, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_m (mask, v3, x, 4);
+ __riscv_th_vsw_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlw.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vx\tv[0-9]+,\s*v[0-9]+,\s*[a-x0-9]+
+** th.vadd\.vx\tv[1-9][0-9]?,\s*v[0-9]+,\s*[a-x0-9]+,\s*v0.t
+** th.vsw.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, int32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlw_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlw_v_i32m1_tumu (mask, v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, x, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_tumu (mask, v3, v2, x, 4);
+ __riscv_th_vsw_v_i32m1 (out, v4, 4);
+}
\ No newline at end of file
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c
new file mode 100644
index 00000000000..aaa75be023d
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlwu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlwu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vsw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, uint32_t x)
+{
+ vuint32m1_t v = __riscv_th_vlwu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlwu_v_u32m1_tu (v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tu (v3, v2, -16, 4);
+ __riscv_th_vsw_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlwu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsw.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlwu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlwu_v_u32m1_m (mask, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_m (mask, v3, -16, 4);
+ __riscv_th_vsw_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlwu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlwu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsw.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlwu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlwu_v_u32m1_tumu (mask, v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tumu (mask, v3, v2, -16, 4);
+ __riscv_th_vsw_v_u32m1 (out, v4, 4);
+}
\ No newline at end of file
--
2.17.1
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
2024-01-10 11:06 ` Re：Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics joshua
2024-01-10 11:08 ` joshua
@ 2024-01-10 11:09 ` juzhe.zhong
2024-01-10 11:14 ` Re：Re：[PATCH " joshua
1 sibling, 1 reply; 17+ messages in thread
From: juzhe.zhong @ 2024-01-10 11:09 UTC (permalink / raw)
To: cooper.joshua, gcc-patches
Cc: Jim Wilson, palmer, andrew, philipp.tomsich, jeffreyalaw,
christoph.muellner, jinma, cooper.qu
So vlb has not only sew = 8 ?
But why do you add intrinsics as follows ?
+DEF_RVV_FUNCTION (th_vlb, th_loadstore_width, full_preds, i8_v_scalar_const_ptr_ops)
Why it is not :
DEF_RVV_FUNCTION (th_vlb, th_loadstore_width, full_preds, all_v_scalar_const_ptr_ops)
?
juzhe.zhong@rivai.ai
发件人： joshua
发送时间： 2024-01-10 19:06
收件人： juzhe.zhong@rivai.ai; gcc-patches
抄送： Jim Wilson; palmer; andrew; philipp.tomsich; jeffreyalaw; christoph.muellner; jinma; cooper.qu
主题： Re：Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
The key difference between vlb/vlh/vlw is not output type too.
Their difference is the range of datatype, not one specific type.
We have dived into the xtheadvector special intrinsics and are
sure about that.
------------------------------------------------------------------
发件人：juzhe.zhong@rivai.ai <juzhe.zhong@rivai.ai>
发送时间：2024年1月10日(星期三) 19:00
收件人："cooper.joshua"<cooper.joshua@linux.alibaba.com>; "gcc-patches"<gcc-patches@gcc.gnu.org>
抄 送：Jim Wilson<jim.wilson.gcc@gmail.com>; palmer<palmer@dabbelt.com>; andrew<andrew@sifive.com>; "philipp.tomsich"<philipp.tomsich@vrull.eu>; jeffreyalaw<jeffreyalaw@gmail.com>; "christoph.muellner"<christoph.muellner@vrull.eu>; jinma<jinma@linux.alibaba.com>; "cooper.qu"<cooper.qu@linux.alibaba.com>
主 题：Re: Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
instance.op_info->args[i].get_tree_type (instance.type.index) is output type.
You can use GDB debug it .
juzhe.zhong@rivai.ai
发件人： joshua
发送时间： 2024-01-10 18:57
收件人： juzhe.zhong@rivai.ai; gcc-patches
抄送： Jim Wilson; palmer; andrew; philipp.tomsich; jeffreyalaw; christoph.muellner; jinma; cooper.qu
主题： Re：Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
Hi Juzhe,
Perhaps things are not as simple as imagined.
The differences between vlb/vlh/vlw is not the same
as vle8/vle16/vle32. "8", "16" or "32" in vle8/vle16/vle32
can be appended from "vle" according to input type.
But vlb/vlh/vlw is different not in input type.
------------------------------------------------------------------
发件人：juzhe.zhong@rivai.ai <juzhe.zhong@rivai.ai>
发送时间：2024年1月10日(星期三) 18:03
收件人："cooper.joshua"<cooper.joshua@linux.alibaba.com>; "gcc-patches"<gcc-patches@gcc.gnu.org>
抄 送：Jim Wilson<jim.wilson.gcc@gmail.com>; palmer<palmer@dabbelt.com>; andrew<andrew@sifive.com>; "philipp.tomsich"<philipp.tomsich@vrull.eu>; jeffreyalaw<jeffreyalaw@gmail.com>; "christoph.muellner"<christoph.muellner@vrull.eu>; jinma<jinma@linux.alibaba.com>; "cooper.qu"<cooper.qu@linux.alibaba.com>
主 题：Re: Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
I mean change these:
+DEF_RVV_FUNCTION (th_vlb, th_loadstore_width, full_preds, i8_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (th_vlh, th_loadstore_width, full_preds, i16_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (th_vlw, th_loadstore_width, full_preds, i32_v_scalar_const_ptr_ops)
into a single:
+DEF_RVV_FUNCTION (th_vl, th_loadstore_width, full_preds, all_v_scalar_const_ptr_ops)
and append "h", "w", or"b" according to
TYPE_UNSIGNED and
GET_MODE_BITSIZE (GET_MODE_INNER (TYPE_MODE (instance.op_info->args[i].get_tree_type (instance.type.index))))
in th_loadstore_width.
It should definitely works, I allow this flexibility in design of the framework.
juzhe.zhong@rivai.ai
发件人： joshua
发送时间： 2024-01-10 17:55
收件人： juzhe.zhong@rivai.ai; gcc-patches
抄送： Jim Wilson; palmer; andrew; philipp.tomsich; jeffreyalaw; christoph.muellner; jinma; cooper.qu
主题： Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
And revise th_loadstore_width, append the name according TYPE_UNSIGNED and
GET_MODE_BITSIZE (GET_MODE_INNER (TYPE_MODE (instance.op_info->args[i].get_tree_type (instance.type.index))))
What do you mean by it? I'm a bit confused.
Changing i8_v_scalar_const_ptr_ops into all_v_scalar_const_ptr_ops
will expand the datatypes that can be used in th_vlb. Can we restrict
again in th_loadstore_width?
------------------------------------------------------------------
发件人：juzhe.zhong@rivai.ai <juzhe.zhong@rivai.ai>
发送时间：2024年1月10日(星期三) 17:35
收件人："cooper.joshua"<cooper.joshua@linux.alibaba.com>; "gcc-patches"<gcc-patches@gcc.gnu.org>
抄 送：Jim Wilson<jim.wilson.gcc@gmail.com>; palmer<palmer@dabbelt.com>; andrew<andrew@sifive.com>; "philipp.tomsich"<philipp.tomsich@vrull.eu>; jeffreyalaw<jeffreyalaw@gmail.com>; "christoph.muellner"<christoph.muellner@vrull.eu>; "cooper.joshua"<cooper.joshua@linux.alibaba.com>; jinma<jinma@linux.alibaba.com>; "cooper.qu"<cooper.qu@linux.alibaba.com>
主 题：Re: [PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
+DEF_RVV_FUNCTION (th_vlb, th_loadstore_width, full_preds, i8_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (th_vlh, th_loadstore_width, full_preds, i16_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (th_vlw, th_loadstore_width, full_preds, i32_v_scalar_const_ptr_ops)
I think we should remove those many data structure you added like: i8_v_scalar_const_ptr_ops
Instead, you should use all_v_scalar_const_ptr_ops
And revise th_loadstore_width, append the name according TYPE_UNSIGNED and
GET_MODE_BITSIZE (GET_MODE_INNER (TYPE_MODE (instance.op_info->args[i].get_tree_type (instance.type.index))))
juzhe.zhong@rivai.ai
From: Jun Sha (Joshua)
Date: 2024-01-10 17:27
To: gcc-patches
CC: jim.wilson.gcc; palmer; andrew; philipp.tomsich; jeffreyalaw; christoph.muellner; juzhe.zhong; Jun Sha (Joshua); Jin Ma; Xianmiao Qu
Subject: [PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
This patch only involves the generation of xtheadvector
special load/store instructions and vext instructions.
gcc/ChangeLog:
* config/riscv/riscv-vector-builtins-bases.cc
(class th_loadstore_width): Define new builtin bases.
(BASE): Define new builtin bases.
* config/riscv/riscv-vector-builtins-bases.h:
Define new builtin class.
* config/riscv/riscv-vector-builtins-functions.def (vlsegff):
Include thead-vector-builtins-functions.def.
* config/riscv/riscv-vector-builtins-shapes.cc
(struct th_loadstore_width_def): Define new builtin shapes.
(struct th_indexed_loadstore_width_def):
Define new builtin shapes.
(SHAPE): Define new builtin shapes.
* config/riscv/riscv-vector-builtins-shapes.h:
Define new builtin shapes.
* config/riscv/riscv-vector-builtins-types.def
(DEF_RVV_I8_OPS): Add datatypes for XTheadVector.
(DEF_RVV_I16_OPS): Add datatypes for XTheadVector.
(DEF_RVV_I32_OPS): Add datatypes for XTheadVector.
(DEF_RVV_U8_OPS): Add datatypes for XTheadVector.
(DEF_RVV_U16_OPS): Add datatypes for XTheadVector.
(DEF_RVV_U32_OPS): Add datatypes for XTheadVector.
(vint8m1_t): Add datatypes for XTheadVector.
(vint8m2_t): Likewise.
(vint8m4_t): Likewise.
(vint8m8_t): Likewise.
(vint16m1_t): Likewise.
(vint16m2_t): Likewise.
(vint16m4_t): Likewise.
(vint16m8_t): Likewise.
(vint32m1_t): Likewise.
(vint32m2_t): Likewise.
(vint32m4_t): Likewise.
(vint32m8_t): Likewise.
(vint64m1_t): Likewise.
(vint64m2_t): Likewise.
(vint64m4_t): Likewise.
(vint64m8_t): Likewise.
(vuint8m1_t): Likewise.
(vuint8m2_t): Likewise.
(vuint8m4_t): Likewise.
(vuint8m8_t): Likewise.
(vuint16m1_t): Likewise.
(vuint16m2_t): Likewise.
(vuint16m4_t): Likewise.
(vuint16m8_t): Likewise.
(vuint32m1_t): Likewise.
(vuint32m2_t): Likewise.
(vuint32m4_t): Likewise.
(vuint32m8_t): Likewise.
(vuint64m1_t): Likewise.
(vuint64m2_t): Likewise.
(vuint64m4_t): Likewise.
(vuint64m8_t): Likewise.
* config/riscv/riscv-vector-builtins.cc
(DEF_RVV_I8_OPS): Add datatypes for XTheadVector.
(DEF_RVV_I16_OPS): Add datatypes for XTheadVector.
(DEF_RVV_I32_OPS): Add datatypes for XTheadVector.
(DEF_RVV_U8_OPS): Add datatypes for XTheadVector.
(DEF_RVV_U16_OPS): Add datatypes for XTheadVector.
(DEF_RVV_U32_OPS): Add datatypes for XTheadVector.
* config/riscv/thead-vector-builtins-functions.def: New file.
* config/riscv/thead-vector.md: Add new patterns.
gcc/testsuite/ChangeLog:
* gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c: New test.
Co-authored-by: Jin Ma <jinma@linux.alibaba.com>
Co-authored-by: Xianmiao Qu <cooper.qu@linux.alibaba.com>
Co-authored-by: Christoph Müllner <christoph.muellner@vrull.eu>
---
.../riscv/riscv-vector-builtins-bases.cc | 139 ++++++++
.../riscv/riscv-vector-builtins-bases.h | 31 ++
.../riscv/riscv-vector-builtins-shapes.cc | 98 ++++++
.../riscv/riscv-vector-builtins-shapes.h | 3 +
.../riscv/riscv-vector-builtins-types.def | 120 +++++++
gcc/config/riscv/riscv-vector-builtins.cc | 311 ++++++++++++++++++
gcc/config/riscv/riscv-vector-builtins.h | 3 +
gcc/config/riscv/t-riscv | 1 +
.../riscv/thead-vector-builtins-functions.def | 39 +++
gcc/config/riscv/thead-vector.md | 253 ++++++++++++++
.../riscv/rvv/xtheadvector/vlb-vsb.c | 68 ++++
.../riscv/rvv/xtheadvector/vlbu-vsb.c | 68 ++++
.../riscv/rvv/xtheadvector/vlh-vsh.c | 68 ++++
.../riscv/rvv/xtheadvector/vlhu-vsh.c | 68 ++++
.../riscv/rvv/xtheadvector/vlw-vsw.c | 68 ++++
.../riscv/rvv/xtheadvector/vlwu-vsw.c | 68 ++++
16 files changed, 1406 insertions(+)
create mode 100644 gcc/config/riscv/thead-vector-builtins-functions.def
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c
diff --git a/gcc/config/riscv/riscv-vector-builtins-bases.cc b/gcc/config/riscv/riscv-vector-builtins-bases.cc
index 46f1a1da33e..5f44f31a12b 100644
--- a/gcc/config/riscv/riscv-vector-builtins-bases.cc
+++ b/gcc/config/riscv/riscv-vector-builtins-bases.cc
@@ -2125,6 +2125,83 @@ public:
}
};
+/* Implements
+ * th.vl(b/h/w)[u].v/th.vs(b/h/w)[u].v/th.vls(b/h/w)[u].v/th.vss(b/h/w)[u].v/
+ * th.vlx(b/h/w)[u].v/th.vs[u]x(b/h/w).v
+ * codegen. */
+template<bool STORE_P, lst_type LST_TYPE, int UNSPEC>
+class th_loadstore_width : public function_base
+{
+public:
+ bool apply_tail_policy_p () const override { return !STORE_P; }
+ bool apply_mask_policy_p () const override { return !STORE_P; }
+
+ unsigned int call_properties (const function_instance &) const override
+ {
+ if (STORE_P)
+ return CP_WRITE_MEMORY;
+ else
+ return CP_READ_MEMORY;
+ }
+
+ bool can_be_overloaded_p (enum predication_type_index pred) const override
+ {
+ if (STORE_P || LST_TYPE == LST_INDEXED)
+ return true;
+ return pred != PRED_TYPE_none;
+ }
+
+ rtx expand (function_expander &e) const override
+ {
+ gcc_assert (TARGET_XTHEADVECTOR);
+ if (LST_TYPE == LST_INDEXED)
+ {
+ if (STORE_P)
+ return e.use_exact_insn (
+ code_for_pred_indexed_store_width (UNSPEC, UNSPEC,
+ e.vector_mode ()));
+ else
+ return e.use_exact_insn (
+ code_for_pred_indexed_load_width (UNSPEC, e.vector_mode ()));
+ }
+ else if (LST_TYPE == LST_STRIDED)
+ {
+ if (STORE_P)
+ return e.use_contiguous_store_insn (
+ code_for_pred_strided_store_width (UNSPEC, e.vector_mode ()));
+ else
+ return e.use_contiguous_load_insn (
+ code_for_pred_strided_load_width (UNSPEC, e.vector_mode ()));
+ }
+ else
+ {
+ if (STORE_P)
+ return e.use_contiguous_store_insn (
+ code_for_pred_store_width (UNSPEC, e.vector_mode ()));
+ else
+ return e.use_contiguous_load_insn (
+ code_for_pred_mov_width (UNSPEC, e.vector_mode ()));
+ }
+ }
+};
+
+/* Implements vext.x.v. */
+class th_extract : public function_base
+{
+public:
+ bool apply_vl_p () const override { return false; }
+ bool apply_tail_policy_p () const override { return false; }
+ bool apply_mask_policy_p () const override { return false; }
+ bool use_mask_predication_p () const override { return false; }
+ bool has_merge_operand_p () const override { return false; }
+
+ rtx expand (function_expander &e) const override
+ {
+ gcc_assert (TARGET_XTHEADVECTOR);
+ return e.use_exact_insn (code_for_pred_th_extract (e.vector_mode ()));
+ }
+};
+
/* Below implements are vector crypto */
/* Implements vandn.[vv,vx] */
class vandn : public function_base
@@ -2587,6 +2664,37 @@ static CONSTEXPR const seg_indexed_load<UNSPEC_ORDERED> vloxseg_obj;
static CONSTEXPR const seg_indexed_store<UNSPEC_UNORDERED> vsuxseg_obj;
static CONSTEXPR const seg_indexed_store<UNSPEC_ORDERED> vsoxseg_obj;
static CONSTEXPR const vlsegff vlsegff_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLB> th_vlb_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLBU> th_vlbu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLH> th_vlh_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLHU> th_vlhu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLW> th_vlw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLWU> th_vlwu_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_UNIT_STRIDE, UNSPEC_TH_VLB> th_vsb_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_UNIT_STRIDE, UNSPEC_TH_VLH> th_vsh_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_UNIT_STRIDE, UNSPEC_TH_VLW> th_vsw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSB> th_vlsb_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSBU> th_vlsbu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSH> th_vlsh_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSHU> th_vlshu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSW> th_vlsw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSWU> th_vlswu_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_STRIDED, UNSPEC_TH_VLSB> th_vssb_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_STRIDED, UNSPEC_TH_VLSH> th_vssh_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_STRIDED, UNSPEC_TH_VLSW> th_vssw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXB> th_vlxb_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXBU> th_vlxbu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXH> th_vlxh_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXHU> th_vlxhu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXW> th_vlxw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXWU> th_vlxwu_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VLXB> th_vsxb_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VLXH> th_vsxh_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VLXW> th_vsxw_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VSUXB> th_vsuxb_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VSUXH> th_vsuxh_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VSUXW> th_vsuxw_obj;
+static CONSTEXPR const th_extract th_vext_x_v_obj;
/* Crypto Vector */
static CONSTEXPR const vandn vandn_obj;
@@ -2878,6 +2986,37 @@ BASE (vloxseg)
BASE (vsuxseg)
BASE (vsoxseg)
BASE (vlsegff)
+BASE (th_vlb)
+BASE (th_vlh)
+BASE (th_vlw)
+BASE (th_vlbu)
+BASE (th_vlhu)
+BASE (th_vlwu)
+BASE (th_vsb)
+BASE (th_vsh)
+BASE (th_vsw)
+BASE (th_vlsb)
+BASE (th_vlsh)
+BASE (th_vlsw)
+BASE (th_vlsbu)
+BASE (th_vlshu)
+BASE (th_vlswu)
+BASE (th_vssb)
+BASE (th_vssh)
+BASE (th_vssw)
+BASE (th_vlxb)
+BASE (th_vlxh)
+BASE (th_vlxw)
+BASE (th_vlxbu)
+BASE (th_vlxhu)
+BASE (th_vlxwu)
+BASE (th_vsxb)
+BASE (th_vsxh)
+BASE (th_vsxw)
+BASE (th_vsuxb)
+BASE (th_vsuxh)
+BASE (th_vsuxw)
+BASE (th_vext_x_v)
/* Crypto vector */
BASE (vandn)
BASE (vbrev)
diff --git a/gcc/config/riscv/riscv-vector-builtins-bases.h b/gcc/config/riscv/riscv-vector-builtins-bases.h
index 1122e3801a7..df43adf9a17 100644
--- a/gcc/config/riscv/riscv-vector-builtins-bases.h
+++ b/gcc/config/riscv/riscv-vector-builtins-bases.h
@@ -299,6 +299,37 @@ extern const function_base *const vloxseg;
extern const function_base *const vsuxseg;
extern const function_base *const vsoxseg;
extern const function_base *const vlsegff;
+extern const function_base *const th_vlb;
+extern const function_base *const th_vlh;
+extern const function_base *const th_vlw;
+extern const function_base *const th_vlbu;
+extern const function_base *const th_vlhu;
+extern const function_base *const th_vlwu;
+extern const function_base *const th_vsb;
+extern const function_base *const th_vsh;
+extern const function_base *const th_vsw;
+extern const function_base *const th_vlsb;
+extern const function_base *const th_vlsh;
+extern const function_base *const th_vlsw;
+extern const function_base *const th_vlsbu;
+extern const function_base *const th_vlshu;
+extern const function_base *const th_vlswu;
+extern const function_base *const th_vssb;
+extern const function_base *const th_vssh;
+extern const function_base *const th_vssw;
+extern const function_base *const th_vlxb;
+extern const function_base *const th_vlxh;
+extern const function_base *const th_vlxw;
+extern const function_base *const th_vlxbu;
+extern const function_base *const th_vlxhu;
+extern const function_base *const th_vlxwu;
+extern const function_base *const th_vsxb;
+extern const function_base *const th_vsxh;
+extern const function_base *const th_vsxw;
+extern const function_base *const th_vsuxb;
+extern const function_base *const th_vsuxh;
+extern const function_base *const th_vsuxw;
+extern const function_base *const th_vext_x_v;
/* Below function_base are Vectro Crypto*/
extern const function_base *const vandn;
extern const function_base *const vbrev;
diff --git a/gcc/config/riscv/riscv-vector-builtins-shapes.cc b/gcc/config/riscv/riscv-vector-builtins-shapes.cc
index 1e4f4d53de6..489a95cf684 100644
--- a/gcc/config/riscv/riscv-vector-builtins-shapes.cc
+++ b/gcc/config/riscv/riscv-vector-builtins-shapes.cc
@@ -211,6 +211,86 @@ struct indexed_loadstore_def : public function_shape
}
};
+/* th_loadstore_width_def class. */
+struct th_loadstore_width_def : public build_base
+{
+ char *get_name (function_builder &b, const function_instance &instance,
+ bool overloaded_p) const override
+ {
+ /* Return nullptr if it can not be overloaded. */
+ if (overloaded_p && !instance.base->can_be_overloaded_p (instance.pred))
+ return nullptr;
+
+ b.append_base_name (instance.base_name);
+
+ /* vop_v --> vop_v_<type>. */
+ if (!overloaded_p)
+ {
+ /* vop --> vop_v. */
+ b.append_name (operand_suffixes[instance.op_info->op]);
+ /* vop_v --> vop_v_<type>. */
+ b.append_name (type_suffixes[instance.type.index].vector);
+ }
+
+ /* According to rvv-intrinsic-doc, it does not add "_m" suffix
+ for vop_m C++ overloaded API. */
+ if (overloaded_p && instance.pred == PRED_TYPE_m)
+ return b.finish_name ();
+ b.append_name (predication_suffixes[instance.pred]);
+ return b.finish_name ();
+ }
+};
+
+
+/* th_indexed_loadstore_width_def class. */
+struct th_indexed_loadstore_width_def : public function_shape
+{
+ void build (function_builder &b,
+ const function_group_info &group) const override
+ {
+ for (unsigned int pred_idx = 0; group.preds[pred_idx] != NUM_PRED_TYPES;
+ ++pred_idx)
+ {
+ for (unsigned int vec_type_idx = 0;
+ group.ops_infos.types[vec_type_idx].index != NUM_VECTOR_TYPES;
+ ++vec_type_idx)
+ {
+ tree index_type = group.ops_infos.args[1].get_tree_type (
+ group.ops_infos.types[vec_type_idx].index);
+ if (!index_type)
+ continue;
+ build_one (b, group, pred_idx, vec_type_idx);
+ }
+ }
+ }
+
+ char *get_name (function_builder &b, const function_instance &instance,
+ bool overloaded_p) const override
+ {
+
+ /* Return nullptr if it can not be overloaded. */
+ if (overloaded_p && !instance.base->can_be_overloaded_p (instance.pred))
+ return nullptr;
+
+ b.append_base_name (instance.base_name);
+ /* vop_v --> vop_v_<type>. */
+ if (!overloaded_p)
+ {
+ /* vop --> vop_v. */
+ b.append_name (operand_suffixes[instance.op_info->op]);
+ /* vop_v --> vop_v_<type>. */
+ b.append_name (type_suffixes[instance.type.index].vector);
+ }
+
+ /* According to rvv-intrinsic-doc, it does not add "_m" suffix
+ for vop_m C++ overloaded API. */
+ if (overloaded_p && instance.pred == PRED_TYPE_m)
+ return b.finish_name ();
+ b.append_name (predication_suffixes[instance.pred]);
+ return b.finish_name ();
+ }
+};
+
/* alu_def class. */
struct alu_def : public build_base
{
@@ -632,6 +712,21 @@ struct reduc_alu_def : public build_base
}
};
+/* th_extract_def class. */
+struct th_extract_def : public build_base
+{
+ char *get_name (function_builder &b, const function_instance &instance,
+ bool overloaded_p) const override
+ {
+ b.append_base_name (instance.base_name);
+ if (overloaded_p)
+ return b.finish_name ();
+ b.append_name (type_suffixes[instance.type.index].vector);
+ b.append_name (type_suffixes[instance.type.index].scalar);
+ return b.finish_name ();
+ }
+};
+
/* scalar_move_def class. */
struct scalar_move_def : public build_base
{
@@ -1094,6 +1189,8 @@ SHAPE(vsetvl, vsetvl)
SHAPE(vsetvl, vsetvlmax)
SHAPE(loadstore, loadstore)
SHAPE(indexed_loadstore, indexed_loadstore)
+SHAPE(th_loadstore_width, th_loadstore_width)
+SHAPE(th_indexed_loadstore_width, th_indexed_loadstore_width)
SHAPE(alu, alu)
SHAPE(alu_frm, alu_frm)
SHAPE(widen_alu, widen_alu)
@@ -1106,6 +1203,7 @@ SHAPE(move, move)
SHAPE(mask_alu, mask_alu)
SHAPE(reduc_alu, reduc_alu)
SHAPE(reduc_alu_frm, reduc_alu_frm)
+SHAPE(th_extract, th_extract)
SHAPE(scalar_move, scalar_move)
SHAPE(vundefined, vundefined)
SHAPE(misc, misc)
diff --git a/gcc/config/riscv/riscv-vector-builtins-shapes.h b/gcc/config/riscv/riscv-vector-builtins-shapes.h
index ac2a28ce017..a7624d0fabd 100644
--- a/gcc/config/riscv/riscv-vector-builtins-shapes.h
+++ b/gcc/config/riscv/riscv-vector-builtins-shapes.h
@@ -28,6 +28,8 @@ extern const function_shape *const vsetvl;
extern const function_shape *const vsetvlmax;
extern const function_shape *const loadstore;
extern const function_shape *const indexed_loadstore;
+extern const function_shape *const th_loadstore_width;
+extern const function_shape *const th_indexed_loadstore_width;
extern const function_shape *const alu;
extern const function_shape *const alu_frm;
extern const function_shape *const widen_alu;
@@ -41,6 +43,7 @@ extern const function_shape *const mask_alu;
extern const function_shape *const reduc_alu;
extern const function_shape *const reduc_alu_frm;
extern const function_shape *const scalar_move;
+extern const function_shape *const th_extract;
extern const function_shape *const vundefined;
extern const function_shape *const misc;
extern const function_shape *const vset;
diff --git a/gcc/config/riscv/riscv-vector-builtins-types.def b/gcc/config/riscv/riscv-vector-builtins-types.def
index 61019a56844..abfeb4fcd9b 100644
--- a/gcc/config/riscv/riscv-vector-builtins-types.def
+++ b/gcc/config/riscv/riscv-vector-builtins-types.def
@@ -24,12 +24,48 @@ along with GCC; see the file COPYING3. If not see
#define DEF_RVV_I_OPS(TYPE, REQUIRE)
#endif
+/* Use "DEF_RVV_I8_OPS" macro include some signed integer (i8/i16/i32/i64)
+ which will be iterated and registered as intrinsic functions. */
+#ifndef DEF_RVV_I8_OPS
+#define DEF_RVV_I8_OPS(TYPE, REQUIRE)
+#endif
+
+/* Use "DEF_RVV_I16_OPS" macro include some signed integer (i16/i32/i64)
+ which will be iterated and registered as intrinsic functions. */
+#ifndef DEF_RVV_I16_OPS
+#define DEF_RVV_I16_OPS(TYPE, REQUIRE)
+#endif
+
+/* Use "DEF_RVV_I32_OPS" macro include some signed integer (i32/i64)
+ which will be iterated and registered as intrinsic functions. */
+#ifndef DEF_RVV_I32_OPS
+#define DEF_RVV_I32_OPS(TYPE, REQUIRE)
+#endif
+
/* Use "DEF_RVV_U_OPS" macro include all unsigned integer which will be
iterated and registered as intrinsic functions. */
#ifndef DEF_RVV_U_OPS
#define DEF_RVV_U_OPS(TYPE, REQUIRE)
#endif
+/* Use "DEF_RVV_U8_OPS" macro include some unsigned integer (i8/i16/i32/i64)
+ which will be iterated and registered as intrinsic functions. */
+#ifndef DEF_RVV_U8_OPS
+#define DEF_RVV_U8_OPS(TYPE, REQUIRE)
+#endif
+
+/* Use "DEF_RVV_U16_OPS" macro include some unsigned integer (i16/i32/i64)
+ which will be iterated and registered as intrinsic functions. */
+#ifndef DEF_RVV_U16_OPS
+#define DEF_RVV_U16_OPS(TYPE, REQUIRE)
+#endif
+
+/* Use "DEF_RVV_U32_OPS" macro include some unsigned integer (i32/i64)
+ which will be iterated and registered as intrinsic functions. */
+#ifndef DEF_RVV_U32_OPS
+#define DEF_RVV_U32_OPS(TYPE, REQUIRE)
+#endif
+
/* Use "DEF_RVV_F_OPS" macro include all floating-point which will be
iterated and registered as intrinsic functions. */
#ifndef DEF_RVV_F_OPS
@@ -374,6 +410,45 @@ DEF_RVV_I_OPS (vint64m2_t, RVV_REQUIRE_ELEN_64)
DEF_RVV_I_OPS (vint64m4_t, RVV_REQUIRE_ELEN_64)
DEF_RVV_I_OPS (vint64m8_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_I8_OPS (vint8m1_t, 0)
+DEF_RVV_I8_OPS (vint8m2_t, 0)
+DEF_RVV_I8_OPS (vint8m4_t, 0)
+DEF_RVV_I8_OPS (vint8m8_t, 0)
+DEF_RVV_I8_OPS (vint16m1_t, 0)
+DEF_RVV_I8_OPS (vint16m2_t, 0)
+DEF_RVV_I8_OPS (vint16m4_t, 0)
+DEF_RVV_I8_OPS (vint16m8_t, 0)
+DEF_RVV_I8_OPS (vint32m1_t, 0)
+DEF_RVV_I8_OPS (vint32m2_t, 0)
+DEF_RVV_I8_OPS (vint32m4_t, 0)
+DEF_RVV_I8_OPS (vint32m8_t, 0)
+DEF_RVV_I8_OPS (vint64m1_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_I8_OPS (vint64m2_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_I8_OPS (vint64m4_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_I8_OPS (vint64m8_t, RVV_REQUIRE_ELEN_64)
+
+DEF_RVV_I16_OPS (vint16m1_t, 0)
+DEF_RVV_I16_OPS (vint16m2_t, 0)
+DEF_RVV_I16_OPS (vint16m4_t, 0)
+DEF_RVV_I16_OPS (vint16m8_t, 0)
+DEF_RVV_I16_OPS (vint32m1_t, 0)
+DEF_RVV_I16_OPS (vint32m2_t, 0)
+DEF_RVV_I16_OPS (vint32m4_t, 0)
+DEF_RVV_I16_OPS (vint32m8_t, 0)
+DEF_RVV_I16_OPS (vint64m1_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_I16_OPS (vint64m2_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_I16_OPS (vint64m4_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_I16_OPS (vint64m8_t, RVV_REQUIRE_ELEN_64)
+
+DEF_RVV_I32_OPS (vint32m1_t, 0)
+DEF_RVV_I32_OPS (vint32m2_t, 0)
+DEF_RVV_I32_OPS (vint32m4_t, 0)
+DEF_RVV_I32_OPS (vint32m8_t, 0)
+DEF_RVV_I32_OPS (vint64m1_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_I32_OPS (vint64m2_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_I32_OPS (vint64m4_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_I32_OPS (vint64m8_t, RVV_REQUIRE_ELEN_64)
+
DEF_RVV_U_OPS (vuint8mf8_t, RVV_REQUIRE_MIN_VLEN_64)
DEF_RVV_U_OPS (vuint8mf4_t, 0)
DEF_RVV_U_OPS (vuint8mf2_t, 0)
@@ -397,6 +472,45 @@ DEF_RVV_U_OPS (vuint64m2_t, RVV_REQUIRE_ELEN_64)
DEF_RVV_U_OPS (vuint64m4_t, RVV_REQUIRE_ELEN_64)
DEF_RVV_U_OPS (vuint64m8_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_U8_OPS (vuint8m1_t, 0)
+DEF_RVV_U8_OPS (vuint8m2_t, 0)
+DEF_RVV_U8_OPS (vuint8m4_t, 0)
+DEF_RVV_U8_OPS (vuint8m8_t, 0)
+DEF_RVV_U8_OPS (vuint16m1_t, 0)
+DEF_RVV_U8_OPS (vuint16m2_t, 0)
+DEF_RVV_U8_OPS (vuint16m4_t, 0)
+DEF_RVV_U8_OPS (vuint16m8_t, 0)
+DEF_RVV_U8_OPS (vuint32m1_t, 0)
+DEF_RVV_U8_OPS (vuint32m2_t, 0)
+DEF_RVV_U8_OPS (vuint32m4_t, 0)
+DEF_RVV_U8_OPS (vuint32m8_t, 0)
+DEF_RVV_U8_OPS (vuint64m1_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_U8_OPS (vuint64m2_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_U8_OPS (vuint64m4_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_U8_OPS (vuint64m8_t, RVV_REQUIRE_ELEN_64)
+
+DEF_RVV_U16_OPS (vuint16m1_t, 0)
+DEF_RVV_U16_OPS (vuint16m2_t, 0)
+DEF_RVV_U16_OPS (vuint16m4_t, 0)
+DEF_RVV_U16_OPS (vuint16m8_t, 0)
+DEF_RVV_U16_OPS (vuint32m1_t, 0)
+DEF_RVV_U16_OPS (vuint32m2_t, 0)
+DEF_RVV_U16_OPS (vuint32m4_t, 0)
+DEF_RVV_U16_OPS (vuint32m8_t, 0)
+DEF_RVV_U16_OPS (vuint64m1_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_U16_OPS (vuint64m2_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_U16_OPS (vuint64m4_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_U16_OPS (vuint64m8_t, RVV_REQUIRE_ELEN_64)
+
+DEF_RVV_U32_OPS (vuint32m1_t, 0)
+DEF_RVV_U32_OPS (vuint32m2_t, 0)
+DEF_RVV_U32_OPS (vuint32m4_t, 0)
+DEF_RVV_U32_OPS (vuint32m8_t, 0)
+DEF_RVV_U32_OPS (vuint64m1_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_U32_OPS (vuint64m2_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_U32_OPS (vuint64m4_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_U32_OPS (vuint64m8_t, RVV_REQUIRE_ELEN_64)
+
DEF_RVV_F_OPS (vfloat16mf4_t, RVV_REQUIRE_ELEN_FP_16 | RVV_REQUIRE_MIN_VLEN_64)
DEF_RVV_F_OPS (vfloat16mf2_t, RVV_REQUIRE_ELEN_FP_16)
DEF_RVV_F_OPS (vfloat16m1_t, RVV_REQUIRE_ELEN_FP_16)
@@ -1379,7 +1493,13 @@ DEF_RVV_CRYPTO_SEW64_OPS (vuint64m4_t, RVV_REQUIRE_ELEN_64)
DEF_RVV_CRYPTO_SEW64_OPS (vuint64m8_t, RVV_REQUIRE_ELEN_64)
#undef DEF_RVV_I_OPS
+#undef DEF_RVV_I8_OPS
+#undef DEF_RVV_I16_OPS
+#undef DEF_RVV_I32_OPS
#undef DEF_RVV_U_OPS
+#undef DEF_RVV_U8_OPS
+#undef DEF_RVV_U16_OPS
+#undef DEF_RVV_U32_OPS
#undef DEF_RVV_F_OPS
#undef DEF_RVV_B_OPS
#undef DEF_RVV_WEXTI_OPS
diff --git a/gcc/config/riscv/riscv-vector-builtins.cc b/gcc/config/riscv/riscv-vector-builtins.cc
index 25e0b6e56de..f429f12dc18 100644
--- a/gcc/config/riscv/riscv-vector-builtins.cc
+++ b/gcc/config/riscv/riscv-vector-builtins.cc
@@ -246,6 +246,63 @@ static const rvv_type_info iu_ops[] = {
#include "riscv-vector-builtins-types.def"
{NUM_VECTOR_TYPES, 0}};
+/* A list of all integer will be registered for intrinsic functions. */
+static const rvv_type_info i8_ops[] = {
+#define DEF_RVV_I8_OPS(TYPE, REQUIRE) {VECTOR_TYPE_##TYPE, REQUIRE},
+#include "riscv-vector-builtins-types.def"
+ {NUM_VECTOR_TYPES, 0}};
+
+/* A list of all integer will be registered for intrinsic functions. */
+static const rvv_type_info i16_ops[] = {
+#define DEF_RVV_I16_OPS(TYPE, REQUIRE) {VECTOR_TYPE_##TYPE, REQUIRE},
+#include "riscv-vector-builtins-types.def"
+ {NUM_VECTOR_TYPES, 0}};
+
+/* A list of all integer will be registered for intrinsic functions. */
+static const rvv_type_info i32_ops[] = {
+#define DEF_RVV_I32_OPS(TYPE, REQUIRE) {VECTOR_TYPE_##TYPE, REQUIRE},
+#include "riscv-vector-builtins-types.def"
+ {NUM_VECTOR_TYPES, 0}};
+
+/* A list of all integer will be registered for intrinsic functions. */
+static const rvv_type_info u8_ops[] = {
+#define DEF_RVV_U8_OPS(TYPE, REQUIRE) {VECTOR_TYPE_##TYPE, REQUIRE},
+#include "riscv-vector-builtins-types.def"
+ {NUM_VECTOR_TYPES, 0}};
+
+/* A list of all integer will be registered for intrinsic functions. */
+static const rvv_type_info u16_ops[] = {
+#define DEF_RVV_U16_OPS(TYPE, REQUIRE) {VECTOR_TYPE_##TYPE, REQUIRE},
+#include "riscv-vector-builtins-types.def"
+ {NUM_VECTOR_TYPES, 0}};
+
+/* A list of all integer will be registered for intrinsic functions. */
+static const rvv_type_info u32_ops[] = {
+#define DEF_RVV_U32_OPS(TYPE, REQUIRE) {VECTOR_TYPE_##TYPE, REQUIRE},
+#include "riscv-vector-builtins-types.def"
+ {NUM_VECTOR_TYPES, 0}};
+
+/* A list of all integer will be registered for intrinsic functions. */
+static const rvv_type_info iu8_ops[] = {
+#define DEF_RVV_I8_OPS(TYPE, REQUIRE) {VECTOR_TYPE_##TYPE, REQUIRE},
+#define DEF_RVV_U8_OPS(TYPE, REQUIRE) {VECTOR_TYPE_##TYPE, REQUIRE},
+#include "riscv-vector-builtins-types.def"
+ {NUM_VECTOR_TYPES, 0}};
+
+/* A list of all integer will be registered for intrinsic functions. */
+static const rvv_type_info iu16_ops[] = {
+#define DEF_RVV_I16_OPS(TYPE, REQUIRE) {VECTOR_TYPE_##TYPE, REQUIRE},
+#define DEF_RVV_U16_OPS(TYPE, REQUIRE) {VECTOR_TYPE_##TYPE, REQUIRE},
+#include "riscv-vector-builtins-types.def"
+ {NUM_VECTOR_TYPES, 0}};
+
+/* A list of all integer will be registered for intrinsic functions. */
+static const rvv_type_info iu32_ops[] = {
+#define DEF_RVV_I32_OPS(TYPE, REQUIRE) {VECTOR_TYPE_##TYPE, REQUIRE},
+#define DEF_RVV_U32_OPS(TYPE, REQUIRE) {VECTOR_TYPE_##TYPE, REQUIRE},
+#include "riscv-vector-builtins-types.def"
+ {NUM_VECTOR_TYPES, 0}};
+
/* A list of all types will be registered for intrinsic functions. */
static const rvv_type_info all_ops[] = {
#define DEF_RVV_I_OPS(TYPE, REQUIRE) {VECTOR_TYPE_##TYPE, REQUIRE},
@@ -934,6 +991,32 @@ static CONSTEXPR const rvv_arg_type_info ext_vcreate_args[]
= {rvv_arg_type_info (RVV_BASE_vector),
rvv_arg_type_info_end};
+/* A list of args for vector_type func (const scalar_type *, size_t)
+ * function. */
+static CONSTEXPR const rvv_arg_type_info scalar_const_ptr_size_args[]
+ = {rvv_arg_type_info (RVV_BASE_scalar_const_ptr),
+ rvv_arg_type_info (RVV_BASE_size), rvv_arg_type_info_end};
+
+/* A list of args for vector_type func (const scalar_type *, eew8_index_type)
+ * function. */
+static CONSTEXPR const rvv_arg_type_info scalar_const_ptr_index_args[]
+ = {rvv_arg_type_info (RVV_BASE_scalar_const_ptr),
+ rvv_arg_type_info (RVV_BASE_unsigned_vector), rvv_arg_type_info_end};
+
+/* A list of args for void func (scalar_type *, eew8_index_type, vector_type)
+ * function. */
+static CONSTEXPR const rvv_arg_type_info scalar_ptr_index_args[]
+ = {rvv_arg_type_info (RVV_BASE_scalar_ptr),
+ rvv_arg_type_info (RVV_BASE_unsigned_vector),
+ rvv_arg_type_info (RVV_BASE_vector), rvv_arg_type_info_end};
+
+/* A list of args for void func (scalar_type *, size_t, vector_type)
+ * function. */
+static CONSTEXPR const rvv_arg_type_info scalar_ptr_size_args[]
+ = {rvv_arg_type_info (RVV_BASE_scalar_ptr),
+ rvv_arg_type_info (RVV_BASE_size), rvv_arg_type_info (RVV_BASE_vector),
+ rvv_arg_type_info_end};
+
/* A list of none preds that will be registered for intrinsic functions. */
static CONSTEXPR const predication_type_index none_preds[]
= {PRED_TYPE_none, NUM_PRED_TYPES};
@@ -1455,6 +1538,14 @@ static CONSTEXPR const rvv_op_info iu_shift_vvv_ops
rvv_arg_type_info (RVV_BASE_vector), /* Return type */
shift_vv_args /* Args */};
+/* A static operand information for scalar_type func (vector_type, size_t)
+ * function registration. */
+static CONSTEXPR const rvv_op_info iu_x_s_u_ops
+ = {iu_ops, /* Types */
+ OP_TYPE_vx, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_scalar), /* Return type */
+ v_size_args /* Args */};
+
/* A static operand information for vector_type func (vector_type, size_t)
* function registration. */
static CONSTEXPR const rvv_op_info iu_shift_vvx_ops
@@ -2638,6 +2729,222 @@ static CONSTEXPR const rvv_op_info all_v_vcreate_lmul4_x2_ops
rvv_arg_type_info (RVV_BASE_vlmul_ext_x2), /* Return type */
ext_vcreate_args /* Args */};
+/* A static operand information for vector_type func (const scalar_type *)
+ * function registration. */
+static CONSTEXPR const rvv_op_info i8_v_scalar_const_ptr_ops
+ = {i8_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *)
+ * function registration. */
+static CONSTEXPR const rvv_op_info i16_v_scalar_const_ptr_ops
+ = {i16_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *)
+ * function registration. */
+static CONSTEXPR const rvv_op_info i32_v_scalar_const_ptr_ops
+ = {i32_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *)
+ * function registration. */
+static CONSTEXPR const rvv_op_info u8_v_scalar_const_ptr_ops
+ = {u8_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *)
+ * function registration. */
+static CONSTEXPR const rvv_op_info u16_v_scalar_const_ptr_ops
+ = {u16_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *)
+ * function registration. */
+static CONSTEXPR const rvv_op_info u32_v_scalar_const_ptr_ops
+ = {u32_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * size_t) function registration. */
+static CONSTEXPR const rvv_op_info i8_v_scalar_const_ptr_size_ops
+ = {i8_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_size_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * size_t) function registration. */
+static CONSTEXPR const rvv_op_info i16_v_scalar_const_ptr_size_ops
+ = {i16_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_size_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * size_t) function registration. */
+static CONSTEXPR const rvv_op_info i32_v_scalar_const_ptr_size_ops
+ = {i32_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_size_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * size_t) function registration. */
+static CONSTEXPR const rvv_op_info u8_v_scalar_const_ptr_size_ops
+ = {u8_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_size_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * size_t) function registration. */
+static CONSTEXPR const rvv_op_info u16_v_scalar_const_ptr_size_ops
+ = {u16_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_size_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * size_t) function registration. */
+static CONSTEXPR const rvv_op_info u32_v_scalar_const_ptr_size_ops
+ = {u32_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_size_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * eew8_index_type) function registration. */
+static CONSTEXPR const rvv_op_info i8_v_scalar_const_ptr_index_ops
+ = {i8_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_index_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * eew8_index_type) function registration. */
+static CONSTEXPR const rvv_op_info u8_v_scalar_const_ptr_index_ops
+ = {u8_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_index_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * eew8_index_type) function registration. */
+static CONSTEXPR const rvv_op_info i16_v_scalar_const_ptr_index_ops
+ = {i16_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_index_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * eew8_index_type) function registration. */
+static CONSTEXPR const rvv_op_info u16_v_scalar_const_ptr_index_ops
+ = {u16_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_index_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * eew8_index_type) function registration. */
+static CONSTEXPR const rvv_op_info i32_v_scalar_const_ptr_index_ops
+ = {i32_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_index_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * eew8_index_type) function registration. */
+static CONSTEXPR const rvv_op_info u32_v_scalar_const_ptr_index_ops
+ = {u32_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_index_args /* Args */};
+
+/* A static operand information for void func (scalar_type *, eew8_index_type,
+ * vector_type) function registration. */
+static CONSTEXPR const rvv_op_info iu8_v_scalar_ptr_index_ops
+ = {iu8_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_void), /* Return type */
+ scalar_ptr_index_args /* Args */};
+
+/* A static operand information for void func (scalar_type *, eew16_index_type,
+ * vector_type) function registration. */
+static CONSTEXPR const rvv_op_info iu16_v_scalar_ptr_index_ops
+ = {iu16_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_void), /* Return type */
+ scalar_ptr_index_args /* Args */};
+
+/* A static operand information for void func (scalar_type *, eew32_index_type,
+ * vector_type) function registration. */
+static CONSTEXPR const rvv_op_info iu32_v_scalar_ptr_index_ops
+ = {iu32_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_void), /* Return type */
+ scalar_ptr_index_args /* Args */};
+
+/* A static operand information for void func (scalar_type *, vector_type,
+ * function registration. */
+static CONSTEXPR const rvv_op_info iu8_v_scalar_ptr_ops
+ = {iu8_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_void), /* Return type */
+ scalar_ptr_args /* Args */};
+
+/* A static operand information for void func (scalar_type *, vector_type)
+ * function registration. */
+static CONSTEXPR const rvv_op_info iu16_v_scalar_ptr_ops
+ = {iu16_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_void), /* Return type */
+ scalar_ptr_args /* Args */};
+
+/* A static operand information for void func (scalar_type *, vector_type)
+ * function registration. */
+static CONSTEXPR const rvv_op_info iu32_v_scalar_ptr_ops
+ = {iu32_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_void), /* Return type */
+ scalar_ptr_args /* Args */};
+
+/* A static operand information for void func (scalar_type *, size_t,
+ * vector_type) function registration. */
+static CONSTEXPR const rvv_op_info iu8_v_scalar_ptr_size_ops
+ = {iu8_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_void), /* Return type */
+ scalar_ptr_size_args /* Args */};
+
+/* A static operand information for void func (scalar_type *, size_t,
+ * vector_type) function registration. */
+static CONSTEXPR const rvv_op_info iu16_v_scalar_ptr_size_ops
+ = {iu16_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_void), /* Return type */
+ scalar_ptr_size_args /* Args */};
+
+/* A static operand information for void func (scalar_type *, size_t,
+ * vector_type) function registration. */
+static CONSTEXPR const rvv_op_info iu32_v_scalar_ptr_size_ops
+ = {iu32_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_void), /* Return type */
+ scalar_ptr_size_args /* Args */};
+
/* A static operand information for vector_type func (vector_type).
Some ins just supports SEW=32, such as crypto vectol Zvkg extension.
* function registration. */
@@ -2816,6 +3123,10 @@ static function_group_info function_groups[] = {
#define DEF_RVV_FUNCTION(NAME, SHAPE, PREDS, OPS_INFO) \
{#NAME, &bases::NAME, &shapes::SHAPE, PREDS, OPS_INFO, REQUIRED_EXTENSIONS},
#include "riscv-vector-builtins-functions.def"
+#undef DEF_RVV_FUNCTION
+#define DEF_RVV_FUNCTION(NAME, SHAPE, PREDS, OPS_INFO) \
+ {#NAME, &bases::NAME, &shapes::SHAPE, PREDS, OPS_INFO, REQUIRED_EXTENSIONS},
+#include "thead-vector-builtins-functions.def"
};
/* The RVV types, with their built-in
diff --git a/gcc/config/riscv/riscv-vector-builtins.h b/gcc/config/riscv/riscv-vector-builtins.h
index 54c8824ff92..a8ee39a3cb2 100644
--- a/gcc/config/riscv/riscv-vector-builtins.h
+++ b/gcc/config/riscv/riscv-vector-builtins.h
@@ -114,6 +114,7 @@ static const unsigned int CP_WRITE_CSR = 1U << 5;
enum required_ext
{
VECTOR_EXT, /* Vector extension */
+ XTHEADVECTOR_EXT, /* XTheadVector extension */
ZVBB_EXT, /* Cryto vector Zvbb sub-ext */
ZVBB_OR_ZVKB_EXT, /* Cryto vector Zvbb or zvkb sub-ext */
ZVBC_EXT, /* Crypto vector Zvbc sub-ext */
@@ -234,6 +235,8 @@ struct function_group_info
{
case VECTOR_EXT:
return TARGET_VECTOR;
+ case XTHEADVECTOR_EXT:
+ return TARGET_XTHEADVECTOR;
case ZVBB_EXT:
return TARGET_ZVBB;
case ZVBB_OR_ZVKB_EXT:
diff --git a/gcc/config/riscv/t-riscv b/gcc/config/riscv/t-riscv
index 32de6b851c1..38494320d8b 100644
--- a/gcc/config/riscv/t-riscv
+++ b/gcc/config/riscv/t-riscv
@@ -1,6 +1,7 @@
RISCV_BUILTINS_H = $(srcdir)/config/riscv/riscv-vector-builtins.h \
$(srcdir)/config/riscv/riscv-vector-builtins.def \
$(srcdir)/config/riscv/riscv-vector-builtins-functions.def \
+ $(srcdir)/config/riscv/thead-vector-builtins-functions.def \
riscv-vector-type-indexer.gen.def
riscv-builtins.o: $(srcdir)/config/riscv/riscv-builtins.cc $(CONFIG_H) \
diff --git a/gcc/config/riscv/thead-vector-builtins-functions.def b/gcc/config/riscv/thead-vector-builtins-functions.def
new file mode 100644
index 00000000000..667820d4c3e
--- /dev/null
+++ b/gcc/config/riscv/thead-vector-builtins-functions.def
@@ -0,0 +1,39 @@
+#ifndef DEF_RVV_FUNCTION
+#define DEF_RVV_FUNCTION(NAME, SHAPE, PREDS, OPS_INFO)
+#endif
+
+#define REQUIRED_EXTENSIONS XTHEADVECTOR_EXT
+DEF_RVV_FUNCTION (th_vlb, th_loadstore_width, full_preds, i8_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (th_vlh, th_loadstore_width, full_preds, i16_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (th_vlw, th_loadstore_width, full_preds, i32_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (th_vlbu, th_loadstore_width, full_preds, u8_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (th_vlhu, th_loadstore_width, full_preds, u16_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (th_vlwu, th_loadstore_width, full_preds, u32_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (th_vsb, th_loadstore_width, none_m_preds, iu8_v_scalar_ptr_ops)
+DEF_RVV_FUNCTION (th_vsh, th_loadstore_width, none_m_preds, iu16_v_scalar_ptr_ops)
+DEF_RVV_FUNCTION (th_vsw, th_loadstore_width, none_m_preds, iu32_v_scalar_ptr_ops)
+DEF_RVV_FUNCTION (th_vlsb, th_loadstore_width, full_preds, i8_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (th_vlsh, th_loadstore_width, full_preds, i16_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (th_vlsw, th_loadstore_width, full_preds, i32_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (th_vlsbu, th_loadstore_width, full_preds, u8_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (th_vlshu, th_loadstore_width, full_preds, u16_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (th_vlswu, th_loadstore_width, full_preds, u32_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (th_vssb, th_loadstore_width, none_m_preds, iu8_v_scalar_ptr_size_ops)
+DEF_RVV_FUNCTION (th_vssh, th_loadstore_width, none_m_preds, iu16_v_scalar_ptr_size_ops)
+DEF_RVV_FUNCTION (th_vssw, th_loadstore_width, none_m_preds, iu32_v_scalar_ptr_size_ops)
+DEF_RVV_FUNCTION (th_vlxb, th_indexed_loadstore_width, full_preds, i8_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (th_vlxh, th_indexed_loadstore_width, full_preds, i16_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (th_vlxw, th_indexed_loadstore_width, full_preds, i32_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (th_vlxbu, th_indexed_loadstore_width, full_preds, u8_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (th_vlxhu, th_indexed_loadstore_width, full_preds, u16_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (th_vlxwu, th_indexed_loadstore_width, full_preds, u32_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (th_vsxb, th_indexed_loadstore_width, none_m_preds, iu8_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (th_vsxh, th_indexed_loadstore_width, none_m_preds, iu16_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (th_vsxw, th_indexed_loadstore_width, none_m_preds, iu32_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (th_vsuxb, th_indexed_loadstore_width, none_m_preds, iu8_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (th_vsuxh, th_indexed_loadstore_width, none_m_preds, iu16_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (th_vsuxw, th_indexed_loadstore_width, none_m_preds, iu32_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (th_vext_x_v, th_extract, none_preds, iu_x_s_u_ops)
+#undef REQUIRED_EXTENSIONS
+
+#undef DEF_RVV_FUNCTION
diff --git a/gcc/config/riscv/thead-vector.md b/gcc/config/riscv/thead-vector.md
index 696b815252d..0f3700d9269 100644
--- a/gcc/config/riscv/thead-vector.md
+++ b/gcc/config/riscv/thead-vector.md
@@ -1,7 +1,95 @@
(define_c_enum "unspec" [
+ UNSPEC_TH_VLB
+ UNSPEC_TH_VLBU
+ UNSPEC_TH_VLH
+ UNSPEC_TH_VLHU
+ UNSPEC_TH_VLW
+ UNSPEC_TH_VLWU
+
+ UNSPEC_TH_VLSB
+ UNSPEC_TH_VLSBU
+ UNSPEC_TH_VLSH
+ UNSPEC_TH_VLSHU
+ UNSPEC_TH_VLSW
+ UNSPEC_TH_VLSWU
+
+ UNSPEC_TH_VLXB
+ UNSPEC_TH_VLXBU
+ UNSPEC_TH_VLXH
+ UNSPEC_TH_VLXHU
+ UNSPEC_TH_VLXW
+ UNSPEC_TH_VLXWU
+
+ UNSPEC_TH_VSUXB
+ UNSPEC_TH_VSUXH
+ UNSPEC_TH_VSUXW
+
UNSPEC_TH_VWLDST
])
+(define_int_iterator UNSPEC_TH_VLMEM_OP [
+ UNSPEC_TH_VLB UNSPEC_TH_VLBU
+ UNSPEC_TH_VLH UNSPEC_TH_VLHU
+ UNSPEC_TH_VLW UNSPEC_TH_VLWU
+])
+
+(define_int_iterator UNSPEC_TH_VLSMEM_OP [
+ UNSPEC_TH_VLSB UNSPEC_TH_VLSBU
+ UNSPEC_TH_VLSH UNSPEC_TH_VLSHU
+ UNSPEC_TH_VLSW UNSPEC_TH_VLSWU
+])
+
+(define_int_iterator UNSPEC_TH_VLXMEM_OP [
+ UNSPEC_TH_VLXB UNSPEC_TH_VLXBU
+ UNSPEC_TH_VLXH UNSPEC_TH_VLXHU
+ UNSPEC_TH_VLXW UNSPEC_TH_VLXWU
+])
+
+(define_int_attr vlmem_op_attr [
+ (UNSPEC_TH_VLB "b") (UNSPEC_TH_VLBU "bu")
+ (UNSPEC_TH_VLH "h") (UNSPEC_TH_VLHU "hu")
+ (UNSPEC_TH_VLW "w") (UNSPEC_TH_VLWU "wu")
+ (UNSPEC_TH_VLSB "b") (UNSPEC_TH_VLSBU "bu")
+ (UNSPEC_TH_VLSH "h") (UNSPEC_TH_VLSHU "hu")
+ (UNSPEC_TH_VLSW "w") (UNSPEC_TH_VLSWU "wu")
+ (UNSPEC_TH_VLXB "b") (UNSPEC_TH_VLXBU "bu")
+ (UNSPEC_TH_VLXH "h") (UNSPEC_TH_VLXHU "hu")
+ (UNSPEC_TH_VLXW "w") (UNSPEC_TH_VLXWU "wu")
+ (UNSPEC_TH_VSUXB "b")
+ (UNSPEC_TH_VSUXH "h")
+ (UNSPEC_TH_VSUXW "w")
+])
+
+(define_int_attr vlmem_order_attr [
+ (UNSPEC_TH_VLXB "")
+ (UNSPEC_TH_VLXH "")
+ (UNSPEC_TH_VLXW "")
+ (UNSPEC_TH_VSUXB "u")
+ (UNSPEC_TH_VSUXH "u")
+ (UNSPEC_TH_VSUXW "u")
+])
+
+(define_int_iterator UNSPEC_TH_VSMEM_OP [
+ UNSPEC_TH_VLB
+ UNSPEC_TH_VLH
+ UNSPEC_TH_VLW
+])
+
+(define_int_iterator UNSPEC_TH_VSSMEM_OP [
+ UNSPEC_TH_VLSB
+ UNSPEC_TH_VLSH
+ UNSPEC_TH_VLSW
+])
+
+(define_int_iterator UNSPEC_TH_VSXMEM_OP [
+ UNSPEC_TH_VLXB
+ UNSPEC_TH_VLXH
+ UNSPEC_TH_VLXW
+ UNSPEC_TH_VSUXB
+ UNSPEC_TH_VSUXH
+ UNSPEC_TH_VSUXW
+])
+
(define_mode_iterator V_VLS_VT [V VLS VT])
(define_mode_iterator V_VB_VLS_VT [V VB VLS VT])
@@ -100,3 +188,168 @@
}
[(set_attr "type" "vldm,vstm,vmalu,vmalu,vmalu")
(set_attr "mode" "<MODE>")])
+
+(define_expand "@pred_mov_width<vlmem_op_attr><mode>"
+ [(set (match_operand:V_VLS 0 "nonimmediate_operand")
+ (if_then_else:V_VLS
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand")
+ (match_operand 4 "vector_length_operand")
+ (match_operand 5 "const_int_operand")
+ (match_operand 6 "const_int_operand")
+ (match_operand 7 "const_int_operand")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLMEM_OP)
+ (match_operand:V_VLS 3 "vector_move_operand")
+ (match_operand:V_VLS 2 "vector_merge_operand")))]
+ "TARGET_XTHEADVECTOR"
+ {})
+
+(define_insn_and_split "*pred_mov_width<vlmem_op_attr><mode>"
+ [(set (match_operand:V_VLS 0 "nonimmediate_operand" "=vr, vr, vd, m, vr, vr")
+ (if_then_else:V_VLS
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1, Wc1, vm, vmWc1, Wc1, Wc1")
+ (match_operand 4 "vector_length_operand" " rK, rK, rK, rK, rK, rK")
+ (match_operand 5 "const_int_operand" " i, i, i, i, i, i")
+ (match_operand 6 "const_int_operand" " i, i, i, i, i, i")
+ (match_operand 7 "const_int_operand" " i, i, i, i, i, i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLMEM_OP)
+ (match_operand:V_VLS 3 "reg_or_mem_operand" " m, m, m, vr, vr, vr")
+ (match_operand:V_VLS 2 "vector_merge_operand" " 0, vu, vu, vu, vu, 0")))]
+ "(TARGET_XTHEADVECTOR
+ && (register_operand (operands[0], <MODE>mode)
+ || register_operand (operands[3], <MODE>mode)))"
+ "@
+ vl<vlmem_op_attr>.v\t%0,%3%p1
+ vl<vlmem_op_attr>.v\t%0,%3
+ vl<vlmem_op_attr>.v\t%0,%3,%1.t
+ vs<vlmem_op_attr>.v\t%3,%0%p1
+ vmv.v.v\t%0,%3
+ vmv.v.v\t%0,%3"
+ "&& register_operand (operands[0], <MODE>mode)
+ && register_operand (operands[3], <MODE>mode)
+ && satisfies_constraint_vu (operands[2])
+ && INTVAL (operands[7]) == riscv_vector::VLMAX"
+ [(set (match_dup 0) (match_dup 3))]
+ ""
+ [(set_attr "type" "vlde,vlde,vlde,vste,vimov,vimov")
+ (set_attr "mode" "<MODE>")])
+
+(define_insn "@pred_store_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "memory_operand" "+m")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1")
+ (match_operand 3 "vector_length_operand" " rK")
+ (match_operand 4 "const_int_operand" " i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VSMEM_OP)
+ (match_operand:VI 2 "register_operand" " vr")
+ (match_dup 0)))]
+ "TARGET_XTHEADVECTOR"
+ "vs<vlmem_op_attr>.v\t%2,%0%p1"
+ [(set_attr "type" "vste")
+ (set_attr "mode" "<MODE>")
+ (set (attr "avl_type_idx") (const_int 4))
+ (set_attr "vl_op_idx" "3")])
+
+(define_insn "@pred_strided_load_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "register_operand" "=vr, vr, vd")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1, Wc1, vm")
+ (match_operand 5 "vector_length_operand" " rK, rK, rK")
+ (match_operand 6 "const_int_operand" " i, i, i")
+ (match_operand 7 "const_int_operand" " i, i, i")
+ (match_operand 8 "const_int_operand" " i, i, i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLSMEM_OP)
+ (unspec:VI
+ [(match_operand:VI 3 "memory_operand" " m, m, m")
+ (match_operand 4 "pmode_reg_or_0_operand" " rJ, rJ, rJ")] UNSPEC_TH_VLSMEM_OP)
+ (match_operand:VI 2 "vector_merge_operand" " 0, vu, vu")))]
+ "TARGET_XTHEADVECTOR"
+ "vls<vlmem_op_attr>.v\t%0,%3,%z4%p1"
+ [(set_attr "type" "vlds")
+ (set_attr "mode" "<MODE>")])
+
+(define_insn "@pred_strided_store_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "memory_operand" "+m")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1")
+ (match_operand 4 "vector_length_operand" " rK")
+ (match_operand 5 "const_int_operand" " i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VSSMEM_OP)
+ (unspec:VI
+ [(match_operand 2 "pmode_reg_or_0_operand" " rJ")
+ (match_operand:VI 3 "register_operand" " vr")] UNSPEC_TH_VSSMEM_OP)
+ (match_dup 0)))]
+ "TARGET_XTHEADVECTOR"
+ "vss<vlmem_op_attr>.v\t%3,%0,%z2%p1"
+ [(set_attr "type" "vsts")
+ (set_attr "mode" "<MODE>")
+ (set (attr "avl_type_idx") (const_int 5))])
+
+(define_insn "@pred_indexed_load_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "register_operand" "=vd, vr,vd, vr")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" " vm,Wc1,vm,Wc1")
+ (match_operand 5 "vector_length_operand" " rK, rK,rK, rK")
+ (match_operand 6 "const_int_operand" " i, i, i, i")
+ (match_operand 7 "const_int_operand" " i, i, i, i")
+ (match_operand 8 "const_int_operand" " i, i, i, i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLXMEM_OP)
+ (unspec:VI
+ [(match_operand 3 "pmode_reg_or_0_operand" " rJ, rJ,rJ, rJ")
+ (mem:BLK (scratch))
+ (match_operand:VI 4 "register_operand" " vr, vr,vr, vr")] UNSPEC_TH_VLXMEM_OP)
+ (match_operand:VI 2 "vector_merge_operand" " vu, vu, 0, 0")))]
+ "TARGET_XTHEADVECTOR"
+ "vlx<vlmem_op_attr>.v\t%0,(%z3),%4%p1"
+ [(set_attr "type" "vldux")
+ (set_attr "mode" "<MODE>")])
+
+(define_insn "@pred_indexed_<vlmem_order_attr>store_width<vlmem_op_attr><mode>"
+ [(set (mem:BLK (scratch))
+ (unspec:BLK
+ [(unspec:<VM>
+ [(match_operand:<VM> 0 "vector_mask_operand" "vmWc1")
+ (match_operand 4 "vector_length_operand" " rK")
+ (match_operand 5 "const_int_operand" " i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VSXMEM_OP)
+ (match_operand 1 "pmode_reg_or_0_operand" " rJ")
+ (match_operand:VI 2 "register_operand" " vr")
+ (match_operand:VI 3 "register_operand" " vr")] UNSPEC_TH_VSXMEM_OP))]
+ "TARGET_XTHEADVECTOR"
+ "vs<vlmem_order_attr>x<vlmem_op_attr>.v\t%3,(%z1),%2%p0"
+ [(set_attr "type" "vstux")
+ (set_attr "mode" "<MODE>")])
+
+(define_expand "@pred_th_extract<mode>"
+ [(set (match_operand:<VEL> 0 "register_operand")
+ (unspec:<VEL>
+ [(vec_select:<VEL>
+ (match_operand:V_VLSI 1 "register_operand")
+ (parallel [(match_operand:DI 2 "register_operand" "r")]))
+ (reg:SI VTYPE_REGNUM)] UNSPEC_VPREDICATE))]
+ "TARGET_XTHEADVECTOR"
+{})
+
+(define_insn "*pred_th_extract<mode>"
+ [(set (match_operand:<VEL> 0 "register_operand" "=r")
+ (unspec:<VEL>
+ [(vec_select:<VEL>
+ (match_operand:V_VLSI 1 "register_operand" "vr")
+ (parallel [(match_operand:DI 2 "register_operand" "r")]))
+ (reg:SI VTYPE_REGNUM)] UNSPEC_VPREDICATE))]
+ "TARGET_XTHEADVECTOR"
+ "vext.x.v\t%0,%1,%2"
+ [(set_attr "type" "vimovvx")
+ (set_attr "mode" "<MODE>")])
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c
new file mode 100644
index 00000000000..4e192bbf025
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+
+** th.vadd\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+
+** th.vsb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out)
+{
+ vint32m1_t v = __riscv_th_vlb_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlb_v_i32m1_tu (v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vv_i32m1 (v2, v2, 4);
+ vint32m1_t v4 = __riscv_vadd_vv_i32m1_tu (v3, v2, v2, 4);
+ __riscv_th_vsb_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlb.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+
+** th.vadd\.vv\tv[1-9][0-9]?,\s*v[0-9]+,\s*v[0-9]+,\s*v0.t
+** th.vsb.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlb_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlb_v_i32m1_m (mask, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vv_i32m1 (v2, v2, 4);
+ vint32m1_t v4 = __riscv_vadd_vv_i32m1_m (mask, v3, v3, 4);
+ __riscv_th_vsb_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlb.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+
+** th.vadd\.vv\tv[1-9][0-9]?,\s*v[0-9]+,\s*v[0-9]+,\s*v0.t
+** th.vsb.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlb_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlb_v_i32m1_tumu (mask, v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vv_i32m1 (v2, v2, 4);
+ vint32m1_t v4 = __riscv_vadd_vv_i32m1_tumu (mask, v3, v2, v2, 4);
+ __riscv_th_vsb_v_i32m1 (out, v4, 4);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c
new file mode 100644
index 00000000000..1538afec68e
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlbu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlbu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vsb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, uint32_t x)
+{
+ vuint32m1_t v = __riscv_th_vlbu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlbu_v_u32m1_tu (v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tu (v3, v2, -16, 4);
+ __riscv_th_vsb_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlbu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsb.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlbu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlbu_v_u32m1_m (mask, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_m (mask, v3, -16, 4);
+ __riscv_th_vsb_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlbu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlbu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsb.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlbu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlbu_v_u32m1_tumu (mask, v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tumu (mask, v3, v2, -16, 4);
+ __riscv_th_vsb_v_u32m1 (out, v4, 4);
+}
\ No newline at end of file
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c
new file mode 100644
index 00000000000..bf4924a1d76
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vsh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, int32_t x)
+{
+ vint32m1_t v = __riscv_th_vlh_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlh_v_i32m1_tu (v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, -16, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_tu (v3, v2, -16, 4);
+ __riscv_th_vsh_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlh.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsh.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, int32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlh_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlh_v_i32m1_m (mask, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, -16, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_m (mask, v3, -16, 4);
+ __riscv_th_vsh_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlh.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsh.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, int32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlh_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlh_v_i32m1_tumu (mask, v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, -16, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_tumu (mask, v3, v2, -16, 4);
+ __riscv_th_vsh_v_i32m1 (out, v4, 4);
+}
\ No newline at end of file
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c
new file mode 100644
index 00000000000..8c451845175
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlhu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlhu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vsh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, uint32_t x)
+{
+ vuint32m1_t v = __riscv_th_vlhu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlhu_v_u32m1_tu (v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tu (v3, v2, -16, 4);
+ __riscv_th_vsh_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlhu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsh.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlhu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlhu_v_u32m1_m (mask, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_m (mask, v3, -16, 4);
+ __riscv_th_vsh_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlhu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlhu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsh.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlhu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlhu_v_u32m1_tumu (mask, v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tumu (mask, v3, v2, -16, 4);
+ __riscv_th_vsh_v_u32m1 (out, v4, 4);
+}
\ No newline at end of file
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c
new file mode 100644
index 00000000000..0f5b09684a5
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vx\tv[0-9]+,\s*v[0-9]+,\s*[a-x0-9]+
+** th.vadd\.vx\tv[0-9]+,\s*v[0-9]+,\s*[a-x0-9]+
+** th.vsw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, int32_t x)
+{
+ vint32m1_t v = __riscv_th_vlw_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlw_v_i32m1_tu (v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, x, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_tu (v3, v2, x, 4);
+ __riscv_th_vsw_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlw.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vx\tv[0-9]+,\s*v[0-9]+,\s*[a-x0-9]+
+** th.vadd\.vx\tv[1-9][0-9]?,\s*v[0-9]+,\s*[a-x0-9]+,\s*v0.t
+** th.vsw.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, int32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlw_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlw_v_i32m1_m (mask, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, x, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_m (mask, v3, x, 4);
+ __riscv_th_vsw_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlw.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vx\tv[0-9]+,\s*v[0-9]+,\s*[a-x0-9]+
+** th.vadd\.vx\tv[1-9][0-9]?,\s*v[0-9]+,\s*[a-x0-9]+,\s*v0.t
+** th.vsw.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, int32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlw_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlw_v_i32m1_tumu (mask, v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, x, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_tumu (mask, v3, v2, x, 4);
+ __riscv_th_vsw_v_i32m1 (out, v4, 4);
+}
\ No newline at end of file
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c
new file mode 100644
index 00000000000..aaa75be023d
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlwu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlwu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vsw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, uint32_t x)
+{
+ vuint32m1_t v = __riscv_th_vlwu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlwu_v_u32m1_tu (v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tu (v3, v2, -16, 4);
+ __riscv_th_vsw_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlwu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsw.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlwu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlwu_v_u32m1_m (mask, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_m (mask, v3, -16, 4);
+ __riscv_th_vsw_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlwu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlwu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsw.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlwu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlwu_v_u32m1_tumu (mask, v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tumu (mask, v3, v2, -16, 4);
+ __riscv_th_vsw_v_u32m1 (out, v4, 4);
+}
\ No newline at end of file
--
2.17.1
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re：Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
2024-01-10 11:09 ` Re：[PATCH " juzhe.zhong
@ 2024-01-10 11:14 ` joshua
2024-01-10 11:16 ` Re：[PATCH " juzhe.zhong
0 siblings, 1 reply; 17+ messages in thread
From: joshua @ 2024-01-10 11:14 UTC (permalink / raw)
To: juzhe.zhong, gcc-patches
Cc: Jim Wilson, palmer, andrew, philipp.tomsich, jeffreyalaw,
christoph.muellner, jinma, cooper.qu
vlb can accept sew=8/16/32/64.
vlh can accept sew=16/32/64;
vlw can accept sew=32/64.
vint8m1_t __riscv_th_vlb_v_i8m1 (const int8_t *a, size_t vl);
vint8m2_t __riscv_th_vlb_v_i8m2 (const int8_t *a, size_t vl);
vint8m4_t __riscv_th_vlb_v_i8m4 (const int8_t *a, size_t vl);
vint8m8_t __riscv_th_vlb_v_i8m8 (const int8_t *a, size_t vl);
vint16m1_t __riscv_th_vlb_v_i16m1 (const int16_t *a, size_t vl);
vint16m2_t __riscv_th_vlb_v_i16m2 (const int16_t *a, size_t vl);
vint16m4_t __riscv_th_vlb_v_i16m4 (const int16_t *a, size_t vl);
vint16m8_t __riscv_th_vlb_v_i16m8 (const int16_t *a, size_t vl);
vint32m1_t __riscv_th_vlb_v_i32m1 (const int32_t *a, size_t vl);
vint32m2_t __riscv_th_vlb_v_i32m2 (const int32_t *a, size_t vl);
vint32m4_t __riscv_th_vlb_v_i32m4 (const int32_t *a, size_t vl);
vint32m8_t __riscv_th_vlb_v_i32m8 (const int32_t *a, size_t vl);
vint64m1_t __riscv_th_vlb_v_i64m1 (const int64_t *a, size_t vl);
vint64m2_t __riscv_th_vlb_v_i64m2 (const int64_t *a, size_t vl);
vint64m4_t __riscv_th_vlb_v_i64m4 (const int64_t *a, size_t vl);
vint64m8_t __riscv_th_vlb_v_i64m8 (const int64_t *a, size_t vl);
vint16m1_t __riscv_th_vlh_v_i16m1 (const int16_t *a, size_t vl);
vint16m2_t __riscv_th_vlh_v_i16m2 (const int16_t *a, size_t vl);
vint16m4_t __riscv_th_vlh_v_i16m4 (const int16_t *a, size_t vl);
vint16m8_t __riscv_th_vlh_v_i16m8 (const int16_t *a, size_t vl);
vint32m1_t __riscv_th_vlh_v_i32m1 (const int32_t *a, size_t vl);
vint32m2_t __riscv_th_vlh_v_i32m2 (const int32_t *a, size_t vl);
vint32m4_t __riscv_th_vlh_v_i32m4 (const int32_t *a, size_t vl);
vint32m8_t __riscv_th_vlh_v_i32m8 (const int32_t *a, size_t vl);
vint64m1_t __riscv_th_vlh_v_i64m1 (const int64_t *a, size_t vl);
vint64m2_t __riscv_th_vlh_v_i64m2 (const int64_t *a, size_t vl);
vint64m4_t __riscv_th_vlh_v_i64m4 (const int64_t *a, size_t vl);
vint64m8_t __riscv_th_vlh_v_i64m8 (const int64_t *a, size_t vl);
vint32m1_t __riscv_th_vlw_v_i32m1 (const int32_t *a, size_t vl);
vint32m2_t __riscv_th_vlw_v_i32m2 (const int32_t *a, size_t vl);
vint32m4_t __riscv_th_vlw_v_i32m4 (const int32_t *a, size_t vl);
vint32m8_t __riscv_th_vlw_v_i32m8 (const int32_t *a, size_t vl);
vint64m1_t __riscv_th_vlw_v_i64m1 (const int64_t *a, size_t vl);
vint64m2_t __riscv_th_vlw_v_i64m2 (const int64_t *a, size_t vl);
vint64m4_t __riscv_th_vlw_v_i64m4 (const int64_t *a, size_t vl);
vint64m8_t __riscv_th_vlw_v_i64m8 (const int64_t *a, size_t vl);
With the exisiting framework, I cannot come up with better way to differentiate
between vlb/vlw.vlh.
------------------------------------------------------------------
发件人：juzhe.zhong@rivai.ai <juzhe.zhong@rivai.ai>
发送时间：2024年1月10日(星期三) 19:09
收件人："cooper.joshua"<cooper.joshua@linux.alibaba.com>; "gcc-patches"<gcc-patches@gcc.gnu.org>
抄 送：Jim Wilson<jim.wilson.gcc@gmail.com>; palmer<palmer@dabbelt.com>; andrew<andrew@sifive.com>; "philipp.tomsich"<philipp.tomsich@vrull.eu>; jeffreyalaw<jeffreyalaw@gmail.com>; "christoph.muellner"<christoph.muellner@vrull.eu>; jinma<jinma@linux.alibaba.com>; "cooper.qu"<cooper.qu@linux.alibaba.com>
主 题：Re: Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
So vlb has not only sew = 8 ?
But why do you add intrinsics as follows ?
+DEF_RVV_FUNCTION (th_vlb, th_loadstore_width, full_preds, i8_v_scalar_const_ptr_ops)
Why it is not :
DEF_RVV_FUNCTION (th_vlb, th_loadstore_width, full_preds, all_v_scalar_const_ptr_ops)
?
juzhe.zhong@rivai.ai
发件人： joshua
发送时间： 2024-01-10 19:06
收件人： juzhe.zhong@rivai.ai; gcc-patches
抄送： Jim Wilson; palmer; andrew; philipp.tomsich; jeffreyalaw; christoph.muellner; jinma; cooper.qu
主题： Re：Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
The key difference between vlb/vlh/vlw is not output type too.
Their difference is the range of datatype, not one specific type.
We have dived into the xtheadvector special intrinsics and are
sure about that.
------------------------------------------------------------------
发件人：juzhe.zhong@rivai.ai <juzhe.zhong@rivai.ai>
发送时间：2024年1月10日(星期三) 19:00
收件人："cooper.joshua"<cooper.joshua@linux.alibaba.com>; "gcc-patches"<gcc-patches@gcc.gnu.org>
抄 送：Jim Wilson<jim.wilson.gcc@gmail.com>; palmer<palmer@dabbelt.com>; andrew<andrew@sifive.com>; "philipp.tomsich"<philipp.tomsich@vrull.eu>; jeffreyalaw<jeffreyalaw@gmail.com>; "christoph.muellner"<christoph.muellner@vrull.eu>; jinma<jinma@linux.alibaba.com>; "cooper.qu"<cooper.qu@linux.alibaba.com>
主 题：Re: Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
instance.op_info->args[i].get_tree_type (instance.type.index) is output type.
You can use GDB debug it .
juzhe.zhong@rivai.ai
发件人： joshua
发送时间： 2024-01-10 18:57
收件人： juzhe.zhong@rivai.ai; gcc-patches
抄送： Jim Wilson; palmer; andrew; philipp.tomsich; jeffreyalaw; christoph.muellner; jinma; cooper.qu
主题： Re：Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
Hi Juzhe,
Perhaps things are not as simple as imagined.
The differences between vlb/vlh/vlw is not the same
as vle8/vle16/vle32. "8", "16" or "32" in vle8/vle16/vle32
can be appended from "vle" according to input type.
But vlb/vlh/vlw is different not in input type.
------------------------------------------------------------------
发件人：juzhe.zhong@rivai.ai <juzhe.zhong@rivai.ai>
发送时间：2024年1月10日(星期三) 18:03
收件人："cooper.joshua"<cooper.joshua@linux.alibaba.com>; "gcc-patches"<gcc-patches@gcc.gnu.org>
抄 送：Jim Wilson<jim.wilson.gcc@gmail.com>; palmer<palmer@dabbelt.com>; andrew<andrew@sifive.com>; "philipp.tomsich"<philipp.tomsich@vrull.eu>; jeffreyalaw<jeffreyalaw@gmail.com>; "christoph.muellner"<christoph.muellner@vrull.eu>; jinma<jinma@linux.alibaba.com>; "cooper.qu"<cooper.qu@linux.alibaba.com>
主 题：Re: Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
I mean change these:
+DEF_RVV_FUNCTION (th_vlb, th_loadstore_width, full_preds, i8_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (th_vlh, th_loadstore_width, full_preds, i16_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (th_vlw, th_loadstore_width, full_preds, i32_v_scalar_const_ptr_ops)
into a single:
+DEF_RVV_FUNCTION (th_vl, th_loadstore_width, full_preds, all_v_scalar_const_ptr_ops)
and append "h", "w", or"b" according to
TYPE_UNSIGNED and
GET_MODE_BITSIZE (GET_MODE_INNER (TYPE_MODE (instance.op_info->args[i].get_tree_type (instance.type.index))))
in th_loadstore_width.
It should definitely works, I allow this flexibility in design of the framework.
juzhe.zhong@rivai.ai
发件人： joshua
发送时间： 2024-01-10 17:55
收件人： juzhe.zhong@rivai.ai; gcc-patches
抄送： Jim Wilson; palmer; andrew; philipp.tomsich; jeffreyalaw; christoph.muellner; jinma; cooper.qu
主题： Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
And revise th_loadstore_width, append the name according TYPE_UNSIGNED and
GET_MODE_BITSIZE (GET_MODE_INNER (TYPE_MODE (instance.op_info->args[i].get_tree_type (instance.type.index))))
What do you mean by it? I'm a bit confused.
Changing i8_v_scalar_const_ptr_ops into all_v_scalar_const_ptr_ops
will expand the datatypes that can be used in th_vlb. Can we restrict
again in th_loadstore_width?
------------------------------------------------------------------
发件人：juzhe.zhong@rivai.ai <juzhe.zhong@rivai.ai>
发送时间：2024年1月10日(星期三) 17:35
收件人："cooper.joshua"<cooper.joshua@linux.alibaba.com>; "gcc-patches"<gcc-patches@gcc.gnu.org>
抄 送：Jim Wilson<jim.wilson.gcc@gmail.com>; palmer<palmer@dabbelt.com>; andrew<andrew@sifive.com>; "philipp.tomsich"<philipp.tomsich@vrull.eu>; jeffreyalaw<jeffreyalaw@gmail.com>; "christoph.muellner"<christoph.muellner@vrull.eu>; "cooper.joshua"<cooper.joshua@linux.alibaba.com>; jinma<jinma@linux.alibaba.com>; "cooper.qu"<cooper.qu@linux.alibaba.com>
主 题：Re: [PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
+DEF_RVV_FUNCTION (th_vlb, th_loadstore_width, full_preds, i8_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (th_vlh, th_loadstore_width, full_preds, i16_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (th_vlw, th_loadstore_width, full_preds, i32_v_scalar_const_ptr_ops)
I think we should remove those many data structure you added like: i8_v_scalar_const_ptr_ops
Instead, you should use all_v_scalar_const_ptr_ops
And revise th_loadstore_width, append the name according TYPE_UNSIGNED and
GET_MODE_BITSIZE (GET_MODE_INNER (TYPE_MODE (instance.op_info->args[i].get_tree_type (instance.type.index))))
juzhe.zhong@rivai.ai
From: Jun Sha (Joshua)
Date: 2024-01-10 17:27
To: gcc-patches
CC: jim.wilson.gcc; palmer; andrew; philipp.tomsich; jeffreyalaw; christoph.muellner; juzhe.zhong; Jun Sha (Joshua); Jin Ma; Xianmiao Qu
Subject: [PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
This patch only involves the generation of xtheadvector
special load/store instructions and vext instructions.
gcc/ChangeLog:
* config/riscv/riscv-vector-builtins-bases.cc
(class th_loadstore_width): Define new builtin bases.
(BASE): Define new builtin bases.
* config/riscv/riscv-vector-builtins-bases.h:
Define new builtin class.
* config/riscv/riscv-vector-builtins-functions.def (vlsegff):
Include thead-vector-builtins-functions.def.
* config/riscv/riscv-vector-builtins-shapes.cc
(struct th_loadstore_width_def): Define new builtin shapes.
(struct th_indexed_loadstore_width_def):
Define new builtin shapes.
(SHAPE): Define new builtin shapes.
* config/riscv/riscv-vector-builtins-shapes.h:
Define new builtin shapes.
* config/riscv/riscv-vector-builtins-types.def
(DEF_RVV_I8_OPS): Add datatypes for XTheadVector.
(DEF_RVV_I16_OPS): Add datatypes for XTheadVector.
(DEF_RVV_I32_OPS): Add datatypes for XTheadVector.
(DEF_RVV_U8_OPS): Add datatypes for XTheadVector.
(DEF_RVV_U16_OPS): Add datatypes for XTheadVector.
(DEF_RVV_U32_OPS): Add datatypes for XTheadVector.
(vint8m1_t): Add datatypes for XTheadVector.
(vint8m2_t): Likewise.
(vint8m4_t): Likewise.
(vint8m8_t): Likewise.
(vint16m1_t): Likewise.
(vint16m2_t): Likewise.
(vint16m4_t): Likewise.
(vint16m8_t): Likewise.
(vint32m1_t): Likewise.
(vint32m2_t): Likewise.
(vint32m4_t): Likewise.
(vint32m8_t): Likewise.
(vint64m1_t): Likewise.
(vint64m2_t): Likewise.
(vint64m4_t): Likewise.
(vint64m8_t): Likewise.
(vuint8m1_t): Likewise.
(vuint8m2_t): Likewise.
(vuint8m4_t): Likewise.
(vuint8m8_t): Likewise.
(vuint16m1_t): Likewise.
(vuint16m2_t): Likewise.
(vuint16m4_t): Likewise.
(vuint16m8_t): Likewise.
(vuint32m1_t): Likewise.
(vuint32m2_t): Likewise.
(vuint32m4_t): Likewise.
(vuint32m8_t): Likewise.
(vuint64m1_t): Likewise.
(vuint64m2_t): Likewise.
(vuint64m4_t): Likewise.
(vuint64m8_t): Likewise.
* config/riscv/riscv-vector-builtins.cc
(DEF_RVV_I8_OPS): Add datatypes for XTheadVector.
(DEF_RVV_I16_OPS): Add datatypes for XTheadVector.
(DEF_RVV_I32_OPS): Add datatypes for XTheadVector.
(DEF_RVV_U8_OPS): Add datatypes for XTheadVector.
(DEF_RVV_U16_OPS): Add datatypes for XTheadVector.
(DEF_RVV_U32_OPS): Add datatypes for XTheadVector.
* config/riscv/thead-vector-builtins-functions.def: New file.
* config/riscv/thead-vector.md: Add new patterns.
gcc/testsuite/ChangeLog:
* gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c: New test.
Co-authored-by: Jin Ma <jinma@linux.alibaba.com>
Co-authored-by: Xianmiao Qu <cooper.qu@linux.alibaba.com>
Co-authored-by: Christoph Müllner <christoph.muellner@vrull.eu>
---
.../riscv/riscv-vector-builtins-bases.cc | 139 ++++++++
.../riscv/riscv-vector-builtins-bases.h | 31 ++
.../riscv/riscv-vector-builtins-shapes.cc | 98 ++++++
.../riscv/riscv-vector-builtins-shapes.h | 3 +
.../riscv/riscv-vector-builtins-types.def | 120 +++++++
gcc/config/riscv/riscv-vector-builtins.cc | 311 ++++++++++++++++++
gcc/config/riscv/riscv-vector-builtins.h | 3 +
gcc/config/riscv/t-riscv | 1 +
.../riscv/thead-vector-builtins-functions.def | 39 +++
gcc/config/riscv/thead-vector.md | 253 ++++++++++++++
.../riscv/rvv/xtheadvector/vlb-vsb.c | 68 ++++
.../riscv/rvv/xtheadvector/vlbu-vsb.c | 68 ++++
.../riscv/rvv/xtheadvector/vlh-vsh.c | 68 ++++
.../riscv/rvv/xtheadvector/vlhu-vsh.c | 68 ++++
.../riscv/rvv/xtheadvector/vlw-vsw.c | 68 ++++
.../riscv/rvv/xtheadvector/vlwu-vsw.c | 68 ++++
16 files changed, 1406 insertions(+)
create mode 100644 gcc/config/riscv/thead-vector-builtins-functions.def
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c
diff --git a/gcc/config/riscv/riscv-vector-builtins-bases.cc b/gcc/config/riscv/riscv-vector-builtins-bases.cc
index 46f1a1da33e..5f44f31a12b 100644
--- a/gcc/config/riscv/riscv-vector-builtins-bases.cc
+++ b/gcc/config/riscv/riscv-vector-builtins-bases.cc
@@ -2125,6 +2125,83 @@ public:
}
};
+/* Implements
+ * th.vl(b/h/w)[u].v/th.vs(b/h/w)[u].v/th.vls(b/h/w)[u].v/th.vss(b/h/w)[u].v/
+ * th.vlx(b/h/w)[u].v/th.vs[u]x(b/h/w).v
+ * codegen. */
+template<bool STORE_P, lst_type LST_TYPE, int UNSPEC>
+class th_loadstore_width : public function_base
+{
+public:
+ bool apply_tail_policy_p () const override { return !STORE_P; }
+ bool apply_mask_policy_p () const override { return !STORE_P; }
+
+ unsigned int call_properties (const function_instance &) const override
+ {
+ if (STORE_P)
+ return CP_WRITE_MEMORY;
+ else
+ return CP_READ_MEMORY;
+ }
+
+ bool can_be_overloaded_p (enum predication_type_index pred) const override
+ {
+ if (STORE_P || LST_TYPE == LST_INDEXED)
+ return true;
+ return pred != PRED_TYPE_none;
+ }
+
+ rtx expand (function_expander &e) const override
+ {
+ gcc_assert (TARGET_XTHEADVECTOR);
+ if (LST_TYPE == LST_INDEXED)
+ {
+ if (STORE_P)
+ return e.use_exact_insn (
+ code_for_pred_indexed_store_width (UNSPEC, UNSPEC,
+ e.vector_mode ()));
+ else
+ return e.use_exact_insn (
+ code_for_pred_indexed_load_width (UNSPEC, e.vector_mode ()));
+ }
+ else if (LST_TYPE == LST_STRIDED)
+ {
+ if (STORE_P)
+ return e.use_contiguous_store_insn (
+ code_for_pred_strided_store_width (UNSPEC, e.vector_mode ()));
+ else
+ return e.use_contiguous_load_insn (
+ code_for_pred_strided_load_width (UNSPEC, e.vector_mode ()));
+ }
+ else
+ {
+ if (STORE_P)
+ return e.use_contiguous_store_insn (
+ code_for_pred_store_width (UNSPEC, e.vector_mode ()));
+ else
+ return e.use_contiguous_load_insn (
+ code_for_pred_mov_width (UNSPEC, e.vector_mode ()));
+ }
+ }
+};
+
+/* Implements vext.x.v. */
+class th_extract : public function_base
+{
+public:
+ bool apply_vl_p () const override { return false; }
+ bool apply_tail_policy_p () const override { return false; }
+ bool apply_mask_policy_p () const override { return false; }
+ bool use_mask_predication_p () const override { return false; }
+ bool has_merge_operand_p () const override { return false; }
+
+ rtx expand (function_expander &e) const override
+ {
+ gcc_assert (TARGET_XTHEADVECTOR);
+ return e.use_exact_insn (code_for_pred_th_extract (e.vector_mode ()));
+ }
+};
+
/* Below implements are vector crypto */
/* Implements vandn.[vv,vx] */
class vandn : public function_base
@@ -2587,6 +2664,37 @@ static CONSTEXPR const seg_indexed_load<UNSPEC_ORDERED> vloxseg_obj;
static CONSTEXPR const seg_indexed_store<UNSPEC_UNORDERED> vsuxseg_obj;
static CONSTEXPR const seg_indexed_store<UNSPEC_ORDERED> vsoxseg_obj;
static CONSTEXPR const vlsegff vlsegff_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLB> th_vlb_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLBU> th_vlbu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLH> th_vlh_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLHU> th_vlhu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLW> th_vlw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLWU> th_vlwu_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_UNIT_STRIDE, UNSPEC_TH_VLB> th_vsb_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_UNIT_STRIDE, UNSPEC_TH_VLH> th_vsh_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_UNIT_STRIDE, UNSPEC_TH_VLW> th_vsw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSB> th_vlsb_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSBU> th_vlsbu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSH> th_vlsh_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSHU> th_vlshu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSW> th_vlsw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSWU> th_vlswu_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_STRIDED, UNSPEC_TH_VLSB> th_vssb_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_STRIDED, UNSPEC_TH_VLSH> th_vssh_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_STRIDED, UNSPEC_TH_VLSW> th_vssw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXB> th_vlxb_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXBU> th_vlxbu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXH> th_vlxh_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXHU> th_vlxhu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXW> th_vlxw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXWU> th_vlxwu_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VLXB> th_vsxb_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VLXH> th_vsxh_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VLXW> th_vsxw_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VSUXB> th_vsuxb_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VSUXH> th_vsuxh_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VSUXW> th_vsuxw_obj;
+static CONSTEXPR const th_extract th_vext_x_v_obj;
/* Crypto Vector */
static CONSTEXPR const vandn vandn_obj;
@@ -2878,6 +2986,37 @@ BASE (vloxseg)
BASE (vsuxseg)
BASE (vsoxseg)
BASE (vlsegff)
+BASE (th_vlb)
+BASE (th_vlh)
+BASE (th_vlw)
+BASE (th_vlbu)
+BASE (th_vlhu)
+BASE (th_vlwu)
+BASE (th_vsb)
+BASE (th_vsh)
+BASE (th_vsw)
+BASE (th_vlsb)
+BASE (th_vlsh)
+BASE (th_vlsw)
+BASE (th_vlsbu)
+BASE (th_vlshu)
+BASE (th_vlswu)
+BASE (th_vssb)
+BASE (th_vssh)
+BASE (th_vssw)
+BASE (th_vlxb)
+BASE (th_vlxh)
+BASE (th_vlxw)
+BASE (th_vlxbu)
+BASE (th_vlxhu)
+BASE (th_vlxwu)
+BASE (th_vsxb)
+BASE (th_vsxh)
+BASE (th_vsxw)
+BASE (th_vsuxb)
+BASE (th_vsuxh)
+BASE (th_vsuxw)
+BASE (th_vext_x_v)
/* Crypto vector */
BASE (vandn)
BASE (vbrev)
diff --git a/gcc/config/riscv/riscv-vector-builtins-bases.h b/gcc/config/riscv/riscv-vector-builtins-bases.h
index 1122e3801a7..df43adf9a17 100644
--- a/gcc/config/riscv/riscv-vector-builtins-bases.h
+++ b/gcc/config/riscv/riscv-vector-builtins-bases.h
@@ -299,6 +299,37 @@ extern const function_base *const vloxseg;
extern const function_base *const vsuxseg;
extern const function_base *const vsoxseg;
extern const function_base *const vlsegff;
+extern const function_base *const th_vlb;
+extern const function_base *const th_vlh;
+extern const function_base *const th_vlw;
+extern const function_base *const th_vlbu;
+extern const function_base *const th_vlhu;
+extern const function_base *const th_vlwu;
+extern const function_base *const th_vsb;
+extern const function_base *const th_vsh;
+extern const function_base *const th_vsw;
+extern const function_base *const th_vlsb;
+extern const function_base *const th_vlsh;
+extern const function_base *const th_vlsw;
+extern const function_base *const th_vlsbu;
+extern const function_base *const th_vlshu;
+extern const function_base *const th_vlswu;
+extern const function_base *const th_vssb;
+extern const function_base *const th_vssh;
+extern const function_base *const th_vssw;
+extern const function_base *const th_vlxb;
+extern const function_base *const th_vlxh;
+extern const function_base *const th_vlxw;
+extern const function_base *const th_vlxbu;
+extern const function_base *const th_vlxhu;
+extern const function_base *const th_vlxwu;
+extern const function_base *const th_vsxb;
+extern const function_base *const th_vsxh;
+extern const function_base *const th_vsxw;
+extern const function_base *const th_vsuxb;
+extern const function_base *const th_vsuxh;
+extern const function_base *const th_vsuxw;
+extern const function_base *const th_vext_x_v;
/* Below function_base are Vectro Crypto*/
extern const function_base *const vandn;
extern const function_base *const vbrev;
diff --git a/gcc/config/riscv/riscv-vector-builtins-shapes.cc b/gcc/config/riscv/riscv-vector-builtins-shapes.cc
index 1e4f4d53de6..489a95cf684 100644
--- a/gcc/config/riscv/riscv-vector-builtins-shapes.cc
+++ b/gcc/config/riscv/riscv-vector-builtins-shapes.cc
@@ -211,6 +211,86 @@ struct indexed_loadstore_def : public function_shape
}
};
+/* th_loadstore_width_def class. */
+struct th_loadstore_width_def : public build_base
+{
+ char *get_name (function_builder &b, const function_instance &instance,
+ bool overloaded_p) const override
+ {
+ /* Return nullptr if it can not be overloaded. */
+ if (overloaded_p && !instance.base->can_be_overloaded_p (instance.pred))
+ return nullptr;
+
+ b.append_base_name (instance.base_name);
+
+ /* vop_v --> vop_v_<type>. */
+ if (!overloaded_p)
+ {
+ /* vop --> vop_v. */
+ b.append_name (operand_suffixes[instance.op_info->op]);
+ /* vop_v --> vop_v_<type>. */
+ b.append_name (type_suffixes[instance.type.index].vector);
+ }
+
+ /* According to rvv-intrinsic-doc, it does not add "_m" suffix
+ for vop_m C++ overloaded API. */
+ if (overloaded_p && instance.pred == PRED_TYPE_m)
+ return b.finish_name ();
+ b.append_name (predication_suffixes[instance.pred]);
+ return b.finish_name ();
+ }
+};
+
+
+/* th_indexed_loadstore_width_def class. */
+struct th_indexed_loadstore_width_def : public function_shape
+{
+ void build (function_builder &b,
+ const function_group_info &group) const override
+ {
+ for (unsigned int pred_idx = 0; group.preds[pred_idx] != NUM_PRED_TYPES;
+ ++pred_idx)
+ {
+ for (unsigned int vec_type_idx = 0;
+ group.ops_infos.types[vec_type_idx].index != NUM_VECTOR_TYPES;
+ ++vec_type_idx)
+ {
+ tree index_type = group.ops_infos.args[1].get_tree_type (
+ group.ops_infos.types[vec_type_idx].index);
+ if (!index_type)
+ continue;
+ build_one (b, group, pred_idx, vec_type_idx);
+ }
+ }
+ }
+
+ char *get_name (function_builder &b, const function_instance &instance,
+ bool overloaded_p) const override
+ {
+
+ /* Return nullptr if it can not be overloaded. */
+ if (overloaded_p && !instance.base->can_be_overloaded_p (instance.pred))
+ return nullptr;
+
+ b.append_base_name (instance.base_name);
+ /* vop_v --> vop_v_<type>. */
+ if (!overloaded_p)
+ {
+ /* vop --> vop_v. */
+ b.append_name (operand_suffixes[instance.op_info->op]);
+ /* vop_v --> vop_v_<type>. */
+ b.append_name (type_suffixes[instance.type.index].vector);
+ }
+
+ /* According to rvv-intrinsic-doc, it does not add "_m" suffix
+ for vop_m C++ overloaded API. */
+ if (overloaded_p && instance.pred == PRED_TYPE_m)
+ return b.finish_name ();
+ b.append_name (predication_suffixes[instance.pred]);
+ return b.finish_name ();
+ }
+};
+
/* alu_def class. */
struct alu_def : public build_base
{
@@ -632,6 +712,21 @@ struct reduc_alu_def : public build_base
}
};
+/* th_extract_def class. */
+struct th_extract_def : public build_base
+{
+ char *get_name (function_builder &b, const function_instance &instance,
+ bool overloaded_p) const override
+ {
+ b.append_base_name (instance.base_name);
+ if (overloaded_p)
+ return b.finish_name ();
+ b.append_name (type_suffixes[instance.type.index].vector);
+ b.append_name (type_suffixes[instance.type.index].scalar);
+ return b.finish_name ();
+ }
+};
+
/* scalar_move_def class. */
struct scalar_move_def : public build_base
{
@@ -1094,6 +1189,8 @@ SHAPE(vsetvl, vsetvl)
SHAPE(vsetvl, vsetvlmax)
SHAPE(loadstore, loadstore)
SHAPE(indexed_loadstore, indexed_loadstore)
+SHAPE(th_loadstore_width, th_loadstore_width)
+SHAPE(th_indexed_loadstore_width, th_indexed_loadstore_width)
SHAPE(alu, alu)
SHAPE(alu_frm, alu_frm)
SHAPE(widen_alu, widen_alu)
@@ -1106,6 +1203,7 @@ SHAPE(move, move)
SHAPE(mask_alu, mask_alu)
SHAPE(reduc_alu, reduc_alu)
SHAPE(reduc_alu_frm, reduc_alu_frm)
+SHAPE(th_extract, th_extract)
SHAPE(scalar_move, scalar_move)
SHAPE(vundefined, vundefined)
SHAPE(misc, misc)
diff --git a/gcc/config/riscv/riscv-vector-builtins-shapes.h b/gcc/config/riscv/riscv-vector-builtins-shapes.h
index ac2a28ce017..a7624d0fabd 100644
--- a/gcc/config/riscv/riscv-vector-builtins-shapes.h
+++ b/gcc/config/riscv/riscv-vector-builtins-shapes.h
@@ -28,6 +28,8 @@ extern const function_shape *const vsetvl;
extern const function_shape *const vsetvlmax;
extern const function_shape *const loadstore;
extern const function_shape *const indexed_loadstore;
+extern const function_shape *const th_loadstore_width;
+extern const function_shape *const th_indexed_loadstore_width;
extern const function_shape *const alu;
extern const function_shape *const alu_frm;
extern const function_shape *const widen_alu;
@@ -41,6 +43,7 @@ extern const function_shape *const mask_alu;
extern const function_shape *const reduc_alu;
extern const function_shape *const reduc_alu_frm;
extern const function_shape *const scalar_move;
+extern const function_shape *const th_extract;
extern const function_shape *const vundefined;
extern const function_shape *const misc;
extern const function_shape *const vset;
diff --git a/gcc/config/riscv/riscv-vector-builtins-types.def b/gcc/config/riscv/riscv-vector-builtins-types.def
index 61019a56844..abfeb4fcd9b 100644
--- a/gcc/config/riscv/riscv-vector-builtins-types.def
+++ b/gcc/config/riscv/riscv-vector-builtins-types.def
@@ -24,12 +24,48 @@ along with GCC; see the file COPYING3. If not see
#define DEF_RVV_I_OPS(TYPE, REQUIRE)
#endif
+/* Use "DEF_RVV_I8_OPS" macro include some signed integer (i8/i16/i32/i64)
+ which will be iterated and registered as intrinsic functions. */
+#ifndef DEF_RVV_I8_OPS
+#define DEF_RVV_I8_OPS(TYPE, REQUIRE)
+#endif
+
+/* Use "DEF_RVV_I16_OPS" macro include some signed integer (i16/i32/i64)
+ which will be iterated and registered as intrinsic functions. */
+#ifndef DEF_RVV_I16_OPS
+#define DEF_RVV_I16_OPS(TYPE, REQUIRE)
+#endif
+
+/* Use "DEF_RVV_I32_OPS" macro include some signed integer (i32/i64)
+ which will be iterated and registered as intrinsic functions. */
+#ifndef DEF_RVV_I32_OPS
+#define DEF_RVV_I32_OPS(TYPE, REQUIRE)
+#endif
+
/* Use "DEF_RVV_U_OPS" macro include all unsigned integer which will be
iterated and registered as intrinsic functions. */
#ifndef DEF_RVV_U_OPS
#define DEF_RVV_U_OPS(TYPE, REQUIRE)
#endif
+/* Use "DEF_RVV_U8_OPS" macro include some unsigned integer (i8/i16/i32/i64)
+ which will be iterated and registered as intrinsic functions. */
+#ifndef DEF_RVV_U8_OPS
+#define DEF_RVV_U8_OPS(TYPE, REQUIRE)
+#endif
+
+/* Use "DEF_RVV_U16_OPS" macro include some unsigned integer (i16/i32/i64)
+ which will be iterated and registered as intrinsic functions. */
+#ifndef DEF_RVV_U16_OPS
+#define DEF_RVV_U16_OPS(TYPE, REQUIRE)
+#endif
+
+/* Use "DEF_RVV_U32_OPS" macro include some unsigned integer (i32/i64)
+ which will be iterated and registered as intrinsic functions. */
+#ifndef DEF_RVV_U32_OPS
+#define DEF_RVV_U32_OPS(TYPE, REQUIRE)
+#endif
+
/* Use "DEF_RVV_F_OPS" macro include all floating-point which will be
iterated and registered as intrinsic functions. */
#ifndef DEF_RVV_F_OPS
@@ -374,6 +410,45 @@ DEF_RVV_I_OPS (vint64m2_t, RVV_REQUIRE_ELEN_64)
DEF_RVV_I_OPS (vint64m4_t, RVV_REQUIRE_ELEN_64)
DEF_RVV_I_OPS (vint64m8_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_I8_OPS (vint8m1_t, 0)
+DEF_RVV_I8_OPS (vint8m2_t, 0)
+DEF_RVV_I8_OPS (vint8m4_t, 0)
+DEF_RVV_I8_OPS (vint8m8_t, 0)
+DEF_RVV_I8_OPS (vint16m1_t, 0)
+DEF_RVV_I8_OPS (vint16m2_t, 0)
+DEF_RVV_I8_OPS (vint16m4_t, 0)
+DEF_RVV_I8_OPS (vint16m8_t, 0)
+DEF_RVV_I8_OPS (vint32m1_t, 0)
+DEF_RVV_I8_OPS (vint32m2_t, 0)
+DEF_RVV_I8_OPS (vint32m4_t, 0)
+DEF_RVV_I8_OPS (vint32m8_t, 0)
+DEF_RVV_I8_OPS (vint64m1_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_I8_OPS (vint64m2_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_I8_OPS (vint64m4_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_I8_OPS (vint64m8_t, RVV_REQUIRE_ELEN_64)
+
+DEF_RVV_I16_OPS (vint16m1_t, 0)
+DEF_RVV_I16_OPS (vint16m2_t, 0)
+DEF_RVV_I16_OPS (vint16m4_t, 0)
+DEF_RVV_I16_OPS (vint16m8_t, 0)
+DEF_RVV_I16_OPS (vint32m1_t, 0)
+DEF_RVV_I16_OPS (vint32m2_t, 0)
+DEF_RVV_I16_OPS (vint32m4_t, 0)
+DEF_RVV_I16_OPS (vint32m8_t, 0)
+DEF_RVV_I16_OPS (vint64m1_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_I16_OPS (vint64m2_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_I16_OPS (vint64m4_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_I16_OPS (vint64m8_t, RVV_REQUIRE_ELEN_64)
+
+DEF_RVV_I32_OPS (vint32m1_t, 0)
+DEF_RVV_I32_OPS (vint32m2_t, 0)
+DEF_RVV_I32_OPS (vint32m4_t, 0)
+DEF_RVV_I32_OPS (vint32m8_t, 0)
+DEF_RVV_I32_OPS (vint64m1_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_I32_OPS (vint64m2_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_I32_OPS (vint64m4_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_I32_OPS (vint64m8_t, RVV_REQUIRE_ELEN_64)
+
DEF_RVV_U_OPS (vuint8mf8_t, RVV_REQUIRE_MIN_VLEN_64)
DEF_RVV_U_OPS (vuint8mf4_t, 0)
DEF_RVV_U_OPS (vuint8mf2_t, 0)
@@ -397,6 +472,45 @@ DEF_RVV_U_OPS (vuint64m2_t, RVV_REQUIRE_ELEN_64)
DEF_RVV_U_OPS (vuint64m4_t, RVV_REQUIRE_ELEN_64)
DEF_RVV_U_OPS (vuint64m8_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_U8_OPS (vuint8m1_t, 0)
+DEF_RVV_U8_OPS (vuint8m2_t, 0)
+DEF_RVV_U8_OPS (vuint8m4_t, 0)
+DEF_RVV_U8_OPS (vuint8m8_t, 0)
+DEF_RVV_U8_OPS (vuint16m1_t, 0)
+DEF_RVV_U8_OPS (vuint16m2_t, 0)
+DEF_RVV_U8_OPS (vuint16m4_t, 0)
+DEF_RVV_U8_OPS (vuint16m8_t, 0)
+DEF_RVV_U8_OPS (vuint32m1_t, 0)
+DEF_RVV_U8_OPS (vuint32m2_t, 0)
+DEF_RVV_U8_OPS (vuint32m4_t, 0)
+DEF_RVV_U8_OPS (vuint32m8_t, 0)
+DEF_RVV_U8_OPS (vuint64m1_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_U8_OPS (vuint64m2_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_U8_OPS (vuint64m4_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_U8_OPS (vuint64m8_t, RVV_REQUIRE_ELEN_64)
+
+DEF_RVV_U16_OPS (vuint16m1_t, 0)
+DEF_RVV_U16_OPS (vuint16m2_t, 0)
+DEF_RVV_U16_OPS (vuint16m4_t, 0)
+DEF_RVV_U16_OPS (vuint16m8_t, 0)
+DEF_RVV_U16_OPS (vuint32m1_t, 0)
+DEF_RVV_U16_OPS (vuint32m2_t, 0)
+DEF_RVV_U16_OPS (vuint32m4_t, 0)
+DEF_RVV_U16_OPS (vuint32m8_t, 0)
+DEF_RVV_U16_OPS (vuint64m1_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_U16_OPS (vuint64m2_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_U16_OPS (vuint64m4_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_U16_OPS (vuint64m8_t, RVV_REQUIRE_ELEN_64)
+
+DEF_RVV_U32_OPS (vuint32m1_t, 0)
+DEF_RVV_U32_OPS (vuint32m2_t, 0)
+DEF_RVV_U32_OPS (vuint32m4_t, 0)
+DEF_RVV_U32_OPS (vuint32m8_t, 0)
+DEF_RVV_U32_OPS (vuint64m1_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_U32_OPS (vuint64m2_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_U32_OPS (vuint64m4_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_U32_OPS (vuint64m8_t, RVV_REQUIRE_ELEN_64)
+
DEF_RVV_F_OPS (vfloat16mf4_t, RVV_REQUIRE_ELEN_FP_16 | RVV_REQUIRE_MIN_VLEN_64)
DEF_RVV_F_OPS (vfloat16mf2_t, RVV_REQUIRE_ELEN_FP_16)
DEF_RVV_F_OPS (vfloat16m1_t, RVV_REQUIRE_ELEN_FP_16)
@@ -1379,7 +1493,13 @@ DEF_RVV_CRYPTO_SEW64_OPS (vuint64m4_t, RVV_REQUIRE_ELEN_64)
DEF_RVV_CRYPTO_SEW64_OPS (vuint64m8_t, RVV_REQUIRE_ELEN_64)
#undef DEF_RVV_I_OPS
+#undef DEF_RVV_I8_OPS
+#undef DEF_RVV_I16_OPS
+#undef DEF_RVV_I32_OPS
#undef DEF_RVV_U_OPS
+#undef DEF_RVV_U8_OPS
+#undef DEF_RVV_U16_OPS
+#undef DEF_RVV_U32_OPS
#undef DEF_RVV_F_OPS
#undef DEF_RVV_B_OPS
#undef DEF_RVV_WEXTI_OPS
diff --git a/gcc/config/riscv/riscv-vector-builtins.cc b/gcc/config/riscv/riscv-vector-builtins.cc
index 25e0b6e56de..f429f12dc18 100644
--- a/gcc/config/riscv/riscv-vector-builtins.cc
+++ b/gcc/config/riscv/riscv-vector-builtins.cc
@@ -246,6 +246,63 @@ static const rvv_type_info iu_ops[] = {
#include "riscv-vector-builtins-types.def"
{NUM_VECTOR_TYPES, 0}};
+/* A list of all integer will be registered for intrinsic functions. */
+static const rvv_type_info i8_ops[] = {
+#define DEF_RVV_I8_OPS(TYPE, REQUIRE) {VECTOR_TYPE_##TYPE, REQUIRE},
+#include "riscv-vector-builtins-types.def"
+ {NUM_VECTOR_TYPES, 0}};
+
+/* A list of all integer will be registered for intrinsic functions. */
+static const rvv_type_info i16_ops[] = {
+#define DEF_RVV_I16_OPS(TYPE, REQUIRE) {VECTOR_TYPE_##TYPE, REQUIRE},
+#include "riscv-vector-builtins-types.def"
+ {NUM_VECTOR_TYPES, 0}};
+
+/* A list of all integer will be registered for intrinsic functions. */
+static const rvv_type_info i32_ops[] = {
+#define DEF_RVV_I32_OPS(TYPE, REQUIRE) {VECTOR_TYPE_##TYPE, REQUIRE},
+#include "riscv-vector-builtins-types.def"
+ {NUM_VECTOR_TYPES, 0}};
+
+/* A list of all integer will be registered for intrinsic functions. */
+static const rvv_type_info u8_ops[] = {
+#define DEF_RVV_U8_OPS(TYPE, REQUIRE) {VECTOR_TYPE_##TYPE, REQUIRE},
+#include "riscv-vector-builtins-types.def"
+ {NUM_VECTOR_TYPES, 0}};
+
+/* A list of all integer will be registered for intrinsic functions. */
+static const rvv_type_info u16_ops[] = {
+#define DEF_RVV_U16_OPS(TYPE, REQUIRE) {VECTOR_TYPE_##TYPE, REQUIRE},
+#include "riscv-vector-builtins-types.def"
+ {NUM_VECTOR_TYPES, 0}};
+
+/* A list of all integer will be registered for intrinsic functions. */
+static const rvv_type_info u32_ops[] = {
+#define DEF_RVV_U32_OPS(TYPE, REQUIRE) {VECTOR_TYPE_##TYPE, REQUIRE},
+#include "riscv-vector-builtins-types.def"
+ {NUM_VECTOR_TYPES, 0}};
+
+/* A list of all integer will be registered for intrinsic functions. */
+static const rvv_type_info iu8_ops[] = {
+#define DEF_RVV_I8_OPS(TYPE, REQUIRE) {VECTOR_TYPE_##TYPE, REQUIRE},
+#define DEF_RVV_U8_OPS(TYPE, REQUIRE) {VECTOR_TYPE_##TYPE, REQUIRE},
+#include "riscv-vector-builtins-types.def"
+ {NUM_VECTOR_TYPES, 0}};
+
+/* A list of all integer will be registered for intrinsic functions. */
+static const rvv_type_info iu16_ops[] = {
+#define DEF_RVV_I16_OPS(TYPE, REQUIRE) {VECTOR_TYPE_##TYPE, REQUIRE},
+#define DEF_RVV_U16_OPS(TYPE, REQUIRE) {VECTOR_TYPE_##TYPE, REQUIRE},
+#include "riscv-vector-builtins-types.def"
+ {NUM_VECTOR_TYPES, 0}};
+
+/* A list of all integer will be registered for intrinsic functions. */
+static const rvv_type_info iu32_ops[] = {
+#define DEF_RVV_I32_OPS(TYPE, REQUIRE) {VECTOR_TYPE_##TYPE, REQUIRE},
+#define DEF_RVV_U32_OPS(TYPE, REQUIRE) {VECTOR_TYPE_##TYPE, REQUIRE},
+#include "riscv-vector-builtins-types.def"
+ {NUM_VECTOR_TYPES, 0}};
+
/* A list of all types will be registered for intrinsic functions. */
static const rvv_type_info all_ops[] = {
#define DEF_RVV_I_OPS(TYPE, REQUIRE) {VECTOR_TYPE_##TYPE, REQUIRE},
@@ -934,6 +991,32 @@ static CONSTEXPR const rvv_arg_type_info ext_vcreate_args[]
= {rvv_arg_type_info (RVV_BASE_vector),
rvv_arg_type_info_end};
+/* A list of args for vector_type func (const scalar_type *, size_t)
+ * function. */
+static CONSTEXPR const rvv_arg_type_info scalar_const_ptr_size_args[]
+ = {rvv_arg_type_info (RVV_BASE_scalar_const_ptr),
+ rvv_arg_type_info (RVV_BASE_size), rvv_arg_type_info_end};
+
+/* A list of args for vector_type func (const scalar_type *, eew8_index_type)
+ * function. */
+static CONSTEXPR const rvv_arg_type_info scalar_const_ptr_index_args[]
+ = {rvv_arg_type_info (RVV_BASE_scalar_const_ptr),
+ rvv_arg_type_info (RVV_BASE_unsigned_vector), rvv_arg_type_info_end};
+
+/* A list of args for void func (scalar_type *, eew8_index_type, vector_type)
+ * function. */
+static CONSTEXPR const rvv_arg_type_info scalar_ptr_index_args[]
+ = {rvv_arg_type_info (RVV_BASE_scalar_ptr),
+ rvv_arg_type_info (RVV_BASE_unsigned_vector),
+ rvv_arg_type_info (RVV_BASE_vector), rvv_arg_type_info_end};
+
+/* A list of args for void func (scalar_type *, size_t, vector_type)
+ * function. */
+static CONSTEXPR const rvv_arg_type_info scalar_ptr_size_args[]
+ = {rvv_arg_type_info (RVV_BASE_scalar_ptr),
+ rvv_arg_type_info (RVV_BASE_size), rvv_arg_type_info (RVV_BASE_vector),
+ rvv_arg_type_info_end};
+
/* A list of none preds that will be registered for intrinsic functions. */
static CONSTEXPR const predication_type_index none_preds[]
= {PRED_TYPE_none, NUM_PRED_TYPES};
@@ -1455,6 +1538,14 @@ static CONSTEXPR const rvv_op_info iu_shift_vvv_ops
rvv_arg_type_info (RVV_BASE_vector), /* Return type */
shift_vv_args /* Args */};
+/* A static operand information for scalar_type func (vector_type, size_t)
+ * function registration. */
+static CONSTEXPR const rvv_op_info iu_x_s_u_ops
+ = {iu_ops, /* Types */
+ OP_TYPE_vx, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_scalar), /* Return type */
+ v_size_args /* Args */};
+
/* A static operand information for vector_type func (vector_type, size_t)
* function registration. */
static CONSTEXPR const rvv_op_info iu_shift_vvx_ops
@@ -2638,6 +2729,222 @@ static CONSTEXPR const rvv_op_info all_v_vcreate_lmul4_x2_ops
rvv_arg_type_info (RVV_BASE_vlmul_ext_x2), /* Return type */
ext_vcreate_args /* Args */};
+/* A static operand information for vector_type func (const scalar_type *)
+ * function registration. */
+static CONSTEXPR const rvv_op_info i8_v_scalar_const_ptr_ops
+ = {i8_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *)
+ * function registration. */
+static CONSTEXPR const rvv_op_info i16_v_scalar_const_ptr_ops
+ = {i16_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *)
+ * function registration. */
+static CONSTEXPR const rvv_op_info i32_v_scalar_const_ptr_ops
+ = {i32_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *)
+ * function registration. */
+static CONSTEXPR const rvv_op_info u8_v_scalar_const_ptr_ops
+ = {u8_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *)
+ * function registration. */
+static CONSTEXPR const rvv_op_info u16_v_scalar_const_ptr_ops
+ = {u16_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *)
+ * function registration. */
+static CONSTEXPR const rvv_op_info u32_v_scalar_const_ptr_ops
+ = {u32_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * size_t) function registration. */
+static CONSTEXPR const rvv_op_info i8_v_scalar_const_ptr_size_ops
+ = {i8_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_size_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * size_t) function registration. */
+static CONSTEXPR const rvv_op_info i16_v_scalar_const_ptr_size_ops
+ = {i16_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_size_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * size_t) function registration. */
+static CONSTEXPR const rvv_op_info i32_v_scalar_const_ptr_size_ops
+ = {i32_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_size_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * size_t) function registration. */
+static CONSTEXPR const rvv_op_info u8_v_scalar_const_ptr_size_ops
+ = {u8_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_size_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * size_t) function registration. */
+static CONSTEXPR const rvv_op_info u16_v_scalar_const_ptr_size_ops
+ = {u16_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_size_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * size_t) function registration. */
+static CONSTEXPR const rvv_op_info u32_v_scalar_const_ptr_size_ops
+ = {u32_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_size_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * eew8_index_type) function registration. */
+static CONSTEXPR const rvv_op_info i8_v_scalar_const_ptr_index_ops
+ = {i8_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_index_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * eew8_index_type) function registration. */
+static CONSTEXPR const rvv_op_info u8_v_scalar_const_ptr_index_ops
+ = {u8_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_index_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * eew8_index_type) function registration. */
+static CONSTEXPR const rvv_op_info i16_v_scalar_const_ptr_index_ops
+ = {i16_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_index_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * eew8_index_type) function registration. */
+static CONSTEXPR const rvv_op_info u16_v_scalar_const_ptr_index_ops
+ = {u16_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_index_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * eew8_index_type) function registration. */
+static CONSTEXPR const rvv_op_info i32_v_scalar_const_ptr_index_ops
+ = {i32_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_index_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * eew8_index_type) function registration. */
+static CONSTEXPR const rvv_op_info u32_v_scalar_const_ptr_index_ops
+ = {u32_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_index_args /* Args */};
+
+/* A static operand information for void func (scalar_type *, eew8_index_type,
+ * vector_type) function registration. */
+static CONSTEXPR const rvv_op_info iu8_v_scalar_ptr_index_ops
+ = {iu8_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_void), /* Return type */
+ scalar_ptr_index_args /* Args */};
+
+/* A static operand information for void func (scalar_type *, eew16_index_type,
+ * vector_type) function registration. */
+static CONSTEXPR const rvv_op_info iu16_v_scalar_ptr_index_ops
+ = {iu16_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_void), /* Return type */
+ scalar_ptr_index_args /* Args */};
+
+/* A static operand information for void func (scalar_type *, eew32_index_type,
+ * vector_type) function registration. */
+static CONSTEXPR const rvv_op_info iu32_v_scalar_ptr_index_ops
+ = {iu32_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_void), /* Return type */
+ scalar_ptr_index_args /* Args */};
+
+/* A static operand information for void func (scalar_type *, vector_type,
+ * function registration. */
+static CONSTEXPR const rvv_op_info iu8_v_scalar_ptr_ops
+ = {iu8_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_void), /* Return type */
+ scalar_ptr_args /* Args */};
+
+/* A static operand information for void func (scalar_type *, vector_type)
+ * function registration. */
+static CONSTEXPR const rvv_op_info iu16_v_scalar_ptr_ops
+ = {iu16_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_void), /* Return type */
+ scalar_ptr_args /* Args */};
+
+/* A static operand information for void func (scalar_type *, vector_type)
+ * function registration. */
+static CONSTEXPR const rvv_op_info iu32_v_scalar_ptr_ops
+ = {iu32_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_void), /* Return type */
+ scalar_ptr_args /* Args */};
+
+/* A static operand information for void func (scalar_type *, size_t,
+ * vector_type) function registration. */
+static CONSTEXPR const rvv_op_info iu8_v_scalar_ptr_size_ops
+ = {iu8_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_void), /* Return type */
+ scalar_ptr_size_args /* Args */};
+
+/* A static operand information for void func (scalar_type *, size_t,
+ * vector_type) function registration. */
+static CONSTEXPR const rvv_op_info iu16_v_scalar_ptr_size_ops
+ = {iu16_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_void), /* Return type */
+ scalar_ptr_size_args /* Args */};
+
+/* A static operand information for void func (scalar_type *, size_t,
+ * vector_type) function registration. */
+static CONSTEXPR const rvv_op_info iu32_v_scalar_ptr_size_ops
+ = {iu32_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_void), /* Return type */
+ scalar_ptr_size_args /* Args */};
+
/* A static operand information for vector_type func (vector_type).
Some ins just supports SEW=32, such as crypto vectol Zvkg extension.
* function registration. */
@@ -2816,6 +3123,10 @@ static function_group_info function_groups[] = {
#define DEF_RVV_FUNCTION(NAME, SHAPE, PREDS, OPS_INFO) \
{#NAME, &bases::NAME, &shapes::SHAPE, PREDS, OPS_INFO, REQUIRED_EXTENSIONS},
#include "riscv-vector-builtins-functions.def"
+#undef DEF_RVV_FUNCTION
+#define DEF_RVV_FUNCTION(NAME, SHAPE, PREDS, OPS_INFO) \
+ {#NAME, &bases::NAME, &shapes::SHAPE, PREDS, OPS_INFO, REQUIRED_EXTENSIONS},
+#include "thead-vector-builtins-functions.def"
};
/* The RVV types, with their built-in
diff --git a/gcc/config/riscv/riscv-vector-builtins.h b/gcc/config/riscv/riscv-vector-builtins.h
index 54c8824ff92..a8ee39a3cb2 100644
--- a/gcc/config/riscv/riscv-vector-builtins.h
+++ b/gcc/config/riscv/riscv-vector-builtins.h
@@ -114,6 +114,7 @@ static const unsigned int CP_WRITE_CSR = 1U << 5;
enum required_ext
{
VECTOR_EXT, /* Vector extension */
+ XTHEADVECTOR_EXT, /* XTheadVector extension */
ZVBB_EXT, /* Cryto vector Zvbb sub-ext */
ZVBB_OR_ZVKB_EXT, /* Cryto vector Zvbb or zvkb sub-ext */
ZVBC_EXT, /* Crypto vector Zvbc sub-ext */
@@ -234,6 +235,8 @@ struct function_group_info
{
case VECTOR_EXT:
return TARGET_VECTOR;
+ case XTHEADVECTOR_EXT:
+ return TARGET_XTHEADVECTOR;
case ZVBB_EXT:
return TARGET_ZVBB;
case ZVBB_OR_ZVKB_EXT:
diff --git a/gcc/config/riscv/t-riscv b/gcc/config/riscv/t-riscv
index 32de6b851c1..38494320d8b 100644
--- a/gcc/config/riscv/t-riscv
+++ b/gcc/config/riscv/t-riscv
@@ -1,6 +1,7 @@
RISCV_BUILTINS_H = $(srcdir)/config/riscv/riscv-vector-builtins.h \
$(srcdir)/config/riscv/riscv-vector-builtins.def \
$(srcdir)/config/riscv/riscv-vector-builtins-functions.def \
+ $(srcdir)/config/riscv/thead-vector-builtins-functions.def \
riscv-vector-type-indexer.gen.def
riscv-builtins.o: $(srcdir)/config/riscv/riscv-builtins.cc $(CONFIG_H) \
diff --git a/gcc/config/riscv/thead-vector-builtins-functions.def b/gcc/config/riscv/thead-vector-builtins-functions.def
new file mode 100644
index 00000000000..667820d4c3e
--- /dev/null
+++ b/gcc/config/riscv/thead-vector-builtins-functions.def
@@ -0,0 +1,39 @@
+#ifndef DEF_RVV_FUNCTION
+#define DEF_RVV_FUNCTION(NAME, SHAPE, PREDS, OPS_INFO)
+#endif
+
+#define REQUIRED_EXTENSIONS XTHEADVECTOR_EXT
+DEF_RVV_FUNCTION (th_vlb, th_loadstore_width, full_preds, i8_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (th_vlh, th_loadstore_width, full_preds, i16_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (th_vlw, th_loadstore_width, full_preds, i32_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (th_vlbu, th_loadstore_width, full_preds, u8_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (th_vlhu, th_loadstore_width, full_preds, u16_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (th_vlwu, th_loadstore_width, full_preds, u32_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (th_vsb, th_loadstore_width, none_m_preds, iu8_v_scalar_ptr_ops)
+DEF_RVV_FUNCTION (th_vsh, th_loadstore_width, none_m_preds, iu16_v_scalar_ptr_ops)
+DEF_RVV_FUNCTION (th_vsw, th_loadstore_width, none_m_preds, iu32_v_scalar_ptr_ops)
+DEF_RVV_FUNCTION (th_vlsb, th_loadstore_width, full_preds, i8_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (th_vlsh, th_loadstore_width, full_preds, i16_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (th_vlsw, th_loadstore_width, full_preds, i32_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (th_vlsbu, th_loadstore_width, full_preds, u8_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (th_vlshu, th_loadstore_width, full_preds, u16_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (th_vlswu, th_loadstore_width, full_preds, u32_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (th_vssb, th_loadstore_width, none_m_preds, iu8_v_scalar_ptr_size_ops)
+DEF_RVV_FUNCTION (th_vssh, th_loadstore_width, none_m_preds, iu16_v_scalar_ptr_size_ops)
+DEF_RVV_FUNCTION (th_vssw, th_loadstore_width, none_m_preds, iu32_v_scalar_ptr_size_ops)
+DEF_RVV_FUNCTION (th_vlxb, th_indexed_loadstore_width, full_preds, i8_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (th_vlxh, th_indexed_loadstore_width, full_preds, i16_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (th_vlxw, th_indexed_loadstore_width, full_preds, i32_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (th_vlxbu, th_indexed_loadstore_width, full_preds, u8_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (th_vlxhu, th_indexed_loadstore_width, full_preds, u16_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (th_vlxwu, th_indexed_loadstore_width, full_preds, u32_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (th_vsxb, th_indexed_loadstore_width, none_m_preds, iu8_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (th_vsxh, th_indexed_loadstore_width, none_m_preds, iu16_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (th_vsxw, th_indexed_loadstore_width, none_m_preds, iu32_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (th_vsuxb, th_indexed_loadstore_width, none_m_preds, iu8_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (th_vsuxh, th_indexed_loadstore_width, none_m_preds, iu16_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (th_vsuxw, th_indexed_loadstore_width, none_m_preds, iu32_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (th_vext_x_v, th_extract, none_preds, iu_x_s_u_ops)
+#undef REQUIRED_EXTENSIONS
+
+#undef DEF_RVV_FUNCTION
diff --git a/gcc/config/riscv/thead-vector.md b/gcc/config/riscv/thead-vector.md
index 696b815252d..0f3700d9269 100644
--- a/gcc/config/riscv/thead-vector.md
+++ b/gcc/config/riscv/thead-vector.md
@@ -1,7 +1,95 @@
(define_c_enum "unspec" [
+ UNSPEC_TH_VLB
+ UNSPEC_TH_VLBU
+ UNSPEC_TH_VLH
+ UNSPEC_TH_VLHU
+ UNSPEC_TH_VLW
+ UNSPEC_TH_VLWU
+
+ UNSPEC_TH_VLSB
+ UNSPEC_TH_VLSBU
+ UNSPEC_TH_VLSH
+ UNSPEC_TH_VLSHU
+ UNSPEC_TH_VLSW
+ UNSPEC_TH_VLSWU
+
+ UNSPEC_TH_VLXB
+ UNSPEC_TH_VLXBU
+ UNSPEC_TH_VLXH
+ UNSPEC_TH_VLXHU
+ UNSPEC_TH_VLXW
+ UNSPEC_TH_VLXWU
+
+ UNSPEC_TH_VSUXB
+ UNSPEC_TH_VSUXH
+ UNSPEC_TH_VSUXW
+
UNSPEC_TH_VWLDST
])
+(define_int_iterator UNSPEC_TH_VLMEM_OP [
+ UNSPEC_TH_VLB UNSPEC_TH_VLBU
+ UNSPEC_TH_VLH UNSPEC_TH_VLHU
+ UNSPEC_TH_VLW UNSPEC_TH_VLWU
+])
+
+(define_int_iterator UNSPEC_TH_VLSMEM_OP [
+ UNSPEC_TH_VLSB UNSPEC_TH_VLSBU
+ UNSPEC_TH_VLSH UNSPEC_TH_VLSHU
+ UNSPEC_TH_VLSW UNSPEC_TH_VLSWU
+])
+
+(define_int_iterator UNSPEC_TH_VLXMEM_OP [
+ UNSPEC_TH_VLXB UNSPEC_TH_VLXBU
+ UNSPEC_TH_VLXH UNSPEC_TH_VLXHU
+ UNSPEC_TH_VLXW UNSPEC_TH_VLXWU
+])
+
+(define_int_attr vlmem_op_attr [
+ (UNSPEC_TH_VLB "b") (UNSPEC_TH_VLBU "bu")
+ (UNSPEC_TH_VLH "h") (UNSPEC_TH_VLHU "hu")
+ (UNSPEC_TH_VLW "w") (UNSPEC_TH_VLWU "wu")
+ (UNSPEC_TH_VLSB "b") (UNSPEC_TH_VLSBU "bu")
+ (UNSPEC_TH_VLSH "h") (UNSPEC_TH_VLSHU "hu")
+ (UNSPEC_TH_VLSW "w") (UNSPEC_TH_VLSWU "wu")
+ (UNSPEC_TH_VLXB "b") (UNSPEC_TH_VLXBU "bu")
+ (UNSPEC_TH_VLXH "h") (UNSPEC_TH_VLXHU "hu")
+ (UNSPEC_TH_VLXW "w") (UNSPEC_TH_VLXWU "wu")
+ (UNSPEC_TH_VSUXB "b")
+ (UNSPEC_TH_VSUXH "h")
+ (UNSPEC_TH_VSUXW "w")
+])
+
+(define_int_attr vlmem_order_attr [
+ (UNSPEC_TH_VLXB "")
+ (UNSPEC_TH_VLXH "")
+ (UNSPEC_TH_VLXW "")
+ (UNSPEC_TH_VSUXB "u")
+ (UNSPEC_TH_VSUXH "u")
+ (UNSPEC_TH_VSUXW "u")
+])
+
+(define_int_iterator UNSPEC_TH_VSMEM_OP [
+ UNSPEC_TH_VLB
+ UNSPEC_TH_VLH
+ UNSPEC_TH_VLW
+])
+
+(define_int_iterator UNSPEC_TH_VSSMEM_OP [
+ UNSPEC_TH_VLSB
+ UNSPEC_TH_VLSH
+ UNSPEC_TH_VLSW
+])
+
+(define_int_iterator UNSPEC_TH_VSXMEM_OP [
+ UNSPEC_TH_VLXB
+ UNSPEC_TH_VLXH
+ UNSPEC_TH_VLXW
+ UNSPEC_TH_VSUXB
+ UNSPEC_TH_VSUXH
+ UNSPEC_TH_VSUXW
+])
+
(define_mode_iterator V_VLS_VT [V VLS VT])
(define_mode_iterator V_VB_VLS_VT [V VB VLS VT])
@@ -100,3 +188,168 @@
}
[(set_attr "type" "vldm,vstm,vmalu,vmalu,vmalu")
(set_attr "mode" "<MODE>")])
+
+(define_expand "@pred_mov_width<vlmem_op_attr><mode>"
+ [(set (match_operand:V_VLS 0 "nonimmediate_operand")
+ (if_then_else:V_VLS
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand")
+ (match_operand 4 "vector_length_operand")
+ (match_operand 5 "const_int_operand")
+ (match_operand 6 "const_int_operand")
+ (match_operand 7 "const_int_operand")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLMEM_OP)
+ (match_operand:V_VLS 3 "vector_move_operand")
+ (match_operand:V_VLS 2 "vector_merge_operand")))]
+ "TARGET_XTHEADVECTOR"
+ {})
+
+(define_insn_and_split "*pred_mov_width<vlmem_op_attr><mode>"
+ [(set (match_operand:V_VLS 0 "nonimmediate_operand" "=vr, vr, vd, m, vr, vr")
+ (if_then_else:V_VLS
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1, Wc1, vm, vmWc1, Wc1, Wc1")
+ (match_operand 4 "vector_length_operand" " rK, rK, rK, rK, rK, rK")
+ (match_operand 5 "const_int_operand" " i, i, i, i, i, i")
+ (match_operand 6 "const_int_operand" " i, i, i, i, i, i")
+ (match_operand 7 "const_int_operand" " i, i, i, i, i, i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLMEM_OP)
+ (match_operand:V_VLS 3 "reg_or_mem_operand" " m, m, m, vr, vr, vr")
+ (match_operand:V_VLS 2 "vector_merge_operand" " 0, vu, vu, vu, vu, 0")))]
+ "(TARGET_XTHEADVECTOR
+ && (register_operand (operands[0], <MODE>mode)
+ || register_operand (operands[3], <MODE>mode)))"
+ "@
+ vl<vlmem_op_attr>.v\t%0,%3%p1
+ vl<vlmem_op_attr>.v\t%0,%3
+ vl<vlmem_op_attr>.v\t%0,%3,%1.t
+ vs<vlmem_op_attr>.v\t%3,%0%p1
+ vmv.v.v\t%0,%3
+ vmv.v.v\t%0,%3"
+ "&& register_operand (operands[0], <MODE>mode)
+ && register_operand (operands[3], <MODE>mode)
+ && satisfies_constraint_vu (operands[2])
+ && INTVAL (operands[7]) == riscv_vector::VLMAX"
+ [(set (match_dup 0) (match_dup 3))]
+ ""
+ [(set_attr "type" "vlde,vlde,vlde,vste,vimov,vimov")
+ (set_attr "mode" "<MODE>")])
+
+(define_insn "@pred_store_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "memory_operand" "+m")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1")
+ (match_operand 3 "vector_length_operand" " rK")
+ (match_operand 4 "const_int_operand" " i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VSMEM_OP)
+ (match_operand:VI 2 "register_operand" " vr")
+ (match_dup 0)))]
+ "TARGET_XTHEADVECTOR"
+ "vs<vlmem_op_attr>.v\t%2,%0%p1"
+ [(set_attr "type" "vste")
+ (set_attr "mode" "<MODE>")
+ (set (attr "avl_type_idx") (const_int 4))
+ (set_attr "vl_op_idx" "3")])
+
+(define_insn "@pred_strided_load_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "register_operand" "=vr, vr, vd")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1, Wc1, vm")
+ (match_operand 5 "vector_length_operand" " rK, rK, rK")
+ (match_operand 6 "const_int_operand" " i, i, i")
+ (match_operand 7 "const_int_operand" " i, i, i")
+ (match_operand 8 "const_int_operand" " i, i, i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLSMEM_OP)
+ (unspec:VI
+ [(match_operand:VI 3 "memory_operand" " m, m, m")
+ (match_operand 4 "pmode_reg_or_0_operand" " rJ, rJ, rJ")] UNSPEC_TH_VLSMEM_OP)
+ (match_operand:VI 2 "vector_merge_operand" " 0, vu, vu")))]
+ "TARGET_XTHEADVECTOR"
+ "vls<vlmem_op_attr>.v\t%0,%3,%z4%p1"
+ [(set_attr "type" "vlds")
+ (set_attr "mode" "<MODE>")])
+
+(define_insn "@pred_strided_store_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "memory_operand" "+m")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1")
+ (match_operand 4 "vector_length_operand" " rK")
+ (match_operand 5 "const_int_operand" " i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VSSMEM_OP)
+ (unspec:VI
+ [(match_operand 2 "pmode_reg_or_0_operand" " rJ")
+ (match_operand:VI 3 "register_operand" " vr")] UNSPEC_TH_VSSMEM_OP)
+ (match_dup 0)))]
+ "TARGET_XTHEADVECTOR"
+ "vss<vlmem_op_attr>.v\t%3,%0,%z2%p1"
+ [(set_attr "type" "vsts")
+ (set_attr "mode" "<MODE>")
+ (set (attr "avl_type_idx") (const_int 5))])
+
+(define_insn "@pred_indexed_load_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "register_operand" "=vd, vr,vd, vr")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" " vm,Wc1,vm,Wc1")
+ (match_operand 5 "vector_length_operand" " rK, rK,rK, rK")
+ (match_operand 6 "const_int_operand" " i, i, i, i")
+ (match_operand 7 "const_int_operand" " i, i, i, i")
+ (match_operand 8 "const_int_operand" " i, i, i, i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLXMEM_OP)
+ (unspec:VI
+ [(match_operand 3 "pmode_reg_or_0_operand" " rJ, rJ,rJ, rJ")
+ (mem:BLK (scratch))
+ (match_operand:VI 4 "register_operand" " vr, vr,vr, vr")] UNSPEC_TH_VLXMEM_OP)
+ (match_operand:VI 2 "vector_merge_operand" " vu, vu, 0, 0")))]
+ "TARGET_XTHEADVECTOR"
+ "vlx<vlmem_op_attr>.v\t%0,(%z3),%4%p1"
+ [(set_attr "type" "vldux")
+ (set_attr "mode" "<MODE>")])
+
+(define_insn "@pred_indexed_<vlmem_order_attr>store_width<vlmem_op_attr><mode>"
+ [(set (mem:BLK (scratch))
+ (unspec:BLK
+ [(unspec:<VM>
+ [(match_operand:<VM> 0 "vector_mask_operand" "vmWc1")
+ (match_operand 4 "vector_length_operand" " rK")
+ (match_operand 5 "const_int_operand" " i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VSXMEM_OP)
+ (match_operand 1 "pmode_reg_or_0_operand" " rJ")
+ (match_operand:VI 2 "register_operand" " vr")
+ (match_operand:VI 3 "register_operand" " vr")] UNSPEC_TH_VSXMEM_OP))]
+ "TARGET_XTHEADVECTOR"
+ "vs<vlmem_order_attr>x<vlmem_op_attr>.v\t%3,(%z1),%2%p0"
+ [(set_attr "type" "vstux")
+ (set_attr "mode" "<MODE>")])
+
+(define_expand "@pred_th_extract<mode>"
+ [(set (match_operand:<VEL> 0 "register_operand")
+ (unspec:<VEL>
+ [(vec_select:<VEL>
+ (match_operand:V_VLSI 1 "register_operand")
+ (parallel [(match_operand:DI 2 "register_operand" "r")]))
+ (reg:SI VTYPE_REGNUM)] UNSPEC_VPREDICATE))]
+ "TARGET_XTHEADVECTOR"
+{})
+
+(define_insn "*pred_th_extract<mode>"
+ [(set (match_operand:<VEL> 0 "register_operand" "=r")
+ (unspec:<VEL>
+ [(vec_select:<VEL>
+ (match_operand:V_VLSI 1 "register_operand" "vr")
+ (parallel [(match_operand:DI 2 "register_operand" "r")]))
+ (reg:SI VTYPE_REGNUM)] UNSPEC_VPREDICATE))]
+ "TARGET_XTHEADVECTOR"
+ "vext.x.v\t%0,%1,%2"
+ [(set_attr "type" "vimovvx")
+ (set_attr "mode" "<MODE>")])
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c
new file mode 100644
index 00000000000..4e192bbf025
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+
+** th.vadd\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+
+** th.vsb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out)
+{
+ vint32m1_t v = __riscv_th_vlb_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlb_v_i32m1_tu (v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vv_i32m1 (v2, v2, 4);
+ vint32m1_t v4 = __riscv_vadd_vv_i32m1_tu (v3, v2, v2, 4);
+ __riscv_th_vsb_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlb.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+
+** th.vadd\.vv\tv[1-9][0-9]?,\s*v[0-9]+,\s*v[0-9]+,\s*v0.t
+** th.vsb.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlb_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlb_v_i32m1_m (mask, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vv_i32m1 (v2, v2, 4);
+ vint32m1_t v4 = __riscv_vadd_vv_i32m1_m (mask, v3, v3, 4);
+ __riscv_th_vsb_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlb.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+
+** th.vadd\.vv\tv[1-9][0-9]?,\s*v[0-9]+,\s*v[0-9]+,\s*v0.t
+** th.vsb.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlb_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlb_v_i32m1_tumu (mask, v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vv_i32m1 (v2, v2, 4);
+ vint32m1_t v4 = __riscv_vadd_vv_i32m1_tumu (mask, v3, v2, v2, 4);
+ __riscv_th_vsb_v_i32m1 (out, v4, 4);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c
new file mode 100644
index 00000000000..1538afec68e
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlbu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlbu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vsb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, uint32_t x)
+{
+ vuint32m1_t v = __riscv_th_vlbu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlbu_v_u32m1_tu (v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tu (v3, v2, -16, 4);
+ __riscv_th_vsb_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlbu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsb.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlbu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlbu_v_u32m1_m (mask, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_m (mask, v3, -16, 4);
+ __riscv_th_vsb_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlbu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlbu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsb.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlbu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlbu_v_u32m1_tumu (mask, v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tumu (mask, v3, v2, -16, 4);
+ __riscv_th_vsb_v_u32m1 (out, v4, 4);
+}
\ No newline at end of file
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c
new file mode 100644
index 00000000000..bf4924a1d76
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vsh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, int32_t x)
+{
+ vint32m1_t v = __riscv_th_vlh_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlh_v_i32m1_tu (v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, -16, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_tu (v3, v2, -16, 4);
+ __riscv_th_vsh_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlh.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsh.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, int32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlh_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlh_v_i32m1_m (mask, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, -16, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_m (mask, v3, -16, 4);
+ __riscv_th_vsh_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlh.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsh.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, int32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlh_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlh_v_i32m1_tumu (mask, v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, -16, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_tumu (mask, v3, v2, -16, 4);
+ __riscv_th_vsh_v_i32m1 (out, v4, 4);
+}
\ No newline at end of file
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c
new file mode 100644
index 00000000000..8c451845175
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlhu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlhu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vsh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, uint32_t x)
+{
+ vuint32m1_t v = __riscv_th_vlhu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlhu_v_u32m1_tu (v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tu (v3, v2, -16, 4);
+ __riscv_th_vsh_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlhu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsh.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlhu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlhu_v_u32m1_m (mask, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_m (mask, v3, -16, 4);
+ __riscv_th_vsh_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlhu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlhu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsh.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlhu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlhu_v_u32m1_tumu (mask, v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tumu (mask, v3, v2, -16, 4);
+ __riscv_th_vsh_v_u32m1 (out, v4, 4);
+}
\ No newline at end of file
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c
new file mode 100644
index 00000000000..0f5b09684a5
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vx\tv[0-9]+,\s*v[0-9]+,\s*[a-x0-9]+
+** th.vadd\.vx\tv[0-9]+,\s*v[0-9]+,\s*[a-x0-9]+
+** th.vsw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, int32_t x)
+{
+ vint32m1_t v = __riscv_th_vlw_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlw_v_i32m1_tu (v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, x, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_tu (v3, v2, x, 4);
+ __riscv_th_vsw_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlw.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vx\tv[0-9]+,\s*v[0-9]+,\s*[a-x0-9]+
+** th.vadd\.vx\tv[1-9][0-9]?,\s*v[0-9]+,\s*[a-x0-9]+,\s*v0.t
+** th.vsw.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, int32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlw_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlw_v_i32m1_m (mask, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, x, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_m (mask, v3, x, 4);
+ __riscv_th_vsw_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlw.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vx\tv[0-9]+,\s*v[0-9]+,\s*[a-x0-9]+
+** th.vadd\.vx\tv[1-9][0-9]?,\s*v[0-9]+,\s*[a-x0-9]+,\s*v0.t
+** th.vsw.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, int32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlw_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlw_v_i32m1_tumu (mask, v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, x, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_tumu (mask, v3, v2, x, 4);
+ __riscv_th_vsw_v_i32m1 (out, v4, 4);
+}
\ No newline at end of file
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c
new file mode 100644
index 00000000000..aaa75be023d
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlwu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlwu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vsw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, uint32_t x)
+{
+ vuint32m1_t v = __riscv_th_vlwu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlwu_v_u32m1_tu (v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tu (v3, v2, -16, 4);
+ __riscv_th_vsw_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlwu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsw.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlwu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlwu_v_u32m1_m (mask, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_m (mask, v3, -16, 4);
+ __riscv_th_vsw_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlwu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlwu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsw.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlwu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlwu_v_u32m1_tumu (mask, v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tumu (mask, v3, v2, -16, 4);
+ __riscv_th_vsw_v_u32m1 (out, v4, 4);
+}
\ No newline at end of file
--
2.17.1
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
2024-01-10 11:14 ` Re：Re：[PATCH " joshua
@ 2024-01-10 11:16 ` juzhe.zhong
0 siblings, 0 replies; 17+ messages in thread
From: juzhe.zhong @ 2024-01-10 11:16 UTC (permalink / raw)
To: cooper.joshua, gcc-patches
Cc: Jim Wilson, palmer, andrew, philipp.tomsich, jeffreyalaw,
christoph.muellner, jinma, cooper.qu
[-- Attachment #1: Type: text/plain, Size: 80575 bytes --]
OK. Use all_v_scalar_const_ptr_ops directly, but blocks the type in shapes.
And remove all data structure like i8_v_scalar_const_ptr_ops
juzhe.zhong@rivai.ai
发件人： joshua
发送时间： 2024-01-10 19:14
收件人： juzhe.zhong@rivai.ai; gcc-patches
抄送： Jim Wilson; palmer; andrew; philipp.tomsich; jeffreyalaw; christoph.muellner; jinma; cooper.qu
主题： Re：Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
vlb can accept sew=8/16/32/64.
vlh can accept sew=16/32/64;
vlw can accept sew=32/64.
vint8m1_t __riscv_th_vlb_v_i8m1 (const int8_t *a, size_t vl);
vint8m2_t __riscv_th_vlb_v_i8m2 (const int8_t *a, size_t vl);
vint8m4_t __riscv_th_vlb_v_i8m4 (const int8_t *a, size_t vl);
vint8m8_t __riscv_th_vlb_v_i8m8 (const int8_t *a, size_t vl);
vint16m1_t __riscv_th_vlb_v_i16m1 (const int16_t *a, size_t vl);
vint16m2_t __riscv_th_vlb_v_i16m2 (const int16_t *a, size_t vl);
vint16m4_t __riscv_th_vlb_v_i16m4 (const int16_t *a, size_t vl);
vint16m8_t __riscv_th_vlb_v_i16m8 (const int16_t *a, size_t vl);
vint32m1_t __riscv_th_vlb_v_i32m1 (const int32_t *a, size_t vl);
vint32m2_t __riscv_th_vlb_v_i32m2 (const int32_t *a, size_t vl);
vint32m4_t __riscv_th_vlb_v_i32m4 (const int32_t *a, size_t vl);
vint32m8_t __riscv_th_vlb_v_i32m8 (const int32_t *a, size_t vl);
vint64m1_t __riscv_th_vlb_v_i64m1 (const int64_t *a, size_t vl);
vint64m2_t __riscv_th_vlb_v_i64m2 (const int64_t *a, size_t vl);
vint64m4_t __riscv_th_vlb_v_i64m4 (const int64_t *a, size_t vl);
vint64m8_t __riscv_th_vlb_v_i64m8 (const int64_t *a, size_t vl);
vint16m1_t __riscv_th_vlh_v_i16m1 (const int16_t *a, size_t vl);
vint16m2_t __riscv_th_vlh_v_i16m2 (const int16_t *a, size_t vl);
vint16m4_t __riscv_th_vlh_v_i16m4 (const int16_t *a, size_t vl);
vint16m8_t __riscv_th_vlh_v_i16m8 (const int16_t *a, size_t vl);
vint32m1_t __riscv_th_vlh_v_i32m1 (const int32_t *a, size_t vl);
vint32m2_t __riscv_th_vlh_v_i32m2 (const int32_t *a, size_t vl);
vint32m4_t __riscv_th_vlh_v_i32m4 (const int32_t *a, size_t vl);
vint32m8_t __riscv_th_vlh_v_i32m8 (const int32_t *a, size_t vl);
vint64m1_t __riscv_th_vlh_v_i64m1 (const int64_t *a, size_t vl);
vint64m2_t __riscv_th_vlh_v_i64m2 (const int64_t *a, size_t vl);
vint64m4_t __riscv_th_vlh_v_i64m4 (const int64_t *a, size_t vl);
vint64m8_t __riscv_th_vlh_v_i64m8 (const int64_t *a, size_t vl);
vint32m1_t __riscv_th_vlw_v_i32m1 (const int32_t *a, size_t vl);
vint32m2_t __riscv_th_vlw_v_i32m2 (const int32_t *a, size_t vl);
vint32m4_t __riscv_th_vlw_v_i32m4 (const int32_t *a, size_t vl);
vint32m8_t __riscv_th_vlw_v_i32m8 (const int32_t *a, size_t vl);
vint64m1_t __riscv_th_vlw_v_i64m1 (const int64_t *a, size_t vl);
vint64m2_t __riscv_th_vlw_v_i64m2 (const int64_t *a, size_t vl);
vint64m4_t __riscv_th_vlw_v_i64m4 (const int64_t *a, size_t vl);
vint64m8_t __riscv_th_vlw_v_i64m8 (const int64_t *a, size_t vl);
With the exisiting framework, I cannot come up with better way to differentiate
between vlb/vlw.vlh.
------------------------------------------------------------------
发件人：juzhe.zhong@rivai.ai <juzhe.zhong@rivai.ai>
发送时间：2024年1月10日(星期三) 19:09
收件人："cooper.joshua"<cooper.joshua@linux.alibaba.com>; "gcc-patches"<gcc-patches@gcc.gnu.org>
抄 送：Jim Wilson<jim.wilson.gcc@gmail.com>; palmer<palmer@dabbelt.com>; andrew<andrew@sifive.com>; "philipp.tomsich"<philipp.tomsich@vrull.eu>; jeffreyalaw<jeffreyalaw@gmail.com>; "christoph.muellner"<christoph.muellner@vrull.eu>; jinma<jinma@linux.alibaba.com>; "cooper.qu"<cooper.qu@linux.alibaba.com>
主 题：Re: Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
So vlb has not only sew = 8 ?
But why do you add intrinsics as follows ?
+DEF_RVV_FUNCTION (th_vlb, th_loadstore_width, full_preds, i8_v_scalar_const_ptr_ops)
Why it is not :
DEF_RVV_FUNCTION (th_vlb, th_loadstore_width, full_preds, all_v_scalar_const_ptr_ops)
?
juzhe.zhong@rivai.ai
发件人： joshua
发送时间： 2024-01-10 19:06
收件人： juzhe.zhong@rivai.ai; gcc-patches
抄送： Jim Wilson; palmer; andrew; philipp.tomsich; jeffreyalaw; christoph.muellner; jinma; cooper.qu
主题： Re：Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
The key difference between vlb/vlh/vlw is not output type too.
Their difference is the range of datatype, not one specific type.
We have dived into the xtheadvector special intrinsics and are
sure about that.
------------------------------------------------------------------
发件人：juzhe.zhong@rivai.ai <juzhe.zhong@rivai.ai>
发送时间：2024年1月10日(星期三) 19:00
收件人："cooper.joshua"<cooper.joshua@linux.alibaba.com>; "gcc-patches"<gcc-patches@gcc.gnu.org>
抄 送：Jim Wilson<jim.wilson.gcc@gmail.com>; palmer<palmer@dabbelt.com>; andrew<andrew@sifive.com>; "philipp.tomsich"<philipp.tomsich@vrull.eu>; jeffreyalaw<jeffreyalaw@gmail.com>; "christoph.muellner"<christoph.muellner@vrull.eu>; jinma<jinma@linux.alibaba.com>; "cooper.qu"<cooper.qu@linux.alibaba.com>
主 题：Re: Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
instance.op_info->args[i].get_tree_type (instance.type.index) is output type.
You can use GDB debug it .
juzhe.zhong@rivai.ai
发件人： joshua
发送时间： 2024-01-10 18:57
收件人： juzhe.zhong@rivai.ai; gcc-patches
抄送： Jim Wilson; palmer; andrew; philipp.tomsich; jeffreyalaw; christoph.muellner; jinma; cooper.qu
主题： Re：Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
Hi Juzhe,
Perhaps things are not as simple as imagined.
The differences between vlb/vlh/vlw is not the same
as vle8/vle16/vle32. "8", "16" or "32" in vle8/vle16/vle32
can be appended from "vle" according to input type.
But vlb/vlh/vlw is different not in input type.
------------------------------------------------------------------
发件人：juzhe.zhong@rivai.ai <juzhe.zhong@rivai.ai>
发送时间：2024年1月10日(星期三) 18:03
收件人："cooper.joshua"<cooper.joshua@linux.alibaba.com>; "gcc-patches"<gcc-patches@gcc.gnu.org>
抄 送：Jim Wilson<jim.wilson.gcc@gmail.com>; palmer<palmer@dabbelt.com>; andrew<andrew@sifive.com>; "philipp.tomsich"<philipp.tomsich@vrull.eu>; jeffreyalaw<jeffreyalaw@gmail.com>; "christoph.muellner"<christoph.muellner@vrull.eu>; jinma<jinma@linux.alibaba.com>; "cooper.qu"<cooper.qu@linux.alibaba.com>
主 题：Re: Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
I mean change these:
+DEF_RVV_FUNCTION (th_vlb, th_loadstore_width, full_preds, i8_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (th_vlh, th_loadstore_width, full_preds, i16_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (th_vlw, th_loadstore_width, full_preds, i32_v_scalar_const_ptr_ops)
into a single:
+DEF_RVV_FUNCTION (th_vl, th_loadstore_width, full_preds, all_v_scalar_const_ptr_ops)
and append "h", "w", or"b" according to
TYPE_UNSIGNED and
GET_MODE_BITSIZE (GET_MODE_INNER (TYPE_MODE (instance.op_info->args[i].get_tree_type (instance.type.index))))
in th_loadstore_width.
It should definitely works, I allow this flexibility in design of the framework.
juzhe.zhong@rivai.ai
发件人： joshua
发送时间： 2024-01-10 17:55
收件人： juzhe.zhong@rivai.ai; gcc-patches
抄送： Jim Wilson; palmer; andrew; philipp.tomsich; jeffreyalaw; christoph.muellner; jinma; cooper.qu
主题： Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
And revise th_loadstore_width, append the name according TYPE_UNSIGNED and
GET_MODE_BITSIZE (GET_MODE_INNER (TYPE_MODE (instance.op_info->args[i].get_tree_type (instance.type.index))))
What do you mean by it? I'm a bit confused.
Changing i8_v_scalar_const_ptr_ops into all_v_scalar_const_ptr_ops
will expand the datatypes that can be used in th_vlb. Can we restrict
again in th_loadstore_width?
------------------------------------------------------------------
发件人：juzhe.zhong@rivai.ai <juzhe.zhong@rivai.ai>
发送时间：2024年1月10日(星期三) 17:35
收件人："cooper.joshua"<cooper.joshua@linux.alibaba.com>; "gcc-patches"<gcc-patches@gcc.gnu.org>
抄 送：Jim Wilson<jim.wilson.gcc@gmail.com>; palmer<palmer@dabbelt.com>; andrew<andrew@sifive.com>; "philipp.tomsich"<philipp.tomsich@vrull.eu>; jeffreyalaw<jeffreyalaw@gmail.com>; "christoph.muellner"<christoph.muellner@vrull.eu>; "cooper.joshua"<cooper.joshua@linux.alibaba.com>; jinma<jinma@linux.alibaba.com>; "cooper.qu"<cooper.qu@linux.alibaba.com>
主 题：Re: [PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
+DEF_RVV_FUNCTION (th_vlb, th_loadstore_width, full_preds, i8_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (th_vlh, th_loadstore_width, full_preds, i16_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (th_vlw, th_loadstore_width, full_preds, i32_v_scalar_const_ptr_ops)
I think we should remove those many data structure you added like: i8_v_scalar_const_ptr_ops
Instead, you should use all_v_scalar_const_ptr_ops
And revise th_loadstore_width, append the name according TYPE_UNSIGNED and
GET_MODE_BITSIZE (GET_MODE_INNER (TYPE_MODE (instance.op_info->args[i].get_tree_type (instance.type.index))))
juzhe.zhong@rivai.ai
From: Jun Sha (Joshua)
Date: 2024-01-10 17:27
To: gcc-patches
CC: jim.wilson.gcc; palmer; andrew; philipp.tomsich; jeffreyalaw; christoph.muellner; juzhe.zhong; Jun Sha (Joshua); Jin Ma; Xianmiao Qu
Subject: [PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
This patch only involves the generation of xtheadvector
special load/store instructions and vext instructions.
gcc/ChangeLog:
* config/riscv/riscv-vector-builtins-bases.cc
(class th_loadstore_width): Define new builtin bases.
(BASE): Define new builtin bases.
* config/riscv/riscv-vector-builtins-bases.h:
Define new builtin class.
* config/riscv/riscv-vector-builtins-functions.def (vlsegff):
Include thead-vector-builtins-functions.def.
* config/riscv/riscv-vector-builtins-shapes.cc
(struct th_loadstore_width_def): Define new builtin shapes.
(struct th_indexed_loadstore_width_def):
Define new builtin shapes.
(SHAPE): Define new builtin shapes.
* config/riscv/riscv-vector-builtins-shapes.h:
Define new builtin shapes.
* config/riscv/riscv-vector-builtins-types.def
(DEF_RVV_I8_OPS): Add datatypes for XTheadVector.
(DEF_RVV_I16_OPS): Add datatypes for XTheadVector.
(DEF_RVV_I32_OPS): Add datatypes for XTheadVector.
(DEF_RVV_U8_OPS): Add datatypes for XTheadVector.
(DEF_RVV_U16_OPS): Add datatypes for XTheadVector.
(DEF_RVV_U32_OPS): Add datatypes for XTheadVector.
(vint8m1_t): Add datatypes for XTheadVector.
(vint8m2_t): Likewise.
(vint8m4_t): Likewise.
(vint8m8_t): Likewise.
(vint16m1_t): Likewise.
(vint16m2_t): Likewise.
(vint16m4_t): Likewise.
(vint16m8_t): Likewise.
(vint32m1_t): Likewise.
(vint32m2_t): Likewise.
(vint32m4_t): Likewise.
(vint32m8_t): Likewise.
(vint64m1_t): Likewise.
(vint64m2_t): Likewise.
(vint64m4_t): Likewise.
(vint64m8_t): Likewise.
(vuint8m1_t): Likewise.
(vuint8m2_t): Likewise.
(vuint8m4_t): Likewise.
(vuint8m8_t): Likewise.
(vuint16m1_t): Likewise.
(vuint16m2_t): Likewise.
(vuint16m4_t): Likewise.
(vuint16m8_t): Likewise.
(vuint32m1_t): Likewise.
(vuint32m2_t): Likewise.
(vuint32m4_t): Likewise.
(vuint32m8_t): Likewise.
(vuint64m1_t): Likewise.
(vuint64m2_t): Likewise.
(vuint64m4_t): Likewise.
(vuint64m8_t): Likewise.
* config/riscv/riscv-vector-builtins.cc
(DEF_RVV_I8_OPS): Add datatypes for XTheadVector.
(DEF_RVV_I16_OPS): Add datatypes for XTheadVector.
(DEF_RVV_I32_OPS): Add datatypes for XTheadVector.
(DEF_RVV_U8_OPS): Add datatypes for XTheadVector.
(DEF_RVV_U16_OPS): Add datatypes for XTheadVector.
(DEF_RVV_U32_OPS): Add datatypes for XTheadVector.
* config/riscv/thead-vector-builtins-functions.def: New file.
* config/riscv/thead-vector.md: Add new patterns.
gcc/testsuite/ChangeLog:
* gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c: New test.
Co-authored-by: Jin Ma <jinma@linux.alibaba.com>
Co-authored-by: Xianmiao Qu <cooper.qu@linux.alibaba.com>
Co-authored-by: Christoph Müllner <christoph.muellner@vrull.eu>
---
.../riscv/riscv-vector-builtins-bases.cc | 139 ++++++++
.../riscv/riscv-vector-builtins-bases.h | 31 ++
.../riscv/riscv-vector-builtins-shapes.cc | 98 ++++++
.../riscv/riscv-vector-builtins-shapes.h | 3 +
.../riscv/riscv-vector-builtins-types.def | 120 +++++++
gcc/config/riscv/riscv-vector-builtins.cc | 311 ++++++++++++++++++
gcc/config/riscv/riscv-vector-builtins.h | 3 +
gcc/config/riscv/t-riscv | 1 +
.../riscv/thead-vector-builtins-functions.def | 39 +++
gcc/config/riscv/thead-vector.md | 253 ++++++++++++++
.../riscv/rvv/xtheadvector/vlb-vsb.c | 68 ++++
.../riscv/rvv/xtheadvector/vlbu-vsb.c | 68 ++++
.../riscv/rvv/xtheadvector/vlh-vsh.c | 68 ++++
.../riscv/rvv/xtheadvector/vlhu-vsh.c | 68 ++++
.../riscv/rvv/xtheadvector/vlw-vsw.c | 68 ++++
.../riscv/rvv/xtheadvector/vlwu-vsw.c | 68 ++++
16 files changed, 1406 insertions(+)
create mode 100644 gcc/config/riscv/thead-vector-builtins-functions.def
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c
diff --git a/gcc/config/riscv/riscv-vector-builtins-bases.cc b/gcc/config/riscv/riscv-vector-builtins-bases.cc
index 46f1a1da33e..5f44f31a12b 100644
--- a/gcc/config/riscv/riscv-vector-builtins-bases.cc
+++ b/gcc/config/riscv/riscv-vector-builtins-bases.cc
@@ -2125,6 +2125,83 @@ public:
}
};
+/* Implements
+ * th.vl(b/h/w)[u].v/th.vs(b/h/w)[u].v/th.vls(b/h/w)[u].v/th.vss(b/h/w)[u].v/
+ * th.vlx(b/h/w)[u].v/th.vs[u]x(b/h/w).v
+ * codegen. */
+template<bool STORE_P, lst_type LST_TYPE, int UNSPEC>
+class th_loadstore_width : public function_base
+{
+public:
+ bool apply_tail_policy_p () const override { return !STORE_P; }
+ bool apply_mask_policy_p () const override { return !STORE_P; }
+
+ unsigned int call_properties (const function_instance &) const override
+ {
+ if (STORE_P)
+ return CP_WRITE_MEMORY;
+ else
+ return CP_READ_MEMORY;
+ }
+
+ bool can_be_overloaded_p (enum predication_type_index pred) const override
+ {
+ if (STORE_P || LST_TYPE == LST_INDEXED)
+ return true;
+ return pred != PRED_TYPE_none;
+ }
+
+ rtx expand (function_expander &e) const override
+ {
+ gcc_assert (TARGET_XTHEADVECTOR);
+ if (LST_TYPE == LST_INDEXED)
+ {
+ if (STORE_P)
+ return e.use_exact_insn (
+ code_for_pred_indexed_store_width (UNSPEC, UNSPEC,
+ e.vector_mode ()));
+ else
+ return e.use_exact_insn (
+ code_for_pred_indexed_load_width (UNSPEC, e.vector_mode ()));
+ }
+ else if (LST_TYPE == LST_STRIDED)
+ {
+ if (STORE_P)
+ return e.use_contiguous_store_insn (
+ code_for_pred_strided_store_width (UNSPEC, e.vector_mode ()));
+ else
+ return e.use_contiguous_load_insn (
+ code_for_pred_strided_load_width (UNSPEC, e.vector_mode ()));
+ }
+ else
+ {
+ if (STORE_P)
+ return e.use_contiguous_store_insn (
+ code_for_pred_store_width (UNSPEC, e.vector_mode ()));
+ else
+ return e.use_contiguous_load_insn (
+ code_for_pred_mov_width (UNSPEC, e.vector_mode ()));
+ }
+ }
+};
+
+/* Implements vext.x.v. */
+class th_extract : public function_base
+{
+public:
+ bool apply_vl_p () const override { return false; }
+ bool apply_tail_policy_p () const override { return false; }
+ bool apply_mask_policy_p () const override { return false; }
+ bool use_mask_predication_p () const override { return false; }
+ bool has_merge_operand_p () const override { return false; }
+
+ rtx expand (function_expander &e) const override
+ {
+ gcc_assert (TARGET_XTHEADVECTOR);
+ return e.use_exact_insn (code_for_pred_th_extract (e.vector_mode ()));
+ }
+};
+
/* Below implements are vector crypto */
/* Implements vandn.[vv,vx] */
class vandn : public function_base
@@ -2587,6 +2664,37 @@ static CONSTEXPR const seg_indexed_load<UNSPEC_ORDERED> vloxseg_obj;
static CONSTEXPR const seg_indexed_store<UNSPEC_UNORDERED> vsuxseg_obj;
static CONSTEXPR const seg_indexed_store<UNSPEC_ORDERED> vsoxseg_obj;
static CONSTEXPR const vlsegff vlsegff_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLB> th_vlb_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLBU> th_vlbu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLH> th_vlh_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLHU> th_vlhu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLW> th_vlw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLWU> th_vlwu_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_UNIT_STRIDE, UNSPEC_TH_VLB> th_vsb_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_UNIT_STRIDE, UNSPEC_TH_VLH> th_vsh_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_UNIT_STRIDE, UNSPEC_TH_VLW> th_vsw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSB> th_vlsb_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSBU> th_vlsbu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSH> th_vlsh_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSHU> th_vlshu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSW> th_vlsw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSWU> th_vlswu_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_STRIDED, UNSPEC_TH_VLSB> th_vssb_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_STRIDED, UNSPEC_TH_VLSH> th_vssh_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_STRIDED, UNSPEC_TH_VLSW> th_vssw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXB> th_vlxb_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXBU> th_vlxbu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXH> th_vlxh_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXHU> th_vlxhu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXW> th_vlxw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXWU> th_vlxwu_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VLXB> th_vsxb_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VLXH> th_vsxh_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VLXW> th_vsxw_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VSUXB> th_vsuxb_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VSUXH> th_vsuxh_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VSUXW> th_vsuxw_obj;
+static CONSTEXPR const th_extract th_vext_x_v_obj;
/* Crypto Vector */
static CONSTEXPR const vandn vandn_obj;
@@ -2878,6 +2986,37 @@ BASE (vloxseg)
BASE (vsuxseg)
BASE (vsoxseg)
BASE (vlsegff)
+BASE (th_vlb)
+BASE (th_vlh)
+BASE (th_vlw)
+BASE (th_vlbu)
+BASE (th_vlhu)
+BASE (th_vlwu)
+BASE (th_vsb)
+BASE (th_vsh)
+BASE (th_vsw)
+BASE (th_vlsb)
+BASE (th_vlsh)
+BASE (th_vlsw)
+BASE (th_vlsbu)
+BASE (th_vlshu)
+BASE (th_vlswu)
+BASE (th_vssb)
+BASE (th_vssh)
+BASE (th_vssw)
+BASE (th_vlxb)
+BASE (th_vlxh)
+BASE (th_vlxw)
+BASE (th_vlxbu)
+BASE (th_vlxhu)
+BASE (th_vlxwu)
+BASE (th_vsxb)
+BASE (th_vsxh)
+BASE (th_vsxw)
+BASE (th_vsuxb)
+BASE (th_vsuxh)
+BASE (th_vsuxw)
+BASE (th_vext_x_v)
/* Crypto vector */
BASE (vandn)
BASE (vbrev)
diff --git a/gcc/config/riscv/riscv-vector-builtins-bases.h b/gcc/config/riscv/riscv-vector-builtins-bases.h
index 1122e3801a7..df43adf9a17 100644
--- a/gcc/config/riscv/riscv-vector-builtins-bases.h
+++ b/gcc/config/riscv/riscv-vector-builtins-bases.h
@@ -299,6 +299,37 @@ extern const function_base *const vloxseg;
extern const function_base *const vsuxseg;
extern const function_base *const vsoxseg;
extern const function_base *const vlsegff;
+extern const function_base *const th_vlb;
+extern const function_base *const th_vlh;
+extern const function_base *const th_vlw;
+extern const function_base *const th_vlbu;
+extern const function_base *const th_vlhu;
+extern const function_base *const th_vlwu;
+extern const function_base *const th_vsb;
+extern const function_base *const th_vsh;
+extern const function_base *const th_vsw;
+extern const function_base *const th_vlsb;
+extern const function_base *const th_vlsh;
+extern const function_base *const th_vlsw;
+extern const function_base *const th_vlsbu;
+extern const function_base *const th_vlshu;
+extern const function_base *const th_vlswu;
+extern const function_base *const th_vssb;
+extern const function_base *const th_vssh;
+extern const function_base *const th_vssw;
+extern const function_base *const th_vlxb;
+extern const function_base *const th_vlxh;
+extern const function_base *const th_vlxw;
+extern const function_base *const th_vlxbu;
+extern const function_base *const th_vlxhu;
+extern const function_base *const th_vlxwu;
+extern const function_base *const th_vsxb;
+extern const function_base *const th_vsxh;
+extern const function_base *const th_vsxw;
+extern const function_base *const th_vsuxb;
+extern const function_base *const th_vsuxh;
+extern const function_base *const th_vsuxw;
+extern const function_base *const th_vext_x_v;
/* Below function_base are Vectro Crypto*/
extern const function_base *const vandn;
extern const function_base *const vbrev;
diff --git a/gcc/config/riscv/riscv-vector-builtins-shapes.cc b/gcc/config/riscv/riscv-vector-builtins-shapes.cc
index 1e4f4d53de6..489a95cf684 100644
--- a/gcc/config/riscv/riscv-vector-builtins-shapes.cc
+++ b/gcc/config/riscv/riscv-vector-builtins-shapes.cc
@@ -211,6 +211,86 @@ struct indexed_loadstore_def : public function_shape
}
};
+/* th_loadstore_width_def class. */
+struct th_loadstore_width_def : public build_base
+{
+ char *get_name (function_builder &b, const function_instance &instance,
+ bool overloaded_p) const override
+ {
+ /* Return nullptr if it can not be overloaded. */
+ if (overloaded_p && !instance.base->can_be_overloaded_p (instance.pred))
+ return nullptr;
+
+ b.append_base_name (instance.base_name);
+
+ /* vop_v --> vop_v_<type>. */
+ if (!overloaded_p)
+ {
+ /* vop --> vop_v. */
+ b.append_name (operand_suffixes[instance.op_info->op]);
+ /* vop_v --> vop_v_<type>. */
+ b.append_name (type_suffixes[instance.type.index].vector);
+ }
+
+ /* According to rvv-intrinsic-doc, it does not add "_m" suffix
+ for vop_m C++ overloaded API. */
+ if (overloaded_p && instance.pred == PRED_TYPE_m)
+ return b.finish_name ();
+ b.append_name (predication_suffixes[instance.pred]);
+ return b.finish_name ();
+ }
+};
+
+
+/* th_indexed_loadstore_width_def class. */
+struct th_indexed_loadstore_width_def : public function_shape
+{
+ void build (function_builder &b,
+ const function_group_info &group) const override
+ {
+ for (unsigned int pred_idx = 0; group.preds[pred_idx] != NUM_PRED_TYPES;
+ ++pred_idx)
+ {
+ for (unsigned int vec_type_idx = 0;
+ group.ops_infos.types[vec_type_idx].index != NUM_VECTOR_TYPES;
+ ++vec_type_idx)
+ {
+ tree index_type = group.ops_infos.args[1].get_tree_type (
+ group.ops_infos.types[vec_type_idx].index);
+ if (!index_type)
+ continue;
+ build_one (b, group, pred_idx, vec_type_idx);
+ }
+ }
+ }
+
+ char *get_name (function_builder &b, const function_instance &instance,
+ bool overloaded_p) const override
+ {
+
+ /* Return nullptr if it can not be overloaded. */
+ if (overloaded_p && !instance.base->can_be_overloaded_p (instance.pred))
+ return nullptr;
+
+ b.append_base_name (instance.base_name);
+ /* vop_v --> vop_v_<type>. */
+ if (!overloaded_p)
+ {
+ /* vop --> vop_v. */
+ b.append_name (operand_suffixes[instance.op_info->op]);
+ /* vop_v --> vop_v_<type>. */
+ b.append_name (type_suffixes[instance.type.index].vector);
+ }
+
+ /* According to rvv-intrinsic-doc, it does not add "_m" suffix
+ for vop_m C++ overloaded API. */
+ if (overloaded_p && instance.pred == PRED_TYPE_m)
+ return b.finish_name ();
+ b.append_name (predication_suffixes[instance.pred]);
+ return b.finish_name ();
+ }
+};
+
/* alu_def class. */
struct alu_def : public build_base
{
@@ -632,6 +712,21 @@ struct reduc_alu_def : public build_base
}
};
+/* th_extract_def class. */
+struct th_extract_def : public build_base
+{
+ char *get_name (function_builder &b, const function_instance &instance,
+ bool overloaded_p) const override
+ {
+ b.append_base_name (instance.base_name);
+ if (overloaded_p)
+ return b.finish_name ();
+ b.append_name (type_suffixes[instance.type.index].vector);
+ b.append_name (type_suffixes[instance.type.index].scalar);
+ return b.finish_name ();
+ }
+};
+
/* scalar_move_def class. */
struct scalar_move_def : public build_base
{
@@ -1094,6 +1189,8 @@ SHAPE(vsetvl, vsetvl)
SHAPE(vsetvl, vsetvlmax)
SHAPE(loadstore, loadstore)
SHAPE(indexed_loadstore, indexed_loadstore)
+SHAPE(th_loadstore_width, th_loadstore_width)
+SHAPE(th_indexed_loadstore_width, th_indexed_loadstore_width)
SHAPE(alu, alu)
SHAPE(alu_frm, alu_frm)
SHAPE(widen_alu, widen_alu)
@@ -1106,6 +1203,7 @@ SHAPE(move, move)
SHAPE(mask_alu, mask_alu)
SHAPE(reduc_alu, reduc_alu)
SHAPE(reduc_alu_frm, reduc_alu_frm)
+SHAPE(th_extract, th_extract)
SHAPE(scalar_move, scalar_move)
SHAPE(vundefined, vundefined)
SHAPE(misc, misc)
diff --git a/gcc/config/riscv/riscv-vector-builtins-shapes.h b/gcc/config/riscv/riscv-vector-builtins-shapes.h
index ac2a28ce017..a7624d0fabd 100644
--- a/gcc/config/riscv/riscv-vector-builtins-shapes.h
+++ b/gcc/config/riscv/riscv-vector-builtins-shapes.h
@@ -28,6 +28,8 @@ extern const function_shape *const vsetvl;
extern const function_shape *const vsetvlmax;
extern const function_shape *const loadstore;
extern const function_shape *const indexed_loadstore;
+extern const function_shape *const th_loadstore_width;
+extern const function_shape *const th_indexed_loadstore_width;
extern const function_shape *const alu;
extern const function_shape *const alu_frm;
extern const function_shape *const widen_alu;
@@ -41,6 +43,7 @@ extern const function_shape *const mask_alu;
extern const function_shape *const reduc_alu;
extern const function_shape *const reduc_alu_frm;
extern const function_shape *const scalar_move;
+extern const function_shape *const th_extract;
extern const function_shape *const vundefined;
extern const function_shape *const misc;
extern const function_shape *const vset;
diff --git a/gcc/config/riscv/riscv-vector-builtins-types.def b/gcc/config/riscv/riscv-vector-builtins-types.def
index 61019a56844..abfeb4fcd9b 100644
--- a/gcc/config/riscv/riscv-vector-builtins-types.def
+++ b/gcc/config/riscv/riscv-vector-builtins-types.def
@@ -24,12 +24,48 @@ along with GCC; see the file COPYING3. If not see
#define DEF_RVV_I_OPS(TYPE, REQUIRE)
#endif
+/* Use "DEF_RVV_I8_OPS" macro include some signed integer (i8/i16/i32/i64)
+ which will be iterated and registered as intrinsic functions. */
+#ifndef DEF_RVV_I8_OPS
+#define DEF_RVV_I8_OPS(TYPE, REQUIRE)
+#endif
+
+/* Use "DEF_RVV_I16_OPS" macro include some signed integer (i16/i32/i64)
+ which will be iterated and registered as intrinsic functions. */
+#ifndef DEF_RVV_I16_OPS
+#define DEF_RVV_I16_OPS(TYPE, REQUIRE)
+#endif
+
+/* Use "DEF_RVV_I32_OPS" macro include some signed integer (i32/i64)
+ which will be iterated and registered as intrinsic functions. */
+#ifndef DEF_RVV_I32_OPS
+#define DEF_RVV_I32_OPS(TYPE, REQUIRE)
+#endif
+
/* Use "DEF_RVV_U_OPS" macro include all unsigned integer which will be
iterated and registered as intrinsic functions. */
#ifndef DEF_RVV_U_OPS
#define DEF_RVV_U_OPS(TYPE, REQUIRE)
#endif
+/* Use "DEF_RVV_U8_OPS" macro include some unsigned integer (i8/i16/i32/i64)
+ which will be iterated and registered as intrinsic functions. */
+#ifndef DEF_RVV_U8_OPS
+#define DEF_RVV_U8_OPS(TYPE, REQUIRE)
+#endif
+
+/* Use "DEF_RVV_U16_OPS" macro include some unsigned integer (i16/i32/i64)
+ which will be iterated and registered as intrinsic functions. */
+#ifndef DEF_RVV_U16_OPS
+#define DEF_RVV_U16_OPS(TYPE, REQUIRE)
+#endif
+
+/* Use "DEF_RVV_U32_OPS" macro include some unsigned integer (i32/i64)
+ which will be iterated and registered as intrinsic functions. */
+#ifndef DEF_RVV_U32_OPS
+#define DEF_RVV_U32_OPS(TYPE, REQUIRE)
+#endif
+
/* Use "DEF_RVV_F_OPS" macro include all floating-point which will be
iterated and registered as intrinsic functions. */
#ifndef DEF_RVV_F_OPS
@@ -374,6 +410,45 @@ DEF_RVV_I_OPS (vint64m2_t, RVV_REQUIRE_ELEN_64)
DEF_RVV_I_OPS (vint64m4_t, RVV_REQUIRE_ELEN_64)
DEF_RVV_I_OPS (vint64m8_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_I8_OPS (vint8m1_t, 0)
+DEF_RVV_I8_OPS (vint8m2_t, 0)
+DEF_RVV_I8_OPS (vint8m4_t, 0)
+DEF_RVV_I8_OPS (vint8m8_t, 0)
+DEF_RVV_I8_OPS (vint16m1_t, 0)
+DEF_RVV_I8_OPS (vint16m2_t, 0)
+DEF_RVV_I8_OPS (vint16m4_t, 0)
+DEF_RVV_I8_OPS (vint16m8_t, 0)
+DEF_RVV_I8_OPS (vint32m1_t, 0)
+DEF_RVV_I8_OPS (vint32m2_t, 0)
+DEF_RVV_I8_OPS (vint32m4_t, 0)
+DEF_RVV_I8_OPS (vint32m8_t, 0)
+DEF_RVV_I8_OPS (vint64m1_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_I8_OPS (vint64m2_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_I8_OPS (vint64m4_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_I8_OPS (vint64m8_t, RVV_REQUIRE_ELEN_64)
+
+DEF_RVV_I16_OPS (vint16m1_t, 0)
+DEF_RVV_I16_OPS (vint16m2_t, 0)
+DEF_RVV_I16_OPS (vint16m4_t, 0)
+DEF_RVV_I16_OPS (vint16m8_t, 0)
+DEF_RVV_I16_OPS (vint32m1_t, 0)
+DEF_RVV_I16_OPS (vint32m2_t, 0)
+DEF_RVV_I16_OPS (vint32m4_t, 0)
+DEF_RVV_I16_OPS (vint32m8_t, 0)
+DEF_RVV_I16_OPS (vint64m1_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_I16_OPS (vint64m2_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_I16_OPS (vint64m4_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_I16_OPS (vint64m8_t, RVV_REQUIRE_ELEN_64)
+
+DEF_RVV_I32_OPS (vint32m1_t, 0)
+DEF_RVV_I32_OPS (vint32m2_t, 0)
+DEF_RVV_I32_OPS (vint32m4_t, 0)
+DEF_RVV_I32_OPS (vint32m8_t, 0)
+DEF_RVV_I32_OPS (vint64m1_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_I32_OPS (vint64m2_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_I32_OPS (vint64m4_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_I32_OPS (vint64m8_t, RVV_REQUIRE_ELEN_64)
+
DEF_RVV_U_OPS (vuint8mf8_t, RVV_REQUIRE_MIN_VLEN_64)
DEF_RVV_U_OPS (vuint8mf4_t, 0)
DEF_RVV_U_OPS (vuint8mf2_t, 0)
@@ -397,6 +472,45 @@ DEF_RVV_U_OPS (vuint64m2_t, RVV_REQUIRE_ELEN_64)
DEF_RVV_U_OPS (vuint64m4_t, RVV_REQUIRE_ELEN_64)
DEF_RVV_U_OPS (vuint64m8_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_U8_OPS (vuint8m1_t, 0)
+DEF_RVV_U8_OPS (vuint8m2_t, 0)
+DEF_RVV_U8_OPS (vuint8m4_t, 0)
+DEF_RVV_U8_OPS (vuint8m8_t, 0)
+DEF_RVV_U8_OPS (vuint16m1_t, 0)
+DEF_RVV_U8_OPS (vuint16m2_t, 0)
+DEF_RVV_U8_OPS (vuint16m4_t, 0)
+DEF_RVV_U8_OPS (vuint16m8_t, 0)
+DEF_RVV_U8_OPS (vuint32m1_t, 0)
+DEF_RVV_U8_OPS (vuint32m2_t, 0)
+DEF_RVV_U8_OPS (vuint32m4_t, 0)
+DEF_RVV_U8_OPS (vuint32m8_t, 0)
+DEF_RVV_U8_OPS (vuint64m1_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_U8_OPS (vuint64m2_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_U8_OPS (vuint64m4_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_U8_OPS (vuint64m8_t, RVV_REQUIRE_ELEN_64)
+
+DEF_RVV_U16_OPS (vuint16m1_t, 0)
+DEF_RVV_U16_OPS (vuint16m2_t, 0)
+DEF_RVV_U16_OPS (vuint16m4_t, 0)
+DEF_RVV_U16_OPS (vuint16m8_t, 0)
+DEF_RVV_U16_OPS (vuint32m1_t, 0)
+DEF_RVV_U16_OPS (vuint32m2_t, 0)
+DEF_RVV_U16_OPS (vuint32m4_t, 0)
+DEF_RVV_U16_OPS (vuint32m8_t, 0)
+DEF_RVV_U16_OPS (vuint64m1_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_U16_OPS (vuint64m2_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_U16_OPS (vuint64m4_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_U16_OPS (vuint64m8_t, RVV_REQUIRE_ELEN_64)
+
+DEF_RVV_U32_OPS (vuint32m1_t, 0)
+DEF_RVV_U32_OPS (vuint32m2_t, 0)
+DEF_RVV_U32_OPS (vuint32m4_t, 0)
+DEF_RVV_U32_OPS (vuint32m8_t, 0)
+DEF_RVV_U32_OPS (vuint64m1_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_U32_OPS (vuint64m2_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_U32_OPS (vuint64m4_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_U32_OPS (vuint64m8_t, RVV_REQUIRE_ELEN_64)
+
DEF_RVV_F_OPS (vfloat16mf4_t, RVV_REQUIRE_ELEN_FP_16 | RVV_REQUIRE_MIN_VLEN_64)
DEF_RVV_F_OPS (vfloat16mf2_t, RVV_REQUIRE_ELEN_FP_16)
DEF_RVV_F_OPS (vfloat16m1_t, RVV_REQUIRE_ELEN_FP_16)
@@ -1379,7 +1493,13 @@ DEF_RVV_CRYPTO_SEW64_OPS (vuint64m4_t, RVV_REQUIRE_ELEN_64)
DEF_RVV_CRYPTO_SEW64_OPS (vuint64m8_t, RVV_REQUIRE_ELEN_64)
#undef DEF_RVV_I_OPS
+#undef DEF_RVV_I8_OPS
+#undef DEF_RVV_I16_OPS
+#undef DEF_RVV_I32_OPS
#undef DEF_RVV_U_OPS
+#undef DEF_RVV_U8_OPS
+#undef DEF_RVV_U16_OPS
+#undef DEF_RVV_U32_OPS
#undef DEF_RVV_F_OPS
#undef DEF_RVV_B_OPS
#undef DEF_RVV_WEXTI_OPS
diff --git a/gcc/config/riscv/riscv-vector-builtins.cc b/gcc/config/riscv/riscv-vector-builtins.cc
index 25e0b6e56de..f429f12dc18 100644
--- a/gcc/config/riscv/riscv-vector-builtins.cc
+++ b/gcc/config/riscv/riscv-vector-builtins.cc
@@ -246,6 +246,63 @@ static const rvv_type_info iu_ops[] = {
#include "riscv-vector-builtins-types.def"
{NUM_VECTOR_TYPES, 0}};
+/* A list of all integer will be registered for intrinsic functions. */
+static const rvv_type_info i8_ops[] = {
+#define DEF_RVV_I8_OPS(TYPE, REQUIRE) {VECTOR_TYPE_##TYPE, REQUIRE},
+#include "riscv-vector-builtins-types.def"
+ {NUM_VECTOR_TYPES, 0}};
+
+/* A list of all integer will be registered for intrinsic functions. */
+static const rvv_type_info i16_ops[] = {
+#define DEF_RVV_I16_OPS(TYPE, REQUIRE) {VECTOR_TYPE_##TYPE, REQUIRE},
+#include "riscv-vector-builtins-types.def"
+ {NUM_VECTOR_TYPES, 0}};
+
+/* A list of all integer will be registered for intrinsic functions. */
+static const rvv_type_info i32_ops[] = {
+#define DEF_RVV_I32_OPS(TYPE, REQUIRE) {VECTOR_TYPE_##TYPE, REQUIRE},
+#include "riscv-vector-builtins-types.def"
+ {NUM_VECTOR_TYPES, 0}};
+
+/* A list of all integer will be registered for intrinsic functions. */
+static const rvv_type_info u8_ops[] = {
+#define DEF_RVV_U8_OPS(TYPE, REQUIRE) {VECTOR_TYPE_##TYPE, REQUIRE},
+#include "riscv-vector-builtins-types.def"
+ {NUM_VECTOR_TYPES, 0}};
+
+/* A list of all integer will be registered for intrinsic functions. */
+static const rvv_type_info u16_ops[] = {
+#define DEF_RVV_U16_OPS(TYPE, REQUIRE) {VECTOR_TYPE_##TYPE, REQUIRE},
+#include "riscv-vector-builtins-types.def"
+ {NUM_VECTOR_TYPES, 0}};
+
+/* A list of all integer will be registered for intrinsic functions. */
+static const rvv_type_info u32_ops[] = {
+#define DEF_RVV_U32_OPS(TYPE, REQUIRE) {VECTOR_TYPE_##TYPE, REQUIRE},
+#include "riscv-vector-builtins-types.def"
+ {NUM_VECTOR_TYPES, 0}};
+
+/* A list of all integer will be registered for intrinsic functions. */
+static const rvv_type_info iu8_ops[] = {
+#define DEF_RVV_I8_OPS(TYPE, REQUIRE) {VECTOR_TYPE_##TYPE, REQUIRE},
+#define DEF_RVV_U8_OPS(TYPE, REQUIRE) {VECTOR_TYPE_##TYPE, REQUIRE},
+#include "riscv-vector-builtins-types.def"
+ {NUM_VECTOR_TYPES, 0}};
+
+/* A list of all integer will be registered for intrinsic functions. */
+static const rvv_type_info iu16_ops[] = {
+#define DEF_RVV_I16_OPS(TYPE, REQUIRE) {VECTOR_TYPE_##TYPE, REQUIRE},
+#define DEF_RVV_U16_OPS(TYPE, REQUIRE) {VECTOR_TYPE_##TYPE, REQUIRE},
+#include "riscv-vector-builtins-types.def"
+ {NUM_VECTOR_TYPES, 0}};
+
+/* A list of all integer will be registered for intrinsic functions. */
+static const rvv_type_info iu32_ops[] = {
+#define DEF_RVV_I32_OPS(TYPE, REQUIRE) {VECTOR_TYPE_##TYPE, REQUIRE},
+#define DEF_RVV_U32_OPS(TYPE, REQUIRE) {VECTOR_TYPE_##TYPE, REQUIRE},
+#include "riscv-vector-builtins-types.def"
+ {NUM_VECTOR_TYPES, 0}};
+
/* A list of all types will be registered for intrinsic functions. */
static const rvv_type_info all_ops[] = {
#define DEF_RVV_I_OPS(TYPE, REQUIRE) {VECTOR_TYPE_##TYPE, REQUIRE},
@@ -934,6 +991,32 @@ static CONSTEXPR const rvv_arg_type_info ext_vcreate_args[]
= {rvv_arg_type_info (RVV_BASE_vector),
rvv_arg_type_info_end};
+/* A list of args for vector_type func (const scalar_type *, size_t)
+ * function. */
+static CONSTEXPR const rvv_arg_type_info scalar_const_ptr_size_args[]
+ = {rvv_arg_type_info (RVV_BASE_scalar_const_ptr),
+ rvv_arg_type_info (RVV_BASE_size), rvv_arg_type_info_end};
+
+/* A list of args for vector_type func (const scalar_type *, eew8_index_type)
+ * function. */
+static CONSTEXPR const rvv_arg_type_info scalar_const_ptr_index_args[]
+ = {rvv_arg_type_info (RVV_BASE_scalar_const_ptr),
+ rvv_arg_type_info (RVV_BASE_unsigned_vector), rvv_arg_type_info_end};
+
+/* A list of args for void func (scalar_type *, eew8_index_type, vector_type)
+ * function. */
+static CONSTEXPR const rvv_arg_type_info scalar_ptr_index_args[]
+ = {rvv_arg_type_info (RVV_BASE_scalar_ptr),
+ rvv_arg_type_info (RVV_BASE_unsigned_vector),
+ rvv_arg_type_info (RVV_BASE_vector), rvv_arg_type_info_end};
+
+/* A list of args for void func (scalar_type *, size_t, vector_type)
+ * function. */
+static CONSTEXPR const rvv_arg_type_info scalar_ptr_size_args[]
+ = {rvv_arg_type_info (RVV_BASE_scalar_ptr),
+ rvv_arg_type_info (RVV_BASE_size), rvv_arg_type_info (RVV_BASE_vector),
+ rvv_arg_type_info_end};
+
/* A list of none preds that will be registered for intrinsic functions. */
static CONSTEXPR const predication_type_index none_preds[]
= {PRED_TYPE_none, NUM_PRED_TYPES};
@@ -1455,6 +1538,14 @@ static CONSTEXPR const rvv_op_info iu_shift_vvv_ops
rvv_arg_type_info (RVV_BASE_vector), /* Return type */
shift_vv_args /* Args */};
+/* A static operand information for scalar_type func (vector_type, size_t)
+ * function registration. */
+static CONSTEXPR const rvv_op_info iu_x_s_u_ops
+ = {iu_ops, /* Types */
+ OP_TYPE_vx, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_scalar), /* Return type */
+ v_size_args /* Args */};
+
/* A static operand information for vector_type func (vector_type, size_t)
* function registration. */
static CONSTEXPR const rvv_op_info iu_shift_vvx_ops
@@ -2638,6 +2729,222 @@ static CONSTEXPR const rvv_op_info all_v_vcreate_lmul4_x2_ops
rvv_arg_type_info (RVV_BASE_vlmul_ext_x2), /* Return type */
ext_vcreate_args /* Args */};
+/* A static operand information for vector_type func (const scalar_type *)
+ * function registration. */
+static CONSTEXPR const rvv_op_info i8_v_scalar_const_ptr_ops
+ = {i8_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *)
+ * function registration. */
+static CONSTEXPR const rvv_op_info i16_v_scalar_const_ptr_ops
+ = {i16_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *)
+ * function registration. */
+static CONSTEXPR const rvv_op_info i32_v_scalar_const_ptr_ops
+ = {i32_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *)
+ * function registration. */
+static CONSTEXPR const rvv_op_info u8_v_scalar_const_ptr_ops
+ = {u8_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *)
+ * function registration. */
+static CONSTEXPR const rvv_op_info u16_v_scalar_const_ptr_ops
+ = {u16_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *)
+ * function registration. */
+static CONSTEXPR const rvv_op_info u32_v_scalar_const_ptr_ops
+ = {u32_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * size_t) function registration. */
+static CONSTEXPR const rvv_op_info i8_v_scalar_const_ptr_size_ops
+ = {i8_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_size_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * size_t) function registration. */
+static CONSTEXPR const rvv_op_info i16_v_scalar_const_ptr_size_ops
+ = {i16_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_size_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * size_t) function registration. */
+static CONSTEXPR const rvv_op_info i32_v_scalar_const_ptr_size_ops
+ = {i32_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_size_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * size_t) function registration. */
+static CONSTEXPR const rvv_op_info u8_v_scalar_const_ptr_size_ops
+ = {u8_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_size_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * size_t) function registration. */
+static CONSTEXPR const rvv_op_info u16_v_scalar_const_ptr_size_ops
+ = {u16_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_size_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * size_t) function registration. */
+static CONSTEXPR const rvv_op_info u32_v_scalar_const_ptr_size_ops
+ = {u32_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_size_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * eew8_index_type) function registration. */
+static CONSTEXPR const rvv_op_info i8_v_scalar_const_ptr_index_ops
+ = {i8_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_index_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * eew8_index_type) function registration. */
+static CONSTEXPR const rvv_op_info u8_v_scalar_const_ptr_index_ops
+ = {u8_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_index_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * eew8_index_type) function registration. */
+static CONSTEXPR const rvv_op_info i16_v_scalar_const_ptr_index_ops
+ = {i16_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_index_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * eew8_index_type) function registration. */
+static CONSTEXPR const rvv_op_info u16_v_scalar_const_ptr_index_ops
+ = {u16_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_index_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * eew8_index_type) function registration. */
+static CONSTEXPR const rvv_op_info i32_v_scalar_const_ptr_index_ops
+ = {i32_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_index_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * eew8_index_type) function registration. */
+static CONSTEXPR const rvv_op_info u32_v_scalar_const_ptr_index_ops
+ = {u32_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_index_args /* Args */};
+
+/* A static operand information for void func (scalar_type *, eew8_index_type,
+ * vector_type) function registration. */
+static CONSTEXPR const rvv_op_info iu8_v_scalar_ptr_index_ops
+ = {iu8_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_void), /* Return type */
+ scalar_ptr_index_args /* Args */};
+
+/* A static operand information for void func (scalar_type *, eew16_index_type,
+ * vector_type) function registration. */
+static CONSTEXPR const rvv_op_info iu16_v_scalar_ptr_index_ops
+ = {iu16_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_void), /* Return type */
+ scalar_ptr_index_args /* Args */};
+
+/* A static operand information for void func (scalar_type *, eew32_index_type,
+ * vector_type) function registration. */
+static CONSTEXPR const rvv_op_info iu32_v_scalar_ptr_index_ops
+ = {iu32_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_void), /* Return type */
+ scalar_ptr_index_args /* Args */};
+
+/* A static operand information for void func (scalar_type *, vector_type,
+ * function registration. */
+static CONSTEXPR const rvv_op_info iu8_v_scalar_ptr_ops
+ = {iu8_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_void), /* Return type */
+ scalar_ptr_args /* Args */};
+
+/* A static operand information for void func (scalar_type *, vector_type)
+ * function registration. */
+static CONSTEXPR const rvv_op_info iu16_v_scalar_ptr_ops
+ = {iu16_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_void), /* Return type */
+ scalar_ptr_args /* Args */};
+
+/* A static operand information for void func (scalar_type *, vector_type)
+ * function registration. */
+static CONSTEXPR const rvv_op_info iu32_v_scalar_ptr_ops
+ = {iu32_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_void), /* Return type */
+ scalar_ptr_args /* Args */};
+
+/* A static operand information for void func (scalar_type *, size_t,
+ * vector_type) function registration. */
+static CONSTEXPR const rvv_op_info iu8_v_scalar_ptr_size_ops
+ = {iu8_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_void), /* Return type */
+ scalar_ptr_size_args /* Args */};
+
+/* A static operand information for void func (scalar_type *, size_t,
+ * vector_type) function registration. */
+static CONSTEXPR const rvv_op_info iu16_v_scalar_ptr_size_ops
+ = {iu16_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_void), /* Return type */
+ scalar_ptr_size_args /* Args */};
+
+/* A static operand information for void func (scalar_type *, size_t,
+ * vector_type) function registration. */
+static CONSTEXPR const rvv_op_info iu32_v_scalar_ptr_size_ops
+ = {iu32_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_void), /* Return type */
+ scalar_ptr_size_args /* Args */};
+
/* A static operand information for vector_type func (vector_type).
Some ins just supports SEW=32, such as crypto vectol Zvkg extension.
* function registration. */
@@ -2816,6 +3123,10 @@ static function_group_info function_groups[] = {
#define DEF_RVV_FUNCTION(NAME, SHAPE, PREDS, OPS_INFO) \
{#NAME, &bases::NAME, &shapes::SHAPE, PREDS, OPS_INFO, REQUIRED_EXTENSIONS},
#include "riscv-vector-builtins-functions.def"
+#undef DEF_RVV_FUNCTION
+#define DEF_RVV_FUNCTION(NAME, SHAPE, PREDS, OPS_INFO) \
+ {#NAME, &bases::NAME, &shapes::SHAPE, PREDS, OPS_INFO, REQUIRED_EXTENSIONS},
+#include "thead-vector-builtins-functions.def"
};
/* The RVV types, with their built-in
diff --git a/gcc/config/riscv/riscv-vector-builtins.h b/gcc/config/riscv/riscv-vector-builtins.h
index 54c8824ff92..a8ee39a3cb2 100644
--- a/gcc/config/riscv/riscv-vector-builtins.h
+++ b/gcc/config/riscv/riscv-vector-builtins.h
@@ -114,6 +114,7 @@ static const unsigned int CP_WRITE_CSR = 1U << 5;
enum required_ext
{
VECTOR_EXT, /* Vector extension */
+ XTHEADVECTOR_EXT, /* XTheadVector extension */
ZVBB_EXT, /* Cryto vector Zvbb sub-ext */
ZVBB_OR_ZVKB_EXT, /* Cryto vector Zvbb or zvkb sub-ext */
ZVBC_EXT, /* Crypto vector Zvbc sub-ext */
@@ -234,6 +235,8 @@ struct function_group_info
{
case VECTOR_EXT:
return TARGET_VECTOR;
+ case XTHEADVECTOR_EXT:
+ return TARGET_XTHEADVECTOR;
case ZVBB_EXT:
return TARGET_ZVBB;
case ZVBB_OR_ZVKB_EXT:
diff --git a/gcc/config/riscv/t-riscv b/gcc/config/riscv/t-riscv
index 32de6b851c1..38494320d8b 100644
--- a/gcc/config/riscv/t-riscv
+++ b/gcc/config/riscv/t-riscv
@@ -1,6 +1,7 @@
RISCV_BUILTINS_H = $(srcdir)/config/riscv/riscv-vector-builtins.h \
$(srcdir)/config/riscv/riscv-vector-builtins.def \
$(srcdir)/config/riscv/riscv-vector-builtins-functions.def \
+ $(srcdir)/config/riscv/thead-vector-builtins-functions.def \
riscv-vector-type-indexer.gen.def
riscv-builtins.o: $(srcdir)/config/riscv/riscv-builtins.cc $(CONFIG_H) \
diff --git a/gcc/config/riscv/thead-vector-builtins-functions.def b/gcc/config/riscv/thead-vector-builtins-functions.def
new file mode 100644
index 00000000000..667820d4c3e
--- /dev/null
+++ b/gcc/config/riscv/thead-vector-builtins-functions.def
@@ -0,0 +1,39 @@
+#ifndef DEF_RVV_FUNCTION
+#define DEF_RVV_FUNCTION(NAME, SHAPE, PREDS, OPS_INFO)
+#endif
+
+#define REQUIRED_EXTENSIONS XTHEADVECTOR_EXT
+DEF_RVV_FUNCTION (th_vlb, th_loadstore_width, full_preds, i8_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (th_vlh, th_loadstore_width, full_preds, i16_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (th_vlw, th_loadstore_width, full_preds, i32_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (th_vlbu, th_loadstore_width, full_preds, u8_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (th_vlhu, th_loadstore_width, full_preds, u16_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (th_vlwu, th_loadstore_width, full_preds, u32_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (th_vsb, th_loadstore_width, none_m_preds, iu8_v_scalar_ptr_ops)
+DEF_RVV_FUNCTION (th_vsh, th_loadstore_width, none_m_preds, iu16_v_scalar_ptr_ops)
+DEF_RVV_FUNCTION (th_vsw, th_loadstore_width, none_m_preds, iu32_v_scalar_ptr_ops)
+DEF_RVV_FUNCTION (th_vlsb, th_loadstore_width, full_preds, i8_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (th_vlsh, th_loadstore_width, full_preds, i16_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (th_vlsw, th_loadstore_width, full_preds, i32_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (th_vlsbu, th_loadstore_width, full_preds, u8_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (th_vlshu, th_loadstore_width, full_preds, u16_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (th_vlswu, th_loadstore_width, full_preds, u32_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (th_vssb, th_loadstore_width, none_m_preds, iu8_v_scalar_ptr_size_ops)
+DEF_RVV_FUNCTION (th_vssh, th_loadstore_width, none_m_preds, iu16_v_scalar_ptr_size_ops)
+DEF_RVV_FUNCTION (th_vssw, th_loadstore_width, none_m_preds, iu32_v_scalar_ptr_size_ops)
+DEF_RVV_FUNCTION (th_vlxb, th_indexed_loadstore_width, full_preds, i8_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (th_vlxh, th_indexed_loadstore_width, full_preds, i16_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (th_vlxw, th_indexed_loadstore_width, full_preds, i32_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (th_vlxbu, th_indexed_loadstore_width, full_preds, u8_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (th_vlxhu, th_indexed_loadstore_width, full_preds, u16_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (th_vlxwu, th_indexed_loadstore_width, full_preds, u32_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (th_vsxb, th_indexed_loadstore_width, none_m_preds, iu8_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (th_vsxh, th_indexed_loadstore_width, none_m_preds, iu16_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (th_vsxw, th_indexed_loadstore_width, none_m_preds, iu32_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (th_vsuxb, th_indexed_loadstore_width, none_m_preds, iu8_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (th_vsuxh, th_indexed_loadstore_width, none_m_preds, iu16_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (th_vsuxw, th_indexed_loadstore_width, none_m_preds, iu32_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (th_vext_x_v, th_extract, none_preds, iu_x_s_u_ops)
+#undef REQUIRED_EXTENSIONS
+
+#undef DEF_RVV_FUNCTION
diff --git a/gcc/config/riscv/thead-vector.md b/gcc/config/riscv/thead-vector.md
index 696b815252d..0f3700d9269 100644
--- a/gcc/config/riscv/thead-vector.md
+++ b/gcc/config/riscv/thead-vector.md
@@ -1,7 +1,95 @@
(define_c_enum "unspec" [
+ UNSPEC_TH_VLB
+ UNSPEC_TH_VLBU
+ UNSPEC_TH_VLH
+ UNSPEC_TH_VLHU
+ UNSPEC_TH_VLW
+ UNSPEC_TH_VLWU
+
+ UNSPEC_TH_VLSB
+ UNSPEC_TH_VLSBU
+ UNSPEC_TH_VLSH
+ UNSPEC_TH_VLSHU
+ UNSPEC_TH_VLSW
+ UNSPEC_TH_VLSWU
+
+ UNSPEC_TH_VLXB
+ UNSPEC_TH_VLXBU
+ UNSPEC_TH_VLXH
+ UNSPEC_TH_VLXHU
+ UNSPEC_TH_VLXW
+ UNSPEC_TH_VLXWU
+
+ UNSPEC_TH_VSUXB
+ UNSPEC_TH_VSUXH
+ UNSPEC_TH_VSUXW
+
UNSPEC_TH_VWLDST
])
+(define_int_iterator UNSPEC_TH_VLMEM_OP [
+ UNSPEC_TH_VLB UNSPEC_TH_VLBU
+ UNSPEC_TH_VLH UNSPEC_TH_VLHU
+ UNSPEC_TH_VLW UNSPEC_TH_VLWU
+])
+
+(define_int_iterator UNSPEC_TH_VLSMEM_OP [
+ UNSPEC_TH_VLSB UNSPEC_TH_VLSBU
+ UNSPEC_TH_VLSH UNSPEC_TH_VLSHU
+ UNSPEC_TH_VLSW UNSPEC_TH_VLSWU
+])
+
+(define_int_iterator UNSPEC_TH_VLXMEM_OP [
+ UNSPEC_TH_VLXB UNSPEC_TH_VLXBU
+ UNSPEC_TH_VLXH UNSPEC_TH_VLXHU
+ UNSPEC_TH_VLXW UNSPEC_TH_VLXWU
+])
+
+(define_int_attr vlmem_op_attr [
+ (UNSPEC_TH_VLB "b") (UNSPEC_TH_VLBU "bu")
+ (UNSPEC_TH_VLH "h") (UNSPEC_TH_VLHU "hu")
+ (UNSPEC_TH_VLW "w") (UNSPEC_TH_VLWU "wu")
+ (UNSPEC_TH_VLSB "b") (UNSPEC_TH_VLSBU "bu")
+ (UNSPEC_TH_VLSH "h") (UNSPEC_TH_VLSHU "hu")
+ (UNSPEC_TH_VLSW "w") (UNSPEC_TH_VLSWU "wu")
+ (UNSPEC_TH_VLXB "b") (UNSPEC_TH_VLXBU "bu")
+ (UNSPEC_TH_VLXH "h") (UNSPEC_TH_VLXHU "hu")
+ (UNSPEC_TH_VLXW "w") (UNSPEC_TH_VLXWU "wu")
+ (UNSPEC_TH_VSUXB "b")
+ (UNSPEC_TH_VSUXH "h")
+ (UNSPEC_TH_VSUXW "w")
+])
+
+(define_int_attr vlmem_order_attr [
+ (UNSPEC_TH_VLXB "")
+ (UNSPEC_TH_VLXH "")
+ (UNSPEC_TH_VLXW "")
+ (UNSPEC_TH_VSUXB "u")
+ (UNSPEC_TH_VSUXH "u")
+ (UNSPEC_TH_VSUXW "u")
+])
+
+(define_int_iterator UNSPEC_TH_VSMEM_OP [
+ UNSPEC_TH_VLB
+ UNSPEC_TH_VLH
+ UNSPEC_TH_VLW
+])
+
+(define_int_iterator UNSPEC_TH_VSSMEM_OP [
+ UNSPEC_TH_VLSB
+ UNSPEC_TH_VLSH
+ UNSPEC_TH_VLSW
+])
+
+(define_int_iterator UNSPEC_TH_VSXMEM_OP [
+ UNSPEC_TH_VLXB
+ UNSPEC_TH_VLXH
+ UNSPEC_TH_VLXW
+ UNSPEC_TH_VSUXB
+ UNSPEC_TH_VSUXH
+ UNSPEC_TH_VSUXW
+])
+
(define_mode_iterator V_VLS_VT [V VLS VT])
(define_mode_iterator V_VB_VLS_VT [V VB VLS VT])
@@ -100,3 +188,168 @@
}
[(set_attr "type" "vldm,vstm,vmalu,vmalu,vmalu")
(set_attr "mode" "<MODE>")])
+
+(define_expand "@pred_mov_width<vlmem_op_attr><mode>"
+ [(set (match_operand:V_VLS 0 "nonimmediate_operand")
+ (if_then_else:V_VLS
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand")
+ (match_operand 4 "vector_length_operand")
+ (match_operand 5 "const_int_operand")
+ (match_operand 6 "const_int_operand")
+ (match_operand 7 "const_int_operand")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLMEM_OP)
+ (match_operand:V_VLS 3 "vector_move_operand")
+ (match_operand:V_VLS 2 "vector_merge_operand")))]
+ "TARGET_XTHEADVECTOR"
+ {})
+
+(define_insn_and_split "*pred_mov_width<vlmem_op_attr><mode>"
+ [(set (match_operand:V_VLS 0 "nonimmediate_operand" "=vr, vr, vd, m, vr, vr")
+ (if_then_else:V_VLS
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1, Wc1, vm, vmWc1, Wc1, Wc1")
+ (match_operand 4 "vector_length_operand" " rK, rK, rK, rK, rK, rK")
+ (match_operand 5 "const_int_operand" " i, i, i, i, i, i")
+ (match_operand 6 "const_int_operand" " i, i, i, i, i, i")
+ (match_operand 7 "const_int_operand" " i, i, i, i, i, i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLMEM_OP)
+ (match_operand:V_VLS 3 "reg_or_mem_operand" " m, m, m, vr, vr, vr")
+ (match_operand:V_VLS 2 "vector_merge_operand" " 0, vu, vu, vu, vu, 0")))]
+ "(TARGET_XTHEADVECTOR
+ && (register_operand (operands[0], <MODE>mode)
+ || register_operand (operands[3], <MODE>mode)))"
+ "@
+ vl<vlmem_op_attr>.v\t%0,%3%p1
+ vl<vlmem_op_attr>.v\t%0,%3
+ vl<vlmem_op_attr>.v\t%0,%3,%1.t
+ vs<vlmem_op_attr>.v\t%3,%0%p1
+ vmv.v.v\t%0,%3
+ vmv.v.v\t%0,%3"
+ "&& register_operand (operands[0], <MODE>mode)
+ && register_operand (operands[3], <MODE>mode)
+ && satisfies_constraint_vu (operands[2])
+ && INTVAL (operands[7]) == riscv_vector::VLMAX"
+ [(set (match_dup 0) (match_dup 3))]
+ ""
+ [(set_attr "type" "vlde,vlde,vlde,vste,vimov,vimov")
+ (set_attr "mode" "<MODE>")])
+
+(define_insn "@pred_store_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "memory_operand" "+m")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1")
+ (match_operand 3 "vector_length_operand" " rK")
+ (match_operand 4 "const_int_operand" " i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VSMEM_OP)
+ (match_operand:VI 2 "register_operand" " vr")
+ (match_dup 0)))]
+ "TARGET_XTHEADVECTOR"
+ "vs<vlmem_op_attr>.v\t%2,%0%p1"
+ [(set_attr "type" "vste")
+ (set_attr "mode" "<MODE>")
+ (set (attr "avl_type_idx") (const_int 4))
+ (set_attr "vl_op_idx" "3")])
+
+(define_insn "@pred_strided_load_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "register_operand" "=vr, vr, vd")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1, Wc1, vm")
+ (match_operand 5 "vector_length_operand" " rK, rK, rK")
+ (match_operand 6 "const_int_operand" " i, i, i")
+ (match_operand 7 "const_int_operand" " i, i, i")
+ (match_operand 8 "const_int_operand" " i, i, i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLSMEM_OP)
+ (unspec:VI
+ [(match_operand:VI 3 "memory_operand" " m, m, m")
+ (match_operand 4 "pmode_reg_or_0_operand" " rJ, rJ, rJ")] UNSPEC_TH_VLSMEM_OP)
+ (match_operand:VI 2 "vector_merge_operand" " 0, vu, vu")))]
+ "TARGET_XTHEADVECTOR"
+ "vls<vlmem_op_attr>.v\t%0,%3,%z4%p1"
+ [(set_attr "type" "vlds")
+ (set_attr "mode" "<MODE>")])
+
+(define_insn "@pred_strided_store_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "memory_operand" "+m")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1")
+ (match_operand 4 "vector_length_operand" " rK")
+ (match_operand 5 "const_int_operand" " i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VSSMEM_OP)
+ (unspec:VI
+ [(match_operand 2 "pmode_reg_or_0_operand" " rJ")
+ (match_operand:VI 3 "register_operand" " vr")] UNSPEC_TH_VSSMEM_OP)
+ (match_dup 0)))]
+ "TARGET_XTHEADVECTOR"
+ "vss<vlmem_op_attr>.v\t%3,%0,%z2%p1"
+ [(set_attr "type" "vsts")
+ (set_attr "mode" "<MODE>")
+ (set (attr "avl_type_idx") (const_int 5))])
+
+(define_insn "@pred_indexed_load_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "register_operand" "=vd, vr,vd, vr")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" " vm,Wc1,vm,Wc1")
+ (match_operand 5 "vector_length_operand" " rK, rK,rK, rK")
+ (match_operand 6 "const_int_operand" " i, i, i, i")
+ (match_operand 7 "const_int_operand" " i, i, i, i")
+ (match_operand 8 "const_int_operand" " i, i, i, i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLXMEM_OP)
+ (unspec:VI
+ [(match_operand 3 "pmode_reg_or_0_operand" " rJ, rJ,rJ, rJ")
+ (mem:BLK (scratch))
+ (match_operand:VI 4 "register_operand" " vr, vr,vr, vr")] UNSPEC_TH_VLXMEM_OP)
+ (match_operand:VI 2 "vector_merge_operand" " vu, vu, 0, 0")))]
+ "TARGET_XTHEADVECTOR"
+ "vlx<vlmem_op_attr>.v\t%0,(%z3),%4%p1"
+ [(set_attr "type" "vldux")
+ (set_attr "mode" "<MODE>")])
+
+(define_insn "@pred_indexed_<vlmem_order_attr>store_width<vlmem_op_attr><mode>"
+ [(set (mem:BLK (scratch))
+ (unspec:BLK
+ [(unspec:<VM>
+ [(match_operand:<VM> 0 "vector_mask_operand" "vmWc1")
+ (match_operand 4 "vector_length_operand" " rK")
+ (match_operand 5 "const_int_operand" " i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VSXMEM_OP)
+ (match_operand 1 "pmode_reg_or_0_operand" " rJ")
+ (match_operand:VI 2 "register_operand" " vr")
+ (match_operand:VI 3 "register_operand" " vr")] UNSPEC_TH_VSXMEM_OP))]
+ "TARGET_XTHEADVECTOR"
+ "vs<vlmem_order_attr>x<vlmem_op_attr>.v\t%3,(%z1),%2%p0"
+ [(set_attr "type" "vstux")
+ (set_attr "mode" "<MODE>")])
+
+(define_expand "@pred_th_extract<mode>"
+ [(set (match_operand:<VEL> 0 "register_operand")
+ (unspec:<VEL>
+ [(vec_select:<VEL>
+ (match_operand:V_VLSI 1 "register_operand")
+ (parallel [(match_operand:DI 2 "register_operand" "r")]))
+ (reg:SI VTYPE_REGNUM)] UNSPEC_VPREDICATE))]
+ "TARGET_XTHEADVECTOR"
+{})
+
+(define_insn "*pred_th_extract<mode>"
+ [(set (match_operand:<VEL> 0 "register_operand" "=r")
+ (unspec:<VEL>
+ [(vec_select:<VEL>
+ (match_operand:V_VLSI 1 "register_operand" "vr")
+ (parallel [(match_operand:DI 2 "register_operand" "r")]))
+ (reg:SI VTYPE_REGNUM)] UNSPEC_VPREDICATE))]
+ "TARGET_XTHEADVECTOR"
+ "vext.x.v\t%0,%1,%2"
+ [(set_attr "type" "vimovvx")
+ (set_attr "mode" "<MODE>")])
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c
new file mode 100644
index 00000000000..4e192bbf025
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+
+** th.vadd\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+
+** th.vsb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out)
+{
+ vint32m1_t v = __riscv_th_vlb_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlb_v_i32m1_tu (v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vv_i32m1 (v2, v2, 4);
+ vint32m1_t v4 = __riscv_vadd_vv_i32m1_tu (v3, v2, v2, 4);
+ __riscv_th_vsb_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlb.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+
+** th.vadd\.vv\tv[1-9][0-9]?,\s*v[0-9]+,\s*v[0-9]+,\s*v0.t
+** th.vsb.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlb_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlb_v_i32m1_m (mask, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vv_i32m1 (v2, v2, 4);
+ vint32m1_t v4 = __riscv_vadd_vv_i32m1_m (mask, v3, v3, 4);
+ __riscv_th_vsb_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlb.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+
+** th.vadd\.vv\tv[1-9][0-9]?,\s*v[0-9]+,\s*v[0-9]+,\s*v0.t
+** th.vsb.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlb_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlb_v_i32m1_tumu (mask, v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vv_i32m1 (v2, v2, 4);
+ vint32m1_t v4 = __riscv_vadd_vv_i32m1_tumu (mask, v3, v2, v2, 4);
+ __riscv_th_vsb_v_i32m1 (out, v4, 4);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c
new file mode 100644
index 00000000000..1538afec68e
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlbu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlbu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vsb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, uint32_t x)
+{
+ vuint32m1_t v = __riscv_th_vlbu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlbu_v_u32m1_tu (v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tu (v3, v2, -16, 4);
+ __riscv_th_vsb_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlbu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsb.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlbu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlbu_v_u32m1_m (mask, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_m (mask, v3, -16, 4);
+ __riscv_th_vsb_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlbu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlbu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsb.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlbu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlbu_v_u32m1_tumu (mask, v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tumu (mask, v3, v2, -16, 4);
+ __riscv_th_vsb_v_u32m1 (out, v4, 4);
+}
\ No newline at end of file
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c
new file mode 100644
index 00000000000..bf4924a1d76
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vsh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, int32_t x)
+{
+ vint32m1_t v = __riscv_th_vlh_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlh_v_i32m1_tu (v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, -16, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_tu (v3, v2, -16, 4);
+ __riscv_th_vsh_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlh.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsh.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, int32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlh_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlh_v_i32m1_m (mask, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, -16, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_m (mask, v3, -16, 4);
+ __riscv_th_vsh_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlh.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsh.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, int32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlh_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlh_v_i32m1_tumu (mask, v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, -16, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_tumu (mask, v3, v2, -16, 4);
+ __riscv_th_vsh_v_i32m1 (out, v4, 4);
+}
\ No newline at end of file
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c
new file mode 100644
index 00000000000..8c451845175
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlhu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlhu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vsh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, uint32_t x)
+{
+ vuint32m1_t v = __riscv_th_vlhu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlhu_v_u32m1_tu (v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tu (v3, v2, -16, 4);
+ __riscv_th_vsh_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlhu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsh.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlhu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlhu_v_u32m1_m (mask, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_m (mask, v3, -16, 4);
+ __riscv_th_vsh_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlhu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlhu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsh.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlhu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlhu_v_u32m1_tumu (mask, v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tumu (mask, v3, v2, -16, 4);
+ __riscv_th_vsh_v_u32m1 (out, v4, 4);
+}
\ No newline at end of file
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c
new file mode 100644
index 00000000000..0f5b09684a5
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vx\tv[0-9]+,\s*v[0-9]+,\s*[a-x0-9]+
+** th.vadd\.vx\tv[0-9]+,\s*v[0-9]+,\s*[a-x0-9]+
+** th.vsw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, int32_t x)
+{
+ vint32m1_t v = __riscv_th_vlw_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlw_v_i32m1_tu (v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, x, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_tu (v3, v2, x, 4);
+ __riscv_th_vsw_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlw.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vx\tv[0-9]+,\s*v[0-9]+,\s*[a-x0-9]+
+** th.vadd\.vx\tv[1-9][0-9]?,\s*v[0-9]+,\s*[a-x0-9]+,\s*v0.t
+** th.vsw.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, int32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlw_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlw_v_i32m1_m (mask, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, x, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_m (mask, v3, x, 4);
+ __riscv_th_vsw_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlw.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vx\tv[0-9]+,\s*v[0-9]+,\s*[a-x0-9]+
+** th.vadd\.vx\tv[1-9][0-9]?,\s*v[0-9]+,\s*[a-x0-9]+,\s*v0.t
+** th.vsw.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, int32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlw_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlw_v_i32m1_tumu (mask, v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, x, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_tumu (mask, v3, v2, x, 4);
+ __riscv_th_vsw_v_i32m1 (out, v4, 4);
+}
\ No newline at end of file
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c
new file mode 100644
index 00000000000..aaa75be023d
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlwu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlwu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vsw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, uint32_t x)
+{
+ vuint32m1_t v = __riscv_th_vlwu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlwu_v_u32m1_tu (v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tu (v3, v2, -16, 4);
+ __riscv_th_vsw_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlwu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsw.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlwu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlwu_v_u32m1_m (mask, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_m (mask, v3, -16, 4);
+ __riscv_th_vsw_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlwu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlwu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsw.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlwu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlwu_v_u32m1_tumu (mask, v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tumu (mask, v3, v2, -16, 4);
+ __riscv_th_vsw_v_u32m1 (out, v4, 4);
+}
\ No newline at end of file
--
2.17.1
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re：Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
2024-01-12 1:14 回复：[PATCH " juzhe.zhong
@ 2024-01-12 3:26 ` joshua
0 siblings, 0 replies; 17+ messages in thread
From: joshua @ 2024-01-12 3:26 UTC (permalink / raw)
To: juzhe.zhong, gcc-patches
Cc: Jim Wilson, palmer, andrew, philipp.tomsich, jeffreyalaw,
christoph.muellner, jinma, cooper.qu
[-- Attachment #1: Type: text/plain, Size: 61749 bytes --]
I have resent all the patches in a new patchset thread.
The TARGET_XTHEADVECTOR for sext/zext in
"Handle differences between XTheadvector and Vector"
have been removed, and all the changelog have been updated.
[PATCH v5] RISC-V: Support XTheadVector extension (gnu.org) <https://gcc.gnu.org/pipermail/gcc-patches/2024-January/642780.html >
------------------------------------------------------------------
发件人：juzhe.zhong@rivai.ai <juzhe.zhong@rivai.ai>
发送时间：2024年1月12日(星期五) 09:14
收件人："cooper.joshua"<cooper.joshua@linux.alibaba.com>; "gcc-patches"<gcc-patches@gcc.gnu.org>
抄 送：Jim Wilson<jim.wilson.gcc@gmail.com>; palmer<palmer@dabbelt.com>; andrew<andrew@sifive.com>; "philipp.tomsich"<philipp.tomsich@vrull.eu>; jeffreyalaw<jeffreyalaw@gmail.com>; "christoph.muellner"<christoph.muellner@vrull.eu>; jinma<jinma@linux.alibaba.com>; "cooper.qu"<cooper.qu@linux.alibaba.com>
主 题：Re: 回复：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
I notice the changelog is incorrect. Plz update the change log:
gcc/ChangeLog: * config.gcc: Add files for XTheadVector intrinsics. * config/riscv/autovec.md: Guard XTheadVector. * config/riscv/riscv-c.cc: Add pragma for XTheadVector. * config/riscv/riscv-string.cc (expand_block_move): Guard XTheadVector. * config/riscv/riscv-string.cc (vls_mode_valid_p): Avoid autovec. * config/riscv/riscv-vector-builtins-shapes.cc (check_type): (build_one): New function. * config/riscv/riscv-vector-builtins.cc (DEF_RVV_FUNCTION): (DEF_THEAD_RVV_FUNCTION): Add new marcos. (check_required_extensions): (handle_pragma_vector): * config/riscv/riscv-vector-builtins.h (RVV_REQUIRE_VECTOR): (RVV_REQUIRE_XTHEADVECTOR): Add RVV_REQUIRE_VECTOR and RVV_REQUIRE_XTHEADVECTOR. (struct function_group_info): * config/riscv/riscv-vector-switch.def (ENTRY): Disable fractional mode for the XTheadVector extension. (TUPLE_ENTRY): Likewise. * config/riscv/riscv-vsetvl.cc: Add functions for xtheadvector. * config/riscv/riscv.cc (riscv_v_ext_vls_mode_p): Guard XTheadVector. (riscv_v_adjust_bytesize): Likewise. (riscv_preferred_simd_mode): Likewsie. (riscv_autovectorize_vector_modes): Likewise. (riscv_vector_mode_supported_any_target_p): Likewise. (TARGET_VECTOR_MODE_SUPPORTED_ANY_TARGET_P): Likewise. * config/riscv/vector.md: Include thead-vector.md. * config/riscv/riscv_th_vector.h: New file. * config/riscv/thead-vector.md: New file.
The changlog doesn't match your patch. I suspect you other patches have same issues.
Plz update them and resend.
juzhe.zhong@rivai.ai
发件人： joshua <mailto:cooper.joshua@linux.alibaba.com >
发送时间： 2024-01-12 08:49
收件人： 钟居哲 <mailto:juzhe.zhong@rivai.ai >; gcc-patches <mailto:gcc-patches@gcc.gnu.org >
抄送： jim.wilson.gcc <mailto:jim.wilson.gcc@gmail.com >; palmer <mailto:palmer@dabbelt.com >; andrew <mailto:andrew@sifive.com >; philipp.tomsich <mailto:philipp.tomsich@vrull.eu >; Jeff Law <mailto:jeffreyalaw@gmail.com >; Christoph Müllner <mailto:christoph.muellner@vrull.eu >; jinma <mailto:jinma@linux.alibaba.com >; Cooper Qu <mailto:cooper.qu@linux.alibaba.com >
主题： 回复：Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
The changes to lmul and sew are only done for vbool,
which is RVVMF64BI/RVVMF32BI/RVVMF16BI, in the
intrinsics with mask. That is because theadvector has no
load/store mask instructions.
lmul and sew in other non-bool types do not need
to be modified. I don’t think directly change get_sew
and get_vlmul is reasonable.
+ (and (eq_attr "type" "vlde,vste,vlsegde,vssegte,vlsegds,vssegts,\ + vlsegdff,vssegtux,vlsegdox,vlsegdux") + (match_test "TARGET_XTHEADVECTOR")) + (const_int INVALID_ATTRIBUTE)Why do you add this ?
theadvector does not have sew in load/storeinstructions. In O2, vector will remove onevsetvl for two consecutive vle with differentsew. However, things will go wrong under thisoptimization for theadvector.
------------------------------------------------------------------
发件人：钟居哲 <juzhe.zhong@rivai.ai>
发送时间：2024年1月12日(星期五) 07:22
收件人："cooper.joshua"<cooper.joshua@linux.alibaba.com>; "gcc-patches"<gcc-patches@gcc.gnu.org>
抄 送："jim.wilson.gcc"<jim.wilson.gcc@gmail.com>; palmer<palmer@dabbelt.com>; andrew<andrew@sifive.com>; "philipp.tomsich"<philipp.tomsich@vrull.eu>; Jeff Law<jeffreyalaw@gmail.com>; "Christoph Müllner"<christoph.muellner@vrull.eu>; jinma<jinma@linux.alibaba.com>; Cooper Qu<cooper.qu@linux.alibaba.com>
主 题：Re: Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
Also, you don't need to change these:
-ENTRY (RVVMF64BI, TARGET_MIN_VLEN > 32, LMUL_F8, 64) -ENTRY (RVVMF32BI, true, LMUL_F4, 32) -ENTRY (RVVMF16BI, true, LMUL_F2, 16) +ENTRY (RVVMF64BI, TARGET_MIN_VLEN > 32, TARGET_XTHEADVECTOR ? LMUL_1 :LMUL_F8, 64) +ENTRY (RVVMF32BI, true, TARGET_XTHEADVECTOR ? LMUL_1 :LMUL_F4, 32) +ENTRY (RVVMF16BI, true, TARGET_XTHEADVECTOR ? LMUL_1 : LMUL_F2 , 16)
You should specify these cases in
enum vlmul_type
get_vlmul (machine_mode mode)
And don't change:
(define_attr "sew" "" - (cond [(eq_attr "mode" "RVVMF64BI,RVVMF32BI,RVVMF16BI,RVVMF8BI,RVVMF4BI,RVVMF2BI,RVVM1BI,\ + (cond [(eq_attr "mode" "RVVMF8BI,RVVMF4BI,RVVMF2BI,RVVM1BI,\ RVVM8QI,RVVM4QI,RVVM2QI,RVVM1QI,RVVMF2QI,RVVMF4QI,RVVMF8QI,\ RVVM1x8QI,RVVMF2x8QI,RVVMF4x8QI,RVVMF8x8QI,\ RVVM1x7QI,RVVMF2x7QI,RVVMF4x7QI,RVVMF8x7QI,\ @@ -99,6 +99,18 @@ V1QI,V2QI,V4QI,V8QI,V16QI,V32QI,V64QI,V128QI,V256QI,V512QI,V1024QI,V2048QI,V4096QI,\ V1BI,V2BI,V4BI,V8BI,V16BI,V32BI,V64BI,V128BI,V256BI,V512BI,V1024BI,V2048BI,V4096BI") (const_int 8) + (eq_attr "mode" "RVVMF16BI") + (if_then_else (match_test "TARGET_XTHEADVECTOR") + (const_int 16) + (const_int 8)) + (eq_attr "mode" "RVVMF32BI") + (if_then_else (match_test "TARGET_XTHEADVECTOR") + (const_int 32) + (const_int 8)) + (eq_attr "mode" "RVVMF64BI") + (if_then_else (match_test "TARGET_XTHEADVECTOR") + (const_int 64) + (const_int 8)) (eq_attr "mode" "RVVM8HI,RVVM4HI,RVVM2HI,RVVM1HI,RVVMF2HI,RVVMF4HI,\ RVVM1x8HI,RVVMF2x8HI,RVVMF4x8HI,\ RVVM1x7HI,RVVMF2x7HI,RVVMF4x7HI,\ @@ -159,9 +171,9 @@ (eq_attr "mode" "RVVM4QI,RVVMF2BI") (symbol_ref "riscv_vector::LMUL_4") (eq_attr "mode" "RVVM2QI,RVVMF4BI") (symbol_ref "riscv_vector::LMUL_2") (eq_attr "mode" "RVVM1QI,RVVMF8BI") (symbol_ref "riscv_vector::LMUL_1") - (eq_attr "mode" "RVVMF2QI,RVVMF16BI") (symbol_ref "riscv_vector::LMUL_F2") - (eq_attr "mode" "RVVMF4QI,RVVMF32BI") (symbol_ref "riscv_vector::LMUL_F4") - (eq_attr "mode" "RVVMF8QI,RVVMF64BI") (symbol_ref "riscv_vector::LMUL_F8") + (eq_attr "mode" "RVVMF2QI,RVVMF16BI") (symbol_ref "TARGET_XTHEADVECTOR ? riscv_vector::LMUL_1 : riscv_vector::LMUL_F2") + (eq_attr "mode" "RVVMF4QI,RVVMF32BI") (symbol_ref "TARGET_XTHEADVECTOR ? riscv_vector::LMUL_1 : riscv_vector::LMUL_F4") + (eq_attr "mode" "RVVMF8QI,RVVMF64BI") (symbol_ref "TARGET_XTHEADVECTOR ? riscv_vector::LMUL_1 : riscv_vector::LMUL_F8")You should change
/* Helper function to get SEW operand. We always have SEW value for
all RVV instructions that have VTYPE OP. */
uint8_t
get_sew (rtx_insn *rinsn)
{
return get_attr_sew (rinsn);
}
/* Helper function to get VLMUL operand. We always have VLMUL value for
all RVV instructions that have VTYPE OP. */
enum vlmul_type
get_vlmul (rtx_insn *rinsn)
{
return (enum vlmul_type) get_attr_vlmul (rinsn);
}
+ (and (eq_attr "type" "vlde,vste,vlsegde,vssegte,vlsegds,vssegts,\ + vlsegdff,vssegtux,vlsegdox,vlsegdux") + (match_test "TARGET_XTHEADVECTOR")) + (const_int INVALID_ATTRIBUTE)Why do you add this ?
juzhe.zhong@rivai.ai
发件人： joshua <mailto:cooper.joshua@linux.alibaba.com >
发送时间： 2024-01-11 22:11
收件人： juzhe.zhong@rivai.ai <mailto:juzhe.zhong@rivai.ai >; gcc-patches <mailto:gcc-patches@gcc.gnu.org >
抄送： Jim Wilson <mailto:jim.wilson.gcc@gmail.com >; palmer <mailto:palmer@dabbelt.com >; andrew <mailto:andrew@sifive.com >; philipp.tomsich <mailto:philipp.tomsich@vrull.eu >; jeffreyalaw <mailto:jeffreyalaw@gmail.com >; christoph.muellner <mailto:christoph.muellner@vrull.eu >; jinma <mailto:jinma@linux.alibaba.com >; cooper.qu <mailto:cooper.qu@linux.alibaba.com >
主题： Re：Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
Is the patch with !TARGET_XTHEADVECTOR for sext/zext
patterns removed OK to commit?
https://gcc.gnu.org/pipermail/gcc-patches/2024-January/642657.html <https://gcc.gnu.org/pipermail/gcc-patches/2024-January/642657.html >
------------------------------------------------------------------
发件人：juzhe.zhong@rivai.ai <juzhe.zhong@rivai.ai>
发送时间：2024年1月11日(星期四) 18:56
收件人："cooper.joshua"<cooper.joshua@linux.alibaba.com>; "gcc-patches"<gcc-patches@gcc.gnu.org>
抄 送：Jim Wilson<jim.wilson.gcc@gmail.com>; palmer<palmer@dabbelt.com>; andrew<andrew@sifive.com>; "philipp.tomsich"<philipp.tomsich@vrull.eu>; jeffreyalaw<jeffreyalaw@gmail.com>; "christoph.muellner"<christoph.muellner@vrull.eu>; jinma<jinma@linux.alibaba.com>; "cooper.qu"<cooper.qu@linux.alibaba.com>
主 题：Re: Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
Yes.
juzhe.zhong@rivai.ai
发件人： joshua
发送时间： 2024-01-11 18:54
收件人： juzhe.zhong@rivai.ai; gcc-patches
抄送： Jim Wilson; palmer; andrew; philipp.tomsich; jeffreyalaw; christoph.muellner; jinma; cooper.qu
主题： Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
Do you mean removing TARGET_XTHEADVECTOR for sext/zext patterns
and then resending the "RISC-V: Handle differences between XTheadvector
and Vector" patch?
------------------------------------------------------------------
发件人：juzhe.zhong@rivai.ai <juzhe.zhong@rivai.ai>
发送时间：2024年1月11日(星期四) 17:57
收件人："cooper.joshua"<cooper.joshua@linux.alibaba.com>; "gcc-patches"<gcc-patches@gcc.gnu.org>
抄 送：Jim Wilson<jim.wilson.gcc@gmail.com>; palmer<palmer@dabbelt.com>; andrew<andrew@sifive.com>; "philipp.tomsich"<philipp.tomsich@vrull.eu>; jeffreyalaw<jeffreyalaw@gmail.com>; "christoph.muellner"<christoph.muellner@vrull.eu>; "cooper.joshua"<cooper.joshua@linux.alibaba.com>; jinma<jinma@linux.alibaba.com>; "cooper.qu"<cooper.qu@linux.alibaba.com>
主 题：Re: [PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
LGTM. Could you resend the patch "RISC-V: Handle differences between XTheadvector and Vector
Thanks.
juzhe.zhong@rivai.ai
From: Jun Sha (Joshua)
Date: 2024-01-11 17:52
To: gcc-patches
CC: jim.wilson.gcc; palmer; andrew; philipp.tomsich; jeffreyalaw; christoph.muellner; juzhe.zhong; Jun Sha (Joshua); Jin Ma; Xianmiao Qu
Subject: [PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
This patch only involves the generation of xtheadvector
special load/store instructions and vext instructions.
gcc/ChangeLog:
* config/riscv/riscv-vector-builtins-bases.cc
(class th_loadstore_width): Define new builtin bases.
(class th_extract): Define new builtin bases.
(BASE): Define new builtin bases.
* config/riscv/riscv-vector-builtins-bases.h:
Define new builtin class.
* config/riscv/riscv-vector-builtins-shapes.cc
(struct th_loadstore_width_def): Define new builtin shapes.
(struct th_indexed_loadstore_width_def):
Define new builtin shapes.
(struct th_extract_def): Define new builtin shapes.
(SHAPE): Define new builtin shapes.
* config/riscv/riscv-vector-builtins-shapes.h:
Define new builtin shapes.
* config/riscv/riscv-vector-builtins.cc (DEF_RVV_FUNCTION):
* config/riscv/riscv-vector-builtins.h (enum required_ext):
(struct function_group_info):
* config/riscv/t-riscv: Add thead-vector-builtins-functions.def
* config/riscv/thead-vector.md
(@pred_mov_width<vlmem_op_attr><mode>): Add new patterns.
(*pred_mov_width<vlmem_op_attr><mode>): Likewise.
(@pred_store_width<vlmem_op_attr><mode>): Likewise.
(@pred_strided_load_width<vlmem_op_attr><mode>): Likewise.
(@pred_strided_store_width<vlmem_op_attr><mode>): Likewise.
(@pred_indexed_load_width<vlmem_op_attr><mode>): Likewise.
(@pred_indexed_<vlmem_order_attr>store_width<vlmem_op_attr><mode>):
(@pred_th_extract<mode>): Likewise.
(*pred_th_extract<mode>): Likewise.
* config/riscv/thead-vector-builtins-functions.def: New file.
gcc/testsuite/ChangeLog:
* gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c: New test.
Co-authored-by: Jin Ma <jinma@linux.alibaba.com>
Co-authored-by: Xianmiao Qu <cooper.qu@linux.alibaba.com>
Co-authored-by: Christoph Müllner <christoph.muellner@vrull.eu>
---
.../riscv/riscv-vector-builtins-bases.cc | 139 ++++++++++
.../riscv/riscv-vector-builtins-bases.h | 31 +++
.../riscv/riscv-vector-builtins-shapes.cc | 160 +++++++++++
.../riscv/riscv-vector-builtins-shapes.h | 3 +
gcc/config/riscv/riscv-vector-builtins.cc | 70 +++++
gcc/config/riscv/riscv-vector-builtins.h | 3 +
gcc/config/riscv/t-riscv | 1 +
.../riscv/thead-vector-builtins-functions.def | 39 +++
gcc/config/riscv/thead-vector.md | 250 ++++++++++++++++++
.../riscv/rvv/xtheadvector/vlb-vsb.c | 68 +++++
.../riscv/rvv/xtheadvector/vlbu-vsb.c | 68 +++++
.../riscv/rvv/xtheadvector/vlh-vsh.c | 68 +++++
.../riscv/rvv/xtheadvector/vlhu-vsh.c | 68 +++++
.../riscv/rvv/xtheadvector/vlw-vsw.c | 68 +++++
.../riscv/rvv/xtheadvector/vlwu-vsw.c | 68 +++++
15 files changed, 1104 insertions(+)
create mode 100644 gcc/config/riscv/thead-vector-builtins-functions.def
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c
diff --git a/gcc/config/riscv/riscv-vector-builtins-bases.cc b/gcc/config/riscv/riscv-vector-builtins-bases.cc
index 1aa6e3c6665..b6f6e4ff37e 100644
--- a/gcc/config/riscv/riscv-vector-builtins-bases.cc
+++ b/gcc/config/riscv/riscv-vector-builtins-bases.cc
@@ -2141,6 +2141,83 @@ public:
}
};
+/* Implements
+ * th.vl(b/h/w)[u].v/th.vs(b/h/w)[u].v/th.vls(b/h/w)[u].v/th.vss(b/h/w)[u].v/
+ * th.vlx(b/h/w)[u].v/th.vs[u]x(b/h/w).v
+ * codegen. */
+template<bool STORE_P, lst_type LST_TYPE, int UNSPEC>
+class th_loadstore_width : public function_base
+{
+public:
+ bool apply_tail_policy_p () const override { return !STORE_P; }
+ bool apply_mask_policy_p () const override { return !STORE_P; }
+
+ unsigned int call_properties (const function_instance &) const override
+ {
+ if (STORE_P)
+ return CP_WRITE_MEMORY;
+ else
+ return CP_READ_MEMORY;
+ }
+
+ bool can_be_overloaded_p (enum predication_type_index pred) const override
+ {
+ if (STORE_P || LST_TYPE == LST_INDEXED)
+ return true;
+ return pred != PRED_TYPE_none;
+ }
+
+ rtx expand (function_expander &e) const override
+ {
+ gcc_assert (TARGET_XTHEADVECTOR);
+ if (LST_TYPE == LST_INDEXED)
+ {
+ if (STORE_P)
+ return e.use_exact_insn (
+ code_for_pred_indexed_store_width (UNSPEC, UNSPEC,
+ e.vector_mode ()));
+ else
+ return e.use_exact_insn (
+ code_for_pred_indexed_load_width (UNSPEC, e.vector_mode ()));
+ }
+ else if (LST_TYPE == LST_STRIDED)
+ {
+ if (STORE_P)
+ return e.use_contiguous_store_insn (
+ code_for_pred_strided_store_width (UNSPEC, e.vector_mode ()));
+ else
+ return e.use_contiguous_load_insn (
+ code_for_pred_strided_load_width (UNSPEC, e.vector_mode ()));
+ }
+ else
+ {
+ if (STORE_P)
+ return e.use_contiguous_store_insn (
+ code_for_pred_store_width (UNSPEC, e.vector_mode ()));
+ else
+ return e.use_contiguous_load_insn (
+ code_for_pred_mov_width (UNSPEC, e.vector_mode ()));
+ }
+ }
+};
+
+/* Implements vext.x.v. */
+class th_extract : public function_base
+{
+public:
+ bool apply_vl_p () const override { return false; }
+ bool apply_tail_policy_p () const override { return false; }
+ bool apply_mask_policy_p () const override { return false; }
+ bool use_mask_predication_p () const override { return false; }
+ bool has_merge_operand_p () const override { return false; }
+
+ rtx expand (function_expander &e) const override
+ {
+ gcc_assert (TARGET_XTHEADVECTOR);
+ return e.use_exact_insn (code_for_pred_th_extract (e.vector_mode ()));
+ }
+};
+
/* Below implements are vector crypto */
/* Implements vandn.[vv,vx] */
class vandn : public function_base
@@ -2603,6 +2680,37 @@ static CONSTEXPR const seg_indexed_load<UNSPEC_ORDERED> vloxseg_obj;
static CONSTEXPR const seg_indexed_store<UNSPEC_UNORDERED> vsuxseg_obj;
static CONSTEXPR const seg_indexed_store<UNSPEC_ORDERED> vsoxseg_obj;
static CONSTEXPR const vlsegff vlsegff_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLB> vlb_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLBU> vlbu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLH> vlh_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLHU> vlhu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLW> vlw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLWU> vlwu_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_UNIT_STRIDE, UNSPEC_TH_VLB> vsb_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_UNIT_STRIDE, UNSPEC_TH_VLH> vsh_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_UNIT_STRIDE, UNSPEC_TH_VLW> vsw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSB> vlsb_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSBU> vlsbu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSH> vlsh_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSHU> vlshu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSW> vlsw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSWU> vlswu_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_STRIDED, UNSPEC_TH_VLSB> vssb_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_STRIDED, UNSPEC_TH_VLSH> vssh_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_STRIDED, UNSPEC_TH_VLSW> vssw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXB> vlxb_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXBU> vlxbu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXH> vlxh_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXHU> vlxhu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXW> vlxw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXWU> vlxwu_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VLXB> vsxb_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VLXH> vsxh_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VLXW> vsxw_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VSUXB> vsuxb_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VSUXH> vsuxh_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VSUXW> vsuxw_obj;
+static CONSTEXPR const th_extract vext_x_v_obj;
/* Crypto Vector */
static CONSTEXPR const vandn vandn_obj;
@@ -2894,6 +3002,37 @@ BASE (vloxseg)
BASE (vsuxseg)
BASE (vsoxseg)
BASE (vlsegff)
+BASE (vlb)
+BASE (vlh)
+BASE (vlw)
+BASE (vlbu)
+BASE (vlhu)
+BASE (vlwu)
+BASE (vsb)
+BASE (vsh)
+BASE (vsw)
+BASE (vlsb)
+BASE (vlsh)
+BASE (vlsw)
+BASE (vlsbu)
+BASE (vlshu)
+BASE (vlswu)
+BASE (vssb)
+BASE (vssh)
+BASE (vssw)
+BASE (vlxb)
+BASE (vlxh)
+BASE (vlxw)
+BASE (vlxbu)
+BASE (vlxhu)
+BASE (vlxwu)
+BASE (vsxb)
+BASE (vsxh)
+BASE (vsxw)
+BASE (vsuxb)
+BASE (vsuxh)
+BASE (vsuxw)
+BASE (vext_x_v)
/* Crypto vector */
BASE (vandn)
BASE (vbrev)
diff --git a/gcc/config/riscv/riscv-vector-builtins-bases.h b/gcc/config/riscv/riscv-vector-builtins-bases.h
index 87c7f43a423..1f2c94d3541 100644
--- a/gcc/config/riscv/riscv-vector-builtins-bases.h
+++ b/gcc/config/riscv/riscv-vector-builtins-bases.h
@@ -280,6 +280,37 @@ extern const function_base *const vloxseg;
extern const function_base *const vsuxseg;
extern const function_base *const vsoxseg;
extern const function_base *const vlsegff;
+extern const function_base *const vlb;
+extern const function_base *const vlh;
+extern const function_base *const vlw;
+extern const function_base *const vlbu;
+extern const function_base *const vlhu;
+extern const function_base *const vlwu;
+extern const function_base *const vsb;
+extern const function_base *const vsh;
+extern const function_base *const vsw;
+extern const function_base *const vlsb;
+extern const function_base *const vlsh;
+extern const function_base *const vlsw;
+extern const function_base *const vlsbu;
+extern const function_base *const vlshu;
+extern const function_base *const vlswu;
+extern const function_base *const vssb;
+extern const function_base *const vssh;
+extern const function_base *const vssw;
+extern const function_base *const vlxb;
+extern const function_base *const vlxh;
+extern const function_base *const vlxw;
+extern const function_base *const vlxbu;
+extern const function_base *const vlxhu;
+extern const function_base *const vlxwu;
+extern const function_base *const vsxb;
+extern const function_base *const vsxh;
+extern const function_base *const vsxw;
+extern const function_base *const vsuxb;
+extern const function_base *const vsuxh;
+extern const function_base *const vsuxw;
+extern const function_base *const vext_x_v;
/* Below function_base are Vectro Crypto*/
extern const function_base *const vandn;
extern const function_base *const vbrev;
diff --git a/gcc/config/riscv/riscv-vector-builtins-shapes.cc b/gcc/config/riscv/riscv-vector-builtins-shapes.cc
index 1e4f4d53de6..8e90b17a94b 100644
--- a/gcc/config/riscv/riscv-vector-builtins-shapes.cc
+++ b/gcc/config/riscv/riscv-vector-builtins-shapes.cc
@@ -211,6 +211,146 @@ struct indexed_loadstore_def : public function_shape
}
};
+/* Add one function instance for GROUP, using operand suffix at index OI,
+ mode suffix at index PAIR && bi and predication suffix at index pred_idx. */
+static void
+build_th_loadstore (function_builder &b, const function_group_info &group,
+ unsigned int pred_idx, unsigned int vec_type_idx)
+{
+ auto_vec<tree, 5> argument_types;
+ function_instance function_instance (group.base_name, *group.base,
+ *group.shape,
+ group.ops_infos.types[vec_type_idx],
+ group.preds[pred_idx], &group.ops_infos);
+ tree return_type = group.ops_infos.ret.get_tree_type (
+ group.ops_infos.types[vec_type_idx].index);
+ b.allocate_argument_types (function_instance, argument_types);
+ b.apply_predication (function_instance, return_type, argument_types);
+
+ if (TARGET_XTHEADVECTOR && !check_type (return_type, argument_types))
+ return;
+
+ tree type = builtin_types[group.ops_infos.types[vec_type_idx].index].vector;
+ if (strstr (group.base_name, "l")
+ && strstr (group.base_name, "u")
+ && !TYPE_UNSIGNED (TREE_TYPE (type)))
+ return;
+
+ if (strstr (group.base_name, "l")
+ && !strstr (group.base_name, "u")
+ && TYPE_UNSIGNED (TREE_TYPE (type)))
+ return;
+
+ machine_mode mode = TYPE_MODE (type);
+ int sew = GET_MODE_BITSIZE (GET_MODE_INNER (mode));
+ if (strstr (group.base_name, "h") && sew == 8)
+ return;
+
+ if (strstr (group.base_name, "w") && (sew == 8 || sew ==16))
+ return;
+
+ b.add_overloaded_function (function_instance, *group.shape);
+ b.add_unique_function (function_instance, (*group.shape), return_type,
+ argument_types);
+}
+
+/* th_loadstore_width_def class. */
+struct th_loadstore_width_def : public build_base
+{
+ void build (function_builder &b,
+ const function_group_info &group) const override
+ {
+ for (unsigned int pred_idx = 0; group.preds[pred_idx] != NUM_PRED_TYPES;
+ ++pred_idx)
+ {
+ for (unsigned int vec_type_idx = 0;
+ group.ops_infos.types[vec_type_idx].index != NUM_VECTOR_TYPES;
+ ++vec_type_idx)
+ {
+ build_th_loadstore (b, group, pred_idx, vec_type_idx);
+ }
+ }
+ }
+
+ char *get_name (function_builder &b, const function_instance &instance,
+ bool overloaded_p) const override
+ {
+ /* Return nullptr if it can not be overloaded. */
+ if (overloaded_p && !instance.base->can_be_overloaded_p (instance.pred))
+ return nullptr;
+
+ b.append_name ("__riscv_th_");
+ b.append_name (instance.base_name);
+
+ /* vop_v --> vop_v_<type>. */
+ if (!overloaded_p)
+ {
+ /* vop --> vop_v. */
+ b.append_name (operand_suffixes[instance.op_info->op]);
+ /* vop_v --> vop_v_<type>. */
+ b.append_name (type_suffixes[instance.type.index].vector);
+ }
+
+ /* According to rvv-intrinsic-doc, it does not add "_m" suffix
+ for vop_m C++ overloaded API. */
+ if (overloaded_p && instance.pred == PRED_TYPE_m)
+ return b.finish_name ();
+ b.append_name (predication_suffixes[instance.pred]);
+ return b.finish_name ();
+ }
+};
+
+
+/* th_indexed_loadstore_width_def class. */
+struct th_indexed_loadstore_width_def : public function_shape
+{
+ void build (function_builder &b,
+ const function_group_info &group) const override
+ {
+ for (unsigned int pred_idx = 0; group.preds[pred_idx] != NUM_PRED_TYPES;
+ ++pred_idx)
+ {
+ for (unsigned int vec_type_idx = 0;
+ group.ops_infos.types[vec_type_idx].index != NUM_VECTOR_TYPES;
+ ++vec_type_idx)
+ {
+ tree index_type = group.ops_infos.args[1].get_tree_type (
+ group.ops_infos.types[vec_type_idx].index);
+ if (!index_type)
+ continue;
+ build_th_loadstore (b, group, pred_idx, vec_type_idx);
+ }
+ }
+ }
+
+ char *get_name (function_builder &b, const function_instance &instance,
+ bool overloaded_p) const override
+ {
+
+ /* Return nullptr if it can not be overloaded. */
+ if (overloaded_p && !instance.base->can_be_overloaded_p (instance.pred))
+ return nullptr;
+
+ b.append_name ("__riscv_th_");
+ b.append_name (instance.base_name);
+ /* vop_v --> vop_v_<type>. */
+ if (!overloaded_p)
+ {
+ /* vop --> vop_v. */
+ b.append_name (operand_suffixes[instance.op_info->op]);
+ /* vop_v --> vop_v_<type>. */
+ b.append_name (type_suffixes[instance.type.index].vector);
+ }
+
+ /* According to rvv-intrinsic-doc, it does not add "_m" suffix
+ for vop_m C++ overloaded API. */
+ if (overloaded_p && instance.pred == PRED_TYPE_m)
+ return b.finish_name ();
+ b.append_name (predication_suffixes[instance.pred]);
+ return b.finish_name ();
+ }
+};
+
/* alu_def class. */
struct alu_def : public build_base
{
@@ -632,6 +772,23 @@ struct reduc_alu_def : public build_base
}
};
+/* th_extract_def class. */
+struct th_extract_def : public build_base
+{
+ char *get_name (function_builder &b, const function_instance &instance,
+ bool overloaded_p) const override
+ {
+ b.append_name ("__riscv_th_");
+ b.append_name (instance.base_name);
+
+ if (overloaded_p)
+ return b.finish_name ();
+ b.append_name (type_suffixes[instance.type.index].vector);
+ b.append_name (type_suffixes[instance.type.index].scalar);
+ return b.finish_name ();
+ }
+};
+
/* scalar_move_def class. */
struct scalar_move_def : public build_base
{
@@ -1094,6 +1251,8 @@ SHAPE(vsetvl, vsetvl)
SHAPE(vsetvl, vsetvlmax)
SHAPE(loadstore, loadstore)
SHAPE(indexed_loadstore, indexed_loadstore)
+SHAPE(th_loadstore_width, th_loadstore_width)
+SHAPE(th_indexed_loadstore_width, th_indexed_loadstore_width)
SHAPE(alu, alu)
SHAPE(alu_frm, alu_frm)
SHAPE(widen_alu, widen_alu)
@@ -1106,6 +1265,7 @@ SHAPE(move, move)
SHAPE(mask_alu, mask_alu)
SHAPE(reduc_alu, reduc_alu)
SHAPE(reduc_alu_frm, reduc_alu_frm)
+SHAPE(th_extract, th_extract)
SHAPE(scalar_move, scalar_move)
SHAPE(vundefined, vundefined)
SHAPE(misc, misc)
diff --git a/gcc/config/riscv/riscv-vector-builtins-shapes.h b/gcc/config/riscv/riscv-vector-builtins-shapes.h
index ac2a28ce017..a7624d0fabd 100644
--- a/gcc/config/riscv/riscv-vector-builtins-shapes.h
+++ b/gcc/config/riscv/riscv-vector-builtins-shapes.h
@@ -28,6 +28,8 @@ extern const function_shape *const vsetvl;
extern const function_shape *const vsetvlmax;
extern const function_shape *const loadstore;
extern const function_shape *const indexed_loadstore;
+extern const function_shape *const th_loadstore_width;
+extern const function_shape *const th_indexed_loadstore_width;
extern const function_shape *const alu;
extern const function_shape *const alu_frm;
extern const function_shape *const widen_alu;
@@ -41,6 +43,7 @@ extern const function_shape *const mask_alu;
extern const function_shape *const reduc_alu;
extern const function_shape *const reduc_alu_frm;
extern const function_shape *const scalar_move;
+extern const function_shape *const th_extract;
extern const function_shape *const vundefined;
extern const function_shape *const misc;
extern const function_shape *const vset;
diff --git a/gcc/config/riscv/riscv-vector-builtins.cc b/gcc/config/riscv/riscv-vector-builtins.cc
index 25e0b6e56de..44b9fec1898 100644
--- a/gcc/config/riscv/riscv-vector-builtins.cc
+++ b/gcc/config/riscv/riscv-vector-builtins.cc
@@ -934,6 +934,32 @@ static CONSTEXPR const rvv_arg_type_info ext_vcreate_args[]
= {rvv_arg_type_info (RVV_BASE_vector),
rvv_arg_type_info_end};
+/* A list of args for vector_type func (const scalar_type *, size_t)
+ * function. */
+static CONSTEXPR const rvv_arg_type_info scalar_const_ptr_size_args[]
+ = {rvv_arg_type_info (RVV_BASE_scalar_const_ptr),
+ rvv_arg_type_info (RVV_BASE_size), rvv_arg_type_info_end};
+
+/* A list of args for vector_type func (const scalar_type *, eew8_index_type)
+ * function. */
+static CONSTEXPR const rvv_arg_type_info scalar_const_ptr_index_args[]
+ = {rvv_arg_type_info (RVV_BASE_scalar_const_ptr),
+ rvv_arg_type_info (RVV_BASE_unsigned_vector), rvv_arg_type_info_end};
+
+/* A list of args for void func (scalar_type *, eew8_index_type, vector_type)
+ * function. */
+static CONSTEXPR const rvv_arg_type_info scalar_ptr_index_args[]
+ = {rvv_arg_type_info (RVV_BASE_scalar_ptr),
+ rvv_arg_type_info (RVV_BASE_unsigned_vector),
+ rvv_arg_type_info (RVV_BASE_vector), rvv_arg_type_info_end};
+
+/* A list of args for void func (scalar_type *, size_t, vector_type)
+ * function. */
+static CONSTEXPR const rvv_arg_type_info scalar_ptr_size_args[]
+ = {rvv_arg_type_info (RVV_BASE_scalar_ptr),
+ rvv_arg_type_info (RVV_BASE_size), rvv_arg_type_info (RVV_BASE_vector),
+ rvv_arg_type_info_end};
+
/* A list of none preds that will be registered for intrinsic functions. */
static CONSTEXPR const predication_type_index none_preds[]
= {PRED_TYPE_none, NUM_PRED_TYPES};
@@ -1455,6 +1481,14 @@ static CONSTEXPR const rvv_op_info iu_shift_vvv_ops
rvv_arg_type_info (RVV_BASE_vector), /* Return type */
shift_vv_args /* Args */};
+/* A static operand information for scalar_type func (vector_type, size_t)
+ * function registration. */
+static CONSTEXPR const rvv_op_info iu_x_s_u_ops
+ = {iu_ops, /* Types */
+ OP_TYPE_vx, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_scalar), /* Return type */
+ v_size_args /* Args */};
+
/* A static operand information for vector_type func (vector_type, size_t)
* function registration. */
static CONSTEXPR const rvv_op_info iu_shift_vvx_ops
@@ -2638,6 +2672,38 @@ static CONSTEXPR const rvv_op_info all_v_vcreate_lmul4_x2_ops
rvv_arg_type_info (RVV_BASE_vlmul_ext_x2), /* Return type */
ext_vcreate_args /* Args */};
+/* A static operand information for vector_type func (const scalar_type *,
+ * size_t) function registration. */
+static CONSTEXPR const rvv_op_info all_v_scalar_const_ptr_size_ops
+ = {all_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_size_args /* Args */};
+
+/* A static operand information for void func (scalar_type *, size_t,
+ * vector_type) function registration. */
+static CONSTEXPR const rvv_op_info all_v_scalar_ptr_size_ops
+ = {all_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_void), /* Return type */
+ scalar_ptr_size_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * index_type) function registration. */
+static CONSTEXPR const rvv_op_info all_v_scalar_const_ptr_index_ops
+ = {all_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_index_args /* Args */};
+
+/* A static operand information for void func (scalar_type *, index_type,
+ * vector_type) function registration. */
+static CONSTEXPR const rvv_op_info all_v_scalar_ptr_index_ops
+ = {all_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_void), /* Return type */
+ scalar_ptr_index_args /* Args */};
+
/* A static operand information for vector_type func (vector_type).
Some ins just supports SEW=32, such as crypto vectol Zvkg extension.
* function registration. */
@@ -2816,6 +2882,10 @@ static function_group_info function_groups[] = {
#define DEF_RVV_FUNCTION(NAME, SHAPE, PREDS, OPS_INFO) \
{#NAME, &bases::NAME, &shapes::SHAPE, PREDS, OPS_INFO, REQUIRED_EXTENSIONS},
#include "riscv-vector-builtins-functions.def"
+#undef DEF_RVV_FUNCTION
+#define DEF_RVV_FUNCTION(NAME, SHAPE, PREDS, OPS_INFO) \
+ {#NAME, &bases::NAME, &shapes::SHAPE, PREDS, OPS_INFO, REQUIRED_EXTENSIONS},
+#include "thead-vector-builtins-functions.def"
};
/* The RVV types, with their built-in
diff --git a/gcc/config/riscv/riscv-vector-builtins.h b/gcc/config/riscv/riscv-vector-builtins.h
index 54c8824ff92..22fed60b4c3 100644
--- a/gcc/config/riscv/riscv-vector-builtins.h
+++ b/gcc/config/riscv/riscv-vector-builtins.h
@@ -123,6 +123,7 @@ enum required_ext
ZVKNHB_EXT, /* Crypto vector Zvknhb sub-ext */
ZVKSED_EXT, /* Crypto vector Zvksed sub-ext */
ZVKSH_EXT, /* Crypto vector Zvksh sub-ext */
+ XTHEADVECTOR_EXT, /* XTheadVector extension */
};
/* Enumerates the RVV operand types. */
@@ -252,6 +253,8 @@ struct function_group_info
return TARGET_ZVKSED;
case ZVKSH_EXT:
return TARGET_ZVKSH;
+ case XTHEADVECTOR_EXT:
+ return TARGET_XTHEADVECTOR;
default:
gcc_unreachable ();
}
diff --git a/gcc/config/riscv/t-riscv b/gcc/config/riscv/t-riscv
index 32de6b851c1..38494320d8b 100644
--- a/gcc/config/riscv/t-riscv
+++ b/gcc/config/riscv/t-riscv
@@ -1,6 +1,7 @@
RISCV_BUILTINS_H = $(srcdir)/config/riscv/riscv-vector-builtins.h \
$(srcdir)/config/riscv/riscv-vector-builtins.def \
$(srcdir)/config/riscv/riscv-vector-builtins-functions.def \
+ $(srcdir)/config/riscv/thead-vector-builtins-functions.def \
riscv-vector-type-indexer.gen.def
riscv-builtins.o: $(srcdir)/config/riscv/riscv-builtins.cc $(CONFIG_H) \
diff --git a/gcc/config/riscv/thead-vector-builtins-functions.def b/gcc/config/riscv/thead-vector-builtins-functions.def
new file mode 100644
index 00000000000..fd3ba29bae9
--- /dev/null
+++ b/gcc/config/riscv/thead-vector-builtins-functions.def
@@ -0,0 +1,39 @@
+#ifndef DEF_RVV_FUNCTION
+#define DEF_RVV_FUNCTION(NAME, SHAPE, PREDS, OPS_INFO)
+#endif
+
+#define REQUIRED_EXTENSIONS XTHEADVECTOR_EXT
+DEF_RVV_FUNCTION (vlb, th_loadstore_width, full_preds, all_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (vlh, th_loadstore_width, full_preds, all_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (vlw, th_loadstore_width, full_preds, all_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (vlbu, th_loadstore_width, full_preds, all_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (vlhu, th_loadstore_width, full_preds, all_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (vlwu, th_loadstore_width, full_preds, all_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (vsb, th_loadstore_width, none_m_preds, all_v_scalar_ptr_ops)
+DEF_RVV_FUNCTION (vsh, th_loadstore_width, none_m_preds, all_v_scalar_ptr_ops)
+DEF_RVV_FUNCTION (vsw, th_loadstore_width, none_m_preds, all_v_scalar_ptr_ops)
+DEF_RVV_FUNCTION (vlsb, th_loadstore_width, full_preds, all_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (vlsh, th_loadstore_width, full_preds, all_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (vlsw, th_loadstore_width, full_preds, all_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (vlsbu, th_loadstore_width, full_preds, all_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (vlshu, th_loadstore_width, full_preds, all_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (vlswu, th_loadstore_width, full_preds, all_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (vssb, th_loadstore_width, none_m_preds, all_v_scalar_ptr_size_ops)
+DEF_RVV_FUNCTION (vssh, th_loadstore_width, none_m_preds, all_v_scalar_ptr_size_ops)
+DEF_RVV_FUNCTION (vssw, th_loadstore_width, none_m_preds, all_v_scalar_ptr_size_ops)
+DEF_RVV_FUNCTION (vlxb, th_indexed_loadstore_width, full_preds, all_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (vlxh, th_indexed_loadstore_width, full_preds, all_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (vlxw, th_indexed_loadstore_width, full_preds, all_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (vlxbu, th_indexed_loadstore_width, full_preds, all_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (vlxhu, th_indexed_loadstore_width, full_preds, all_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (vlxwu, th_indexed_loadstore_width, full_preds, all_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (vsxb, th_indexed_loadstore_width, none_m_preds, all_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (vsxh, th_indexed_loadstore_width, none_m_preds, all_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (vsxw, th_indexed_loadstore_width, none_m_preds, all_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (vsuxb, th_indexed_loadstore_width, none_m_preds, all_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (vsuxh, th_indexed_loadstore_width, none_m_preds, all_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (vsuxw, th_indexed_loadstore_width, none_m_preds, all_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (vext_x_v, th_extract, none_preds, iu_x_s_u_ops)
+#undef REQUIRED_EXTENSIONS
+
+#undef DEF_RVV_FUNCTION
diff --git a/gcc/config/riscv/thead-vector.md b/gcc/config/riscv/thead-vector.md
index 696b815252d..5fe9ba08c4e 100644
--- a/gcc/config/riscv/thead-vector.md
+++ b/gcc/config/riscv/thead-vector.md
@@ -1,7 +1,95 @@
(define_c_enum "unspec" [
+ UNSPEC_TH_VLB
+ UNSPEC_TH_VLBU
+ UNSPEC_TH_VLH
+ UNSPEC_TH_VLHU
+ UNSPEC_TH_VLW
+ UNSPEC_TH_VLWU
+
+ UNSPEC_TH_VLSB
+ UNSPEC_TH_VLSBU
+ UNSPEC_TH_VLSH
+ UNSPEC_TH_VLSHU
+ UNSPEC_TH_VLSW
+ UNSPEC_TH_VLSWU
+
+ UNSPEC_TH_VLXB
+ UNSPEC_TH_VLXBU
+ UNSPEC_TH_VLXH
+ UNSPEC_TH_VLXHU
+ UNSPEC_TH_VLXW
+ UNSPEC_TH_VLXWU
+
+ UNSPEC_TH_VSUXB
+ UNSPEC_TH_VSUXH
+ UNSPEC_TH_VSUXW
+
UNSPEC_TH_VWLDST
])
+(define_int_iterator UNSPEC_TH_VLMEM_OP [
+ UNSPEC_TH_VLB UNSPEC_TH_VLBU
+ UNSPEC_TH_VLH UNSPEC_TH_VLHU
+ UNSPEC_TH_VLW UNSPEC_TH_VLWU
+])
+
+(define_int_iterator UNSPEC_TH_VLSMEM_OP [
+ UNSPEC_TH_VLSB UNSPEC_TH_VLSBU
+ UNSPEC_TH_VLSH UNSPEC_TH_VLSHU
+ UNSPEC_TH_VLSW UNSPEC_TH_VLSWU
+])
+
+(define_int_iterator UNSPEC_TH_VLXMEM_OP [
+ UNSPEC_TH_VLXB UNSPEC_TH_VLXBU
+ UNSPEC_TH_VLXH UNSPEC_TH_VLXHU
+ UNSPEC_TH_VLXW UNSPEC_TH_VLXWU
+])
+
+(define_int_attr vlmem_op_attr [
+ (UNSPEC_TH_VLB "b") (UNSPEC_TH_VLBU "bu")
+ (UNSPEC_TH_VLH "h") (UNSPEC_TH_VLHU "hu")
+ (UNSPEC_TH_VLW "w") (UNSPEC_TH_VLWU "wu")
+ (UNSPEC_TH_VLSB "b") (UNSPEC_TH_VLSBU "bu")
+ (UNSPEC_TH_VLSH "h") (UNSPEC_TH_VLSHU "hu")
+ (UNSPEC_TH_VLSW "w") (UNSPEC_TH_VLSWU "wu")
+ (UNSPEC_TH_VLXB "b") (UNSPEC_TH_VLXBU "bu")
+ (UNSPEC_TH_VLXH "h") (UNSPEC_TH_VLXHU "hu")
+ (UNSPEC_TH_VLXW "w") (UNSPEC_TH_VLXWU "wu")
+ (UNSPEC_TH_VSUXB "b")
+ (UNSPEC_TH_VSUXH "h")
+ (UNSPEC_TH_VSUXW "w")
+])
+
+(define_int_attr vlmem_order_attr [
+ (UNSPEC_TH_VLXB "")
+ (UNSPEC_TH_VLXH "")
+ (UNSPEC_TH_VLXW "")
+ (UNSPEC_TH_VSUXB "u")
+ (UNSPEC_TH_VSUXH "u")
+ (UNSPEC_TH_VSUXW "u")
+])
+
+(define_int_iterator UNSPEC_TH_VSMEM_OP [
+ UNSPEC_TH_VLB
+ UNSPEC_TH_VLH
+ UNSPEC_TH_VLW
+])
+
+(define_int_iterator UNSPEC_TH_VSSMEM_OP [
+ UNSPEC_TH_VLSB
+ UNSPEC_TH_VLSH
+ UNSPEC_TH_VLSW
+])
+
+(define_int_iterator UNSPEC_TH_VSXMEM_OP [
+ UNSPEC_TH_VLXB
+ UNSPEC_TH_VLXH
+ UNSPEC_TH_VLXW
+ UNSPEC_TH_VSUXB
+ UNSPEC_TH_VSUXH
+ UNSPEC_TH_VSUXW
+])
+
(define_mode_iterator V_VLS_VT [V VLS VT])
(define_mode_iterator V_VB_VLS_VT [V VB VLS VT])
@@ -100,3 +188,165 @@
}
[(set_attr "type" "vldm,vstm,vmalu,vmalu,vmalu")
(set_attr "mode" "<MODE>")])
+
+(define_expand "@pred_mov_width<vlmem_op_attr><mode>"
+ [(set (match_operand:V_VLS 0 "nonimmediate_operand")
+ (if_then_else:V_VLS
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand")
+ (match_operand 4 "vector_length_operand")
+ (match_operand 5 "const_int_operand")
+ (match_operand 6 "const_int_operand")
+ (match_operand 7 "const_int_operand")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLMEM_OP)
+ (match_operand:V_VLS 3 "vector_move_operand")
+ (match_operand:V_VLS 2 "vector_merge_operand")))]
+ "TARGET_XTHEADVECTOR"
+ {})
+
+(define_insn_and_split "*pred_mov_width<vlmem_op_attr><mode>"
+ [(set (match_operand:V_VLS 0 "nonimmediate_operand" "=vr, vr, vd, m, vr, vr")
+ (if_then_else:V_VLS
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1, Wc1, vm, vmWc1, Wc1, Wc1")
+ (match_operand 4 "vector_length_operand" " rK, rK, rK, rK, rK, rK")
+ (match_operand 5 "const_int_operand" " i, i, i, i, i, i")
+ (match_operand 6 "const_int_operand" " i, i, i, i, i, i")
+ (match_operand 7 "const_int_operand" " i, i, i, i, i, i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLMEM_OP)
+ (match_operand:V_VLS 3 "reg_or_mem_operand" " m, m, m, vr, vr, vr")
+ (match_operand:V_VLS 2 "vector_merge_operand" " 0, vu, vu, vu, vu, 0")))]
+ "(TARGET_XTHEADVECTOR
+ && (register_operand (operands[0], <MODE>mode)
+ || register_operand (operands[3], <MODE>mode)))"
+ "@
+ vl<vlmem_op_attr>.v\t%0,%3%p1
+ vl<vlmem_op_attr>.v\t%0,%3
+ vl<vlmem_op_attr>.v\t%0,%3,%1.t
+ vs<vlmem_op_attr>.v\t%3,%0%p1
+ vmv.v.v\t%0,%3
+ vmv.v.v\t%0,%3"
+ "&& riscv_vector::whole_reg_to_reg_move_p (operands, <MODE>mode, 7)"
+ [(set (match_dup 0) (match_dup 3))]
+ ""
+ [(set_attr "type" "vlde,vlde,vlde,vste,vimov,vimov")
+ (set_attr "mode" "<MODE>")])
+
+(define_insn "@pred_store_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "memory_operand" "+m")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1")
+ (match_operand 3 "vector_length_operand" " rK")
+ (match_operand 4 "const_int_operand" " i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VSMEM_OP)
+ (match_operand:VI 2 "register_operand" " vr")
+ (match_dup 0)))]
+ "TARGET_XTHEADVECTOR"
+ "vs<vlmem_op_attr>.v\t%2,%0%p1"
+ [(set_attr "type" "vste")
+ (set_attr "mode" "<MODE>")
+ (set (attr "avl_type_idx") (const_int 4))
+ (set_attr "vl_op_idx" "3")])
+
+(define_insn "@pred_strided_load_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "register_operand" "=vr, vr, vd")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1, Wc1, vm")
+ (match_operand 5 "vector_length_operand" " rK, rK, rK")
+ (match_operand 6 "const_int_operand" " i, i, i")
+ (match_operand 7 "const_int_operand" " i, i, i")
+ (match_operand 8 "const_int_operand" " i, i, i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLSMEM_OP)
+ (unspec:VI
+ [(match_operand:VI 3 "memory_operand" " m, m, m")
+ (match_operand 4 "pmode_reg_or_0_operand" " rJ, rJ, rJ")] UNSPEC_TH_VLSMEM_OP)
+ (match_operand:VI 2 "vector_merge_operand" " 0, vu, vu")))]
+ "TARGET_XTHEADVECTOR"
+ "vls<vlmem_op_attr>.v\t%0,%3,%z4%p1"
+ [(set_attr "type" "vlds")
+ (set_attr "mode" "<MODE>")])
+
+(define_insn "@pred_strided_store_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "memory_operand" "+m")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1")
+ (match_operand 4 "vector_length_operand" " rK")
+ (match_operand 5 "const_int_operand" " i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VSSMEM_OP)
+ (unspec:VI
+ [(match_operand 2 "pmode_reg_or_0_operand" " rJ")
+ (match_operand:VI 3 "register_operand" " vr")] UNSPEC_TH_VSSMEM_OP)
+ (match_dup 0)))]
+ "TARGET_XTHEADVECTOR"
+ "vss<vlmem_op_attr>.v\t%3,%0,%z2%p1"
+ [(set_attr "type" "vsts")
+ (set_attr "mode" "<MODE>")
+ (set (attr "avl_type_idx") (const_int 5))])
+
+(define_insn "@pred_indexed_load_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "register_operand" "=vd, vr,vd, vr")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" " vm,Wc1,vm,Wc1")
+ (match_operand 5 "vector_length_operand" " rK, rK,rK, rK")
+ (match_operand 6 "const_int_operand" " i, i, i, i")
+ (match_operand 7 "const_int_operand" " i, i, i, i")
+ (match_operand 8 "const_int_operand" " i, i, i, i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLXMEM_OP)
+ (unspec:VI
+ [(match_operand 3 "pmode_reg_or_0_operand" " rJ, rJ,rJ, rJ")
+ (mem:BLK (scratch))
+ (match_operand:VI 4 "register_operand" " vr, vr,vr, vr")] UNSPEC_TH_VLXMEM_OP)
+ (match_operand:VI 2 "vector_merge_operand" " vu, vu, 0, 0")))]
+ "TARGET_XTHEADVECTOR"
+ "vlx<vlmem_op_attr>.v\t%0,(%z3),%4%p1"
+ [(set_attr "type" "vldux")
+ (set_attr "mode" "<MODE>")])
+
+(define_insn "@pred_indexed_<vlmem_order_attr>store_width<vlmem_op_attr><mode>"
+ [(set (mem:BLK (scratch))
+ (unspec:BLK
+ [(unspec:<VM>
+ [(match_operand:<VM> 0 "vector_mask_operand" "vmWc1")
+ (match_operand 4 "vector_length_operand" " rK")
+ (match_operand 5 "const_int_operand" " i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VSXMEM_OP)
+ (match_operand 1 "pmode_reg_or_0_operand" " rJ")
+ (match_operand:VI 2 "register_operand" " vr")
+ (match_operand:VI 3 "register_operand" " vr")] UNSPEC_TH_VSXMEM_OP))]
+ "TARGET_XTHEADVECTOR"
+ "vs<vlmem_order_attr>x<vlmem_op_attr>.v\t%3,(%z1),%2%p0"
+ [(set_attr "type" "vstux")
+ (set_attr "mode" "<MODE>")])
+
+(define_expand "@pred_th_extract<mode>"
+ [(set (match_operand:<VEL> 0 "register_operand")
+ (unspec:<VEL>
+ [(vec_select:<VEL>
+ (match_operand:V_VLSI 1 "register_operand")
+ (parallel [(match_operand:DI 2 "register_operand" "r")]))
+ (reg:SI VTYPE_REGNUM)] UNSPEC_VPREDICATE))]
+ "TARGET_XTHEADVECTOR"
+{})
+
+(define_insn "*pred_th_extract<mode>"
+ [(set (match_operand:<VEL> 0 "register_operand" "=r")
+ (unspec:<VEL>
+ [(vec_select:<VEL>
+ (match_operand:V_VLSI 1 "register_operand" "vr")
+ (parallel [(match_operand:DI 2 "register_operand" "r")]))
+ (reg:SI VTYPE_REGNUM)] UNSPEC_VPREDICATE))]
+ "TARGET_XTHEADVECTOR"
+ "vext.x.v\t%0,%1,%2"
+ [(set_attr "type" "vimovvx")
+ (set_attr "mode" "<MODE>")])
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c
new file mode 100644
index 00000000000..3c12c124597
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_th_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+
+** th.vadd\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+
+** th.vsb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out)
+{
+ vint32m1_t v = __riscv_th_vlb_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlb_v_i32m1_tu (v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vv_i32m1 (v2, v2, 4);
+ vint32m1_t v4 = __riscv_vadd_vv_i32m1_tu (v3, v2, v2, 4);
+ __riscv_th_vsb_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlb.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+
+** th.vadd\.vv\tv[1-9][0-9]?,\s*v[0-9]+,\s*v[0-9]+,\s*v0.t
+** th.vsb.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlb_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlb_v_i32m1_m (mask, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vv_i32m1 (v2, v2, 4);
+ vint32m1_t v4 = __riscv_vadd_vv_i32m1_m (mask, v3, v3, 4);
+ __riscv_th_vsb_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlb.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+
+** th.vadd\.vv\tv[1-9][0-9]?,\s*v[0-9]+,\s*v[0-9]+,\s*v0.t
+** th.vsb.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlb_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlb_v_i32m1_tumu (mask, v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vv_i32m1 (v2, v2, 4);
+ vint32m1_t v4 = __riscv_vadd_vv_i32m1_tumu (mask, v3, v2, v2, 4);
+ __riscv_th_vsb_v_i32m1 (out, v4, 4);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c
new file mode 100644
index 00000000000..30bef369375
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_th_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlbu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlbu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vsb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, uint32_t x)
+{
+ vuint32m1_t v = __riscv_th_vlbu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlbu_v_u32m1_tu (v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tu (v3, v2, -16, 4);
+ __riscv_th_vsb_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlbu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsb.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlbu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlbu_v_u32m1_m (mask, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_m (mask, v3, -16, 4);
+ __riscv_th_vsb_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlbu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlbu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsb.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlbu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlbu_v_u32m1_tumu (mask, v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tumu (mask, v3, v2, -16, 4);
+ __riscv_th_vsb_v_u32m1 (out, v4, 4);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c
new file mode 100644
index 00000000000..3c8b5ccc16b
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_th_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vsh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, int32_t x)
+{
+ vint32m1_t v = __riscv_th_vlh_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlh_v_i32m1_tu (v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, -16, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_tu (v3, v2, -16, 4);
+ __riscv_th_vsh_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlh.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsh.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, int32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlh_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlh_v_i32m1_m (mask, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, -16, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_m (mask, v3, -16, 4);
+ __riscv_th_vsh_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlh.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsh.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, int32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlh_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlh_v_i32m1_tumu (mask, v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, -16, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_tumu (mask, v3, v2, -16, 4);
+ __riscv_th_vsh_v_i32m1 (out, v4, 4);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c
new file mode 100644
index 00000000000..b7c00404f18
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_th_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlhu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlhu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vsh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, uint32_t x)
+{
+ vuint32m1_t v = __riscv_th_vlhu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlhu_v_u32m1_tu (v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tu (v3, v2, -16, 4);
+ __riscv_th_vsh_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlhu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsh.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlhu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlhu_v_u32m1_m (mask, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_m (mask, v3, -16, 4);
+ __riscv_th_vsh_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlhu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlhu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsh.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlhu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlhu_v_u32m1_tumu (mask, v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tumu (mask, v3, v2, -16, 4);
+ __riscv_th_vsh_v_u32m1 (out, v4, 4);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c
new file mode 100644
index 00000000000..17a53012acf
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_th_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vx\tv[0-9]+,\s*v[0-9]+,\s*[a-x0-9]+
+** th.vadd\.vx\tv[0-9]+,\s*v[0-9]+,\s*[a-x0-9]+
+** th.vsw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, int32_t x)
+{
+ vint32m1_t v = __riscv_th_vlw_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlw_v_i32m1_tu (v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, x, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_tu (v3, v2, x, 4);
+ __riscv_th_vsw_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlw.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vx\tv[0-9]+,\s*v[0-9]+,\s*[a-x0-9]+
+** th.vadd\.vx\tv[1-9][0-9]?,\s*v[0-9]+,\s*[a-x0-9]+,\s*v0.t
+** th.vsw.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, int32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlw_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlw_v_i32m1_m (mask, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, x, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_m (mask, v3, x, 4);
+ __riscv_th_vsw_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlw.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vx\tv[0-9]+,\s*v[0-9]+,\s*[a-x0-9]+
+** th.vadd\.vx\tv[1-9][0-9]?,\s*v[0-9]+,\s*[a-x0-9]+,\s*v0.t
+** th.vsw.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, int32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlw_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlw_v_i32m1_tumu (mask, v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, x, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_tumu (mask, v3, v2, x, 4);
+ __riscv_th_vsw_v_i32m1 (out, v4, 4);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c
new file mode 100644
index 00000000000..b187cfc852b
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_th_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlwu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlwu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vsw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, uint32_t x)
+{
+ vuint32m1_t v = __riscv_th_vlwu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlwu_v_u32m1_tu (v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tu (v3, v2, -16, 4);
+ __riscv_th_vsw_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlwu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsw.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlwu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlwu_v_u32m1_m (mask, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_m (mask, v3, -16, 4);
+ __riscv_th_vsw_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlwu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlwu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsw.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlwu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlwu_v_u32m1_tumu (mask, v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tumu (mask, v3, v2, -16, 4);
+ __riscv_th_vsw_v_u32m1 (out, v4, 4);
+}
--
2.17.1
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re：Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
2024-01-11 10:55 Re：[PATCH " juzhe.zhong
@ 2024-01-11 14:11 ` joshua
0 siblings, 0 replies; 17+ messages in thread
From: joshua @ 2024-01-11 14:11 UTC (permalink / raw)
To: juzhe.zhong, gcc-patches
Cc: Jim Wilson, palmer, andrew, philipp.tomsich, jeffreyalaw,
christoph.muellner, jinma, cooper.qu
Is the patch with !TARGET_XTHEADVECTOR for sext/zext
patterns removed OK to commit?
https://gcc.gnu.org/pipermail/gcc-patches/2024-January/642657.html
------------------------------------------------------------------
发件人：juzhe.zhong@rivai.ai <juzhe.zhong@rivai.ai>
发送时间：2024年1月11日(星期四) 18:56
收件人："cooper.joshua"<cooper.joshua@linux.alibaba.com>; "gcc-patches"<gcc-patches@gcc.gnu.org>
抄 送：Jim Wilson<jim.wilson.gcc@gmail.com>; palmer<palmer@dabbelt.com>; andrew<andrew@sifive.com>; "philipp.tomsich"<philipp.tomsich@vrull.eu>; jeffreyalaw<jeffreyalaw@gmail.com>; "christoph.muellner"<christoph.muellner@vrull.eu>; jinma<jinma@linux.alibaba.com>; "cooper.qu"<cooper.qu@linux.alibaba.com>
主 题：Re: Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
Yes.
juzhe.zhong@rivai.ai
发件人： joshua
发送时间： 2024-01-11 18:54
收件人： juzhe.zhong@rivai.ai; gcc-patches
抄送： Jim Wilson; palmer; andrew; philipp.tomsich; jeffreyalaw; christoph.muellner; jinma; cooper.qu
主题： Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
Do you mean removing TARGET_XTHEADVECTOR for sext/zext patterns
and then resending the "RISC-V: Handle differences between XTheadvector
and Vector" patch?
------------------------------------------------------------------
发件人：juzhe.zhong@rivai.ai <juzhe.zhong@rivai.ai>
发送时间：2024年1月11日(星期四) 17:57
收件人："cooper.joshua"<cooper.joshua@linux.alibaba.com>; "gcc-patches"<gcc-patches@gcc.gnu.org>
抄 送：Jim Wilson<jim.wilson.gcc@gmail.com>; palmer<palmer@dabbelt.com>; andrew<andrew@sifive.com>; "philipp.tomsich"<philipp.tomsich@vrull.eu>; jeffreyalaw<jeffreyalaw@gmail.com>; "christoph.muellner"<christoph.muellner@vrull.eu>; "cooper.joshua"<cooper.joshua@linux.alibaba.com>; jinma<jinma@linux.alibaba.com>; "cooper.qu"<cooper.qu@linux.alibaba.com>
主 题：Re: [PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
LGTM. Could you resend the patch "RISC-V: Handle differences between XTheadvector and Vector
Thanks.
juzhe.zhong@rivai.ai
From: Jun Sha (Joshua)
Date: 2024-01-11 17:52
To: gcc-patches
CC: jim.wilson.gcc; palmer; andrew; philipp.tomsich; jeffreyalaw; christoph.muellner; juzhe.zhong; Jun Sha (Joshua); Jin Ma; Xianmiao Qu
Subject: [PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
This patch only involves the generation of xtheadvector
special load/store instructions and vext instructions.
gcc/ChangeLog:
* config/riscv/riscv-vector-builtins-bases.cc
(class th_loadstore_width): Define new builtin bases.
(class th_extract): Define new builtin bases.
(BASE): Define new builtin bases.
* config/riscv/riscv-vector-builtins-bases.h:
Define new builtin class.
* config/riscv/riscv-vector-builtins-shapes.cc
(struct th_loadstore_width_def): Define new builtin shapes.
(struct th_indexed_loadstore_width_def):
Define new builtin shapes.
(struct th_extract_def): Define new builtin shapes.
(SHAPE): Define new builtin shapes.
* config/riscv/riscv-vector-builtins-shapes.h:
Define new builtin shapes.
* config/riscv/riscv-vector-builtins.cc (DEF_RVV_FUNCTION):
* config/riscv/riscv-vector-builtins.h (enum required_ext):
(struct function_group_info):
* config/riscv/t-riscv: Add thead-vector-builtins-functions.def
* config/riscv/thead-vector.md
(@pred_mov_width<vlmem_op_attr><mode>): Add new patterns.
(*pred_mov_width<vlmem_op_attr><mode>): Likewise.
(@pred_store_width<vlmem_op_attr><mode>): Likewise.
(@pred_strided_load_width<vlmem_op_attr><mode>): Likewise.
(@pred_strided_store_width<vlmem_op_attr><mode>): Likewise.
(@pred_indexed_load_width<vlmem_op_attr><mode>): Likewise.
(@pred_indexed_<vlmem_order_attr>store_width<vlmem_op_attr><mode>):
(@pred_th_extract<mode>): Likewise.
(*pred_th_extract<mode>): Likewise.
* config/riscv/thead-vector-builtins-functions.def: New file.
gcc/testsuite/ChangeLog:
* gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c: New test.
Co-authored-by: Jin Ma <jinma@linux.alibaba.com>
Co-authored-by: Xianmiao Qu <cooper.qu@linux.alibaba.com>
Co-authored-by: Christoph Müllner <christoph.muellner@vrull.eu>
---
.../riscv/riscv-vector-builtins-bases.cc | 139 ++++++++++
.../riscv/riscv-vector-builtins-bases.h | 31 +++
.../riscv/riscv-vector-builtins-shapes.cc | 160 +++++++++++
.../riscv/riscv-vector-builtins-shapes.h | 3 +
gcc/config/riscv/riscv-vector-builtins.cc | 70 +++++
gcc/config/riscv/riscv-vector-builtins.h | 3 +
gcc/config/riscv/t-riscv | 1 +
.../riscv/thead-vector-builtins-functions.def | 39 +++
gcc/config/riscv/thead-vector.md | 250 ++++++++++++++++++
.../riscv/rvv/xtheadvector/vlb-vsb.c | 68 +++++
.../riscv/rvv/xtheadvector/vlbu-vsb.c | 68 +++++
.../riscv/rvv/xtheadvector/vlh-vsh.c | 68 +++++
.../riscv/rvv/xtheadvector/vlhu-vsh.c | 68 +++++
.../riscv/rvv/xtheadvector/vlw-vsw.c | 68 +++++
.../riscv/rvv/xtheadvector/vlwu-vsw.c | 68 +++++
15 files changed, 1104 insertions(+)
create mode 100644 gcc/config/riscv/thead-vector-builtins-functions.def
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c
diff --git a/gcc/config/riscv/riscv-vector-builtins-bases.cc b/gcc/config/riscv/riscv-vector-builtins-bases.cc
index 1aa6e3c6665..b6f6e4ff37e 100644
--- a/gcc/config/riscv/riscv-vector-builtins-bases.cc
+++ b/gcc/config/riscv/riscv-vector-builtins-bases.cc
@@ -2141,6 +2141,83 @@ public:
}
};
+/* Implements
+ * th.vl(b/h/w)[u].v/th.vs(b/h/w)[u].v/th.vls(b/h/w)[u].v/th.vss(b/h/w)[u].v/
+ * th.vlx(b/h/w)[u].v/th.vs[u]x(b/h/w).v
+ * codegen. */
+template<bool STORE_P, lst_type LST_TYPE, int UNSPEC>
+class th_loadstore_width : public function_base
+{
+public:
+ bool apply_tail_policy_p () const override { return !STORE_P; }
+ bool apply_mask_policy_p () const override { return !STORE_P; }
+
+ unsigned int call_properties (const function_instance &) const override
+ {
+ if (STORE_P)
+ return CP_WRITE_MEMORY;
+ else
+ return CP_READ_MEMORY;
+ }
+
+ bool can_be_overloaded_p (enum predication_type_index pred) const override
+ {
+ if (STORE_P || LST_TYPE == LST_INDEXED)
+ return true;
+ return pred != PRED_TYPE_none;
+ }
+
+ rtx expand (function_expander &e) const override
+ {
+ gcc_assert (TARGET_XTHEADVECTOR);
+ if (LST_TYPE == LST_INDEXED)
+ {
+ if (STORE_P)
+ return e.use_exact_insn (
+ code_for_pred_indexed_store_width (UNSPEC, UNSPEC,
+ e.vector_mode ()));
+ else
+ return e.use_exact_insn (
+ code_for_pred_indexed_load_width (UNSPEC, e.vector_mode ()));
+ }
+ else if (LST_TYPE == LST_STRIDED)
+ {
+ if (STORE_P)
+ return e.use_contiguous_store_insn (
+ code_for_pred_strided_store_width (UNSPEC, e.vector_mode ()));
+ else
+ return e.use_contiguous_load_insn (
+ code_for_pred_strided_load_width (UNSPEC, e.vector_mode ()));
+ }
+ else
+ {
+ if (STORE_P)
+ return e.use_contiguous_store_insn (
+ code_for_pred_store_width (UNSPEC, e.vector_mode ()));
+ else
+ return e.use_contiguous_load_insn (
+ code_for_pred_mov_width (UNSPEC, e.vector_mode ()));
+ }
+ }
+};
+
+/* Implements vext.x.v. */
+class th_extract : public function_base
+{
+public:
+ bool apply_vl_p () const override { return false; }
+ bool apply_tail_policy_p () const override { return false; }
+ bool apply_mask_policy_p () const override { return false; }
+ bool use_mask_predication_p () const override { return false; }
+ bool has_merge_operand_p () const override { return false; }
+
+ rtx expand (function_expander &e) const override
+ {
+ gcc_assert (TARGET_XTHEADVECTOR);
+ return e.use_exact_insn (code_for_pred_th_extract (e.vector_mode ()));
+ }
+};
+
/* Below implements are vector crypto */
/* Implements vandn.[vv,vx] */
class vandn : public function_base
@@ -2603,6 +2680,37 @@ static CONSTEXPR const seg_indexed_load<UNSPEC_ORDERED> vloxseg_obj;
static CONSTEXPR const seg_indexed_store<UNSPEC_UNORDERED> vsuxseg_obj;
static CONSTEXPR const seg_indexed_store<UNSPEC_ORDERED> vsoxseg_obj;
static CONSTEXPR const vlsegff vlsegff_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLB> vlb_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLBU> vlbu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLH> vlh_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLHU> vlhu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLW> vlw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLWU> vlwu_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_UNIT_STRIDE, UNSPEC_TH_VLB> vsb_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_UNIT_STRIDE, UNSPEC_TH_VLH> vsh_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_UNIT_STRIDE, UNSPEC_TH_VLW> vsw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSB> vlsb_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSBU> vlsbu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSH> vlsh_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSHU> vlshu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSW> vlsw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSWU> vlswu_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_STRIDED, UNSPEC_TH_VLSB> vssb_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_STRIDED, UNSPEC_TH_VLSH> vssh_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_STRIDED, UNSPEC_TH_VLSW> vssw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXB> vlxb_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXBU> vlxbu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXH> vlxh_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXHU> vlxhu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXW> vlxw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXWU> vlxwu_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VLXB> vsxb_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VLXH> vsxh_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VLXW> vsxw_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VSUXB> vsuxb_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VSUXH> vsuxh_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VSUXW> vsuxw_obj;
+static CONSTEXPR const th_extract vext_x_v_obj;
/* Crypto Vector */
static CONSTEXPR const vandn vandn_obj;
@@ -2894,6 +3002,37 @@ BASE (vloxseg)
BASE (vsuxseg)
BASE (vsoxseg)
BASE (vlsegff)
+BASE (vlb)
+BASE (vlh)
+BASE (vlw)
+BASE (vlbu)
+BASE (vlhu)
+BASE (vlwu)
+BASE (vsb)
+BASE (vsh)
+BASE (vsw)
+BASE (vlsb)
+BASE (vlsh)
+BASE (vlsw)
+BASE (vlsbu)
+BASE (vlshu)
+BASE (vlswu)
+BASE (vssb)
+BASE (vssh)
+BASE (vssw)
+BASE (vlxb)
+BASE (vlxh)
+BASE (vlxw)
+BASE (vlxbu)
+BASE (vlxhu)
+BASE (vlxwu)
+BASE (vsxb)
+BASE (vsxh)
+BASE (vsxw)
+BASE (vsuxb)
+BASE (vsuxh)
+BASE (vsuxw)
+BASE (vext_x_v)
/* Crypto vector */
BASE (vandn)
BASE (vbrev)
diff --git a/gcc/config/riscv/riscv-vector-builtins-bases.h b/gcc/config/riscv/riscv-vector-builtins-bases.h
index 87c7f43a423..1f2c94d3541 100644
--- a/gcc/config/riscv/riscv-vector-builtins-bases.h
+++ b/gcc/config/riscv/riscv-vector-builtins-bases.h
@@ -280,6 +280,37 @@ extern const function_base *const vloxseg;
extern const function_base *const vsuxseg;
extern const function_base *const vsoxseg;
extern const function_base *const vlsegff;
+extern const function_base *const vlb;
+extern const function_base *const vlh;
+extern const function_base *const vlw;
+extern const function_base *const vlbu;
+extern const function_base *const vlhu;
+extern const function_base *const vlwu;
+extern const function_base *const vsb;
+extern const function_base *const vsh;
+extern const function_base *const vsw;
+extern const function_base *const vlsb;
+extern const function_base *const vlsh;
+extern const function_base *const vlsw;
+extern const function_base *const vlsbu;
+extern const function_base *const vlshu;
+extern const function_base *const vlswu;
+extern const function_base *const vssb;
+extern const function_base *const vssh;
+extern const function_base *const vssw;
+extern const function_base *const vlxb;
+extern const function_base *const vlxh;
+extern const function_base *const vlxw;
+extern const function_base *const vlxbu;
+extern const function_base *const vlxhu;
+extern const function_base *const vlxwu;
+extern const function_base *const vsxb;
+extern const function_base *const vsxh;
+extern const function_base *const vsxw;
+extern const function_base *const vsuxb;
+extern const function_base *const vsuxh;
+extern const function_base *const vsuxw;
+extern const function_base *const vext_x_v;
/* Below function_base are Vectro Crypto*/
extern const function_base *const vandn;
extern const function_base *const vbrev;
diff --git a/gcc/config/riscv/riscv-vector-builtins-shapes.cc b/gcc/config/riscv/riscv-vector-builtins-shapes.cc
index 1e4f4d53de6..8e90b17a94b 100644
--- a/gcc/config/riscv/riscv-vector-builtins-shapes.cc
+++ b/gcc/config/riscv/riscv-vector-builtins-shapes.cc
@@ -211,6 +211,146 @@ struct indexed_loadstore_def : public function_shape
}
};
+/* Add one function instance for GROUP, using operand suffix at index OI,
+ mode suffix at index PAIR && bi and predication suffix at index pred_idx. */
+static void
+build_th_loadstore (function_builder &b, const function_group_info &group,
+ unsigned int pred_idx, unsigned int vec_type_idx)
+{
+ auto_vec<tree, 5> argument_types;
+ function_instance function_instance (group.base_name, *group.base,
+ *group.shape,
+ group.ops_infos.types[vec_type_idx],
+ group.preds[pred_idx], &group.ops_infos);
+ tree return_type = group.ops_infos.ret.get_tree_type (
+ group.ops_infos.types[vec_type_idx].index);
+ b.allocate_argument_types (function_instance, argument_types);
+ b.apply_predication (function_instance, return_type, argument_types);
+
+ if (TARGET_XTHEADVECTOR && !check_type (return_type, argument_types))
+ return;
+
+ tree type = builtin_types[group.ops_infos.types[vec_type_idx].index].vector;
+ if (strstr (group.base_name, "l")
+ && strstr (group.base_name, "u")
+ && !TYPE_UNSIGNED (TREE_TYPE (type)))
+ return;
+
+ if (strstr (group.base_name, "l")
+ && !strstr (group.base_name, "u")
+ && TYPE_UNSIGNED (TREE_TYPE (type)))
+ return;
+
+ machine_mode mode = TYPE_MODE (type);
+ int sew = GET_MODE_BITSIZE (GET_MODE_INNER (mode));
+ if (strstr (group.base_name, "h") && sew == 8)
+ return;
+
+ if (strstr (group.base_name, "w") && (sew == 8 || sew ==16))
+ return;
+
+ b.add_overloaded_function (function_instance, *group.shape);
+ b.add_unique_function (function_instance, (*group.shape), return_type,
+ argument_types);
+}
+
+/* th_loadstore_width_def class. */
+struct th_loadstore_width_def : public build_base
+{
+ void build (function_builder &b,
+ const function_group_info &group) const override
+ {
+ for (unsigned int pred_idx = 0; group.preds[pred_idx] != NUM_PRED_TYPES;
+ ++pred_idx)
+ {
+ for (unsigned int vec_type_idx = 0;
+ group.ops_infos.types[vec_type_idx].index != NUM_VECTOR_TYPES;
+ ++vec_type_idx)
+ {
+ build_th_loadstore (b, group, pred_idx, vec_type_idx);
+ }
+ }
+ }
+
+ char *get_name (function_builder &b, const function_instance &instance,
+ bool overloaded_p) const override
+ {
+ /* Return nullptr if it can not be overloaded. */
+ if (overloaded_p && !instance.base->can_be_overloaded_p (instance.pred))
+ return nullptr;
+
+ b.append_name ("__riscv_th_");
+ b.append_name (instance.base_name);
+
+ /* vop_v --> vop_v_<type>. */
+ if (!overloaded_p)
+ {
+ /* vop --> vop_v. */
+ b.append_name (operand_suffixes[instance.op_info->op]);
+ /* vop_v --> vop_v_<type>. */
+ b.append_name (type_suffixes[instance.type.index].vector);
+ }
+
+ /* According to rvv-intrinsic-doc, it does not add "_m" suffix
+ for vop_m C++ overloaded API. */
+ if (overloaded_p && instance.pred == PRED_TYPE_m)
+ return b.finish_name ();
+ b.append_name (predication_suffixes[instance.pred]);
+ return b.finish_name ();
+ }
+};
+
+
+/* th_indexed_loadstore_width_def class. */
+struct th_indexed_loadstore_width_def : public function_shape
+{
+ void build (function_builder &b,
+ const function_group_info &group) const override
+ {
+ for (unsigned int pred_idx = 0; group.preds[pred_idx] != NUM_PRED_TYPES;
+ ++pred_idx)
+ {
+ for (unsigned int vec_type_idx = 0;
+ group.ops_infos.types[vec_type_idx].index != NUM_VECTOR_TYPES;
+ ++vec_type_idx)
+ {
+ tree index_type = group.ops_infos.args[1].get_tree_type (
+ group.ops_infos.types[vec_type_idx].index);
+ if (!index_type)
+ continue;
+ build_th_loadstore (b, group, pred_idx, vec_type_idx);
+ }
+ }
+ }
+
+ char *get_name (function_builder &b, const function_instance &instance,
+ bool overloaded_p) const override
+ {
+
+ /* Return nullptr if it can not be overloaded. */
+ if (overloaded_p && !instance.base->can_be_overloaded_p (instance.pred))
+ return nullptr;
+
+ b.append_name ("__riscv_th_");
+ b.append_name (instance.base_name);
+ /* vop_v --> vop_v_<type>. */
+ if (!overloaded_p)
+ {
+ /* vop --> vop_v. */
+ b.append_name (operand_suffixes[instance.op_info->op]);
+ /* vop_v --> vop_v_<type>. */
+ b.append_name (type_suffixes[instance.type.index].vector);
+ }
+
+ /* According to rvv-intrinsic-doc, it does not add "_m" suffix
+ for vop_m C++ overloaded API. */
+ if (overloaded_p && instance.pred == PRED_TYPE_m)
+ return b.finish_name ();
+ b.append_name (predication_suffixes[instance.pred]);
+ return b.finish_name ();
+ }
+};
+
/* alu_def class. */
struct alu_def : public build_base
{
@@ -632,6 +772,23 @@ struct reduc_alu_def : public build_base
}
};
+/* th_extract_def class. */
+struct th_extract_def : public build_base
+{
+ char *get_name (function_builder &b, const function_instance &instance,
+ bool overloaded_p) const override
+ {
+ b.append_name ("__riscv_th_");
+ b.append_name (instance.base_name);
+
+ if (overloaded_p)
+ return b.finish_name ();
+ b.append_name (type_suffixes[instance.type.index].vector);
+ b.append_name (type_suffixes[instance.type.index].scalar);
+ return b.finish_name ();
+ }
+};
+
/* scalar_move_def class. */
struct scalar_move_def : public build_base
{
@@ -1094,6 +1251,8 @@ SHAPE(vsetvl, vsetvl)
SHAPE(vsetvl, vsetvlmax)
SHAPE(loadstore, loadstore)
SHAPE(indexed_loadstore, indexed_loadstore)
+SHAPE(th_loadstore_width, th_loadstore_width)
+SHAPE(th_indexed_loadstore_width, th_indexed_loadstore_width)
SHAPE(alu, alu)
SHAPE(alu_frm, alu_frm)
SHAPE(widen_alu, widen_alu)
@@ -1106,6 +1265,7 @@ SHAPE(move, move)
SHAPE(mask_alu, mask_alu)
SHAPE(reduc_alu, reduc_alu)
SHAPE(reduc_alu_frm, reduc_alu_frm)
+SHAPE(th_extract, th_extract)
SHAPE(scalar_move, scalar_move)
SHAPE(vundefined, vundefined)
SHAPE(misc, misc)
diff --git a/gcc/config/riscv/riscv-vector-builtins-shapes.h b/gcc/config/riscv/riscv-vector-builtins-shapes.h
index ac2a28ce017..a7624d0fabd 100644
--- a/gcc/config/riscv/riscv-vector-builtins-shapes.h
+++ b/gcc/config/riscv/riscv-vector-builtins-shapes.h
@@ -28,6 +28,8 @@ extern const function_shape *const vsetvl;
extern const function_shape *const vsetvlmax;
extern const function_shape *const loadstore;
extern const function_shape *const indexed_loadstore;
+extern const function_shape *const th_loadstore_width;
+extern const function_shape *const th_indexed_loadstore_width;
extern const function_shape *const alu;
extern const function_shape *const alu_frm;
extern const function_shape *const widen_alu;
@@ -41,6 +43,7 @@ extern const function_shape *const mask_alu;
extern const function_shape *const reduc_alu;
extern const function_shape *const reduc_alu_frm;
extern const function_shape *const scalar_move;
+extern const function_shape *const th_extract;
extern const function_shape *const vundefined;
extern const function_shape *const misc;
extern const function_shape *const vset;
diff --git a/gcc/config/riscv/riscv-vector-builtins.cc b/gcc/config/riscv/riscv-vector-builtins.cc
index 25e0b6e56de..44b9fec1898 100644
--- a/gcc/config/riscv/riscv-vector-builtins.cc
+++ b/gcc/config/riscv/riscv-vector-builtins.cc
@@ -934,6 +934,32 @@ static CONSTEXPR const rvv_arg_type_info ext_vcreate_args[]
= {rvv_arg_type_info (RVV_BASE_vector),
rvv_arg_type_info_end};
+/* A list of args for vector_type func (const scalar_type *, size_t)
+ * function. */
+static CONSTEXPR const rvv_arg_type_info scalar_const_ptr_size_args[]
+ = {rvv_arg_type_info (RVV_BASE_scalar_const_ptr),
+ rvv_arg_type_info (RVV_BASE_size), rvv_arg_type_info_end};
+
+/* A list of args for vector_type func (const scalar_type *, eew8_index_type)
+ * function. */
+static CONSTEXPR const rvv_arg_type_info scalar_const_ptr_index_args[]
+ = {rvv_arg_type_info (RVV_BASE_scalar_const_ptr),
+ rvv_arg_type_info (RVV_BASE_unsigned_vector), rvv_arg_type_info_end};
+
+/* A list of args for void func (scalar_type *, eew8_index_type, vector_type)
+ * function. */
+static CONSTEXPR const rvv_arg_type_info scalar_ptr_index_args[]
+ = {rvv_arg_type_info (RVV_BASE_scalar_ptr),
+ rvv_arg_type_info (RVV_BASE_unsigned_vector),
+ rvv_arg_type_info (RVV_BASE_vector), rvv_arg_type_info_end};
+
+/* A list of args for void func (scalar_type *, size_t, vector_type)
+ * function. */
+static CONSTEXPR const rvv_arg_type_info scalar_ptr_size_args[]
+ = {rvv_arg_type_info (RVV_BASE_scalar_ptr),
+ rvv_arg_type_info (RVV_BASE_size), rvv_arg_type_info (RVV_BASE_vector),
+ rvv_arg_type_info_end};
+
/* A list of none preds that will be registered for intrinsic functions. */
static CONSTEXPR const predication_type_index none_preds[]
= {PRED_TYPE_none, NUM_PRED_TYPES};
@@ -1455,6 +1481,14 @@ static CONSTEXPR const rvv_op_info iu_shift_vvv_ops
rvv_arg_type_info (RVV_BASE_vector), /* Return type */
shift_vv_args /* Args */};
+/* A static operand information for scalar_type func (vector_type, size_t)
+ * function registration. */
+static CONSTEXPR const rvv_op_info iu_x_s_u_ops
+ = {iu_ops, /* Types */
+ OP_TYPE_vx, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_scalar), /* Return type */
+ v_size_args /* Args */};
+
/* A static operand information for vector_type func (vector_type, size_t)
* function registration. */
static CONSTEXPR const rvv_op_info iu_shift_vvx_ops
@@ -2638,6 +2672,38 @@ static CONSTEXPR const rvv_op_info all_v_vcreate_lmul4_x2_ops
rvv_arg_type_info (RVV_BASE_vlmul_ext_x2), /* Return type */
ext_vcreate_args /* Args */};
+/* A static operand information for vector_type func (const scalar_type *,
+ * size_t) function registration. */
+static CONSTEXPR const rvv_op_info all_v_scalar_const_ptr_size_ops
+ = {all_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_size_args /* Args */};
+
+/* A static operand information for void func (scalar_type *, size_t,
+ * vector_type) function registration. */
+static CONSTEXPR const rvv_op_info all_v_scalar_ptr_size_ops
+ = {all_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_void), /* Return type */
+ scalar_ptr_size_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * index_type) function registration. */
+static CONSTEXPR const rvv_op_info all_v_scalar_const_ptr_index_ops
+ = {all_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_index_args /* Args */};
+
+/* A static operand information for void func (scalar_type *, index_type,
+ * vector_type) function registration. */
+static CONSTEXPR const rvv_op_info all_v_scalar_ptr_index_ops
+ = {all_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_void), /* Return type */
+ scalar_ptr_index_args /* Args */};
+
/* A static operand information for vector_type func (vector_type).
Some ins just supports SEW=32, such as crypto vectol Zvkg extension.
* function registration. */
@@ -2816,6 +2882,10 @@ static function_group_info function_groups[] = {
#define DEF_RVV_FUNCTION(NAME, SHAPE, PREDS, OPS_INFO) \
{#NAME, &bases::NAME, &shapes::SHAPE, PREDS, OPS_INFO, REQUIRED_EXTENSIONS},
#include "riscv-vector-builtins-functions.def"
+#undef DEF_RVV_FUNCTION
+#define DEF_RVV_FUNCTION(NAME, SHAPE, PREDS, OPS_INFO) \
+ {#NAME, &bases::NAME, &shapes::SHAPE, PREDS, OPS_INFO, REQUIRED_EXTENSIONS},
+#include "thead-vector-builtins-functions.def"
};
/* The RVV types, with their built-in
diff --git a/gcc/config/riscv/riscv-vector-builtins.h b/gcc/config/riscv/riscv-vector-builtins.h
index 54c8824ff92..22fed60b4c3 100644
--- a/gcc/config/riscv/riscv-vector-builtins.h
+++ b/gcc/config/riscv/riscv-vector-builtins.h
@@ -123,6 +123,7 @@ enum required_ext
ZVKNHB_EXT, /* Crypto vector Zvknhb sub-ext */
ZVKSED_EXT, /* Crypto vector Zvksed sub-ext */
ZVKSH_EXT, /* Crypto vector Zvksh sub-ext */
+ XTHEADVECTOR_EXT, /* XTheadVector extension */
};
/* Enumerates the RVV operand types. */
@@ -252,6 +253,8 @@ struct function_group_info
return TARGET_ZVKSED;
case ZVKSH_EXT:
return TARGET_ZVKSH;
+ case XTHEADVECTOR_EXT:
+ return TARGET_XTHEADVECTOR;
default:
gcc_unreachable ();
}
diff --git a/gcc/config/riscv/t-riscv b/gcc/config/riscv/t-riscv
index 32de6b851c1..38494320d8b 100644
--- a/gcc/config/riscv/t-riscv
+++ b/gcc/config/riscv/t-riscv
@@ -1,6 +1,7 @@
RISCV_BUILTINS_H = $(srcdir)/config/riscv/riscv-vector-builtins.h \
$(srcdir)/config/riscv/riscv-vector-builtins.def \
$(srcdir)/config/riscv/riscv-vector-builtins-functions.def \
+ $(srcdir)/config/riscv/thead-vector-builtins-functions.def \
riscv-vector-type-indexer.gen.def
riscv-builtins.o: $(srcdir)/config/riscv/riscv-builtins.cc $(CONFIG_H) \
diff --git a/gcc/config/riscv/thead-vector-builtins-functions.def b/gcc/config/riscv/thead-vector-builtins-functions.def
new file mode 100644
index 00000000000..fd3ba29bae9
--- /dev/null
+++ b/gcc/config/riscv/thead-vector-builtins-functions.def
@@ -0,0 +1,39 @@
+#ifndef DEF_RVV_FUNCTION
+#define DEF_RVV_FUNCTION(NAME, SHAPE, PREDS, OPS_INFO)
+#endif
+
+#define REQUIRED_EXTENSIONS XTHEADVECTOR_EXT
+DEF_RVV_FUNCTION (vlb, th_loadstore_width, full_preds, all_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (vlh, th_loadstore_width, full_preds, all_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (vlw, th_loadstore_width, full_preds, all_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (vlbu, th_loadstore_width, full_preds, all_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (vlhu, th_loadstore_width, full_preds, all_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (vlwu, th_loadstore_width, full_preds, all_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (vsb, th_loadstore_width, none_m_preds, all_v_scalar_ptr_ops)
+DEF_RVV_FUNCTION (vsh, th_loadstore_width, none_m_preds, all_v_scalar_ptr_ops)
+DEF_RVV_FUNCTION (vsw, th_loadstore_width, none_m_preds, all_v_scalar_ptr_ops)
+DEF_RVV_FUNCTION (vlsb, th_loadstore_width, full_preds, all_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (vlsh, th_loadstore_width, full_preds, all_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (vlsw, th_loadstore_width, full_preds, all_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (vlsbu, th_loadstore_width, full_preds, all_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (vlshu, th_loadstore_width, full_preds, all_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (vlswu, th_loadstore_width, full_preds, all_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (vssb, th_loadstore_width, none_m_preds, all_v_scalar_ptr_size_ops)
+DEF_RVV_FUNCTION (vssh, th_loadstore_width, none_m_preds, all_v_scalar_ptr_size_ops)
+DEF_RVV_FUNCTION (vssw, th_loadstore_width, none_m_preds, all_v_scalar_ptr_size_ops)
+DEF_RVV_FUNCTION (vlxb, th_indexed_loadstore_width, full_preds, all_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (vlxh, th_indexed_loadstore_width, full_preds, all_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (vlxw, th_indexed_loadstore_width, full_preds, all_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (vlxbu, th_indexed_loadstore_width, full_preds, all_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (vlxhu, th_indexed_loadstore_width, full_preds, all_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (vlxwu, th_indexed_loadstore_width, full_preds, all_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (vsxb, th_indexed_loadstore_width, none_m_preds, all_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (vsxh, th_indexed_loadstore_width, none_m_preds, all_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (vsxw, th_indexed_loadstore_width, none_m_preds, all_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (vsuxb, th_indexed_loadstore_width, none_m_preds, all_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (vsuxh, th_indexed_loadstore_width, none_m_preds, all_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (vsuxw, th_indexed_loadstore_width, none_m_preds, all_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (vext_x_v, th_extract, none_preds, iu_x_s_u_ops)
+#undef REQUIRED_EXTENSIONS
+
+#undef DEF_RVV_FUNCTION
diff --git a/gcc/config/riscv/thead-vector.md b/gcc/config/riscv/thead-vector.md
index 696b815252d..5fe9ba08c4e 100644
--- a/gcc/config/riscv/thead-vector.md
+++ b/gcc/config/riscv/thead-vector.md
@@ -1,7 +1,95 @@
(define_c_enum "unspec" [
+ UNSPEC_TH_VLB
+ UNSPEC_TH_VLBU
+ UNSPEC_TH_VLH
+ UNSPEC_TH_VLHU
+ UNSPEC_TH_VLW
+ UNSPEC_TH_VLWU
+
+ UNSPEC_TH_VLSB
+ UNSPEC_TH_VLSBU
+ UNSPEC_TH_VLSH
+ UNSPEC_TH_VLSHU
+ UNSPEC_TH_VLSW
+ UNSPEC_TH_VLSWU
+
+ UNSPEC_TH_VLXB
+ UNSPEC_TH_VLXBU
+ UNSPEC_TH_VLXH
+ UNSPEC_TH_VLXHU
+ UNSPEC_TH_VLXW
+ UNSPEC_TH_VLXWU
+
+ UNSPEC_TH_VSUXB
+ UNSPEC_TH_VSUXH
+ UNSPEC_TH_VSUXW
+
UNSPEC_TH_VWLDST
])
+(define_int_iterator UNSPEC_TH_VLMEM_OP [
+ UNSPEC_TH_VLB UNSPEC_TH_VLBU
+ UNSPEC_TH_VLH UNSPEC_TH_VLHU
+ UNSPEC_TH_VLW UNSPEC_TH_VLWU
+])
+
+(define_int_iterator UNSPEC_TH_VLSMEM_OP [
+ UNSPEC_TH_VLSB UNSPEC_TH_VLSBU
+ UNSPEC_TH_VLSH UNSPEC_TH_VLSHU
+ UNSPEC_TH_VLSW UNSPEC_TH_VLSWU
+])
+
+(define_int_iterator UNSPEC_TH_VLXMEM_OP [
+ UNSPEC_TH_VLXB UNSPEC_TH_VLXBU
+ UNSPEC_TH_VLXH UNSPEC_TH_VLXHU
+ UNSPEC_TH_VLXW UNSPEC_TH_VLXWU
+])
+
+(define_int_attr vlmem_op_attr [
+ (UNSPEC_TH_VLB "b") (UNSPEC_TH_VLBU "bu")
+ (UNSPEC_TH_VLH "h") (UNSPEC_TH_VLHU "hu")
+ (UNSPEC_TH_VLW "w") (UNSPEC_TH_VLWU "wu")
+ (UNSPEC_TH_VLSB "b") (UNSPEC_TH_VLSBU "bu")
+ (UNSPEC_TH_VLSH "h") (UNSPEC_TH_VLSHU "hu")
+ (UNSPEC_TH_VLSW "w") (UNSPEC_TH_VLSWU "wu")
+ (UNSPEC_TH_VLXB "b") (UNSPEC_TH_VLXBU "bu")
+ (UNSPEC_TH_VLXH "h") (UNSPEC_TH_VLXHU "hu")
+ (UNSPEC_TH_VLXW "w") (UNSPEC_TH_VLXWU "wu")
+ (UNSPEC_TH_VSUXB "b")
+ (UNSPEC_TH_VSUXH "h")
+ (UNSPEC_TH_VSUXW "w")
+])
+
+(define_int_attr vlmem_order_attr [
+ (UNSPEC_TH_VLXB "")
+ (UNSPEC_TH_VLXH "")
+ (UNSPEC_TH_VLXW "")
+ (UNSPEC_TH_VSUXB "u")
+ (UNSPEC_TH_VSUXH "u")
+ (UNSPEC_TH_VSUXW "u")
+])
+
+(define_int_iterator UNSPEC_TH_VSMEM_OP [
+ UNSPEC_TH_VLB
+ UNSPEC_TH_VLH
+ UNSPEC_TH_VLW
+])
+
+(define_int_iterator UNSPEC_TH_VSSMEM_OP [
+ UNSPEC_TH_VLSB
+ UNSPEC_TH_VLSH
+ UNSPEC_TH_VLSW
+])
+
+(define_int_iterator UNSPEC_TH_VSXMEM_OP [
+ UNSPEC_TH_VLXB
+ UNSPEC_TH_VLXH
+ UNSPEC_TH_VLXW
+ UNSPEC_TH_VSUXB
+ UNSPEC_TH_VSUXH
+ UNSPEC_TH_VSUXW
+])
+
(define_mode_iterator V_VLS_VT [V VLS VT])
(define_mode_iterator V_VB_VLS_VT [V VB VLS VT])
@@ -100,3 +188,165 @@
}
[(set_attr "type" "vldm,vstm,vmalu,vmalu,vmalu")
(set_attr "mode" "<MODE>")])
+
+(define_expand "@pred_mov_width<vlmem_op_attr><mode>"
+ [(set (match_operand:V_VLS 0 "nonimmediate_operand")
+ (if_then_else:V_VLS
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand")
+ (match_operand 4 "vector_length_operand")
+ (match_operand 5 "const_int_operand")
+ (match_operand 6 "const_int_operand")
+ (match_operand 7 "const_int_operand")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLMEM_OP)
+ (match_operand:V_VLS 3 "vector_move_operand")
+ (match_operand:V_VLS 2 "vector_merge_operand")))]
+ "TARGET_XTHEADVECTOR"
+ {})
+
+(define_insn_and_split "*pred_mov_width<vlmem_op_attr><mode>"
+ [(set (match_operand:V_VLS 0 "nonimmediate_operand" "=vr, vr, vd, m, vr, vr")
+ (if_then_else:V_VLS
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1, Wc1, vm, vmWc1, Wc1, Wc1")
+ (match_operand 4 "vector_length_operand" " rK, rK, rK, rK, rK, rK")
+ (match_operand 5 "const_int_operand" " i, i, i, i, i, i")
+ (match_operand 6 "const_int_operand" " i, i, i, i, i, i")
+ (match_operand 7 "const_int_operand" " i, i, i, i, i, i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLMEM_OP)
+ (match_operand:V_VLS 3 "reg_or_mem_operand" " m, m, m, vr, vr, vr")
+ (match_operand:V_VLS 2 "vector_merge_operand" " 0, vu, vu, vu, vu, 0")))]
+ "(TARGET_XTHEADVECTOR
+ && (register_operand (operands[0], <MODE>mode)
+ || register_operand (operands[3], <MODE>mode)))"
+ "@
+ vl<vlmem_op_attr>.v\t%0,%3%p1
+ vl<vlmem_op_attr>.v\t%0,%3
+ vl<vlmem_op_attr>.v\t%0,%3,%1.t
+ vs<vlmem_op_attr>.v\t%3,%0%p1
+ vmv.v.v\t%0,%3
+ vmv.v.v\t%0,%3"
+ "&& riscv_vector::whole_reg_to_reg_move_p (operands, <MODE>mode, 7)"
+ [(set (match_dup 0) (match_dup 3))]
+ ""
+ [(set_attr "type" "vlde,vlde,vlde,vste,vimov,vimov")
+ (set_attr "mode" "<MODE>")])
+
+(define_insn "@pred_store_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "memory_operand" "+m")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1")
+ (match_operand 3 "vector_length_operand" " rK")
+ (match_operand 4 "const_int_operand" " i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VSMEM_OP)
+ (match_operand:VI 2 "register_operand" " vr")
+ (match_dup 0)))]
+ "TARGET_XTHEADVECTOR"
+ "vs<vlmem_op_attr>.v\t%2,%0%p1"
+ [(set_attr "type" "vste")
+ (set_attr "mode" "<MODE>")
+ (set (attr "avl_type_idx") (const_int 4))
+ (set_attr "vl_op_idx" "3")])
+
+(define_insn "@pred_strided_load_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "register_operand" "=vr, vr, vd")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1, Wc1, vm")
+ (match_operand 5 "vector_length_operand" " rK, rK, rK")
+ (match_operand 6 "const_int_operand" " i, i, i")
+ (match_operand 7 "const_int_operand" " i, i, i")
+ (match_operand 8 "const_int_operand" " i, i, i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLSMEM_OP)
+ (unspec:VI
+ [(match_operand:VI 3 "memory_operand" " m, m, m")
+ (match_operand 4 "pmode_reg_or_0_operand" " rJ, rJ, rJ")] UNSPEC_TH_VLSMEM_OP)
+ (match_operand:VI 2 "vector_merge_operand" " 0, vu, vu")))]
+ "TARGET_XTHEADVECTOR"
+ "vls<vlmem_op_attr>.v\t%0,%3,%z4%p1"
+ [(set_attr "type" "vlds")
+ (set_attr "mode" "<MODE>")])
+
+(define_insn "@pred_strided_store_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "memory_operand" "+m")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1")
+ (match_operand 4 "vector_length_operand" " rK")
+ (match_operand 5 "const_int_operand" " i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VSSMEM_OP)
+ (unspec:VI
+ [(match_operand 2 "pmode_reg_or_0_operand" " rJ")
+ (match_operand:VI 3 "register_operand" " vr")] UNSPEC_TH_VSSMEM_OP)
+ (match_dup 0)))]
+ "TARGET_XTHEADVECTOR"
+ "vss<vlmem_op_attr>.v\t%3,%0,%z2%p1"
+ [(set_attr "type" "vsts")
+ (set_attr "mode" "<MODE>")
+ (set (attr "avl_type_idx") (const_int 5))])
+
+(define_insn "@pred_indexed_load_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "register_operand" "=vd, vr,vd, vr")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" " vm,Wc1,vm,Wc1")
+ (match_operand 5 "vector_length_operand" " rK, rK,rK, rK")
+ (match_operand 6 "const_int_operand" " i, i, i, i")
+ (match_operand 7 "const_int_operand" " i, i, i, i")
+ (match_operand 8 "const_int_operand" " i, i, i, i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLXMEM_OP)
+ (unspec:VI
+ [(match_operand 3 "pmode_reg_or_0_operand" " rJ, rJ,rJ, rJ")
+ (mem:BLK (scratch))
+ (match_operand:VI 4 "register_operand" " vr, vr,vr, vr")] UNSPEC_TH_VLXMEM_OP)
+ (match_operand:VI 2 "vector_merge_operand" " vu, vu, 0, 0")))]
+ "TARGET_XTHEADVECTOR"
+ "vlx<vlmem_op_attr>.v\t%0,(%z3),%4%p1"
+ [(set_attr "type" "vldux")
+ (set_attr "mode" "<MODE>")])
+
+(define_insn "@pred_indexed_<vlmem_order_attr>store_width<vlmem_op_attr><mode>"
+ [(set (mem:BLK (scratch))
+ (unspec:BLK
+ [(unspec:<VM>
+ [(match_operand:<VM> 0 "vector_mask_operand" "vmWc1")
+ (match_operand 4 "vector_length_operand" " rK")
+ (match_operand 5 "const_int_operand" " i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VSXMEM_OP)
+ (match_operand 1 "pmode_reg_or_0_operand" " rJ")
+ (match_operand:VI 2 "register_operand" " vr")
+ (match_operand:VI 3 "register_operand" " vr")] UNSPEC_TH_VSXMEM_OP))]
+ "TARGET_XTHEADVECTOR"
+ "vs<vlmem_order_attr>x<vlmem_op_attr>.v\t%3,(%z1),%2%p0"
+ [(set_attr "type" "vstux")
+ (set_attr "mode" "<MODE>")])
+
+(define_expand "@pred_th_extract<mode>"
+ [(set (match_operand:<VEL> 0 "register_operand")
+ (unspec:<VEL>
+ [(vec_select:<VEL>
+ (match_operand:V_VLSI 1 "register_operand")
+ (parallel [(match_operand:DI 2 "register_operand" "r")]))
+ (reg:SI VTYPE_REGNUM)] UNSPEC_VPREDICATE))]
+ "TARGET_XTHEADVECTOR"
+{})
+
+(define_insn "*pred_th_extract<mode>"
+ [(set (match_operand:<VEL> 0 "register_operand" "=r")
+ (unspec:<VEL>
+ [(vec_select:<VEL>
+ (match_operand:V_VLSI 1 "register_operand" "vr")
+ (parallel [(match_operand:DI 2 "register_operand" "r")]))
+ (reg:SI VTYPE_REGNUM)] UNSPEC_VPREDICATE))]
+ "TARGET_XTHEADVECTOR"
+ "vext.x.v\t%0,%1,%2"
+ [(set_attr "type" "vimovvx")
+ (set_attr "mode" "<MODE>")])
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c
new file mode 100644
index 00000000000..3c12c124597
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_th_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+
+** th.vadd\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+
+** th.vsb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out)
+{
+ vint32m1_t v = __riscv_th_vlb_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlb_v_i32m1_tu (v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vv_i32m1 (v2, v2, 4);
+ vint32m1_t v4 = __riscv_vadd_vv_i32m1_tu (v3, v2, v2, 4);
+ __riscv_th_vsb_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlb.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+
+** th.vadd\.vv\tv[1-9][0-9]?,\s*v[0-9]+,\s*v[0-9]+,\s*v0.t
+** th.vsb.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlb_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlb_v_i32m1_m (mask, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vv_i32m1 (v2, v2, 4);
+ vint32m1_t v4 = __riscv_vadd_vv_i32m1_m (mask, v3, v3, 4);
+ __riscv_th_vsb_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlb.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+
+** th.vadd\.vv\tv[1-9][0-9]?,\s*v[0-9]+,\s*v[0-9]+,\s*v0.t
+** th.vsb.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlb_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlb_v_i32m1_tumu (mask, v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vv_i32m1 (v2, v2, 4);
+ vint32m1_t v4 = __riscv_vadd_vv_i32m1_tumu (mask, v3, v2, v2, 4);
+ __riscv_th_vsb_v_i32m1 (out, v4, 4);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c
new file mode 100644
index 00000000000..30bef369375
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_th_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlbu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlbu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vsb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, uint32_t x)
+{
+ vuint32m1_t v = __riscv_th_vlbu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlbu_v_u32m1_tu (v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tu (v3, v2, -16, 4);
+ __riscv_th_vsb_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlbu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsb.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlbu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlbu_v_u32m1_m (mask, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_m (mask, v3, -16, 4);
+ __riscv_th_vsb_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlbu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlbu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsb.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlbu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlbu_v_u32m1_tumu (mask, v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tumu (mask, v3, v2, -16, 4);
+ __riscv_th_vsb_v_u32m1 (out, v4, 4);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c
new file mode 100644
index 00000000000..3c8b5ccc16b
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_th_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vsh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, int32_t x)
+{
+ vint32m1_t v = __riscv_th_vlh_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlh_v_i32m1_tu (v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, -16, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_tu (v3, v2, -16, 4);
+ __riscv_th_vsh_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlh.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsh.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, int32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlh_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlh_v_i32m1_m (mask, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, -16, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_m (mask, v3, -16, 4);
+ __riscv_th_vsh_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlh.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsh.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, int32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlh_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlh_v_i32m1_tumu (mask, v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, -16, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_tumu (mask, v3, v2, -16, 4);
+ __riscv_th_vsh_v_i32m1 (out, v4, 4);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c
new file mode 100644
index 00000000000..b7c00404f18
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_th_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlhu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlhu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vsh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, uint32_t x)
+{
+ vuint32m1_t v = __riscv_th_vlhu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlhu_v_u32m1_tu (v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tu (v3, v2, -16, 4);
+ __riscv_th_vsh_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlhu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsh.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlhu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlhu_v_u32m1_m (mask, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_m (mask, v3, -16, 4);
+ __riscv_th_vsh_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlhu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlhu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsh.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlhu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlhu_v_u32m1_tumu (mask, v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tumu (mask, v3, v2, -16, 4);
+ __riscv_th_vsh_v_u32m1 (out, v4, 4);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c
new file mode 100644
index 00000000000..17a53012acf
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_th_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vx\tv[0-9]+,\s*v[0-9]+,\s*[a-x0-9]+
+** th.vadd\.vx\tv[0-9]+,\s*v[0-9]+,\s*[a-x0-9]+
+** th.vsw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, int32_t x)
+{
+ vint32m1_t v = __riscv_th_vlw_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlw_v_i32m1_tu (v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, x, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_tu (v3, v2, x, 4);
+ __riscv_th_vsw_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlw.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vx\tv[0-9]+,\s*v[0-9]+,\s*[a-x0-9]+
+** th.vadd\.vx\tv[1-9][0-9]?,\s*v[0-9]+,\s*[a-x0-9]+,\s*v0.t
+** th.vsw.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, int32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlw_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlw_v_i32m1_m (mask, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, x, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_m (mask, v3, x, 4);
+ __riscv_th_vsw_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlw.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vx\tv[0-9]+,\s*v[0-9]+,\s*[a-x0-9]+
+** th.vadd\.vx\tv[1-9][0-9]?,\s*v[0-9]+,\s*[a-x0-9]+,\s*v0.t
+** th.vsw.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, int32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlw_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlw_v_i32m1_tumu (mask, v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, x, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_tumu (mask, v3, v2, x, 4);
+ __riscv_th_vsw_v_i32m1 (out, v4, 4);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c
new file mode 100644
index 00000000000..b187cfc852b
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_th_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlwu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlwu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vsw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, uint32_t x)
+{
+ vuint32m1_t v = __riscv_th_vlwu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlwu_v_u32m1_tu (v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tu (v3, v2, -16, 4);
+ __riscv_th_vsw_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlwu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsw.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlwu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlwu_v_u32m1_m (mask, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_m (mask, v3, -16, 4);
+ __riscv_th_vsw_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlwu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlwu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsw.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlwu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlwu_v_u32m1_tumu (mask, v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tumu (mask, v3, v2, -16, 4);
+ __riscv_th_vsw_v_u32m1 (out, v4, 4);
+}
--
2.17.1
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re：Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
2024-01-11 12:33 Re：[PATCH " juzhe.zhong
@ 2024-01-11 12:36 ` joshua
0 siblings, 0 replies; 17+ messages in thread
From: joshua @ 2024-01-11 12:36 UTC (permalink / raw)
To: juzhe.zhong, gcc-patches
Cc: Jim Wilson, palmer, andrew, philipp.tomsich, jeffreyalaw,
christoph.muellner, jinma, cooper.qu
Perhaps not. I suggest that is the limitation of rvv 0.7.1.
I will consult my colleagues who are familiar with using rvv 0.7.1 tomorrow.
------------------------------------------------------------------
发件人：juzhe.zhong@rivai.ai <juzhe.zhong@rivai.ai>
发送时间：2024年1月11日(星期四) 20:33
收件人："cooper.joshua"<cooper.joshua@linux.alibaba.com>; "gcc-patches"<gcc-patches@gcc.gnu.org>
抄 送：Jim Wilson<jim.wilson.gcc@gmail.com>; palmer<palmer@dabbelt.com>; andrew<andrew@sifive.com>; "philipp.tomsich"<philipp.tomsich@vrull.eu>; jeffreyalaw<jeffreyalaw@gmail.com>; "christoph.muellner"<christoph.muellner@vrull.eu>; jinma<jinma@linux.alibaba.com>; "cooper.qu"<cooper.qu@linux.alibaba.com>
主 题：Re: Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
Does theadvector has extension instructions ?
Show me the pattern.
juzhe.zhong@rivai.ai
发件人： joshua
发送时间： 2024-01-11 20:31
收件人： juzhe.zhong@rivai.ai; gcc-patches
抄送： Jim Wilson; palmer; andrew; philipp.tomsich; jeffreyalaw; christoph.muellner; jinma; cooper.qu
主题： Re：Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
Yes.
(insn 14 13 15 2 (set (reg:QI 147)
(const_int 1 [0x1])) {*movqi_internal}
(nil))
------------------------------------------------------------------
发件人：juzhe.zhong@rivai.ai <juzhe.zhong@rivai.ai>
发送时间：2024年1月11日(星期四) 20:28
收件人："cooper.joshua"<cooper.joshua@linux.alibaba.com>; "gcc-patches"<gcc-patches@gcc.gnu.org>
抄 送：Jim Wilson<jim.wilson.gcc@gmail.com>; palmer<palmer@dabbelt.com>; andrew<andrew@sifive.com>; "philipp.tomsich"<philipp.tomsich@vrull.eu>; jeffreyalaw<jeffreyalaw@gmail.com>; "christoph.muellner"<christoph.muellner@vrull.eu>; jinma<jinma@linux.alibaba.com>; "cooper.qu"<cooper.qu@linux.alibaba.com>
主 题：Re: Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
(vec_duplicate:RVVM1QI (reg:QI 147))))
Find the RTL define pseudo 147 to me.
I guess it is 1.
juzhe.zhong@rivai.ai
发件人： joshua
发送时间： 2024-01-11 20:18
收件人： juzhe.zhong@rivai.ai; gcc-patches
抄送： Jim Wilson; palmer; andrew; philipp.tomsich; jeffreyalaw; christoph.muellner; jinma; cooper.qu
主题： Re：Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
No, we have debugged and the code won't enter this autovec pattern.
The root cause is CSE, in a very early pass.
What I sent to you just now is the dump result which shows the difference.
------------------------------------------------------------------
发件人：juzhe.zhong@rivai.ai <juzhe.zhong@rivai.ai>
发送时间：2024年1月11日(星期四) 20:13
收件人："cooper.joshua"<cooper.joshua@linux.alibaba.com>; "gcc-patches"<gcc-patches@gcc.gnu.org>
抄 送：Jim Wilson<jim.wilson.gcc@gmail.com>; palmer<palmer@dabbelt.com>; andrew<andrew@sifive.com>; "philipp.tomsich"<philipp.tomsich@vrull.eu>; jeffreyalaw<jeffreyalaw@gmail.com>; "christoph.muellner"<christoph.muellner@vrull.eu>; jinma<jinma@linux.alibaba.com>; "cooper.qu"<cooper.qu@linux.alibaba.com>
主 题：Re: Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
I guess it is because of patterns from autovectorization pattern:
;; Use define_insn_and_split to define vsext.vf2/vzext.vf2 will help
;; to combine instructions as below:
;; vsext.vf2 + vsext.vf2 + vadd.vv ==> vwadd.vv
(define_insn_and_split "<optab><v_double_trunc><mode>2"
[(set (match_operand:VWEXTI 0 "register_operand" "=&vr")
(any_extend:VWEXTI
(match_operand:<V_DOUBLE_TRUNC> 1 "register_operand" "vr")))]
"TARGET_VECTOR && can_create_pseudo_p ()"
"#"
"&& 1"
[(const_int 0)]
{
insn_code icode = code_for_pred_vf2 (<CODE>, <MODE>mode);
riscv_vector::emit_vlmax_insn (icode, riscv_vector::UNARY_OP, operands);
DONE;
}
[(set_attr "type" "vext")
(set_attr "mode" "<MODE>")])
juzhe.zhong@rivai.ai
发件人： joshua
发送时间： 2024-01-11 20:05
收件人： juzhe.zhong@rivai.ai; gcc-patches
抄送： Jim Wilson; palmer; andrew; philipp.tomsich; jeffreyalaw; christoph.muellner; jinma; cooper.qu
主题： Re：Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
without CSE:
(insn 16 15 17 2 (set (reg/v:RVVM2HI 137 [ output_var_0 ])
(if_then_else:RVVM2HI (unspec:RVVMF8BI [
(const_vector:RVVMF8BI repeat [
(const_int 1 [0x1])
])
(reg:DI 146)
(const_int 2 [0x2]) repeated x2
(const_int 0 [0])
(reg:SI 66 vl)
(reg:SI 67 vtype)
] UNSPEC_VPREDICATE)
(mult:RVVM2HI (sign_extend:RVVM2HI (reg/v:RVVM1QI 136 [ op1 ]))
(sign_extend:RVVM2HI (vec_duplicate:RVVM1QI (reg:QI 147))))
(unspec:RVVM2HI [
(reg:SI 0 zero)
] UNSPEC_VUNDEF))) {pred_dual_widen_mulsrvvm2hi_scalar}
with CSE:
(insn 16 15 17 2 (set (reg/v:RVVM2HI 137 [ output_var_0 ])
(if_then_else:RVVM2HI (unspec:RVVMF8BI [
(const_vector:RVVMF8BI repeat [
(const_int 1 [0x1])
])
(reg:DI 146)
(const_int 2 [0x2]) repeated x2
(const_int 0 [0])
(reg:SI 66 vl)
(reg:SI 67 vtype)
] UNSPEC_VPREDICATE)
(sign_extend:RVVM2HI (reg/v:RVVM1QI 136 [ op1 ]))
(unspec:RVVM2HI [
(reg:SI 0 zero)
] UNSPEC_VUNDEF))) {pred_extendrvvm2hi_vf2}
------------------------------------------------------------------
发件人：juzhe.zhong@rivai.ai <juzhe.zhong@rivai.ai>
发送时间：2024年1月11日(星期四) 17:32
收件人："cooper.joshua"<cooper.joshua@linux.alibaba.com>; "gcc-patches"<gcc-patches@gcc.gnu.org>
抄 送：Jim Wilson<jim.wilson.gcc@gmail.com>; palmer<palmer@dabbelt.com>; andrew<andrew@sifive.com>; "philipp.tomsich"<philipp.tomsich@vrull.eu>; jeffreyalaw<jeffreyalaw@gmail.com>; "christoph.muellner"<christoph.muellner@vrull.eu>; jinma<jinma@linux.alibaba.com>; "cooper.qu"<cooper.qu@linux.alibaba.com>
主 题：Re: Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
Ok. Let's hold on "RISC-V: Handle differences between XTheadvector and Vector" patch
until you can reproduce the issue for me.
juzhe.zhong@rivai.ai
发件人： joshua
发送时间： 2024-01-11 17:29
收件人： juzhe.zhong@rivai.ai; gcc-patches
抄送： Jim Wilson; palmer; andrew; philipp.tomsich; jeffreyalaw; christoph.muellner; jinma; cooper.qu
主题： Re：Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
The sext/zext issue is not related to xtheadvector-special patterns.
I added !TARGET_XTHEADVECTOR to sext/zext patterns in
"RISC-V: Handle differences between XTheadvector and Vector"
That is caused by the vwmul pattern, but I cannot reproduce it right now.
------------------------------------------------------------------
发件人：juzhe.zhong@rivai.ai <juzhe.zhong@rivai.ai>
发送时间：2024年1月11日(星期四) 17:24
收件人："cooper.joshua"<cooper.joshua@linux.alibaba.com>; "gcc-patches"<gcc-patches@gcc.gnu.org>
抄 送：Jim Wilson<jim.wilson.gcc@gmail.com>; palmer<palmer@dabbelt.com>; andrew<andrew@sifive.com>; "philipp.tomsich"<philipp.tomsich@vrull.eu>; jeffreyalaw<jeffreyalaw@gmail.com>; "christoph.muellner"<christoph.muellner@vrull.eu>; jinma<jinma@linux.alibaba.com>; "cooper.qu"<cooper.qu@linux.alibaba.com>
主 题：Re: Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
I don't see any patterns are possible CSE into sext/zext patterns:
+(define_expand "@pred_mov_width<vlmem_op_attr><mode>"
+ [(set (match_operand:V_VLS 0 "nonimmediate_operand")
+ (if_then_else:V_VLS
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand")
+ (match_operand 4 "vector_length_operand")
+ (match_operand 5 "const_int_operand")
+ (match_operand 6 "const_int_operand")
+ (match_operand 7 "const_int_operand")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLMEM_OP)
+ (match_operand:V_VLS 3 "vector_move_operand")
+ (match_operand:V_VLS 2 "vector_merge_operand")))]
+ "TARGET_XTHEADVECTOR"
+ {})
+
+(define_insn_and_split "*pred_mov_width<vlmem_op_attr><mode>"
+ [(set (match_operand:V_VLS 0 "nonimmediate_operand" "=vr, vr, vd, m, vr, vr")
+ (if_then_else:V_VLS
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1, Wc1, vm, vmWc1, Wc1, Wc1")
+ (match_operand 4 "vector_length_operand" " rK, rK, rK, rK, rK, rK")
+ (match_operand 5 "const_int_operand" " i, i, i, i, i, i")
+ (match_operand 6 "const_int_operand" " i, i, i, i, i, i")
+ (match_operand 7 "const_int_operand" " i, i, i, i, i, i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLMEM_OP)
+ (match_operand:V_VLS 3 "reg_or_mem_operand" " m, m, m, vr, vr, vr")
+ (match_operand:V_VLS 2 "vector_merge_operand" " 0, vu, vu, vu, vu, 0")))]
+ "(TARGET_XTHEADVECTOR
+ && (register_operand (operands[0], <MODE>mode)
+ || register_operand (operands[3], <MODE>mode)))"
+ "@
+ vl<vlmem_op_attr>.v\t%0,%3%p1
+ vl<vlmem_op_attr>.v\t%0,%3
+ vl<vlmem_op_attr>.v\t%0,%3,%1.t
+ vs<vlmem_op_attr>.v\t%3,%0%p1
+ vmv.v.v\t%0,%3
+ vmv.v.v\t%0,%3"
+ "&& register_operand (operands[0], <MODE>mode)
+ && register_operand (operands[3], <MODE>mode)
+ && satisfies_constraint_vu (operands[2])
+ && INTVAL (operands[7]) == riscv_vector::VLMAX"
+ [(set (match_dup 0) (match_dup 3))]
+ ""
+ [(set_attr "type" "vlde,vlde,vlde,vste,vimov,vimov")
+ (set_attr "mode" "<MODE>")])
+
+(define_insn "@pred_store_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "memory_operand" "+m")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1")
+ (match_operand 3 "vector_length_operand" " rK")
+ (match_operand 4 "const_int_operand" " i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VSMEM_OP)
+ (match_operand:VI 2 "register_operand" " vr")
+ (match_dup 0)))]
+ "TARGET_XTHEADVECTOR"
+ "vs<vlmem_op_attr>.v\t%2,%0%p1"
+ [(set_attr "type" "vste")
+ (set_attr "mode" "<MODE>")
+ (set (attr "avl_type_idx") (const_int 4))
+ (set_attr "vl_op_idx" "3")])
+
+(define_insn "@pred_strided_load_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "register_operand" "=vr, vr, vd")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1, Wc1, vm")
+ (match_operand 5 "vector_length_operand" " rK, rK, rK")
+ (match_operand 6 "const_int_operand" " i, i, i")
+ (match_operand 7 "const_int_operand" " i, i, i")
+ (match_operand 8 "const_int_operand" " i, i, i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLSMEM_OP)
+ (unspec:VI
+ [(match_operand:VI 3 "memory_operand" " m, m, m")
+ (match_operand 4 "pmode_reg_or_0_operand" " rJ, rJ, rJ")] UNSPEC_TH_VLSMEM_OP)
+ (match_operand:VI 2 "vector_merge_operand" " 0, vu, vu")))]
+ "TARGET_XTHEADVECTOR"
+ "vls<vlmem_op_attr>.v\t%0,%3,%z4%p1"
+ [(set_attr "type" "vlds")
+ (set_attr "mode" "<MODE>")])
+
+(define_insn "@pred_strided_store_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "memory_operand" "+m")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1")
+ (match_operand 4 "vector_length_operand" " rK")
+ (match_operand 5 "const_int_operand" " i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VSSMEM_OP)
+ (unspec:VI
+ [(match_operand 2 "pmode_reg_or_0_operand" " rJ")
+ (match_operand:VI 3 "register_operand" " vr")] UNSPEC_TH_VSSMEM_OP)
+ (match_dup 0)))]
+ "TARGET_XTHEADVECTOR"
+ "vss<vlmem_op_attr>.v\t%3,%0,%z2%p1"
+ [(set_attr "type" "vsts")
+ (set_attr "mode" "<MODE>")
+ (set (attr "avl_type_idx") (const_int 5))])
+
+(define_insn "@pred_indexed_load_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "register_operand" "=vd, vr,vd, vr")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" " vm,Wc1,vm,Wc1")
+ (match_operand 5 "vector_length_operand" " rK, rK,rK, rK")
+ (match_operand 6 "const_int_operand" " i, i, i, i")
+ (match_operand 7 "const_int_operand" " i, i, i, i")
+ (match_operand 8 "const_int_operand" " i, i, i, i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLXMEM_OP)
+ (unspec:VI
+ [(match_operand 3 "pmode_reg_or_0_operand" " rJ, rJ,rJ, rJ")
+ (mem:BLK (scratch))
+ (match_operand:VI 4 "register_operand" " vr, vr,vr, vr")] UNSPEC_TH_VLXMEM_OP)
+ (match_operand:VI 2 "vector_merge_operand" " vu, vu, 0, 0")))]
+ "TARGET_XTHEADVECTOR"
+ "vlx<vlmem_op_attr>.v\t%0,(%z3),%4%p1"
+ [(set_attr "type" "vldux")
+ (set_attr "mode" "<MODE>")])
+
+(define_insn "@pred_indexed_<vlmem_order_attr>store_width<vlmem_op_attr><mode>"
+ [(set (mem:BLK (scratch))
+ (unspec:BLK
+ [(unspec:<VM>
+ [(match_operand:<VM> 0 "vector_mask_operand" "vmWc1")
+ (match_operand 4 "vector_length_operand" " rK")
+ (match_operand 5 "const_int_operand" " i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VSXMEM_OP)
+ (match_operand 1 "pmode_reg_or_0_operand" " rJ")
+ (match_operand:VI 2 "register_operand" " vr")
+ (match_operand:VI 3 "register_operand" " vr")] UNSPEC_TH_VSXMEM_OP))]
+ "TARGET_XTHEADVECTOR"
+ "vs<vlmem_order_attr>x<vlmem_op_attr>.v\t%3,(%z1),%2%p0"
+ [(set_attr "type" "vstux")
+ (set_attr "mode" "<MODE>")])
+
+(define_expand "@pred_th_extract<mode>"
+ [(set (match_operand:<VEL> 0 "register_operand")
+ (unspec:<VEL>
+ [(vec_select:<VEL>
+ (match_operand:V_VLSI 1 "register_operand")
+ (parallel [(match_operand:DI 2 "register_operand" "r")]))
+ (reg:SI VTYPE_REGNUM)] UNSPEC_VPREDICATE))]
+ "TARGET_XTHEADVECTOR"
+{})
+
+(define_insn "*pred_th_extract<mode>"
+ [(set (match_operand:<VEL> 0 "register_operand" "=r")
+ (unspec:<VEL>
+ [(vec_select:<VEL>
+ (match_operand:V_VLSI 1 "register_operand" "vr")
+ (parallel [(match_operand:DI 2 "register_operand" "r")]))
+ (reg:SI VTYPE_REGNUM)] UNSPEC_VPREDICATE))]
+ "TARGET_XTHEADVECTOR"
+ "vext.x.v\t%0,%1,%2"
+ [(set_attr "type" "vimovvx")
+ (set_attr "mode" "<MODE>")])
juzhe.zhong@rivai.ai
发件人： joshua
发送时间： 2024-01-11 17:21
收件人： juzhe.zhong@rivai.ai; gcc-patches
抄送： Jim Wilson; palmer; andrew; philipp.tomsich; jeffreyalaw; christoph.muellner; jinma; cooper.qu
主题： Re：Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
"I didn't see theadvector-specific extension patterns. Could you show me?"
They are all in the file thead-vector.md.
For the sext/zext issue, perhaps I need some time to reproduce that optimization,
but I can clearly remember it is related to vwmul.
------------------------------------------------------------------
发件人：juzhe.zhong@rivai.ai <juzhe.zhong@rivai.ai>
发送时间：2024年1月11日(星期四) 17:17
收件人："cooper.joshua"<cooper.joshua@linux.alibaba.com>; "gcc-patches"<gcc-patches@gcc.gnu.org>
抄 送：Jim Wilson<jim.wilson.gcc@gmail.com>; palmer<palmer@dabbelt.com>; andrew<andrew@sifive.com>; "philipp.tomsich"<philipp.tomsich@vrull.eu>; jeffreyalaw<jeffreyalaw@gmail.com>; "christoph.muellner"<christoph.muellner@vrull.eu>; jinma<jinma@linux.alibaba.com>; "cooper.qu"<cooper.qu@linux.alibaba.com>
主 题：Re: Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
You mean which pattern optimized sext/vzext pattern?
I didn't see theadvector-specific extension patterns. Could you show me?
juzhe.zhong@rivai.ai
发件人： joshua
发送时间： 2024-01-11 17:14
收件人： juzhe.zhong@rivai.ai; gcc-patches
抄送： Jim Wilson; palmer; andrew; philipp.tomsich; jeffreyalaw; christoph.muellner; jinma; cooper.qu
主题： Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
To be specific, in CSE pass, the initial pattern will be optimized into the sext/zext pattern.
------------------------------------------------------------------
发件人：joshua <cooper.joshua@linux.alibaba.com>
发送时间：2024年1月11日(星期四) 17:11
收件人："juzhe.zhong@rivai.ai"<juzhe.zhong@rivai.ai>; "gcc-patches"<gcc-patches@gcc.gnu.org>
抄 送：Jim Wilson<jim.wilson.gcc@gmail.com>; palmer<palmer@dabbelt.com>; andrew<andrew@sifive.com>; "philipp.tomsich"<philipp.tomsich@vrull.eu>; jeffreyalaw<jeffreyalaw@gmail.com>; "christoph.muellner"<christoph.muellner@vrull.eu>; jinma<jinma@linux.alibaba.com>; "cooper.qu"<cooper.qu@linux.alibaba.com>
主 题：Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
sext/zext will be generated in O2 even without corresponding intrinsics.
------------------------------------------------------------------
发件人：juzhe.zhong@rivai.ai <juzhe.zhong@rivai.ai>
发送时间：2024年1月11日(星期四) 17:07
收件人："cooper.joshua"<cooper.joshua@linux.alibaba.com>; "gcc-patches"<gcc-patches@gcc.gnu.org>
抄 送：Jim Wilson<jim.wilson.gcc@gmail.com>; palmer<palmer@dabbelt.com>; andrew<andrew@sifive.com>; "philipp.tomsich"<philipp.tomsich@vrull.eu>; jeffreyalaw<jeffreyalaw@gmail.com>; "christoph.muellner"<christoph.muellner@vrull.eu>; "cooper.joshua"<cooper.joshua@linux.alibaba.com>; jinma<jinma@linux.alibaba.com>; "cooper.qu"<cooper.qu@linux.alibaba.com>
主 题：Re: [PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
enum required_ext
{
VECTOR_EXT, /* Vector extension */
+ XTHEADVECTOR_EXT, /* XTheadVector extension */
ZVBB_EXT, /* Cryto vector Zvbb sub-ext */
ZVBB_OR_ZVKB_EXT, /* Cryto vector Zvbb or zvkb sub-ext */
ZVBC_EXT, /* Crypto vector Zvbc sub-ext */
Add theadvector to the end of the enum.
+ case XTHEADVECTOR_EXT:
+ return TARGET_XTHEADVECTOR;
Same.
+ "&& register_operand (operands[0], <MODE>mode)
+ && register_operand (operands[3], <MODE>mode)
+ && satisfies_constraint_vu (operands[2])
+ && INTVAL (operands[7]) == riscv_vector::VLMAX"
You can use whole_reg_to_reg_move_p
Btw, I review again : RISC-V: Handle differences between XTheadvector and Vector
(any_extend:VWEXTI (match_operand:<V_DOUBLE_TRUNC> 3 "register_operand" "W21,W21,W21,W21,W42,W42,W42,W42,W84,W84,W84,W84, vr, vr")) (match_operand:VWEXTI 2 "vector_merge_operand" " vu, vu, 0, 0, vu, vu, 0, 0, vu, vu, 0, 0, vu, 0")))]- "TARGET_VECTOR"+ "TARGET_VECTOR && !TARGET_XTHEADVECTOR" "v<sz>ext.vf2\t%0,%3%p1" [(set_attr "type" "vext") (set_attr "mode" "<MODE>")@@ -3713,7 +3744,7 @@ (any_extend:VQEXTI (match_operand:<V_QUAD_TRUNC> 3 "register_operand" "W43,W43,W43,W43,W86,W86,W86,W86, vr, vr")) (match_operand:VQEXTI 2 "vector_merge_operand" " vu, vu, 0, 0, vu, vu, 0, 0, vu, 0")))]- "TARGET_VECTOR"+ "TARGET_VECTOR && !TARGET_XTHEADVECTOR" "v<sz>ext.vf4\t%0,%3%p1" [(set_attr "type" "vext") (set_attr "mode" "<MODE>")@@ -3734,7 +3765,7 @@ (any_extend:VOEXTI (match_operand:<V_OCT_TRUNC> 3 "register_operand" "W87,W87,W87,W87, vr, vr")) (match_operand:VOEXTI 2 "vector_merge_operand" " vu, vu, 0, 0, vu, 0")))]- "TARGET_VECTOR"+ "TARGET_VECTOR && !TARGET_XTHEADVECTOR" "v<sz>ext.vf8\t%0,%3%p1" [(set_attr "type" "vext") (set_attr "mode" "<MODE>")
Why do you add these !TARGERT_XTHEADVECRTOR ?
juzhe.zhong@rivai.ai
From: Jun Sha (Joshua)
Date: 2024-01-11 16:46
To: gcc-patches
CC: jim.wilson.gcc; palmer; andrew; philipp.tomsich; jeffreyalaw; christoph.muellner; juzhe.zhong; Jun Sha (Joshua); Jin Ma; Xianmiao Qu
Subject: [PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
This patch only involves the generation of xtheadvector
special load/store instructions and vext instructions.
gcc/ChangeLog:
* config/riscv/riscv-vector-builtins-bases.cc
(class th_loadstore_width): Define new builtin bases.
(class th_extract): Define new builtin bases.
(BASE): Define new builtin bases.
* config/riscv/riscv-vector-builtins-bases.h:
Define new builtin class.
* config/riscv/riscv-vector-builtins-shapes.cc
(struct th_loadstore_width_def): Define new builtin shapes.
(struct th_indexed_loadstore_width_def):
Define new builtin shapes.
(struct th_extract_def): Define new builtin shapes.
(SHAPE): Define new builtin shapes.
* config/riscv/riscv-vector-builtins-shapes.h:
Define new builtin shapes.
* config/riscv/riscv-vector-builtins.cc (DEF_RVV_FUNCTION):
* config/riscv/riscv-vector-builtins.h (enum required_ext):
(struct function_group_info):
* config/riscv/t-riscv: Add thead-vector-builtins-functions.def
* config/riscv/thead-vector.md
(@pred_mov_width<vlmem_op_attr><mode>): Add new patterns.
(*pred_mov_width<vlmem_op_attr><mode>): Likewise.
(@pred_store_width<vlmem_op_attr><mode>): Likewise.
(@pred_strided_load_width<vlmem_op_attr><mode>): Likewise.
(@pred_strided_store_width<vlmem_op_attr><mode>): Likewise.
(@pred_indexed_load_width<vlmem_op_attr><mode>): Likewise.
(@pred_indexed_<vlmem_order_attr>store_width<vlmem_op_attr><mode>):
(@pred_th_extract<mode>): Likewise.
(*pred_th_extract<mode>): Likewise.
* config/riscv/thead-vector-builtins-functions.def: New file.
gcc/testsuite/ChangeLog:
* gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c: New test.
Co-authored-by: Jin Ma <jinma@linux.alibaba.com>
Co-authored-by: Xianmiao Qu <cooper.qu@linux.alibaba.com>
Co-authored-by: Christoph Müllner <christoph.muellner@vrull.eu>
---
.../riscv/riscv-vector-builtins-bases.cc | 139 ++++++++++
.../riscv/riscv-vector-builtins-bases.h | 31 +++
.../riscv/riscv-vector-builtins-shapes.cc | 160 +++++++++++
.../riscv/riscv-vector-builtins-shapes.h | 3 +
gcc/config/riscv/riscv-vector-builtins.cc | 70 +++++
gcc/config/riscv/riscv-vector-builtins.h | 3 +
gcc/config/riscv/t-riscv | 1 +
.../riscv/thead-vector-builtins-functions.def | 39 +++
gcc/config/riscv/thead-vector.md | 253 ++++++++++++++++++
.../riscv/rvv/xtheadvector/vlb-vsb.c | 68 +++++
.../riscv/rvv/xtheadvector/vlbu-vsb.c | 68 +++++
.../riscv/rvv/xtheadvector/vlh-vsh.c | 68 +++++
.../riscv/rvv/xtheadvector/vlhu-vsh.c | 68 +++++
.../riscv/rvv/xtheadvector/vlw-vsw.c | 68 +++++
.../riscv/rvv/xtheadvector/vlwu-vsw.c | 68 +++++
15 files changed, 1107 insertions(+)
create mode 100644 gcc/config/riscv/thead-vector-builtins-functions.def
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c
diff --git a/gcc/config/riscv/riscv-vector-builtins-bases.cc b/gcc/config/riscv/riscv-vector-builtins-bases.cc
index 46f1a1da33e..3eba7943757 100644
--- a/gcc/config/riscv/riscv-vector-builtins-bases.cc
+++ b/gcc/config/riscv/riscv-vector-builtins-bases.cc
@@ -2125,6 +2125,83 @@ public:
}
};
+/* Implements
+ * th.vl(b/h/w)[u].v/th.vs(b/h/w)[u].v/th.vls(b/h/w)[u].v/th.vss(b/h/w)[u].v/
+ * th.vlx(b/h/w)[u].v/th.vs[u]x(b/h/w).v
+ * codegen. */
+template<bool STORE_P, lst_type LST_TYPE, int UNSPEC>
+class th_loadstore_width : public function_base
+{
+public:
+ bool apply_tail_policy_p () const override { return !STORE_P; }
+ bool apply_mask_policy_p () const override { return !STORE_P; }
+
+ unsigned int call_properties (const function_instance &) const override
+ {
+ if (STORE_P)
+ return CP_WRITE_MEMORY;
+ else
+ return CP_READ_MEMORY;
+ }
+
+ bool can_be_overloaded_p (enum predication_type_index pred) const override
+ {
+ if (STORE_P || LST_TYPE == LST_INDEXED)
+ return true;
+ return pred != PRED_TYPE_none;
+ }
+
+ rtx expand (function_expander &e) const override
+ {
+ gcc_assert (TARGET_XTHEADVECTOR);
+ if (LST_TYPE == LST_INDEXED)
+ {
+ if (STORE_P)
+ return e.use_exact_insn (
+ code_for_pred_indexed_store_width (UNSPEC, UNSPEC,
+ e.vector_mode ()));
+ else
+ return e.use_exact_insn (
+ code_for_pred_indexed_load_width (UNSPEC, e.vector_mode ()));
+ }
+ else if (LST_TYPE == LST_STRIDED)
+ {
+ if (STORE_P)
+ return e.use_contiguous_store_insn (
+ code_for_pred_strided_store_width (UNSPEC, e.vector_mode ()));
+ else
+ return e.use_contiguous_load_insn (
+ code_for_pred_strided_load_width (UNSPEC, e.vector_mode ()));
+ }
+ else
+ {
+ if (STORE_P)
+ return e.use_contiguous_store_insn (
+ code_for_pred_store_width (UNSPEC, e.vector_mode ()));
+ else
+ return e.use_contiguous_load_insn (
+ code_for_pred_mov_width (UNSPEC, e.vector_mode ()));
+ }
+ }
+};
+
+/* Implements vext.x.v. */
+class th_extract : public function_base
+{
+public:
+ bool apply_vl_p () const override { return false; }
+ bool apply_tail_policy_p () const override { return false; }
+ bool apply_mask_policy_p () const override { return false; }
+ bool use_mask_predication_p () const override { return false; }
+ bool has_merge_operand_p () const override { return false; }
+
+ rtx expand (function_expander &e) const override
+ {
+ gcc_assert (TARGET_XTHEADVECTOR);
+ return e.use_exact_insn (code_for_pred_th_extract (e.vector_mode ()));
+ }
+};
+
/* Below implements are vector crypto */
/* Implements vandn.[vv,vx] */
class vandn : public function_base
@@ -2587,6 +2664,37 @@ static CONSTEXPR const seg_indexed_load<UNSPEC_ORDERED> vloxseg_obj;
static CONSTEXPR const seg_indexed_store<UNSPEC_UNORDERED> vsuxseg_obj;
static CONSTEXPR const seg_indexed_store<UNSPEC_ORDERED> vsoxseg_obj;
static CONSTEXPR const vlsegff vlsegff_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLB> vlb_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLBU> vlbu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLH> vlh_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLHU> vlhu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLW> vlw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLWU> vlwu_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_UNIT_STRIDE, UNSPEC_TH_VLB> vsb_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_UNIT_STRIDE, UNSPEC_TH_VLH> vsh_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_UNIT_STRIDE, UNSPEC_TH_VLW> vsw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSB> vlsb_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSBU> vlsbu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSH> vlsh_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSHU> vlshu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSW> vlsw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSWU> vlswu_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_STRIDED, UNSPEC_TH_VLSB> vssb_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_STRIDED, UNSPEC_TH_VLSH> vssh_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_STRIDED, UNSPEC_TH_VLSW> vssw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXB> vlxb_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXBU> vlxbu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXH> vlxh_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXHU> vlxhu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXW> vlxw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXWU> vlxwu_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VLXB> vsxb_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VLXH> vsxh_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VLXW> vsxw_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VSUXB> vsuxb_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VSUXH> vsuxh_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VSUXW> vsuxw_obj;
+static CONSTEXPR const th_extract vext_x_v_obj;
/* Crypto Vector */
static CONSTEXPR const vandn vandn_obj;
@@ -2878,6 +2986,37 @@ BASE (vloxseg)
BASE (vsuxseg)
BASE (vsoxseg)
BASE (vlsegff)
+BASE (vlb)
+BASE (vlh)
+BASE (vlw)
+BASE (vlbu)
+BASE (vlhu)
+BASE (vlwu)
+BASE (vsb)
+BASE (vsh)
+BASE (vsw)
+BASE (vlsb)
+BASE (vlsh)
+BASE (vlsw)
+BASE (vlsbu)
+BASE (vlshu)
+BASE (vlswu)
+BASE (vssb)
+BASE (vssh)
+BASE (vssw)
+BASE (vlxb)
+BASE (vlxh)
+BASE (vlxw)
+BASE (vlxbu)
+BASE (vlxhu)
+BASE (vlxwu)
+BASE (vsxb)
+BASE (vsxh)
+BASE (vsxw)
+BASE (vsuxb)
+BASE (vsuxh)
+BASE (vsuxw)
+BASE (vext_x_v)
/* Crypto vector */
BASE (vandn)
BASE (vbrev)
diff --git a/gcc/config/riscv/riscv-vector-builtins-bases.h b/gcc/config/riscv/riscv-vector-builtins-bases.h
index 1122e3801a7..565a0311d2b 100644
--- a/gcc/config/riscv/riscv-vector-builtins-bases.h
+++ b/gcc/config/riscv/riscv-vector-builtins-bases.h
@@ -299,6 +299,37 @@ extern const function_base *const vloxseg;
extern const function_base *const vsuxseg;
extern const function_base *const vsoxseg;
extern const function_base *const vlsegff;
+extern const function_base *const vlb;
+extern const function_base *const vlh;
+extern const function_base *const vlw;
+extern const function_base *const vlbu;
+extern const function_base *const vlhu;
+extern const function_base *const vlwu;
+extern const function_base *const vsb;
+extern const function_base *const vsh;
+extern const function_base *const vsw;
+extern const function_base *const vlsb;
+extern const function_base *const vlsh;
+extern const function_base *const vlsw;
+extern const function_base *const vlsbu;
+extern const function_base *const vlshu;
+extern const function_base *const vlswu;
+extern const function_base *const vssb;
+extern const function_base *const vssh;
+extern const function_base *const vssw;
+extern const function_base *const vlxb;
+extern const function_base *const vlxh;
+extern const function_base *const vlxw;
+extern const function_base *const vlxbu;
+extern const function_base *const vlxhu;
+extern const function_base *const vlxwu;
+extern const function_base *const vsxb;
+extern const function_base *const vsxh;
+extern const function_base *const vsxw;
+extern const function_base *const vsuxb;
+extern const function_base *const vsuxh;
+extern const function_base *const vsuxw;
+extern const function_base *const vext_x_v;
/* Below function_base are Vectro Crypto*/
extern const function_base *const vandn;
extern const function_base *const vbrev;
diff --git a/gcc/config/riscv/riscv-vector-builtins-shapes.cc b/gcc/config/riscv/riscv-vector-builtins-shapes.cc
index 1e4f4d53de6..8e90b17a94b 100644
--- a/gcc/config/riscv/riscv-vector-builtins-shapes.cc
+++ b/gcc/config/riscv/riscv-vector-builtins-shapes.cc
@@ -211,6 +211,146 @@ struct indexed_loadstore_def : public function_shape
}
};
+/* Add one function instance for GROUP, using operand suffix at index OI,
+ mode suffix at index PAIR && bi and predication suffix at index pred_idx. */
+static void
+build_th_loadstore (function_builder &b, const function_group_info &group,
+ unsigned int pred_idx, unsigned int vec_type_idx)
+{
+ auto_vec<tree, 5> argument_types;
+ function_instance function_instance (group.base_name, *group.base,
+ *group.shape,
+ group.ops_infos.types[vec_type_idx],
+ group.preds[pred_idx], &group.ops_infos);
+ tree return_type = group.ops_infos.ret.get_tree_type (
+ group.ops_infos.types[vec_type_idx].index);
+ b.allocate_argument_types (function_instance, argument_types);
+ b.apply_predication (function_instance, return_type, argument_types);
+
+ if (TARGET_XTHEADVECTOR && !check_type (return_type, argument_types))
+ return;
+
+ tree type = builtin_types[group.ops_infos.types[vec_type_idx].index].vector;
+ if (strstr (group.base_name, "l")
+ && strstr (group.base_name, "u")
+ && !TYPE_UNSIGNED (TREE_TYPE (type)))
+ return;
+
+ if (strstr (group.base_name, "l")
+ && !strstr (group.base_name, "u")
+ && TYPE_UNSIGNED (TREE_TYPE (type)))
+ return;
+
+ machine_mode mode = TYPE_MODE (type);
+ int sew = GET_MODE_BITSIZE (GET_MODE_INNER (mode));
+ if (strstr (group.base_name, "h") && sew == 8)
+ return;
+
+ if (strstr (group.base_name, "w") && (sew == 8 || sew ==16))
+ return;
+
+ b.add_overloaded_function (function_instance, *group.shape);
+ b.add_unique_function (function_instance, (*group.shape), return_type,
+ argument_types);
+}
+
+/* th_loadstore_width_def class. */
+struct th_loadstore_width_def : public build_base
+{
+ void build (function_builder &b,
+ const function_group_info &group) const override
+ {
+ for (unsigned int pred_idx = 0; group.preds[pred_idx] != NUM_PRED_TYPES;
+ ++pred_idx)
+ {
+ for (unsigned int vec_type_idx = 0;
+ group.ops_infos.types[vec_type_idx].index != NUM_VECTOR_TYPES;
+ ++vec_type_idx)
+ {
+ build_th_loadstore (b, group, pred_idx, vec_type_idx);
+ }
+ }
+ }
+
+ char *get_name (function_builder &b, const function_instance &instance,
+ bool overloaded_p) const override
+ {
+ /* Return nullptr if it can not be overloaded. */
+ if (overloaded_p && !instance.base->can_be_overloaded_p (instance.pred))
+ return nullptr;
+
+ b.append_name ("__riscv_th_");
+ b.append_name (instance.base_name);
+
+ /* vop_v --> vop_v_<type>. */
+ if (!overloaded_p)
+ {
+ /* vop --> vop_v. */
+ b.append_name (operand_suffixes[instance.op_info->op]);
+ /* vop_v --> vop_v_<type>. */
+ b.append_name (type_suffixes[instance.type.index].vector);
+ }
+
+ /* According to rvv-intrinsic-doc, it does not add "_m" suffix
+ for vop_m C++ overloaded API. */
+ if (overloaded_p && instance.pred == PRED_TYPE_m)
+ return b.finish_name ();
+ b.append_name (predication_suffixes[instance.pred]);
+ return b.finish_name ();
+ }
+};
+
+
+/* th_indexed_loadstore_width_def class. */
+struct th_indexed_loadstore_width_def : public function_shape
+{
+ void build (function_builder &b,
+ const function_group_info &group) const override
+ {
+ for (unsigned int pred_idx = 0; group.preds[pred_idx] != NUM_PRED_TYPES;
+ ++pred_idx)
+ {
+ for (unsigned int vec_type_idx = 0;
+ group.ops_infos.types[vec_type_idx].index != NUM_VECTOR_TYPES;
+ ++vec_type_idx)
+ {
+ tree index_type = group.ops_infos.args[1].get_tree_type (
+ group.ops_infos.types[vec_type_idx].index);
+ if (!index_type)
+ continue;
+ build_th_loadstore (b, group, pred_idx, vec_type_idx);
+ }
+ }
+ }
+
+ char *get_name (function_builder &b, const function_instance &instance,
+ bool overloaded_p) const override
+ {
+
+ /* Return nullptr if it can not be overloaded. */
+ if (overloaded_p && !instance.base->can_be_overloaded_p (instance.pred))
+ return nullptr;
+
+ b.append_name ("__riscv_th_");
+ b.append_name (instance.base_name);
+ /* vop_v --> vop_v_<type>. */
+ if (!overloaded_p)
+ {
+ /* vop --> vop_v. */
+ b.append_name (operand_suffixes[instance.op_info->op]);
+ /* vop_v --> vop_v_<type>. */
+ b.append_name (type_suffixes[instance.type.index].vector);
+ }
+
+ /* According to rvv-intrinsic-doc, it does not add "_m" suffix
+ for vop_m C++ overloaded API. */
+ if (overloaded_p && instance.pred == PRED_TYPE_m)
+ return b.finish_name ();
+ b.append_name (predication_suffixes[instance.pred]);
+ return b.finish_name ();
+ }
+};
+
/* alu_def class. */
struct alu_def : public build_base
{
@@ -632,6 +772,23 @@ struct reduc_alu_def : public build_base
}
};
+/* th_extract_def class. */
+struct th_extract_def : public build_base
+{
+ char *get_name (function_builder &b, const function_instance &instance,
+ bool overloaded_p) const override
+ {
+ b.append_name ("__riscv_th_");
+ b.append_name (instance.base_name);
+
+ if (overloaded_p)
+ return b.finish_name ();
+ b.append_name (type_suffixes[instance.type.index].vector);
+ b.append_name (type_suffixes[instance.type.index].scalar);
+ return b.finish_name ();
+ }
+};
+
/* scalar_move_def class. */
struct scalar_move_def : public build_base
{
@@ -1094,6 +1251,8 @@ SHAPE(vsetvl, vsetvl)
SHAPE(vsetvl, vsetvlmax)
SHAPE(loadstore, loadstore)
SHAPE(indexed_loadstore, indexed_loadstore)
+SHAPE(th_loadstore_width, th_loadstore_width)
+SHAPE(th_indexed_loadstore_width, th_indexed_loadstore_width)
SHAPE(alu, alu)
SHAPE(alu_frm, alu_frm)
SHAPE(widen_alu, widen_alu)
@@ -1106,6 +1265,7 @@ SHAPE(move, move)
SHAPE(mask_alu, mask_alu)
SHAPE(reduc_alu, reduc_alu)
SHAPE(reduc_alu_frm, reduc_alu_frm)
+SHAPE(th_extract, th_extract)
SHAPE(scalar_move, scalar_move)
SHAPE(vundefined, vundefined)
SHAPE(misc, misc)
diff --git a/gcc/config/riscv/riscv-vector-builtins-shapes.h b/gcc/config/riscv/riscv-vector-builtins-shapes.h
index ac2a28ce017..a7624d0fabd 100644
--- a/gcc/config/riscv/riscv-vector-builtins-shapes.h
+++ b/gcc/config/riscv/riscv-vector-builtins-shapes.h
@@ -28,6 +28,8 @@ extern const function_shape *const vsetvl;
extern const function_shape *const vsetvlmax;
extern const function_shape *const loadstore;
extern const function_shape *const indexed_loadstore;
+extern const function_shape *const th_loadstore_width;
+extern const function_shape *const th_indexed_loadstore_width;
extern const function_shape *const alu;
extern const function_shape *const alu_frm;
extern const function_shape *const widen_alu;
@@ -41,6 +43,7 @@ extern const function_shape *const mask_alu;
extern const function_shape *const reduc_alu;
extern const function_shape *const reduc_alu_frm;
extern const function_shape *const scalar_move;
+extern const function_shape *const th_extract;
extern const function_shape *const vundefined;
extern const function_shape *const misc;
extern const function_shape *const vset;
diff --git a/gcc/config/riscv/riscv-vector-builtins.cc b/gcc/config/riscv/riscv-vector-builtins.cc
index 25e0b6e56de..44b9fec1898 100644
--- a/gcc/config/riscv/riscv-vector-builtins.cc
+++ b/gcc/config/riscv/riscv-vector-builtins.cc
@@ -934,6 +934,32 @@ static CONSTEXPR const rvv_arg_type_info ext_vcreate_args[]
= {rvv_arg_type_info (RVV_BASE_vector),
rvv_arg_type_info_end};
+/* A list of args for vector_type func (const scalar_type *, size_t)
+ * function. */
+static CONSTEXPR const rvv_arg_type_info scalar_const_ptr_size_args[]
+ = {rvv_arg_type_info (RVV_BASE_scalar_const_ptr),
+ rvv_arg_type_info (RVV_BASE_size), rvv_arg_type_info_end};
+
+/* A list of args for vector_type func (const scalar_type *, eew8_index_type)
+ * function. */
+static CONSTEXPR const rvv_arg_type_info scalar_const_ptr_index_args[]
+ = {rvv_arg_type_info (RVV_BASE_scalar_const_ptr),
+ rvv_arg_type_info (RVV_BASE_unsigned_vector), rvv_arg_type_info_end};
+
+/* A list of args for void func (scalar_type *, eew8_index_type, vector_type)
+ * function. */
+static CONSTEXPR const rvv_arg_type_info scalar_ptr_index_args[]
+ = {rvv_arg_type_info (RVV_BASE_scalar_ptr),
+ rvv_arg_type_info (RVV_BASE_unsigned_vector),
+ rvv_arg_type_info (RVV_BASE_vector), rvv_arg_type_info_end};
+
+/* A list of args for void func (scalar_type *, size_t, vector_type)
+ * function. */
+static CONSTEXPR const rvv_arg_type_info scalar_ptr_size_args[]
+ = {rvv_arg_type_info (RVV_BASE_scalar_ptr),
+ rvv_arg_type_info (RVV_BASE_size), rvv_arg_type_info (RVV_BASE_vector),
+ rvv_arg_type_info_end};
+
/* A list of none preds that will be registered for intrinsic functions. */
static CONSTEXPR const predication_type_index none_preds[]
= {PRED_TYPE_none, NUM_PRED_TYPES};
@@ -1455,6 +1481,14 @@ static CONSTEXPR const rvv_op_info iu_shift_vvv_ops
rvv_arg_type_info (RVV_BASE_vector), /* Return type */
shift_vv_args /* Args */};
+/* A static operand information for scalar_type func (vector_type, size_t)
+ * function registration. */
+static CONSTEXPR const rvv_op_info iu_x_s_u_ops
+ = {iu_ops, /* Types */
+ OP_TYPE_vx, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_scalar), /* Return type */
+ v_size_args /* Args */};
+
/* A static operand information for vector_type func (vector_type, size_t)
* function registration. */
static CONSTEXPR const rvv_op_info iu_shift_vvx_ops
@@ -2638,6 +2672,38 @@ static CONSTEXPR const rvv_op_info all_v_vcreate_lmul4_x2_ops
rvv_arg_type_info (RVV_BASE_vlmul_ext_x2), /* Return type */
ext_vcreate_args /* Args */};
+/* A static operand information for vector_type func (const scalar_type *,
+ * size_t) function registration. */
+static CONSTEXPR const rvv_op_info all_v_scalar_const_ptr_size_ops
+ = {all_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_size_args /* Args */};
+
+/* A static operand information for void func (scalar_type *, size_t,
+ * vector_type) function registration. */
+static CONSTEXPR const rvv_op_info all_v_scalar_ptr_size_ops
+ = {all_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_void), /* Return type */
+ scalar_ptr_size_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * index_type) function registration. */
+static CONSTEXPR const rvv_op_info all_v_scalar_const_ptr_index_ops
+ = {all_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_index_args /* Args */};
+
+/* A static operand information for void func (scalar_type *, index_type,
+ * vector_type) function registration. */
+static CONSTEXPR const rvv_op_info all_v_scalar_ptr_index_ops
+ = {all_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_void), /* Return type */
+ scalar_ptr_index_args /* Args */};
+
/* A static operand information for vector_type func (vector_type).
Some ins just supports SEW=32, such as crypto vectol Zvkg extension.
* function registration. */
@@ -2816,6 +2882,10 @@ static function_group_info function_groups[] = {
#define DEF_RVV_FUNCTION(NAME, SHAPE, PREDS, OPS_INFO) \
{#NAME, &bases::NAME, &shapes::SHAPE, PREDS, OPS_INFO, REQUIRED_EXTENSIONS},
#include "riscv-vector-builtins-functions.def"
+#undef DEF_RVV_FUNCTION
+#define DEF_RVV_FUNCTION(NAME, SHAPE, PREDS, OPS_INFO) \
+ {#NAME, &bases::NAME, &shapes::SHAPE, PREDS, OPS_INFO, REQUIRED_EXTENSIONS},
+#include "thead-vector-builtins-functions.def"
};
/* The RVV types, with their built-in
diff --git a/gcc/config/riscv/riscv-vector-builtins.h b/gcc/config/riscv/riscv-vector-builtins.h
index 54c8824ff92..a8ee39a3cb2 100644
--- a/gcc/config/riscv/riscv-vector-builtins.h
+++ b/gcc/config/riscv/riscv-vector-builtins.h
@@ -114,6 +114,7 @@ static const unsigned int CP_WRITE_CSR = 1U << 5;
enum required_ext
{
VECTOR_EXT, /* Vector extension */
+ XTHEADVECTOR_EXT, /* XTheadVector extension */
ZVBB_EXT, /* Cryto vector Zvbb sub-ext */
ZVBB_OR_ZVKB_EXT, /* Cryto vector Zvbb or zvkb sub-ext */
ZVBC_EXT, /* Crypto vector Zvbc sub-ext */
@@ -234,6 +235,8 @@ struct function_group_info
{
case VECTOR_EXT:
return TARGET_VECTOR;
+ case XTHEADVECTOR_EXT:
+ return TARGET_XTHEADVECTOR;
case ZVBB_EXT:
return TARGET_ZVBB;
case ZVBB_OR_ZVKB_EXT:
diff --git a/gcc/config/riscv/t-riscv b/gcc/config/riscv/t-riscv
index 32de6b851c1..38494320d8b 100644
--- a/gcc/config/riscv/t-riscv
+++ b/gcc/config/riscv/t-riscv
@@ -1,6 +1,7 @@
RISCV_BUILTINS_H = $(srcdir)/config/riscv/riscv-vector-builtins.h \
$(srcdir)/config/riscv/riscv-vector-builtins.def \
$(srcdir)/config/riscv/riscv-vector-builtins-functions.def \
+ $(srcdir)/config/riscv/thead-vector-builtins-functions.def \
riscv-vector-type-indexer.gen.def
riscv-builtins.o: $(srcdir)/config/riscv/riscv-builtins.cc $(CONFIG_H) \
diff --git a/gcc/config/riscv/thead-vector-builtins-functions.def b/gcc/config/riscv/thead-vector-builtins-functions.def
new file mode 100644
index 00000000000..fd3ba29bae9
--- /dev/null
+++ b/gcc/config/riscv/thead-vector-builtins-functions.def
@@ -0,0 +1,39 @@
+#ifndef DEF_RVV_FUNCTION
+#define DEF_RVV_FUNCTION(NAME, SHAPE, PREDS, OPS_INFO)
+#endif
+
+#define REQUIRED_EXTENSIONS XTHEADVECTOR_EXT
+DEF_RVV_FUNCTION (vlb, th_loadstore_width, full_preds, all_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (vlh, th_loadstore_width, full_preds, all_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (vlw, th_loadstore_width, full_preds, all_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (vlbu, th_loadstore_width, full_preds, all_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (vlhu, th_loadstore_width, full_preds, all_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (vlwu, th_loadstore_width, full_preds, all_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (vsb, th_loadstore_width, none_m_preds, all_v_scalar_ptr_ops)
+DEF_RVV_FUNCTION (vsh, th_loadstore_width, none_m_preds, all_v_scalar_ptr_ops)
+DEF_RVV_FUNCTION (vsw, th_loadstore_width, none_m_preds, all_v_scalar_ptr_ops)
+DEF_RVV_FUNCTION (vlsb, th_loadstore_width, full_preds, all_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (vlsh, th_loadstore_width, full_preds, all_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (vlsw, th_loadstore_width, full_preds, all_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (vlsbu, th_loadstore_width, full_preds, all_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (vlshu, th_loadstore_width, full_preds, all_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (vlswu, th_loadstore_width, full_preds, all_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (vssb, th_loadstore_width, none_m_preds, all_v_scalar_ptr_size_ops)
+DEF_RVV_FUNCTION (vssh, th_loadstore_width, none_m_preds, all_v_scalar_ptr_size_ops)
+DEF_RVV_FUNCTION (vssw, th_loadstore_width, none_m_preds, all_v_scalar_ptr_size_ops)
+DEF_RVV_FUNCTION (vlxb, th_indexed_loadstore_width, full_preds, all_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (vlxh, th_indexed_loadstore_width, full_preds, all_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (vlxw, th_indexed_loadstore_width, full_preds, all_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (vlxbu, th_indexed_loadstore_width, full_preds, all_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (vlxhu, th_indexed_loadstore_width, full_preds, all_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (vlxwu, th_indexed_loadstore_width, full_preds, all_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (vsxb, th_indexed_loadstore_width, none_m_preds, all_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (vsxh, th_indexed_loadstore_width, none_m_preds, all_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (vsxw, th_indexed_loadstore_width, none_m_preds, all_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (vsuxb, th_indexed_loadstore_width, none_m_preds, all_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (vsuxh, th_indexed_loadstore_width, none_m_preds, all_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (vsuxw, th_indexed_loadstore_width, none_m_preds, all_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (vext_x_v, th_extract, none_preds, iu_x_s_u_ops)
+#undef REQUIRED_EXTENSIONS
+
+#undef DEF_RVV_FUNCTION
diff --git a/gcc/config/riscv/thead-vector.md b/gcc/config/riscv/thead-vector.md
index 696b815252d..0f3700d9269 100644
--- a/gcc/config/riscv/thead-vector.md
+++ b/gcc/config/riscv/thead-vector.md
@@ -1,7 +1,95 @@
(define_c_enum "unspec" [
+ UNSPEC_TH_VLB
+ UNSPEC_TH_VLBU
+ UNSPEC_TH_VLH
+ UNSPEC_TH_VLHU
+ UNSPEC_TH_VLW
+ UNSPEC_TH_VLWU
+
+ UNSPEC_TH_VLSB
+ UNSPEC_TH_VLSBU
+ UNSPEC_TH_VLSH
+ UNSPEC_TH_VLSHU
+ UNSPEC_TH_VLSW
+ UNSPEC_TH_VLSWU
+
+ UNSPEC_TH_VLXB
+ UNSPEC_TH_VLXBU
+ UNSPEC_TH_VLXH
+ UNSPEC_TH_VLXHU
+ UNSPEC_TH_VLXW
+ UNSPEC_TH_VLXWU
+
+ UNSPEC_TH_VSUXB
+ UNSPEC_TH_VSUXH
+ UNSPEC_TH_VSUXW
+
UNSPEC_TH_VWLDST
])
+(define_int_iterator UNSPEC_TH_VLMEM_OP [
+ UNSPEC_TH_VLB UNSPEC_TH_VLBU
+ UNSPEC_TH_VLH UNSPEC_TH_VLHU
+ UNSPEC_TH_VLW UNSPEC_TH_VLWU
+])
+
+(define_int_iterator UNSPEC_TH_VLSMEM_OP [
+ UNSPEC_TH_VLSB UNSPEC_TH_VLSBU
+ UNSPEC_TH_VLSH UNSPEC_TH_VLSHU
+ UNSPEC_TH_VLSW UNSPEC_TH_VLSWU
+])
+
+(define_int_iterator UNSPEC_TH_VLXMEM_OP [
+ UNSPEC_TH_VLXB UNSPEC_TH_VLXBU
+ UNSPEC_TH_VLXH UNSPEC_TH_VLXHU
+ UNSPEC_TH_VLXW UNSPEC_TH_VLXWU
+])
+
+(define_int_attr vlmem_op_attr [
+ (UNSPEC_TH_VLB "b") (UNSPEC_TH_VLBU "bu")
+ (UNSPEC_TH_VLH "h") (UNSPEC_TH_VLHU "hu")
+ (UNSPEC_TH_VLW "w") (UNSPEC_TH_VLWU "wu")
+ (UNSPEC_TH_VLSB "b") (UNSPEC_TH_VLSBU "bu")
+ (UNSPEC_TH_VLSH "h") (UNSPEC_TH_VLSHU "hu")
+ (UNSPEC_TH_VLSW "w") (UNSPEC_TH_VLSWU "wu")
+ (UNSPEC_TH_VLXB "b") (UNSPEC_TH_VLXBU "bu")
+ (UNSPEC_TH_VLXH "h") (UNSPEC_TH_VLXHU "hu")
+ (UNSPEC_TH_VLXW "w") (UNSPEC_TH_VLXWU "wu")
+ (UNSPEC_TH_VSUXB "b")
+ (UNSPEC_TH_VSUXH "h")
+ (UNSPEC_TH_VSUXW "w")
+])
+
+(define_int_attr vlmem_order_attr [
+ (UNSPEC_TH_VLXB "")
+ (UNSPEC_TH_VLXH "")
+ (UNSPEC_TH_VLXW "")
+ (UNSPEC_TH_VSUXB "u")
+ (UNSPEC_TH_VSUXH "u")
+ (UNSPEC_TH_VSUXW "u")
+])
+
+(define_int_iterator UNSPEC_TH_VSMEM_OP [
+ UNSPEC_TH_VLB
+ UNSPEC_TH_VLH
+ UNSPEC_TH_VLW
+])
+
+(define_int_iterator UNSPEC_TH_VSSMEM_OP [
+ UNSPEC_TH_VLSB
+ UNSPEC_TH_VLSH
+ UNSPEC_TH_VLSW
+])
+
+(define_int_iterator UNSPEC_TH_VSXMEM_OP [
+ UNSPEC_TH_VLXB
+ UNSPEC_TH_VLXH
+ UNSPEC_TH_VLXW
+ UNSPEC_TH_VSUXB
+ UNSPEC_TH_VSUXH
+ UNSPEC_TH_VSUXW
+])
+
(define_mode_iterator V_VLS_VT [V VLS VT])
(define_mode_iterator V_VB_VLS_VT [V VB VLS VT])
@@ -100,3 +188,168 @@
}
[(set_attr "type" "vldm,vstm,vmalu,vmalu,vmalu")
(set_attr "mode" "<MODE>")])
+
+(define_expand "@pred_mov_width<vlmem_op_attr><mode>"
+ [(set (match_operand:V_VLS 0 "nonimmediate_operand")
+ (if_then_else:V_VLS
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand")
+ (match_operand 4 "vector_length_operand")
+ (match_operand 5 "const_int_operand")
+ (match_operand 6 "const_int_operand")
+ (match_operand 7 "const_int_operand")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLMEM_OP)
+ (match_operand:V_VLS 3 "vector_move_operand")
+ (match_operand:V_VLS 2 "vector_merge_operand")))]
+ "TARGET_XTHEADVECTOR"
+ {})
+
+(define_insn_and_split "*pred_mov_width<vlmem_op_attr><mode>"
+ [(set (match_operand:V_VLS 0 "nonimmediate_operand" "=vr, vr, vd, m, vr, vr")
+ (if_then_else:V_VLS
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1, Wc1, vm, vmWc1, Wc1, Wc1")
+ (match_operand 4 "vector_length_operand" " rK, rK, rK, rK, rK, rK")
+ (match_operand 5 "const_int_operand" " i, i, i, i, i, i")
+ (match_operand 6 "const_int_operand" " i, i, i, i, i, i")
+ (match_operand 7 "const_int_operand" " i, i, i, i, i, i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLMEM_OP)
+ (match_operand:V_VLS 3 "reg_or_mem_operand" " m, m, m, vr, vr, vr")
+ (match_operand:V_VLS 2 "vector_merge_operand" " 0, vu, vu, vu, vu, 0")))]
+ "(TARGET_XTHEADVECTOR
+ && (register_operand (operands[0], <MODE>mode)
+ || register_operand (operands[3], <MODE>mode)))"
+ "@
+ vl<vlmem_op_attr>.v\t%0,%3%p1
+ vl<vlmem_op_attr>.v\t%0,%3
+ vl<vlmem_op_attr>.v\t%0,%3,%1.t
+ vs<vlmem_op_attr>.v\t%3,%0%p1
+ vmv.v.v\t%0,%3
+ vmv.v.v\t%0,%3"
+ "&& register_operand (operands[0], <MODE>mode)
+ && register_operand (operands[3], <MODE>mode)
+ && satisfies_constraint_vu (operands[2])
+ && INTVAL (operands[7]) == riscv_vector::VLMAX"
+ [(set (match_dup 0) (match_dup 3))]
+ ""
+ [(set_attr "type" "vlde,vlde,vlde,vste,vimov,vimov")
+ (set_attr "mode" "<MODE>")])
+
+(define_insn "@pred_store_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "memory_operand" "+m")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1")
+ (match_operand 3 "vector_length_operand" " rK")
+ (match_operand 4 "const_int_operand" " i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VSMEM_OP)
+ (match_operand:VI 2 "register_operand" " vr")
+ (match_dup 0)))]
+ "TARGET_XTHEADVECTOR"
+ "vs<vlmem_op_attr>.v\t%2,%0%p1"
+ [(set_attr "type" "vste")
+ (set_attr "mode" "<MODE>")
+ (set (attr "avl_type_idx") (const_int 4))
+ (set_attr "vl_op_idx" "3")])
+
+(define_insn "@pred_strided_load_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "register_operand" "=vr, vr, vd")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1, Wc1, vm")
+ (match_operand 5 "vector_length_operand" " rK, rK, rK")
+ (match_operand 6 "const_int_operand" " i, i, i")
+ (match_operand 7 "const_int_operand" " i, i, i")
+ (match_operand 8 "const_int_operand" " i, i, i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLSMEM_OP)
+ (unspec:VI
+ [(match_operand:VI 3 "memory_operand" " m, m, m")
+ (match_operand 4 "pmode_reg_or_0_operand" " rJ, rJ, rJ")] UNSPEC_TH_VLSMEM_OP)
+ (match_operand:VI 2 "vector_merge_operand" " 0, vu, vu")))]
+ "TARGET_XTHEADVECTOR"
+ "vls<vlmem_op_attr>.v\t%0,%3,%z4%p1"
+ [(set_attr "type" "vlds")
+ (set_attr "mode" "<MODE>")])
+
+(define_insn "@pred_strided_store_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "memory_operand" "+m")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1")
+ (match_operand 4 "vector_length_operand" " rK")
+ (match_operand 5 "const_int_operand" " i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VSSMEM_OP)
+ (unspec:VI
+ [(match_operand 2 "pmode_reg_or_0_operand" " rJ")
+ (match_operand:VI 3 "register_operand" " vr")] UNSPEC_TH_VSSMEM_OP)
+ (match_dup 0)))]
+ "TARGET_XTHEADVECTOR"
+ "vss<vlmem_op_attr>.v\t%3,%0,%z2%p1"
+ [(set_attr "type" "vsts")
+ (set_attr "mode" "<MODE>")
+ (set (attr "avl_type_idx") (const_int 5))])
+
+(define_insn "@pred_indexed_load_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "register_operand" "=vd, vr,vd, vr")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" " vm,Wc1,vm,Wc1")
+ (match_operand 5 "vector_length_operand" " rK, rK,rK, rK")
+ (match_operand 6 "const_int_operand" " i, i, i, i")
+ (match_operand 7 "const_int_operand" " i, i, i, i")
+ (match_operand 8 "const_int_operand" " i, i, i, i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLXMEM_OP)
+ (unspec:VI
+ [(match_operand 3 "pmode_reg_or_0_operand" " rJ, rJ,rJ, rJ")
+ (mem:BLK (scratch))
+ (match_operand:VI 4 "register_operand" " vr, vr,vr, vr")] UNSPEC_TH_VLXMEM_OP)
+ (match_operand:VI 2 "vector_merge_operand" " vu, vu, 0, 0")))]
+ "TARGET_XTHEADVECTOR"
+ "vlx<vlmem_op_attr>.v\t%0,(%z3),%4%p1"
+ [(set_attr "type" "vldux")
+ (set_attr "mode" "<MODE>")])
+
+(define_insn "@pred_indexed_<vlmem_order_attr>store_width<vlmem_op_attr><mode>"
+ [(set (mem:BLK (scratch))
+ (unspec:BLK
+ [(unspec:<VM>
+ [(match_operand:<VM> 0 "vector_mask_operand" "vmWc1")
+ (match_operand 4 "vector_length_operand" " rK")
+ (match_operand 5 "const_int_operand" " i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VSXMEM_OP)
+ (match_operand 1 "pmode_reg_or_0_operand" " rJ")
+ (match_operand:VI 2 "register_operand" " vr")
+ (match_operand:VI 3 "register_operand" " vr")] UNSPEC_TH_VSXMEM_OP))]
+ "TARGET_XTHEADVECTOR"
+ "vs<vlmem_order_attr>x<vlmem_op_attr>.v\t%3,(%z1),%2%p0"
+ [(set_attr "type" "vstux")
+ (set_attr "mode" "<MODE>")])
+
+(define_expand "@pred_th_extract<mode>"
+ [(set (match_operand:<VEL> 0 "register_operand")
+ (unspec:<VEL>
+ [(vec_select:<VEL>
+ (match_operand:V_VLSI 1 "register_operand")
+ (parallel [(match_operand:DI 2 "register_operand" "r")]))
+ (reg:SI VTYPE_REGNUM)] UNSPEC_VPREDICATE))]
+ "TARGET_XTHEADVECTOR"
+{})
+
+(define_insn "*pred_th_extract<mode>"
+ [(set (match_operand:<VEL> 0 "register_operand" "=r")
+ (unspec:<VEL>
+ [(vec_select:<VEL>
+ (match_operand:V_VLSI 1 "register_operand" "vr")
+ (parallel [(match_operand:DI 2 "register_operand" "r")]))
+ (reg:SI VTYPE_REGNUM)] UNSPEC_VPREDICATE))]
+ "TARGET_XTHEADVECTOR"
+ "vext.x.v\t%0,%1,%2"
+ [(set_attr "type" "vimovvx")
+ (set_attr "mode" "<MODE>")])
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c
new file mode 100644
index 00000000000..3c12c124597
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_th_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+
+** th.vadd\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+
+** th.vsb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out)
+{
+ vint32m1_t v = __riscv_th_vlb_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlb_v_i32m1_tu (v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vv_i32m1 (v2, v2, 4);
+ vint32m1_t v4 = __riscv_vadd_vv_i32m1_tu (v3, v2, v2, 4);
+ __riscv_th_vsb_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlb.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+
+** th.vadd\.vv\tv[1-9][0-9]?,\s*v[0-9]+,\s*v[0-9]+,\s*v0.t
+** th.vsb.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlb_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlb_v_i32m1_m (mask, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vv_i32m1 (v2, v2, 4);
+ vint32m1_t v4 = __riscv_vadd_vv_i32m1_m (mask, v3, v3, 4);
+ __riscv_th_vsb_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlb.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+
+** th.vadd\.vv\tv[1-9][0-9]?,\s*v[0-9]+,\s*v[0-9]+,\s*v0.t
+** th.vsb.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlb_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlb_v_i32m1_tumu (mask, v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vv_i32m1 (v2, v2, 4);
+ vint32m1_t v4 = __riscv_vadd_vv_i32m1_tumu (mask, v3, v2, v2, 4);
+ __riscv_th_vsb_v_i32m1 (out, v4, 4);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c
new file mode 100644
index 00000000000..30bef369375
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_th_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlbu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlbu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vsb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, uint32_t x)
+{
+ vuint32m1_t v = __riscv_th_vlbu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlbu_v_u32m1_tu (v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tu (v3, v2, -16, 4);
+ __riscv_th_vsb_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlbu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsb.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlbu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlbu_v_u32m1_m (mask, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_m (mask, v3, -16, 4);
+ __riscv_th_vsb_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlbu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlbu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsb.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlbu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlbu_v_u32m1_tumu (mask, v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tumu (mask, v3, v2, -16, 4);
+ __riscv_th_vsb_v_u32m1 (out, v4, 4);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c
new file mode 100644
index 00000000000..3c8b5ccc16b
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_th_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vsh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, int32_t x)
+{
+ vint32m1_t v = __riscv_th_vlh_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlh_v_i32m1_tu (v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, -16, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_tu (v3, v2, -16, 4);
+ __riscv_th_vsh_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlh.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsh.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, int32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlh_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlh_v_i32m1_m (mask, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, -16, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_m (mask, v3, -16, 4);
+ __riscv_th_vsh_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlh.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsh.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, int32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlh_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlh_v_i32m1_tumu (mask, v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, -16, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_tumu (mask, v3, v2, -16, 4);
+ __riscv_th_vsh_v_i32m1 (out, v4, 4);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c
new file mode 100644
index 00000000000..b7c00404f18
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_th_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlhu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlhu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vsh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, uint32_t x)
+{
+ vuint32m1_t v = __riscv_th_vlhu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlhu_v_u32m1_tu (v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tu (v3, v2, -16, 4);
+ __riscv_th_vsh_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlhu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsh.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlhu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlhu_v_u32m1_m (mask, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_m (mask, v3, -16, 4);
+ __riscv_th_vsh_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlhu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlhu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsh.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlhu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlhu_v_u32m1_tumu (mask, v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tumu (mask, v3, v2, -16, 4);
+ __riscv_th_vsh_v_u32m1 (out, v4, 4);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c
new file mode 100644
index 00000000000..17a53012acf
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_th_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vx\tv[0-9]+,\s*v[0-9]+,\s*[a-x0-9]+
+** th.vadd\.vx\tv[0-9]+,\s*v[0-9]+,\s*[a-x0-9]+
+** th.vsw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, int32_t x)
+{
+ vint32m1_t v = __riscv_th_vlw_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlw_v_i32m1_tu (v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, x, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_tu (v3, v2, x, 4);
+ __riscv_th_vsw_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlw.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vx\tv[0-9]+,\s*v[0-9]+,\s*[a-x0-9]+
+** th.vadd\.vx\tv[1-9][0-9]?,\s*v[0-9]+,\s*[a-x0-9]+,\s*v0.t
+** th.vsw.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, int32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlw_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlw_v_i32m1_m (mask, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, x, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_m (mask, v3, x, 4);
+ __riscv_th_vsw_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlw.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vx\tv[0-9]+,\s*v[0-9]+,\s*[a-x0-9]+
+** th.vadd\.vx\tv[1-9][0-9]?,\s*v[0-9]+,\s*[a-x0-9]+,\s*v0.t
+** th.vsw.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, int32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlw_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlw_v_i32m1_tumu (mask, v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, x, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_tumu (mask, v3, v2, x, 4);
+ __riscv_th_vsw_v_i32m1 (out, v4, 4);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c
new file mode 100644
index 00000000000..b187cfc852b
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_th_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlwu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlwu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vsw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, uint32_t x)
+{
+ vuint32m1_t v = __riscv_th_vlwu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlwu_v_u32m1_tu (v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tu (v3, v2, -16, 4);
+ __riscv_th_vsw_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlwu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsw.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlwu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlwu_v_u32m1_m (mask, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_m (mask, v3, -16, 4);
+ __riscv_th_vsw_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlwu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlwu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsw.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlwu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlwu_v_u32m1_tumu (mask, v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tumu (mask, v3, v2, -16, 4);
+ __riscv_th_vsw_v_u32m1 (out, v4, 4);
+}
--
2.17.1
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re：Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
2024-01-11 12:28 Re：[PATCH " juzhe.zhong
@ 2024-01-11 12:31 ` joshua
0 siblings, 0 replies; 17+ messages in thread
From: joshua @ 2024-01-11 12:31 UTC (permalink / raw)
To: juzhe.zhong, gcc-patches
Cc: Jim Wilson, palmer, andrew, philipp.tomsich, jeffreyalaw,
christoph.muellner, jinma, cooper.qu
Yes.
(insn 14 13 15 2 (set (reg:QI 147)
(const_int 1 [0x1])) {*movqi_internal}
(nil))
------------------------------------------------------------------
发件人：juzhe.zhong@rivai.ai <juzhe.zhong@rivai.ai>
发送时间：2024年1月11日(星期四) 20:28
收件人："cooper.joshua"<cooper.joshua@linux.alibaba.com>; "gcc-patches"<gcc-patches@gcc.gnu.org>
抄 送：Jim Wilson<jim.wilson.gcc@gmail.com>; palmer<palmer@dabbelt.com>; andrew<andrew@sifive.com>; "philipp.tomsich"<philipp.tomsich@vrull.eu>; jeffreyalaw<jeffreyalaw@gmail.com>; "christoph.muellner"<christoph.muellner@vrull.eu>; jinma<jinma@linux.alibaba.com>; "cooper.qu"<cooper.qu@linux.alibaba.com>
主 题：Re: Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
(vec_duplicate:RVVM1QI (reg:QI 147))))
Find the RTL define pseudo 147 to me.
I guess it is 1.
juzhe.zhong@rivai.ai
发件人： joshua
发送时间： 2024-01-11 20:18
收件人： juzhe.zhong@rivai.ai; gcc-patches
抄送： Jim Wilson; palmer; andrew; philipp.tomsich; jeffreyalaw; christoph.muellner; jinma; cooper.qu
主题： Re：Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
No, we have debugged and the code won't enter this autovec pattern.
The root cause is CSE, in a very early pass.
What I sent to you just now is the dump result which shows the difference.
------------------------------------------------------------------
发件人：juzhe.zhong@rivai.ai <juzhe.zhong@rivai.ai>
发送时间：2024年1月11日(星期四) 20:13
收件人："cooper.joshua"<cooper.joshua@linux.alibaba.com>; "gcc-patches"<gcc-patches@gcc.gnu.org>
抄 送：Jim Wilson<jim.wilson.gcc@gmail.com>; palmer<palmer@dabbelt.com>; andrew<andrew@sifive.com>; "philipp.tomsich"<philipp.tomsich@vrull.eu>; jeffreyalaw<jeffreyalaw@gmail.com>; "christoph.muellner"<christoph.muellner@vrull.eu>; jinma<jinma@linux.alibaba.com>; "cooper.qu"<cooper.qu@linux.alibaba.com>
主 题：Re: Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
I guess it is because of patterns from autovectorization pattern:
;; Use define_insn_and_split to define vsext.vf2/vzext.vf2 will help
;; to combine instructions as below:
;; vsext.vf2 + vsext.vf2 + vadd.vv ==> vwadd.vv
(define_insn_and_split "<optab><v_double_trunc><mode>2"
[(set (match_operand:VWEXTI 0 "register_operand" "=&vr")
(any_extend:VWEXTI
(match_operand:<V_DOUBLE_TRUNC> 1 "register_operand" "vr")))]
"TARGET_VECTOR && can_create_pseudo_p ()"
"#"
"&& 1"
[(const_int 0)]
{
insn_code icode = code_for_pred_vf2 (<CODE>, <MODE>mode);
riscv_vector::emit_vlmax_insn (icode, riscv_vector::UNARY_OP, operands);
DONE;
}
[(set_attr "type" "vext")
(set_attr "mode" "<MODE>")])
juzhe.zhong@rivai.ai
发件人： joshua
发送时间： 2024-01-11 20:05
收件人： juzhe.zhong@rivai.ai; gcc-patches
抄送： Jim Wilson; palmer; andrew; philipp.tomsich; jeffreyalaw; christoph.muellner; jinma; cooper.qu
主题： Re：Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
without CSE:
(insn 16 15 17 2 (set (reg/v:RVVM2HI 137 [ output_var_0 ])
(if_then_else:RVVM2HI (unspec:RVVMF8BI [
(const_vector:RVVMF8BI repeat [
(const_int 1 [0x1])
])
(reg:DI 146)
(const_int 2 [0x2]) repeated x2
(const_int 0 [0])
(reg:SI 66 vl)
(reg:SI 67 vtype)
] UNSPEC_VPREDICATE)
(mult:RVVM2HI (sign_extend:RVVM2HI (reg/v:RVVM1QI 136 [ op1 ]))
(sign_extend:RVVM2HI (vec_duplicate:RVVM1QI (reg:QI 147))))
(unspec:RVVM2HI [
(reg:SI 0 zero)
] UNSPEC_VUNDEF))) {pred_dual_widen_mulsrvvm2hi_scalar}
with CSE:
(insn 16 15 17 2 (set (reg/v:RVVM2HI 137 [ output_var_0 ])
(if_then_else:RVVM2HI (unspec:RVVMF8BI [
(const_vector:RVVMF8BI repeat [
(const_int 1 [0x1])
])
(reg:DI 146)
(const_int 2 [0x2]) repeated x2
(const_int 0 [0])
(reg:SI 66 vl)
(reg:SI 67 vtype)
] UNSPEC_VPREDICATE)
(sign_extend:RVVM2HI (reg/v:RVVM1QI 136 [ op1 ]))
(unspec:RVVM2HI [
(reg:SI 0 zero)
] UNSPEC_VUNDEF))) {pred_extendrvvm2hi_vf2}
------------------------------------------------------------------
发件人：juzhe.zhong@rivai.ai <juzhe.zhong@rivai.ai>
发送时间：2024年1月11日(星期四) 17:32
收件人："cooper.joshua"<cooper.joshua@linux.alibaba.com>; "gcc-patches"<gcc-patches@gcc.gnu.org>
抄 送：Jim Wilson<jim.wilson.gcc@gmail.com>; palmer<palmer@dabbelt.com>; andrew<andrew@sifive.com>; "philipp.tomsich"<philipp.tomsich@vrull.eu>; jeffreyalaw<jeffreyalaw@gmail.com>; "christoph.muellner"<christoph.muellner@vrull.eu>; jinma<jinma@linux.alibaba.com>; "cooper.qu"<cooper.qu@linux.alibaba.com>
主 题：Re: Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
Ok. Let's hold on "RISC-V: Handle differences between XTheadvector and Vector" patch
until you can reproduce the issue for me.
juzhe.zhong@rivai.ai
发件人： joshua
发送时间： 2024-01-11 17:29
收件人： juzhe.zhong@rivai.ai; gcc-patches
抄送： Jim Wilson; palmer; andrew; philipp.tomsich; jeffreyalaw; christoph.muellner; jinma; cooper.qu
主题： Re：Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
The sext/zext issue is not related to xtheadvector-special patterns.
I added !TARGET_XTHEADVECTOR to sext/zext patterns in
"RISC-V: Handle differences between XTheadvector and Vector"
That is caused by the vwmul pattern, but I cannot reproduce it right now.
------------------------------------------------------------------
发件人：juzhe.zhong@rivai.ai <juzhe.zhong@rivai.ai>
发送时间：2024年1月11日(星期四) 17:24
收件人："cooper.joshua"<cooper.joshua@linux.alibaba.com>; "gcc-patches"<gcc-patches@gcc.gnu.org>
抄 送：Jim Wilson<jim.wilson.gcc@gmail.com>; palmer<palmer@dabbelt.com>; andrew<andrew@sifive.com>; "philipp.tomsich"<philipp.tomsich@vrull.eu>; jeffreyalaw<jeffreyalaw@gmail.com>; "christoph.muellner"<christoph.muellner@vrull.eu>; jinma<jinma@linux.alibaba.com>; "cooper.qu"<cooper.qu@linux.alibaba.com>
主 题：Re: Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
I don't see any patterns are possible CSE into sext/zext patterns:
+(define_expand "@pred_mov_width<vlmem_op_attr><mode>"
+ [(set (match_operand:V_VLS 0 "nonimmediate_operand")
+ (if_then_else:V_VLS
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand")
+ (match_operand 4 "vector_length_operand")
+ (match_operand 5 "const_int_operand")
+ (match_operand 6 "const_int_operand")
+ (match_operand 7 "const_int_operand")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLMEM_OP)
+ (match_operand:V_VLS 3 "vector_move_operand")
+ (match_operand:V_VLS 2 "vector_merge_operand")))]
+ "TARGET_XTHEADVECTOR"
+ {})
+
+(define_insn_and_split "*pred_mov_width<vlmem_op_attr><mode>"
+ [(set (match_operand:V_VLS 0 "nonimmediate_operand" "=vr, vr, vd, m, vr, vr")
+ (if_then_else:V_VLS
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1, Wc1, vm, vmWc1, Wc1, Wc1")
+ (match_operand 4 "vector_length_operand" " rK, rK, rK, rK, rK, rK")
+ (match_operand 5 "const_int_operand" " i, i, i, i, i, i")
+ (match_operand 6 "const_int_operand" " i, i, i, i, i, i")
+ (match_operand 7 "const_int_operand" " i, i, i, i, i, i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLMEM_OP)
+ (match_operand:V_VLS 3 "reg_or_mem_operand" " m, m, m, vr, vr, vr")
+ (match_operand:V_VLS 2 "vector_merge_operand" " 0, vu, vu, vu, vu, 0")))]
+ "(TARGET_XTHEADVECTOR
+ && (register_operand (operands[0], <MODE>mode)
+ || register_operand (operands[3], <MODE>mode)))"
+ "@
+ vl<vlmem_op_attr>.v\t%0,%3%p1
+ vl<vlmem_op_attr>.v\t%0,%3
+ vl<vlmem_op_attr>.v\t%0,%3,%1.t
+ vs<vlmem_op_attr>.v\t%3,%0%p1
+ vmv.v.v\t%0,%3
+ vmv.v.v\t%0,%3"
+ "&& register_operand (operands[0], <MODE>mode)
+ && register_operand (operands[3], <MODE>mode)
+ && satisfies_constraint_vu (operands[2])
+ && INTVAL (operands[7]) == riscv_vector::VLMAX"
+ [(set (match_dup 0) (match_dup 3))]
+ ""
+ [(set_attr "type" "vlde,vlde,vlde,vste,vimov,vimov")
+ (set_attr "mode" "<MODE>")])
+
+(define_insn "@pred_store_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "memory_operand" "+m")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1")
+ (match_operand 3 "vector_length_operand" " rK")
+ (match_operand 4 "const_int_operand" " i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VSMEM_OP)
+ (match_operand:VI 2 "register_operand" " vr")
+ (match_dup 0)))]
+ "TARGET_XTHEADVECTOR"
+ "vs<vlmem_op_attr>.v\t%2,%0%p1"
+ [(set_attr "type" "vste")
+ (set_attr "mode" "<MODE>")
+ (set (attr "avl_type_idx") (const_int 4))
+ (set_attr "vl_op_idx" "3")])
+
+(define_insn "@pred_strided_load_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "register_operand" "=vr, vr, vd")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1, Wc1, vm")
+ (match_operand 5 "vector_length_operand" " rK, rK, rK")
+ (match_operand 6 "const_int_operand" " i, i, i")
+ (match_operand 7 "const_int_operand" " i, i, i")
+ (match_operand 8 "const_int_operand" " i, i, i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLSMEM_OP)
+ (unspec:VI
+ [(match_operand:VI 3 "memory_operand" " m, m, m")
+ (match_operand 4 "pmode_reg_or_0_operand" " rJ, rJ, rJ")] UNSPEC_TH_VLSMEM_OP)
+ (match_operand:VI 2 "vector_merge_operand" " 0, vu, vu")))]
+ "TARGET_XTHEADVECTOR"
+ "vls<vlmem_op_attr>.v\t%0,%3,%z4%p1"
+ [(set_attr "type" "vlds")
+ (set_attr "mode" "<MODE>")])
+
+(define_insn "@pred_strided_store_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "memory_operand" "+m")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1")
+ (match_operand 4 "vector_length_operand" " rK")
+ (match_operand 5 "const_int_operand" " i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VSSMEM_OP)
+ (unspec:VI
+ [(match_operand 2 "pmode_reg_or_0_operand" " rJ")
+ (match_operand:VI 3 "register_operand" " vr")] UNSPEC_TH_VSSMEM_OP)
+ (match_dup 0)))]
+ "TARGET_XTHEADVECTOR"
+ "vss<vlmem_op_attr>.v\t%3,%0,%z2%p1"
+ [(set_attr "type" "vsts")
+ (set_attr "mode" "<MODE>")
+ (set (attr "avl_type_idx") (const_int 5))])
+
+(define_insn "@pred_indexed_load_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "register_operand" "=vd, vr,vd, vr")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" " vm,Wc1,vm,Wc1")
+ (match_operand 5 "vector_length_operand" " rK, rK,rK, rK")
+ (match_operand 6 "const_int_operand" " i, i, i, i")
+ (match_operand 7 "const_int_operand" " i, i, i, i")
+ (match_operand 8 "const_int_operand" " i, i, i, i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLXMEM_OP)
+ (unspec:VI
+ [(match_operand 3 "pmode_reg_or_0_operand" " rJ, rJ,rJ, rJ")
+ (mem:BLK (scratch))
+ (match_operand:VI 4 "register_operand" " vr, vr,vr, vr")] UNSPEC_TH_VLXMEM_OP)
+ (match_operand:VI 2 "vector_merge_operand" " vu, vu, 0, 0")))]
+ "TARGET_XTHEADVECTOR"
+ "vlx<vlmem_op_attr>.v\t%0,(%z3),%4%p1"
+ [(set_attr "type" "vldux")
+ (set_attr "mode" "<MODE>")])
+
+(define_insn "@pred_indexed_<vlmem_order_attr>store_width<vlmem_op_attr><mode>"
+ [(set (mem:BLK (scratch))
+ (unspec:BLK
+ [(unspec:<VM>
+ [(match_operand:<VM> 0 "vector_mask_operand" "vmWc1")
+ (match_operand 4 "vector_length_operand" " rK")
+ (match_operand 5 "const_int_operand" " i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VSXMEM_OP)
+ (match_operand 1 "pmode_reg_or_0_operand" " rJ")
+ (match_operand:VI 2 "register_operand" " vr")
+ (match_operand:VI 3 "register_operand" " vr")] UNSPEC_TH_VSXMEM_OP))]
+ "TARGET_XTHEADVECTOR"
+ "vs<vlmem_order_attr>x<vlmem_op_attr>.v\t%3,(%z1),%2%p0"
+ [(set_attr "type" "vstux")
+ (set_attr "mode" "<MODE>")])
+
+(define_expand "@pred_th_extract<mode>"
+ [(set (match_operand:<VEL> 0 "register_operand")
+ (unspec:<VEL>
+ [(vec_select:<VEL>
+ (match_operand:V_VLSI 1 "register_operand")
+ (parallel [(match_operand:DI 2 "register_operand" "r")]))
+ (reg:SI VTYPE_REGNUM)] UNSPEC_VPREDICATE))]
+ "TARGET_XTHEADVECTOR"
+{})
+
+(define_insn "*pred_th_extract<mode>"
+ [(set (match_operand:<VEL> 0 "register_operand" "=r")
+ (unspec:<VEL>
+ [(vec_select:<VEL>
+ (match_operand:V_VLSI 1 "register_operand" "vr")
+ (parallel [(match_operand:DI 2 "register_operand" "r")]))
+ (reg:SI VTYPE_REGNUM)] UNSPEC_VPREDICATE))]
+ "TARGET_XTHEADVECTOR"
+ "vext.x.v\t%0,%1,%2"
+ [(set_attr "type" "vimovvx")
+ (set_attr "mode" "<MODE>")])
juzhe.zhong@rivai.ai
发件人： joshua
发送时间： 2024-01-11 17:21
收件人： juzhe.zhong@rivai.ai; gcc-patches
抄送： Jim Wilson; palmer; andrew; philipp.tomsich; jeffreyalaw; christoph.muellner; jinma; cooper.qu
主题： Re：Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
"I didn't see theadvector-specific extension patterns. Could you show me?"
They are all in the file thead-vector.md.
For the sext/zext issue, perhaps I need some time to reproduce that optimization,
but I can clearly remember it is related to vwmul.
------------------------------------------------------------------
发件人：juzhe.zhong@rivai.ai <juzhe.zhong@rivai.ai>
发送时间：2024年1月11日(星期四) 17:17
收件人："cooper.joshua"<cooper.joshua@linux.alibaba.com>; "gcc-patches"<gcc-patches@gcc.gnu.org>
抄 送：Jim Wilson<jim.wilson.gcc@gmail.com>; palmer<palmer@dabbelt.com>; andrew<andrew@sifive.com>; "philipp.tomsich"<philipp.tomsich@vrull.eu>; jeffreyalaw<jeffreyalaw@gmail.com>; "christoph.muellner"<christoph.muellner@vrull.eu>; jinma<jinma@linux.alibaba.com>; "cooper.qu"<cooper.qu@linux.alibaba.com>
主 题：Re: Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
You mean which pattern optimized sext/vzext pattern?
I didn't see theadvector-specific extension patterns. Could you show me?
juzhe.zhong@rivai.ai
发件人： joshua
发送时间： 2024-01-11 17:14
收件人： juzhe.zhong@rivai.ai; gcc-patches
抄送： Jim Wilson; palmer; andrew; philipp.tomsich; jeffreyalaw; christoph.muellner; jinma; cooper.qu
主题： Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
To be specific, in CSE pass, the initial pattern will be optimized into the sext/zext pattern.
------------------------------------------------------------------
发件人：joshua <cooper.joshua@linux.alibaba.com>
发送时间：2024年1月11日(星期四) 17:11
收件人："juzhe.zhong@rivai.ai"<juzhe.zhong@rivai.ai>; "gcc-patches"<gcc-patches@gcc.gnu.org>
抄 送：Jim Wilson<jim.wilson.gcc@gmail.com>; palmer<palmer@dabbelt.com>; andrew<andrew@sifive.com>; "philipp.tomsich"<philipp.tomsich@vrull.eu>; jeffreyalaw<jeffreyalaw@gmail.com>; "christoph.muellner"<christoph.muellner@vrull.eu>; jinma<jinma@linux.alibaba.com>; "cooper.qu"<cooper.qu@linux.alibaba.com>
主 题：Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
sext/zext will be generated in O2 even without corresponding intrinsics.
------------------------------------------------------------------
发件人：juzhe.zhong@rivai.ai <juzhe.zhong@rivai.ai>
发送时间：2024年1月11日(星期四) 17:07
收件人："cooper.joshua"<cooper.joshua@linux.alibaba.com>; "gcc-patches"<gcc-patches@gcc.gnu.org>
抄 送：Jim Wilson<jim.wilson.gcc@gmail.com>; palmer<palmer@dabbelt.com>; andrew<andrew@sifive.com>; "philipp.tomsich"<philipp.tomsich@vrull.eu>; jeffreyalaw<jeffreyalaw@gmail.com>; "christoph.muellner"<christoph.muellner@vrull.eu>; "cooper.joshua"<cooper.joshua@linux.alibaba.com>; jinma<jinma@linux.alibaba.com>; "cooper.qu"<cooper.qu@linux.alibaba.com>
主 题：Re: [PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
enum required_ext
{
VECTOR_EXT, /* Vector extension */
+ XTHEADVECTOR_EXT, /* XTheadVector extension */
ZVBB_EXT, /* Cryto vector Zvbb sub-ext */
ZVBB_OR_ZVKB_EXT, /* Cryto vector Zvbb or zvkb sub-ext */
ZVBC_EXT, /* Crypto vector Zvbc sub-ext */
Add theadvector to the end of the enum.
+ case XTHEADVECTOR_EXT:
+ return TARGET_XTHEADVECTOR;
Same.
+ "&& register_operand (operands[0], <MODE>mode)
+ && register_operand (operands[3], <MODE>mode)
+ && satisfies_constraint_vu (operands[2])
+ && INTVAL (operands[7]) == riscv_vector::VLMAX"
You can use whole_reg_to_reg_move_p
Btw, I review again : RISC-V: Handle differences between XTheadvector and Vector
(any_extend:VWEXTI (match_operand:<V_DOUBLE_TRUNC> 3 "register_operand" "W21,W21,W21,W21,W42,W42,W42,W42,W84,W84,W84,W84, vr, vr")) (match_operand:VWEXTI 2 "vector_merge_operand" " vu, vu, 0, 0, vu, vu, 0, 0, vu, vu, 0, 0, vu, 0")))]- "TARGET_VECTOR"+ "TARGET_VECTOR && !TARGET_XTHEADVECTOR" "v<sz>ext.vf2\t%0,%3%p1" [(set_attr "type" "vext") (set_attr "mode" "<MODE>")@@ -3713,7 +3744,7 @@ (any_extend:VQEXTI (match_operand:<V_QUAD_TRUNC> 3 "register_operand" "W43,W43,W43,W43,W86,W86,W86,W86, vr, vr")) (match_operand:VQEXTI 2 "vector_merge_operand" " vu, vu, 0, 0, vu, vu, 0, 0, vu, 0")))]- "TARGET_VECTOR"+ "TARGET_VECTOR && !TARGET_XTHEADVECTOR" "v<sz>ext.vf4\t%0,%3%p1" [(set_attr "type" "vext") (set_attr "mode" "<MODE>")@@ -3734,7 +3765,7 @@ (any_extend:VOEXTI (match_operand:<V_OCT_TRUNC> 3 "register_operand" "W87,W87,W87,W87, vr, vr")) (match_operand:VOEXTI 2 "vector_merge_operand" " vu, vu, 0, 0, vu, 0")))]- "TARGET_VECTOR"+ "TARGET_VECTOR && !TARGET_XTHEADVECTOR" "v<sz>ext.vf8\t%0,%3%p1" [(set_attr "type" "vext") (set_attr "mode" "<MODE>")
Why do you add these !TARGERT_XTHEADVECRTOR ?
juzhe.zhong@rivai.ai
From: Jun Sha (Joshua)
Date: 2024-01-11 16:46
To: gcc-patches
CC: jim.wilson.gcc; palmer; andrew; philipp.tomsich; jeffreyalaw; christoph.muellner; juzhe.zhong; Jun Sha (Joshua); Jin Ma; Xianmiao Qu
Subject: [PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
This patch only involves the generation of xtheadvector
special load/store instructions and vext instructions.
gcc/ChangeLog:
* config/riscv/riscv-vector-builtins-bases.cc
(class th_loadstore_width): Define new builtin bases.
(class th_extract): Define new builtin bases.
(BASE): Define new builtin bases.
* config/riscv/riscv-vector-builtins-bases.h:
Define new builtin class.
* config/riscv/riscv-vector-builtins-shapes.cc
(struct th_loadstore_width_def): Define new builtin shapes.
(struct th_indexed_loadstore_width_def):
Define new builtin shapes.
(struct th_extract_def): Define new builtin shapes.
(SHAPE): Define new builtin shapes.
* config/riscv/riscv-vector-builtins-shapes.h:
Define new builtin shapes.
* config/riscv/riscv-vector-builtins.cc (DEF_RVV_FUNCTION):
* config/riscv/riscv-vector-builtins.h (enum required_ext):
(struct function_group_info):
* config/riscv/t-riscv: Add thead-vector-builtins-functions.def
* config/riscv/thead-vector.md
(@pred_mov_width<vlmem_op_attr><mode>): Add new patterns.
(*pred_mov_width<vlmem_op_attr><mode>): Likewise.
(@pred_store_width<vlmem_op_attr><mode>): Likewise.
(@pred_strided_load_width<vlmem_op_attr><mode>): Likewise.
(@pred_strided_store_width<vlmem_op_attr><mode>): Likewise.
(@pred_indexed_load_width<vlmem_op_attr><mode>): Likewise.
(@pred_indexed_<vlmem_order_attr>store_width<vlmem_op_attr><mode>):
(@pred_th_extract<mode>): Likewise.
(*pred_th_extract<mode>): Likewise.
* config/riscv/thead-vector-builtins-functions.def: New file.
gcc/testsuite/ChangeLog:
* gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c: New test.
Co-authored-by: Jin Ma <jinma@linux.alibaba.com>
Co-authored-by: Xianmiao Qu <cooper.qu@linux.alibaba.com>
Co-authored-by: Christoph Müllner <christoph.muellner@vrull.eu>
---
.../riscv/riscv-vector-builtins-bases.cc | 139 ++++++++++
.../riscv/riscv-vector-builtins-bases.h | 31 +++
.../riscv/riscv-vector-builtins-shapes.cc | 160 +++++++++++
.../riscv/riscv-vector-builtins-shapes.h | 3 +
gcc/config/riscv/riscv-vector-builtins.cc | 70 +++++
gcc/config/riscv/riscv-vector-builtins.h | 3 +
gcc/config/riscv/t-riscv | 1 +
.../riscv/thead-vector-builtins-functions.def | 39 +++
gcc/config/riscv/thead-vector.md | 253 ++++++++++++++++++
.../riscv/rvv/xtheadvector/vlb-vsb.c | 68 +++++
.../riscv/rvv/xtheadvector/vlbu-vsb.c | 68 +++++
.../riscv/rvv/xtheadvector/vlh-vsh.c | 68 +++++
.../riscv/rvv/xtheadvector/vlhu-vsh.c | 68 +++++
.../riscv/rvv/xtheadvector/vlw-vsw.c | 68 +++++
.../riscv/rvv/xtheadvector/vlwu-vsw.c | 68 +++++
15 files changed, 1107 insertions(+)
create mode 100644 gcc/config/riscv/thead-vector-builtins-functions.def
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c
diff --git a/gcc/config/riscv/riscv-vector-builtins-bases.cc b/gcc/config/riscv/riscv-vector-builtins-bases.cc
index 46f1a1da33e..3eba7943757 100644
--- a/gcc/config/riscv/riscv-vector-builtins-bases.cc
+++ b/gcc/config/riscv/riscv-vector-builtins-bases.cc
@@ -2125,6 +2125,83 @@ public:
}
};
+/* Implements
+ * th.vl(b/h/w)[u].v/th.vs(b/h/w)[u].v/th.vls(b/h/w)[u].v/th.vss(b/h/w)[u].v/
+ * th.vlx(b/h/w)[u].v/th.vs[u]x(b/h/w).v
+ * codegen. */
+template<bool STORE_P, lst_type LST_TYPE, int UNSPEC>
+class th_loadstore_width : public function_base
+{
+public:
+ bool apply_tail_policy_p () const override { return !STORE_P; }
+ bool apply_mask_policy_p () const override { return !STORE_P; }
+
+ unsigned int call_properties (const function_instance &) const override
+ {
+ if (STORE_P)
+ return CP_WRITE_MEMORY;
+ else
+ return CP_READ_MEMORY;
+ }
+
+ bool can_be_overloaded_p (enum predication_type_index pred) const override
+ {
+ if (STORE_P || LST_TYPE == LST_INDEXED)
+ return true;
+ return pred != PRED_TYPE_none;
+ }
+
+ rtx expand (function_expander &e) const override
+ {
+ gcc_assert (TARGET_XTHEADVECTOR);
+ if (LST_TYPE == LST_INDEXED)
+ {
+ if (STORE_P)
+ return e.use_exact_insn (
+ code_for_pred_indexed_store_width (UNSPEC, UNSPEC,
+ e.vector_mode ()));
+ else
+ return e.use_exact_insn (
+ code_for_pred_indexed_load_width (UNSPEC, e.vector_mode ()));
+ }
+ else if (LST_TYPE == LST_STRIDED)
+ {
+ if (STORE_P)
+ return e.use_contiguous_store_insn (
+ code_for_pred_strided_store_width (UNSPEC, e.vector_mode ()));
+ else
+ return e.use_contiguous_load_insn (
+ code_for_pred_strided_load_width (UNSPEC, e.vector_mode ()));
+ }
+ else
+ {
+ if (STORE_P)
+ return e.use_contiguous_store_insn (
+ code_for_pred_store_width (UNSPEC, e.vector_mode ()));
+ else
+ return e.use_contiguous_load_insn (
+ code_for_pred_mov_width (UNSPEC, e.vector_mode ()));
+ }
+ }
+};
+
+/* Implements vext.x.v. */
+class th_extract : public function_base
+{
+public:
+ bool apply_vl_p () const override { return false; }
+ bool apply_tail_policy_p () const override { return false; }
+ bool apply_mask_policy_p () const override { return false; }
+ bool use_mask_predication_p () const override { return false; }
+ bool has_merge_operand_p () const override { return false; }
+
+ rtx expand (function_expander &e) const override
+ {
+ gcc_assert (TARGET_XTHEADVECTOR);
+ return e.use_exact_insn (code_for_pred_th_extract (e.vector_mode ()));
+ }
+};
+
/* Below implements are vector crypto */
/* Implements vandn.[vv,vx] */
class vandn : public function_base
@@ -2587,6 +2664,37 @@ static CONSTEXPR const seg_indexed_load<UNSPEC_ORDERED> vloxseg_obj;
static CONSTEXPR const seg_indexed_store<UNSPEC_UNORDERED> vsuxseg_obj;
static CONSTEXPR const seg_indexed_store<UNSPEC_ORDERED> vsoxseg_obj;
static CONSTEXPR const vlsegff vlsegff_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLB> vlb_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLBU> vlbu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLH> vlh_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLHU> vlhu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLW> vlw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLWU> vlwu_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_UNIT_STRIDE, UNSPEC_TH_VLB> vsb_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_UNIT_STRIDE, UNSPEC_TH_VLH> vsh_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_UNIT_STRIDE, UNSPEC_TH_VLW> vsw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSB> vlsb_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSBU> vlsbu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSH> vlsh_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSHU> vlshu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSW> vlsw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSWU> vlswu_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_STRIDED, UNSPEC_TH_VLSB> vssb_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_STRIDED, UNSPEC_TH_VLSH> vssh_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_STRIDED, UNSPEC_TH_VLSW> vssw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXB> vlxb_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXBU> vlxbu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXH> vlxh_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXHU> vlxhu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXW> vlxw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXWU> vlxwu_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VLXB> vsxb_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VLXH> vsxh_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VLXW> vsxw_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VSUXB> vsuxb_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VSUXH> vsuxh_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VSUXW> vsuxw_obj;
+static CONSTEXPR const th_extract vext_x_v_obj;
/* Crypto Vector */
static CONSTEXPR const vandn vandn_obj;
@@ -2878,6 +2986,37 @@ BASE (vloxseg)
BASE (vsuxseg)
BASE (vsoxseg)
BASE (vlsegff)
+BASE (vlb)
+BASE (vlh)
+BASE (vlw)
+BASE (vlbu)
+BASE (vlhu)
+BASE (vlwu)
+BASE (vsb)
+BASE (vsh)
+BASE (vsw)
+BASE (vlsb)
+BASE (vlsh)
+BASE (vlsw)
+BASE (vlsbu)
+BASE (vlshu)
+BASE (vlswu)
+BASE (vssb)
+BASE (vssh)
+BASE (vssw)
+BASE (vlxb)
+BASE (vlxh)
+BASE (vlxw)
+BASE (vlxbu)
+BASE (vlxhu)
+BASE (vlxwu)
+BASE (vsxb)
+BASE (vsxh)
+BASE (vsxw)
+BASE (vsuxb)
+BASE (vsuxh)
+BASE (vsuxw)
+BASE (vext_x_v)
/* Crypto vector */
BASE (vandn)
BASE (vbrev)
diff --git a/gcc/config/riscv/riscv-vector-builtins-bases.h b/gcc/config/riscv/riscv-vector-builtins-bases.h
index 1122e3801a7..565a0311d2b 100644
--- a/gcc/config/riscv/riscv-vector-builtins-bases.h
+++ b/gcc/config/riscv/riscv-vector-builtins-bases.h
@@ -299,6 +299,37 @@ extern const function_base *const vloxseg;
extern const function_base *const vsuxseg;
extern const function_base *const vsoxseg;
extern const function_base *const vlsegff;
+extern const function_base *const vlb;
+extern const function_base *const vlh;
+extern const function_base *const vlw;
+extern const function_base *const vlbu;
+extern const function_base *const vlhu;
+extern const function_base *const vlwu;
+extern const function_base *const vsb;
+extern const function_base *const vsh;
+extern const function_base *const vsw;
+extern const function_base *const vlsb;
+extern const function_base *const vlsh;
+extern const function_base *const vlsw;
+extern const function_base *const vlsbu;
+extern const function_base *const vlshu;
+extern const function_base *const vlswu;
+extern const function_base *const vssb;
+extern const function_base *const vssh;
+extern const function_base *const vssw;
+extern const function_base *const vlxb;
+extern const function_base *const vlxh;
+extern const function_base *const vlxw;
+extern const function_base *const vlxbu;
+extern const function_base *const vlxhu;
+extern const function_base *const vlxwu;
+extern const function_base *const vsxb;
+extern const function_base *const vsxh;
+extern const function_base *const vsxw;
+extern const function_base *const vsuxb;
+extern const function_base *const vsuxh;
+extern const function_base *const vsuxw;
+extern const function_base *const vext_x_v;
/* Below function_base are Vectro Crypto*/
extern const function_base *const vandn;
extern const function_base *const vbrev;
diff --git a/gcc/config/riscv/riscv-vector-builtins-shapes.cc b/gcc/config/riscv/riscv-vector-builtins-shapes.cc
index 1e4f4d53de6..8e90b17a94b 100644
--- a/gcc/config/riscv/riscv-vector-builtins-shapes.cc
+++ b/gcc/config/riscv/riscv-vector-builtins-shapes.cc
@@ -211,6 +211,146 @@ struct indexed_loadstore_def : public function_shape
}
};
+/* Add one function instance for GROUP, using operand suffix at index OI,
+ mode suffix at index PAIR && bi and predication suffix at index pred_idx. */
+static void
+build_th_loadstore (function_builder &b, const function_group_info &group,
+ unsigned int pred_idx, unsigned int vec_type_idx)
+{
+ auto_vec<tree, 5> argument_types;
+ function_instance function_instance (group.base_name, *group.base,
+ *group.shape,
+ group.ops_infos.types[vec_type_idx],
+ group.preds[pred_idx], &group.ops_infos);
+ tree return_type = group.ops_infos.ret.get_tree_type (
+ group.ops_infos.types[vec_type_idx].index);
+ b.allocate_argument_types (function_instance, argument_types);
+ b.apply_predication (function_instance, return_type, argument_types);
+
+ if (TARGET_XTHEADVECTOR && !check_type (return_type, argument_types))
+ return;
+
+ tree type = builtin_types[group.ops_infos.types[vec_type_idx].index].vector;
+ if (strstr (group.base_name, "l")
+ && strstr (group.base_name, "u")
+ && !TYPE_UNSIGNED (TREE_TYPE (type)))
+ return;
+
+ if (strstr (group.base_name, "l")
+ && !strstr (group.base_name, "u")
+ && TYPE_UNSIGNED (TREE_TYPE (type)))
+ return;
+
+ machine_mode mode = TYPE_MODE (type);
+ int sew = GET_MODE_BITSIZE (GET_MODE_INNER (mode));
+ if (strstr (group.base_name, "h") && sew == 8)
+ return;
+
+ if (strstr (group.base_name, "w") && (sew == 8 || sew ==16))
+ return;
+
+ b.add_overloaded_function (function_instance, *group.shape);
+ b.add_unique_function (function_instance, (*group.shape), return_type,
+ argument_types);
+}
+
+/* th_loadstore_width_def class. */
+struct th_loadstore_width_def : public build_base
+{
+ void build (function_builder &b,
+ const function_group_info &group) const override
+ {
+ for (unsigned int pred_idx = 0; group.preds[pred_idx] != NUM_PRED_TYPES;
+ ++pred_idx)
+ {
+ for (unsigned int vec_type_idx = 0;
+ group.ops_infos.types[vec_type_idx].index != NUM_VECTOR_TYPES;
+ ++vec_type_idx)
+ {
+ build_th_loadstore (b, group, pred_idx, vec_type_idx);
+ }
+ }
+ }
+
+ char *get_name (function_builder &b, const function_instance &instance,
+ bool overloaded_p) const override
+ {
+ /* Return nullptr if it can not be overloaded. */
+ if (overloaded_p && !instance.base->can_be_overloaded_p (instance.pred))
+ return nullptr;
+
+ b.append_name ("__riscv_th_");
+ b.append_name (instance.base_name);
+
+ /* vop_v --> vop_v_<type>. */
+ if (!overloaded_p)
+ {
+ /* vop --> vop_v. */
+ b.append_name (operand_suffixes[instance.op_info->op]);
+ /* vop_v --> vop_v_<type>. */
+ b.append_name (type_suffixes[instance.type.index].vector);
+ }
+
+ /* According to rvv-intrinsic-doc, it does not add "_m" suffix
+ for vop_m C++ overloaded API. */
+ if (overloaded_p && instance.pred == PRED_TYPE_m)
+ return b.finish_name ();
+ b.append_name (predication_suffixes[instance.pred]);
+ return b.finish_name ();
+ }
+};
+
+
+/* th_indexed_loadstore_width_def class. */
+struct th_indexed_loadstore_width_def : public function_shape
+{
+ void build (function_builder &b,
+ const function_group_info &group) const override
+ {
+ for (unsigned int pred_idx = 0; group.preds[pred_idx] != NUM_PRED_TYPES;
+ ++pred_idx)
+ {
+ for (unsigned int vec_type_idx = 0;
+ group.ops_infos.types[vec_type_idx].index != NUM_VECTOR_TYPES;
+ ++vec_type_idx)
+ {
+ tree index_type = group.ops_infos.args[1].get_tree_type (
+ group.ops_infos.types[vec_type_idx].index);
+ if (!index_type)
+ continue;
+ build_th_loadstore (b, group, pred_idx, vec_type_idx);
+ }
+ }
+ }
+
+ char *get_name (function_builder &b, const function_instance &instance,
+ bool overloaded_p) const override
+ {
+
+ /* Return nullptr if it can not be overloaded. */
+ if (overloaded_p && !instance.base->can_be_overloaded_p (instance.pred))
+ return nullptr;
+
+ b.append_name ("__riscv_th_");
+ b.append_name (instance.base_name);
+ /* vop_v --> vop_v_<type>. */
+ if (!overloaded_p)
+ {
+ /* vop --> vop_v. */
+ b.append_name (operand_suffixes[instance.op_info->op]);
+ /* vop_v --> vop_v_<type>. */
+ b.append_name (type_suffixes[instance.type.index].vector);
+ }
+
+ /* According to rvv-intrinsic-doc, it does not add "_m" suffix
+ for vop_m C++ overloaded API. */
+ if (overloaded_p && instance.pred == PRED_TYPE_m)
+ return b.finish_name ();
+ b.append_name (predication_suffixes[instance.pred]);
+ return b.finish_name ();
+ }
+};
+
/* alu_def class. */
struct alu_def : public build_base
{
@@ -632,6 +772,23 @@ struct reduc_alu_def : public build_base
}
};
+/* th_extract_def class. */
+struct th_extract_def : public build_base
+{
+ char *get_name (function_builder &b, const function_instance &instance,
+ bool overloaded_p) const override
+ {
+ b.append_name ("__riscv_th_");
+ b.append_name (instance.base_name);
+
+ if (overloaded_p)
+ return b.finish_name ();
+ b.append_name (type_suffixes[instance.type.index].vector);
+ b.append_name (type_suffixes[instance.type.index].scalar);
+ return b.finish_name ();
+ }
+};
+
/* scalar_move_def class. */
struct scalar_move_def : public build_base
{
@@ -1094,6 +1251,8 @@ SHAPE(vsetvl, vsetvl)
SHAPE(vsetvl, vsetvlmax)
SHAPE(loadstore, loadstore)
SHAPE(indexed_loadstore, indexed_loadstore)
+SHAPE(th_loadstore_width, th_loadstore_width)
+SHAPE(th_indexed_loadstore_width, th_indexed_loadstore_width)
SHAPE(alu, alu)
SHAPE(alu_frm, alu_frm)
SHAPE(widen_alu, widen_alu)
@@ -1106,6 +1265,7 @@ SHAPE(move, move)
SHAPE(mask_alu, mask_alu)
SHAPE(reduc_alu, reduc_alu)
SHAPE(reduc_alu_frm, reduc_alu_frm)
+SHAPE(th_extract, th_extract)
SHAPE(scalar_move, scalar_move)
SHAPE(vundefined, vundefined)
SHAPE(misc, misc)
diff --git a/gcc/config/riscv/riscv-vector-builtins-shapes.h b/gcc/config/riscv/riscv-vector-builtins-shapes.h
index ac2a28ce017..a7624d0fabd 100644
--- a/gcc/config/riscv/riscv-vector-builtins-shapes.h
+++ b/gcc/config/riscv/riscv-vector-builtins-shapes.h
@@ -28,6 +28,8 @@ extern const function_shape *const vsetvl;
extern const function_shape *const vsetvlmax;
extern const function_shape *const loadstore;
extern const function_shape *const indexed_loadstore;
+extern const function_shape *const th_loadstore_width;
+extern const function_shape *const th_indexed_loadstore_width;
extern const function_shape *const alu;
extern const function_shape *const alu_frm;
extern const function_shape *const widen_alu;
@@ -41,6 +43,7 @@ extern const function_shape *const mask_alu;
extern const function_shape *const reduc_alu;
extern const function_shape *const reduc_alu_frm;
extern const function_shape *const scalar_move;
+extern const function_shape *const th_extract;
extern const function_shape *const vundefined;
extern const function_shape *const misc;
extern const function_shape *const vset;
diff --git a/gcc/config/riscv/riscv-vector-builtins.cc b/gcc/config/riscv/riscv-vector-builtins.cc
index 25e0b6e56de..44b9fec1898 100644
--- a/gcc/config/riscv/riscv-vector-builtins.cc
+++ b/gcc/config/riscv/riscv-vector-builtins.cc
@@ -934,6 +934,32 @@ static CONSTEXPR const rvv_arg_type_info ext_vcreate_args[]
= {rvv_arg_type_info (RVV_BASE_vector),
rvv_arg_type_info_end};
+/* A list of args for vector_type func (const scalar_type *, size_t)
+ * function. */
+static CONSTEXPR const rvv_arg_type_info scalar_const_ptr_size_args[]
+ = {rvv_arg_type_info (RVV_BASE_scalar_const_ptr),
+ rvv_arg_type_info (RVV_BASE_size), rvv_arg_type_info_end};
+
+/* A list of args for vector_type func (const scalar_type *, eew8_index_type)
+ * function. */
+static CONSTEXPR const rvv_arg_type_info scalar_const_ptr_index_args[]
+ = {rvv_arg_type_info (RVV_BASE_scalar_const_ptr),
+ rvv_arg_type_info (RVV_BASE_unsigned_vector), rvv_arg_type_info_end};
+
+/* A list of args for void func (scalar_type *, eew8_index_type, vector_type)
+ * function. */
+static CONSTEXPR const rvv_arg_type_info scalar_ptr_index_args[]
+ = {rvv_arg_type_info (RVV_BASE_scalar_ptr),
+ rvv_arg_type_info (RVV_BASE_unsigned_vector),
+ rvv_arg_type_info (RVV_BASE_vector), rvv_arg_type_info_end};
+
+/* A list of args for void func (scalar_type *, size_t, vector_type)
+ * function. */
+static CONSTEXPR const rvv_arg_type_info scalar_ptr_size_args[]
+ = {rvv_arg_type_info (RVV_BASE_scalar_ptr),
+ rvv_arg_type_info (RVV_BASE_size), rvv_arg_type_info (RVV_BASE_vector),
+ rvv_arg_type_info_end};
+
/* A list of none preds that will be registered for intrinsic functions. */
static CONSTEXPR const predication_type_index none_preds[]
= {PRED_TYPE_none, NUM_PRED_TYPES};
@@ -1455,6 +1481,14 @@ static CONSTEXPR const rvv_op_info iu_shift_vvv_ops
rvv_arg_type_info (RVV_BASE_vector), /* Return type */
shift_vv_args /* Args */};
+/* A static operand information for scalar_type func (vector_type, size_t)
+ * function registration. */
+static CONSTEXPR const rvv_op_info iu_x_s_u_ops
+ = {iu_ops, /* Types */
+ OP_TYPE_vx, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_scalar), /* Return type */
+ v_size_args /* Args */};
+
/* A static operand information for vector_type func (vector_type, size_t)
* function registration. */
static CONSTEXPR const rvv_op_info iu_shift_vvx_ops
@@ -2638,6 +2672,38 @@ static CONSTEXPR const rvv_op_info all_v_vcreate_lmul4_x2_ops
rvv_arg_type_info (RVV_BASE_vlmul_ext_x2), /* Return type */
ext_vcreate_args /* Args */};
+/* A static operand information for vector_type func (const scalar_type *,
+ * size_t) function registration. */
+static CONSTEXPR const rvv_op_info all_v_scalar_const_ptr_size_ops
+ = {all_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_size_args /* Args */};
+
+/* A static operand information for void func (scalar_type *, size_t,
+ * vector_type) function registration. */
+static CONSTEXPR const rvv_op_info all_v_scalar_ptr_size_ops
+ = {all_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_void), /* Return type */
+ scalar_ptr_size_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * index_type) function registration. */
+static CONSTEXPR const rvv_op_info all_v_scalar_const_ptr_index_ops
+ = {all_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_index_args /* Args */};
+
+/* A static operand information for void func (scalar_type *, index_type,
+ * vector_type) function registration. */
+static CONSTEXPR const rvv_op_info all_v_scalar_ptr_index_ops
+ = {all_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_void), /* Return type */
+ scalar_ptr_index_args /* Args */};
+
/* A static operand information for vector_type func (vector_type).
Some ins just supports SEW=32, such as crypto vectol Zvkg extension.
* function registration. */
@@ -2816,6 +2882,10 @@ static function_group_info function_groups[] = {
#define DEF_RVV_FUNCTION(NAME, SHAPE, PREDS, OPS_INFO) \
{#NAME, &bases::NAME, &shapes::SHAPE, PREDS, OPS_INFO, REQUIRED_EXTENSIONS},
#include "riscv-vector-builtins-functions.def"
+#undef DEF_RVV_FUNCTION
+#define DEF_RVV_FUNCTION(NAME, SHAPE, PREDS, OPS_INFO) \
+ {#NAME, &bases::NAME, &shapes::SHAPE, PREDS, OPS_INFO, REQUIRED_EXTENSIONS},
+#include "thead-vector-builtins-functions.def"
};
/* The RVV types, with their built-in
diff --git a/gcc/config/riscv/riscv-vector-builtins.h b/gcc/config/riscv/riscv-vector-builtins.h
index 54c8824ff92..a8ee39a3cb2 100644
--- a/gcc/config/riscv/riscv-vector-builtins.h
+++ b/gcc/config/riscv/riscv-vector-builtins.h
@@ -114,6 +114,7 @@ static const unsigned int CP_WRITE_CSR = 1U << 5;
enum required_ext
{
VECTOR_EXT, /* Vector extension */
+ XTHEADVECTOR_EXT, /* XTheadVector extension */
ZVBB_EXT, /* Cryto vector Zvbb sub-ext */
ZVBB_OR_ZVKB_EXT, /* Cryto vector Zvbb or zvkb sub-ext */
ZVBC_EXT, /* Crypto vector Zvbc sub-ext */
@@ -234,6 +235,8 @@ struct function_group_info
{
case VECTOR_EXT:
return TARGET_VECTOR;
+ case XTHEADVECTOR_EXT:
+ return TARGET_XTHEADVECTOR;
case ZVBB_EXT:
return TARGET_ZVBB;
case ZVBB_OR_ZVKB_EXT:
diff --git a/gcc/config/riscv/t-riscv b/gcc/config/riscv/t-riscv
index 32de6b851c1..38494320d8b 100644
--- a/gcc/config/riscv/t-riscv
+++ b/gcc/config/riscv/t-riscv
@@ -1,6 +1,7 @@
RISCV_BUILTINS_H = $(srcdir)/config/riscv/riscv-vector-builtins.h \
$(srcdir)/config/riscv/riscv-vector-builtins.def \
$(srcdir)/config/riscv/riscv-vector-builtins-functions.def \
+ $(srcdir)/config/riscv/thead-vector-builtins-functions.def \
riscv-vector-type-indexer.gen.def
riscv-builtins.o: $(srcdir)/config/riscv/riscv-builtins.cc $(CONFIG_H) \
diff --git a/gcc/config/riscv/thead-vector-builtins-functions.def b/gcc/config/riscv/thead-vector-builtins-functions.def
new file mode 100644
index 00000000000..fd3ba29bae9
--- /dev/null
+++ b/gcc/config/riscv/thead-vector-builtins-functions.def
@@ -0,0 +1,39 @@
+#ifndef DEF_RVV_FUNCTION
+#define DEF_RVV_FUNCTION(NAME, SHAPE, PREDS, OPS_INFO)
+#endif
+
+#define REQUIRED_EXTENSIONS XTHEADVECTOR_EXT
+DEF_RVV_FUNCTION (vlb, th_loadstore_width, full_preds, all_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (vlh, th_loadstore_width, full_preds, all_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (vlw, th_loadstore_width, full_preds, all_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (vlbu, th_loadstore_width, full_preds, all_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (vlhu, th_loadstore_width, full_preds, all_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (vlwu, th_loadstore_width, full_preds, all_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (vsb, th_loadstore_width, none_m_preds, all_v_scalar_ptr_ops)
+DEF_RVV_FUNCTION (vsh, th_loadstore_width, none_m_preds, all_v_scalar_ptr_ops)
+DEF_RVV_FUNCTION (vsw, th_loadstore_width, none_m_preds, all_v_scalar_ptr_ops)
+DEF_RVV_FUNCTION (vlsb, th_loadstore_width, full_preds, all_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (vlsh, th_loadstore_width, full_preds, all_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (vlsw, th_loadstore_width, full_preds, all_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (vlsbu, th_loadstore_width, full_preds, all_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (vlshu, th_loadstore_width, full_preds, all_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (vlswu, th_loadstore_width, full_preds, all_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (vssb, th_loadstore_width, none_m_preds, all_v_scalar_ptr_size_ops)
+DEF_RVV_FUNCTION (vssh, th_loadstore_width, none_m_preds, all_v_scalar_ptr_size_ops)
+DEF_RVV_FUNCTION (vssw, th_loadstore_width, none_m_preds, all_v_scalar_ptr_size_ops)
+DEF_RVV_FUNCTION (vlxb, th_indexed_loadstore_width, full_preds, all_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (vlxh, th_indexed_loadstore_width, full_preds, all_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (vlxw, th_indexed_loadstore_width, full_preds, all_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (vlxbu, th_indexed_loadstore_width, full_preds, all_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (vlxhu, th_indexed_loadstore_width, full_preds, all_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (vlxwu, th_indexed_loadstore_width, full_preds, all_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (vsxb, th_indexed_loadstore_width, none_m_preds, all_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (vsxh, th_indexed_loadstore_width, none_m_preds, all_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (vsxw, th_indexed_loadstore_width, none_m_preds, all_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (vsuxb, th_indexed_loadstore_width, none_m_preds, all_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (vsuxh, th_indexed_loadstore_width, none_m_preds, all_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (vsuxw, th_indexed_loadstore_width, none_m_preds, all_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (vext_x_v, th_extract, none_preds, iu_x_s_u_ops)
+#undef REQUIRED_EXTENSIONS
+
+#undef DEF_RVV_FUNCTION
diff --git a/gcc/config/riscv/thead-vector.md b/gcc/config/riscv/thead-vector.md
index 696b815252d..0f3700d9269 100644
--- a/gcc/config/riscv/thead-vector.md
+++ b/gcc/config/riscv/thead-vector.md
@@ -1,7 +1,95 @@
(define_c_enum "unspec" [
+ UNSPEC_TH_VLB
+ UNSPEC_TH_VLBU
+ UNSPEC_TH_VLH
+ UNSPEC_TH_VLHU
+ UNSPEC_TH_VLW
+ UNSPEC_TH_VLWU
+
+ UNSPEC_TH_VLSB
+ UNSPEC_TH_VLSBU
+ UNSPEC_TH_VLSH
+ UNSPEC_TH_VLSHU
+ UNSPEC_TH_VLSW
+ UNSPEC_TH_VLSWU
+
+ UNSPEC_TH_VLXB
+ UNSPEC_TH_VLXBU
+ UNSPEC_TH_VLXH
+ UNSPEC_TH_VLXHU
+ UNSPEC_TH_VLXW
+ UNSPEC_TH_VLXWU
+
+ UNSPEC_TH_VSUXB
+ UNSPEC_TH_VSUXH
+ UNSPEC_TH_VSUXW
+
UNSPEC_TH_VWLDST
])
+(define_int_iterator UNSPEC_TH_VLMEM_OP [
+ UNSPEC_TH_VLB UNSPEC_TH_VLBU
+ UNSPEC_TH_VLH UNSPEC_TH_VLHU
+ UNSPEC_TH_VLW UNSPEC_TH_VLWU
+])
+
+(define_int_iterator UNSPEC_TH_VLSMEM_OP [
+ UNSPEC_TH_VLSB UNSPEC_TH_VLSBU
+ UNSPEC_TH_VLSH UNSPEC_TH_VLSHU
+ UNSPEC_TH_VLSW UNSPEC_TH_VLSWU
+])
+
+(define_int_iterator UNSPEC_TH_VLXMEM_OP [
+ UNSPEC_TH_VLXB UNSPEC_TH_VLXBU
+ UNSPEC_TH_VLXH UNSPEC_TH_VLXHU
+ UNSPEC_TH_VLXW UNSPEC_TH_VLXWU
+])
+
+(define_int_attr vlmem_op_attr [
+ (UNSPEC_TH_VLB "b") (UNSPEC_TH_VLBU "bu")
+ (UNSPEC_TH_VLH "h") (UNSPEC_TH_VLHU "hu")
+ (UNSPEC_TH_VLW "w") (UNSPEC_TH_VLWU "wu")
+ (UNSPEC_TH_VLSB "b") (UNSPEC_TH_VLSBU "bu")
+ (UNSPEC_TH_VLSH "h") (UNSPEC_TH_VLSHU "hu")
+ (UNSPEC_TH_VLSW "w") (UNSPEC_TH_VLSWU "wu")
+ (UNSPEC_TH_VLXB "b") (UNSPEC_TH_VLXBU "bu")
+ (UNSPEC_TH_VLXH "h") (UNSPEC_TH_VLXHU "hu")
+ (UNSPEC_TH_VLXW "w") (UNSPEC_TH_VLXWU "wu")
+ (UNSPEC_TH_VSUXB "b")
+ (UNSPEC_TH_VSUXH "h")
+ (UNSPEC_TH_VSUXW "w")
+])
+
+(define_int_attr vlmem_order_attr [
+ (UNSPEC_TH_VLXB "")
+ (UNSPEC_TH_VLXH "")
+ (UNSPEC_TH_VLXW "")
+ (UNSPEC_TH_VSUXB "u")
+ (UNSPEC_TH_VSUXH "u")
+ (UNSPEC_TH_VSUXW "u")
+])
+
+(define_int_iterator UNSPEC_TH_VSMEM_OP [
+ UNSPEC_TH_VLB
+ UNSPEC_TH_VLH
+ UNSPEC_TH_VLW
+])
+
+(define_int_iterator UNSPEC_TH_VSSMEM_OP [
+ UNSPEC_TH_VLSB
+ UNSPEC_TH_VLSH
+ UNSPEC_TH_VLSW
+])
+
+(define_int_iterator UNSPEC_TH_VSXMEM_OP [
+ UNSPEC_TH_VLXB
+ UNSPEC_TH_VLXH
+ UNSPEC_TH_VLXW
+ UNSPEC_TH_VSUXB
+ UNSPEC_TH_VSUXH
+ UNSPEC_TH_VSUXW
+])
+
(define_mode_iterator V_VLS_VT [V VLS VT])
(define_mode_iterator V_VB_VLS_VT [V VB VLS VT])
@@ -100,3 +188,168 @@
}
[(set_attr "type" "vldm,vstm,vmalu,vmalu,vmalu")
(set_attr "mode" "<MODE>")])
+
+(define_expand "@pred_mov_width<vlmem_op_attr><mode>"
+ [(set (match_operand:V_VLS 0 "nonimmediate_operand")
+ (if_then_else:V_VLS
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand")
+ (match_operand 4 "vector_length_operand")
+ (match_operand 5 "const_int_operand")
+ (match_operand 6 "const_int_operand")
+ (match_operand 7 "const_int_operand")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLMEM_OP)
+ (match_operand:V_VLS 3 "vector_move_operand")
+ (match_operand:V_VLS 2 "vector_merge_operand")))]
+ "TARGET_XTHEADVECTOR"
+ {})
+
+(define_insn_and_split "*pred_mov_width<vlmem_op_attr><mode>"
+ [(set (match_operand:V_VLS 0 "nonimmediate_operand" "=vr, vr, vd, m, vr, vr")
+ (if_then_else:V_VLS
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1, Wc1, vm, vmWc1, Wc1, Wc1")
+ (match_operand 4 "vector_length_operand" " rK, rK, rK, rK, rK, rK")
+ (match_operand 5 "const_int_operand" " i, i, i, i, i, i")
+ (match_operand 6 "const_int_operand" " i, i, i, i, i, i")
+ (match_operand 7 "const_int_operand" " i, i, i, i, i, i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLMEM_OP)
+ (match_operand:V_VLS 3 "reg_or_mem_operand" " m, m, m, vr, vr, vr")
+ (match_operand:V_VLS 2 "vector_merge_operand" " 0, vu, vu, vu, vu, 0")))]
+ "(TARGET_XTHEADVECTOR
+ && (register_operand (operands[0], <MODE>mode)
+ || register_operand (operands[3], <MODE>mode)))"
+ "@
+ vl<vlmem_op_attr>.v\t%0,%3%p1
+ vl<vlmem_op_attr>.v\t%0,%3
+ vl<vlmem_op_attr>.v\t%0,%3,%1.t
+ vs<vlmem_op_attr>.v\t%3,%0%p1
+ vmv.v.v\t%0,%3
+ vmv.v.v\t%0,%3"
+ "&& register_operand (operands[0], <MODE>mode)
+ && register_operand (operands[3], <MODE>mode)
+ && satisfies_constraint_vu (operands[2])
+ && INTVAL (operands[7]) == riscv_vector::VLMAX"
+ [(set (match_dup 0) (match_dup 3))]
+ ""
+ [(set_attr "type" "vlde,vlde,vlde,vste,vimov,vimov")
+ (set_attr "mode" "<MODE>")])
+
+(define_insn "@pred_store_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "memory_operand" "+m")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1")
+ (match_operand 3 "vector_length_operand" " rK")
+ (match_operand 4 "const_int_operand" " i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VSMEM_OP)
+ (match_operand:VI 2 "register_operand" " vr")
+ (match_dup 0)))]
+ "TARGET_XTHEADVECTOR"
+ "vs<vlmem_op_attr>.v\t%2,%0%p1"
+ [(set_attr "type" "vste")
+ (set_attr "mode" "<MODE>")
+ (set (attr "avl_type_idx") (const_int 4))
+ (set_attr "vl_op_idx" "3")])
+
+(define_insn "@pred_strided_load_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "register_operand" "=vr, vr, vd")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1, Wc1, vm")
+ (match_operand 5 "vector_length_operand" " rK, rK, rK")
+ (match_operand 6 "const_int_operand" " i, i, i")
+ (match_operand 7 "const_int_operand" " i, i, i")
+ (match_operand 8 "const_int_operand" " i, i, i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLSMEM_OP)
+ (unspec:VI
+ [(match_operand:VI 3 "memory_operand" " m, m, m")
+ (match_operand 4 "pmode_reg_or_0_operand" " rJ, rJ, rJ")] UNSPEC_TH_VLSMEM_OP)
+ (match_operand:VI 2 "vector_merge_operand" " 0, vu, vu")))]
+ "TARGET_XTHEADVECTOR"
+ "vls<vlmem_op_attr>.v\t%0,%3,%z4%p1"
+ [(set_attr "type" "vlds")
+ (set_attr "mode" "<MODE>")])
+
+(define_insn "@pred_strided_store_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "memory_operand" "+m")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1")
+ (match_operand 4 "vector_length_operand" " rK")
+ (match_operand 5 "const_int_operand" " i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VSSMEM_OP)
+ (unspec:VI
+ [(match_operand 2 "pmode_reg_or_0_operand" " rJ")
+ (match_operand:VI 3 "register_operand" " vr")] UNSPEC_TH_VSSMEM_OP)
+ (match_dup 0)))]
+ "TARGET_XTHEADVECTOR"
+ "vss<vlmem_op_attr>.v\t%3,%0,%z2%p1"
+ [(set_attr "type" "vsts")
+ (set_attr "mode" "<MODE>")
+ (set (attr "avl_type_idx") (const_int 5))])
+
+(define_insn "@pred_indexed_load_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "register_operand" "=vd, vr,vd, vr")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" " vm,Wc1,vm,Wc1")
+ (match_operand 5 "vector_length_operand" " rK, rK,rK, rK")
+ (match_operand 6 "const_int_operand" " i, i, i, i")
+ (match_operand 7 "const_int_operand" " i, i, i, i")
+ (match_operand 8 "const_int_operand" " i, i, i, i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLXMEM_OP)
+ (unspec:VI
+ [(match_operand 3 "pmode_reg_or_0_operand" " rJ, rJ,rJ, rJ")
+ (mem:BLK (scratch))
+ (match_operand:VI 4 "register_operand" " vr, vr,vr, vr")] UNSPEC_TH_VLXMEM_OP)
+ (match_operand:VI 2 "vector_merge_operand" " vu, vu, 0, 0")))]
+ "TARGET_XTHEADVECTOR"
+ "vlx<vlmem_op_attr>.v\t%0,(%z3),%4%p1"
+ [(set_attr "type" "vldux")
+ (set_attr "mode" "<MODE>")])
+
+(define_insn "@pred_indexed_<vlmem_order_attr>store_width<vlmem_op_attr><mode>"
+ [(set (mem:BLK (scratch))
+ (unspec:BLK
+ [(unspec:<VM>
+ [(match_operand:<VM> 0 "vector_mask_operand" "vmWc1")
+ (match_operand 4 "vector_length_operand" " rK")
+ (match_operand 5 "const_int_operand" " i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VSXMEM_OP)
+ (match_operand 1 "pmode_reg_or_0_operand" " rJ")
+ (match_operand:VI 2 "register_operand" " vr")
+ (match_operand:VI 3 "register_operand" " vr")] UNSPEC_TH_VSXMEM_OP))]
+ "TARGET_XTHEADVECTOR"
+ "vs<vlmem_order_attr>x<vlmem_op_attr>.v\t%3,(%z1),%2%p0"
+ [(set_attr "type" "vstux")
+ (set_attr "mode" "<MODE>")])
+
+(define_expand "@pred_th_extract<mode>"
+ [(set (match_operand:<VEL> 0 "register_operand")
+ (unspec:<VEL>
+ [(vec_select:<VEL>
+ (match_operand:V_VLSI 1 "register_operand")
+ (parallel [(match_operand:DI 2 "register_operand" "r")]))
+ (reg:SI VTYPE_REGNUM)] UNSPEC_VPREDICATE))]
+ "TARGET_XTHEADVECTOR"
+{})
+
+(define_insn "*pred_th_extract<mode>"
+ [(set (match_operand:<VEL> 0 "register_operand" "=r")
+ (unspec:<VEL>
+ [(vec_select:<VEL>
+ (match_operand:V_VLSI 1 "register_operand" "vr")
+ (parallel [(match_operand:DI 2 "register_operand" "r")]))
+ (reg:SI VTYPE_REGNUM)] UNSPEC_VPREDICATE))]
+ "TARGET_XTHEADVECTOR"
+ "vext.x.v\t%0,%1,%2"
+ [(set_attr "type" "vimovvx")
+ (set_attr "mode" "<MODE>")])
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c
new file mode 100644
index 00000000000..3c12c124597
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_th_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+
+** th.vadd\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+
+** th.vsb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out)
+{
+ vint32m1_t v = __riscv_th_vlb_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlb_v_i32m1_tu (v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vv_i32m1 (v2, v2, 4);
+ vint32m1_t v4 = __riscv_vadd_vv_i32m1_tu (v3, v2, v2, 4);
+ __riscv_th_vsb_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlb.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+
+** th.vadd\.vv\tv[1-9][0-9]?,\s*v[0-9]+,\s*v[0-9]+,\s*v0.t
+** th.vsb.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlb_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlb_v_i32m1_m (mask, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vv_i32m1 (v2, v2, 4);
+ vint32m1_t v4 = __riscv_vadd_vv_i32m1_m (mask, v3, v3, 4);
+ __riscv_th_vsb_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlb.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+
+** th.vadd\.vv\tv[1-9][0-9]?,\s*v[0-9]+,\s*v[0-9]+,\s*v0.t
+** th.vsb.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlb_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlb_v_i32m1_tumu (mask, v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vv_i32m1 (v2, v2, 4);
+ vint32m1_t v4 = __riscv_vadd_vv_i32m1_tumu (mask, v3, v2, v2, 4);
+ __riscv_th_vsb_v_i32m1 (out, v4, 4);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c
new file mode 100644
index 00000000000..30bef369375
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_th_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlbu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlbu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vsb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, uint32_t x)
+{
+ vuint32m1_t v = __riscv_th_vlbu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlbu_v_u32m1_tu (v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tu (v3, v2, -16, 4);
+ __riscv_th_vsb_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlbu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsb.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlbu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlbu_v_u32m1_m (mask, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_m (mask, v3, -16, 4);
+ __riscv_th_vsb_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlbu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlbu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsb.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlbu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlbu_v_u32m1_tumu (mask, v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tumu (mask, v3, v2, -16, 4);
+ __riscv_th_vsb_v_u32m1 (out, v4, 4);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c
new file mode 100644
index 00000000000..3c8b5ccc16b
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_th_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vsh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, int32_t x)
+{
+ vint32m1_t v = __riscv_th_vlh_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlh_v_i32m1_tu (v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, -16, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_tu (v3, v2, -16, 4);
+ __riscv_th_vsh_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlh.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsh.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, int32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlh_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlh_v_i32m1_m (mask, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, -16, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_m (mask, v3, -16, 4);
+ __riscv_th_vsh_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlh.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsh.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, int32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlh_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlh_v_i32m1_tumu (mask, v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, -16, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_tumu (mask, v3, v2, -16, 4);
+ __riscv_th_vsh_v_i32m1 (out, v4, 4);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c
new file mode 100644
index 00000000000..b7c00404f18
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_th_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlhu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlhu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vsh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, uint32_t x)
+{
+ vuint32m1_t v = __riscv_th_vlhu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlhu_v_u32m1_tu (v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tu (v3, v2, -16, 4);
+ __riscv_th_vsh_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlhu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsh.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlhu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlhu_v_u32m1_m (mask, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_m (mask, v3, -16, 4);
+ __riscv_th_vsh_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlhu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlhu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsh.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlhu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlhu_v_u32m1_tumu (mask, v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tumu (mask, v3, v2, -16, 4);
+ __riscv_th_vsh_v_u32m1 (out, v4, 4);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c
new file mode 100644
index 00000000000..17a53012acf
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_th_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vx\tv[0-9]+,\s*v[0-9]+,\s*[a-x0-9]+
+** th.vadd\.vx\tv[0-9]+,\s*v[0-9]+,\s*[a-x0-9]+
+** th.vsw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, int32_t x)
+{
+ vint32m1_t v = __riscv_th_vlw_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlw_v_i32m1_tu (v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, x, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_tu (v3, v2, x, 4);
+ __riscv_th_vsw_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlw.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vx\tv[0-9]+,\s*v[0-9]+,\s*[a-x0-9]+
+** th.vadd\.vx\tv[1-9][0-9]?,\s*v[0-9]+,\s*[a-x0-9]+,\s*v0.t
+** th.vsw.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, int32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlw_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlw_v_i32m1_m (mask, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, x, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_m (mask, v3, x, 4);
+ __riscv_th_vsw_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlw.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vx\tv[0-9]+,\s*v[0-9]+,\s*[a-x0-9]+
+** th.vadd\.vx\tv[1-9][0-9]?,\s*v[0-9]+,\s*[a-x0-9]+,\s*v0.t
+** th.vsw.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, int32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlw_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlw_v_i32m1_tumu (mask, v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, x, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_tumu (mask, v3, v2, x, 4);
+ __riscv_th_vsw_v_i32m1 (out, v4, 4);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c
new file mode 100644
index 00000000000..b187cfc852b
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_th_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlwu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlwu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vsw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, uint32_t x)
+{
+ vuint32m1_t v = __riscv_th_vlwu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlwu_v_u32m1_tu (v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tu (v3, v2, -16, 4);
+ __riscv_th_vsw_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlwu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsw.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlwu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlwu_v_u32m1_m (mask, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_m (mask, v3, -16, 4);
+ __riscv_th_vsw_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlwu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlwu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsw.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlwu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlwu_v_u32m1_tumu (mask, v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tumu (mask, v3, v2, -16, 4);
+ __riscv_th_vsw_v_u32m1 (out, v4, 4);
+}
--
2.17.1
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re：Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
2024-01-11 12:13 Re：[PATCH " juzhe.zhong
@ 2024-01-11 12:18 ` joshua
0 siblings, 0 replies; 17+ messages in thread
From: joshua @ 2024-01-11 12:18 UTC (permalink / raw)
To: juzhe.zhong, gcc-patches
Cc: Jim Wilson, palmer, andrew, philipp.tomsich, jeffreyalaw,
christoph.muellner, jinma, cooper.qu
No, we have debugged and the code won't enter this autovec pattern.
The root cause is CSE, in a very early pass.
What I sent to you just now is the dump result which shows the difference.
------------------------------------------------------------------
发件人：juzhe.zhong@rivai.ai <juzhe.zhong@rivai.ai>
发送时间：2024年1月11日(星期四) 20:13
收件人："cooper.joshua"<cooper.joshua@linux.alibaba.com>; "gcc-patches"<gcc-patches@gcc.gnu.org>
抄 送：Jim Wilson<jim.wilson.gcc@gmail.com>; palmer<palmer@dabbelt.com>; andrew<andrew@sifive.com>; "philipp.tomsich"<philipp.tomsich@vrull.eu>; jeffreyalaw<jeffreyalaw@gmail.com>; "christoph.muellner"<christoph.muellner@vrull.eu>; jinma<jinma@linux.alibaba.com>; "cooper.qu"<cooper.qu@linux.alibaba.com>
主 题：Re: Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
I guess it is because of patterns from autovectorization pattern:
;; Use define_insn_and_split to define vsext.vf2/vzext.vf2 will help
;; to combine instructions as below:
;; vsext.vf2 + vsext.vf2 + vadd.vv ==> vwadd.vv
(define_insn_and_split "<optab><v_double_trunc><mode>2"
[(set (match_operand:VWEXTI 0 "register_operand" "=&vr")
(any_extend:VWEXTI
(match_operand:<V_DOUBLE_TRUNC> 1 "register_operand" "vr")))]
"TARGET_VECTOR && can_create_pseudo_p ()"
"#"
"&& 1"
[(const_int 0)]
{
insn_code icode = code_for_pred_vf2 (<CODE>, <MODE>mode);
riscv_vector::emit_vlmax_insn (icode, riscv_vector::UNARY_OP, operands);
DONE;
}
[(set_attr "type" "vext")
(set_attr "mode" "<MODE>")])
juzhe.zhong@rivai.ai
发件人： joshua
发送时间： 2024-01-11 20:05
收件人： juzhe.zhong@rivai.ai; gcc-patches
抄送： Jim Wilson; palmer; andrew; philipp.tomsich; jeffreyalaw; christoph.muellner; jinma; cooper.qu
主题： Re：Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
without CSE:
(insn 16 15 17 2 (set (reg/v:RVVM2HI 137 [ output_var_0 ])
(if_then_else:RVVM2HI (unspec:RVVMF8BI [
(const_vector:RVVMF8BI repeat [
(const_int 1 [0x1])
])
(reg:DI 146)
(const_int 2 [0x2]) repeated x2
(const_int 0 [0])
(reg:SI 66 vl)
(reg:SI 67 vtype)
] UNSPEC_VPREDICATE)
(mult:RVVM2HI (sign_extend:RVVM2HI (reg/v:RVVM1QI 136 [ op1 ]))
(sign_extend:RVVM2HI (vec_duplicate:RVVM1QI (reg:QI 147))))
(unspec:RVVM2HI [
(reg:SI 0 zero)
] UNSPEC_VUNDEF))) {pred_dual_widen_mulsrvvm2hi_scalar}
with CSE:
(insn 16 15 17 2 (set (reg/v:RVVM2HI 137 [ output_var_0 ])
(if_then_else:RVVM2HI (unspec:RVVMF8BI [
(const_vector:RVVMF8BI repeat [
(const_int 1 [0x1])
])
(reg:DI 146)
(const_int 2 [0x2]) repeated x2
(const_int 0 [0])
(reg:SI 66 vl)
(reg:SI 67 vtype)
] UNSPEC_VPREDICATE)
(sign_extend:RVVM2HI (reg/v:RVVM1QI 136 [ op1 ]))
(unspec:RVVM2HI [
(reg:SI 0 zero)
] UNSPEC_VUNDEF))) {pred_extendrvvm2hi_vf2}
------------------------------------------------------------------
发件人：juzhe.zhong@rivai.ai <juzhe.zhong@rivai.ai>
发送时间：2024年1月11日(星期四) 17:32
收件人："cooper.joshua"<cooper.joshua@linux.alibaba.com>; "gcc-patches"<gcc-patches@gcc.gnu.org>
抄 送：Jim Wilson<jim.wilson.gcc@gmail.com>; palmer<palmer@dabbelt.com>; andrew<andrew@sifive.com>; "philipp.tomsich"<philipp.tomsich@vrull.eu>; jeffreyalaw<jeffreyalaw@gmail.com>; "christoph.muellner"<christoph.muellner@vrull.eu>; jinma<jinma@linux.alibaba.com>; "cooper.qu"<cooper.qu@linux.alibaba.com>
主 题：Re: Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
Ok. Let's hold on "RISC-V: Handle differences between XTheadvector and Vector" patch
until you can reproduce the issue for me.
juzhe.zhong@rivai.ai
发件人： joshua
发送时间： 2024-01-11 17:29
收件人： juzhe.zhong@rivai.ai; gcc-patches
抄送： Jim Wilson; palmer; andrew; philipp.tomsich; jeffreyalaw; christoph.muellner; jinma; cooper.qu
主题： Re：Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
The sext/zext issue is not related to xtheadvector-special patterns.
I added !TARGET_XTHEADVECTOR to sext/zext patterns in
"RISC-V: Handle differences between XTheadvector and Vector"
That is caused by the vwmul pattern, but I cannot reproduce it right now.
------------------------------------------------------------------
发件人：juzhe.zhong@rivai.ai <juzhe.zhong@rivai.ai>
发送时间：2024年1月11日(星期四) 17:24
收件人："cooper.joshua"<cooper.joshua@linux.alibaba.com>; "gcc-patches"<gcc-patches@gcc.gnu.org>
抄 送：Jim Wilson<jim.wilson.gcc@gmail.com>; palmer<palmer@dabbelt.com>; andrew<andrew@sifive.com>; "philipp.tomsich"<philipp.tomsich@vrull.eu>; jeffreyalaw<jeffreyalaw@gmail.com>; "christoph.muellner"<christoph.muellner@vrull.eu>; jinma<jinma@linux.alibaba.com>; "cooper.qu"<cooper.qu@linux.alibaba.com>
主 题：Re: Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
I don't see any patterns are possible CSE into sext/zext patterns:
+(define_expand "@pred_mov_width<vlmem_op_attr><mode>"
+ [(set (match_operand:V_VLS 0 "nonimmediate_operand")
+ (if_then_else:V_VLS
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand")
+ (match_operand 4 "vector_length_operand")
+ (match_operand 5 "const_int_operand")
+ (match_operand 6 "const_int_operand")
+ (match_operand 7 "const_int_operand")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLMEM_OP)
+ (match_operand:V_VLS 3 "vector_move_operand")
+ (match_operand:V_VLS 2 "vector_merge_operand")))]
+ "TARGET_XTHEADVECTOR"
+ {})
+
+(define_insn_and_split "*pred_mov_width<vlmem_op_attr><mode>"
+ [(set (match_operand:V_VLS 0 "nonimmediate_operand" "=vr, vr, vd, m, vr, vr")
+ (if_then_else:V_VLS
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1, Wc1, vm, vmWc1, Wc1, Wc1")
+ (match_operand 4 "vector_length_operand" " rK, rK, rK, rK, rK, rK")
+ (match_operand 5 "const_int_operand" " i, i, i, i, i, i")
+ (match_operand 6 "const_int_operand" " i, i, i, i, i, i")
+ (match_operand 7 "const_int_operand" " i, i, i, i, i, i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLMEM_OP)
+ (match_operand:V_VLS 3 "reg_or_mem_operand" " m, m, m, vr, vr, vr")
+ (match_operand:V_VLS 2 "vector_merge_operand" " 0, vu, vu, vu, vu, 0")))]
+ "(TARGET_XTHEADVECTOR
+ && (register_operand (operands[0], <MODE>mode)
+ || register_operand (operands[3], <MODE>mode)))"
+ "@
+ vl<vlmem_op_attr>.v\t%0,%3%p1
+ vl<vlmem_op_attr>.v\t%0,%3
+ vl<vlmem_op_attr>.v\t%0,%3,%1.t
+ vs<vlmem_op_attr>.v\t%3,%0%p1
+ vmv.v.v\t%0,%3
+ vmv.v.v\t%0,%3"
+ "&& register_operand (operands[0], <MODE>mode)
+ && register_operand (operands[3], <MODE>mode)
+ && satisfies_constraint_vu (operands[2])
+ && INTVAL (operands[7]) == riscv_vector::VLMAX"
+ [(set (match_dup 0) (match_dup 3))]
+ ""
+ [(set_attr "type" "vlde,vlde,vlde,vste,vimov,vimov")
+ (set_attr "mode" "<MODE>")])
+
+(define_insn "@pred_store_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "memory_operand" "+m")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1")
+ (match_operand 3 "vector_length_operand" " rK")
+ (match_operand 4 "const_int_operand" " i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VSMEM_OP)
+ (match_operand:VI 2 "register_operand" " vr")
+ (match_dup 0)))]
+ "TARGET_XTHEADVECTOR"
+ "vs<vlmem_op_attr>.v\t%2,%0%p1"
+ [(set_attr "type" "vste")
+ (set_attr "mode" "<MODE>")
+ (set (attr "avl_type_idx") (const_int 4))
+ (set_attr "vl_op_idx" "3")])
+
+(define_insn "@pred_strided_load_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "register_operand" "=vr, vr, vd")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1, Wc1, vm")
+ (match_operand 5 "vector_length_operand" " rK, rK, rK")
+ (match_operand 6 "const_int_operand" " i, i, i")
+ (match_operand 7 "const_int_operand" " i, i, i")
+ (match_operand 8 "const_int_operand" " i, i, i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLSMEM_OP)
+ (unspec:VI
+ [(match_operand:VI 3 "memory_operand" " m, m, m")
+ (match_operand 4 "pmode_reg_or_0_operand" " rJ, rJ, rJ")] UNSPEC_TH_VLSMEM_OP)
+ (match_operand:VI 2 "vector_merge_operand" " 0, vu, vu")))]
+ "TARGET_XTHEADVECTOR"
+ "vls<vlmem_op_attr>.v\t%0,%3,%z4%p1"
+ [(set_attr "type" "vlds")
+ (set_attr "mode" "<MODE>")])
+
+(define_insn "@pred_strided_store_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "memory_operand" "+m")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1")
+ (match_operand 4 "vector_length_operand" " rK")
+ (match_operand 5 "const_int_operand" " i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VSSMEM_OP)
+ (unspec:VI
+ [(match_operand 2 "pmode_reg_or_0_operand" " rJ")
+ (match_operand:VI 3 "register_operand" " vr")] UNSPEC_TH_VSSMEM_OP)
+ (match_dup 0)))]
+ "TARGET_XTHEADVECTOR"
+ "vss<vlmem_op_attr>.v\t%3,%0,%z2%p1"
+ [(set_attr "type" "vsts")
+ (set_attr "mode" "<MODE>")
+ (set (attr "avl_type_idx") (const_int 5))])
+
+(define_insn "@pred_indexed_load_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "register_operand" "=vd, vr,vd, vr")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" " vm,Wc1,vm,Wc1")
+ (match_operand 5 "vector_length_operand" " rK, rK,rK, rK")
+ (match_operand 6 "const_int_operand" " i, i, i, i")
+ (match_operand 7 "const_int_operand" " i, i, i, i")
+ (match_operand 8 "const_int_operand" " i, i, i, i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLXMEM_OP)
+ (unspec:VI
+ [(match_operand 3 "pmode_reg_or_0_operand" " rJ, rJ,rJ, rJ")
+ (mem:BLK (scratch))
+ (match_operand:VI 4 "register_operand" " vr, vr,vr, vr")] UNSPEC_TH_VLXMEM_OP)
+ (match_operand:VI 2 "vector_merge_operand" " vu, vu, 0, 0")))]
+ "TARGET_XTHEADVECTOR"
+ "vlx<vlmem_op_attr>.v\t%0,(%z3),%4%p1"
+ [(set_attr "type" "vldux")
+ (set_attr "mode" "<MODE>")])
+
+(define_insn "@pred_indexed_<vlmem_order_attr>store_width<vlmem_op_attr><mode>"
+ [(set (mem:BLK (scratch))
+ (unspec:BLK
+ [(unspec:<VM>
+ [(match_operand:<VM> 0 "vector_mask_operand" "vmWc1")
+ (match_operand 4 "vector_length_operand" " rK")
+ (match_operand 5 "const_int_operand" " i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VSXMEM_OP)
+ (match_operand 1 "pmode_reg_or_0_operand" " rJ")
+ (match_operand:VI 2 "register_operand" " vr")
+ (match_operand:VI 3 "register_operand" " vr")] UNSPEC_TH_VSXMEM_OP))]
+ "TARGET_XTHEADVECTOR"
+ "vs<vlmem_order_attr>x<vlmem_op_attr>.v\t%3,(%z1),%2%p0"
+ [(set_attr "type" "vstux")
+ (set_attr "mode" "<MODE>")])
+
+(define_expand "@pred_th_extract<mode>"
+ [(set (match_operand:<VEL> 0 "register_operand")
+ (unspec:<VEL>
+ [(vec_select:<VEL>
+ (match_operand:V_VLSI 1 "register_operand")
+ (parallel [(match_operand:DI 2 "register_operand" "r")]))
+ (reg:SI VTYPE_REGNUM)] UNSPEC_VPREDICATE))]
+ "TARGET_XTHEADVECTOR"
+{})
+
+(define_insn "*pred_th_extract<mode>"
+ [(set (match_operand:<VEL> 0 "register_operand" "=r")
+ (unspec:<VEL>
+ [(vec_select:<VEL>
+ (match_operand:V_VLSI 1 "register_operand" "vr")
+ (parallel [(match_operand:DI 2 "register_operand" "r")]))
+ (reg:SI VTYPE_REGNUM)] UNSPEC_VPREDICATE))]
+ "TARGET_XTHEADVECTOR"
+ "vext.x.v\t%0,%1,%2"
+ [(set_attr "type" "vimovvx")
+ (set_attr "mode" "<MODE>")])
juzhe.zhong@rivai.ai
发件人： joshua
发送时间： 2024-01-11 17:21
收件人： juzhe.zhong@rivai.ai; gcc-patches
抄送： Jim Wilson; palmer; andrew; philipp.tomsich; jeffreyalaw; christoph.muellner; jinma; cooper.qu
主题： Re：Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
"I didn't see theadvector-specific extension patterns. Could you show me?"
They are all in the file thead-vector.md.
For the sext/zext issue, perhaps I need some time to reproduce that optimization,
but I can clearly remember it is related to vwmul.
------------------------------------------------------------------
发件人：juzhe.zhong@rivai.ai <juzhe.zhong@rivai.ai>
发送时间：2024年1月11日(星期四) 17:17
收件人："cooper.joshua"<cooper.joshua@linux.alibaba.com>; "gcc-patches"<gcc-patches@gcc.gnu.org>
抄 送：Jim Wilson<jim.wilson.gcc@gmail.com>; palmer<palmer@dabbelt.com>; andrew<andrew@sifive.com>; "philipp.tomsich"<philipp.tomsich@vrull.eu>; jeffreyalaw<jeffreyalaw@gmail.com>; "christoph.muellner"<christoph.muellner@vrull.eu>; jinma<jinma@linux.alibaba.com>; "cooper.qu"<cooper.qu@linux.alibaba.com>
主 题：Re: Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
You mean which pattern optimized sext/vzext pattern?
I didn't see theadvector-specific extension patterns. Could you show me?
juzhe.zhong@rivai.ai
发件人： joshua
发送时间： 2024-01-11 17:14
收件人： juzhe.zhong@rivai.ai; gcc-patches
抄送： Jim Wilson; palmer; andrew; philipp.tomsich; jeffreyalaw; christoph.muellner; jinma; cooper.qu
主题： Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
To be specific, in CSE pass, the initial pattern will be optimized into the sext/zext pattern.
------------------------------------------------------------------
发件人：joshua <cooper.joshua@linux.alibaba.com>
发送时间：2024年1月11日(星期四) 17:11
收件人："juzhe.zhong@rivai.ai"<juzhe.zhong@rivai.ai>; "gcc-patches"<gcc-patches@gcc.gnu.org>
抄 送：Jim Wilson<jim.wilson.gcc@gmail.com>; palmer<palmer@dabbelt.com>; andrew<andrew@sifive.com>; "philipp.tomsich"<philipp.tomsich@vrull.eu>; jeffreyalaw<jeffreyalaw@gmail.com>; "christoph.muellner"<christoph.muellner@vrull.eu>; jinma<jinma@linux.alibaba.com>; "cooper.qu"<cooper.qu@linux.alibaba.com>
主 题：Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
sext/zext will be generated in O2 even without corresponding intrinsics.
------------------------------------------------------------------
发件人：juzhe.zhong@rivai.ai <juzhe.zhong@rivai.ai>
发送时间：2024年1月11日(星期四) 17:07
收件人："cooper.joshua"<cooper.joshua@linux.alibaba.com>; "gcc-patches"<gcc-patches@gcc.gnu.org>
抄 送：Jim Wilson<jim.wilson.gcc@gmail.com>; palmer<palmer@dabbelt.com>; andrew<andrew@sifive.com>; "philipp.tomsich"<philipp.tomsich@vrull.eu>; jeffreyalaw<jeffreyalaw@gmail.com>; "christoph.muellner"<christoph.muellner@vrull.eu>; "cooper.joshua"<cooper.joshua@linux.alibaba.com>; jinma<jinma@linux.alibaba.com>; "cooper.qu"<cooper.qu@linux.alibaba.com>
主 题：Re: [PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
enum required_ext
{
VECTOR_EXT, /* Vector extension */
+ XTHEADVECTOR_EXT, /* XTheadVector extension */
ZVBB_EXT, /* Cryto vector Zvbb sub-ext */
ZVBB_OR_ZVKB_EXT, /* Cryto vector Zvbb or zvkb sub-ext */
ZVBC_EXT, /* Crypto vector Zvbc sub-ext */
Add theadvector to the end of the enum.
+ case XTHEADVECTOR_EXT:
+ return TARGET_XTHEADVECTOR;
Same.
+ "&& register_operand (operands[0], <MODE>mode)
+ && register_operand (operands[3], <MODE>mode)
+ && satisfies_constraint_vu (operands[2])
+ && INTVAL (operands[7]) == riscv_vector::VLMAX"
You can use whole_reg_to_reg_move_p
Btw, I review again : RISC-V: Handle differences between XTheadvector and Vector
(any_extend:VWEXTI (match_operand:<V_DOUBLE_TRUNC> 3 "register_operand" "W21,W21,W21,W21,W42,W42,W42,W42,W84,W84,W84,W84, vr, vr")) (match_operand:VWEXTI 2 "vector_merge_operand" " vu, vu, 0, 0, vu, vu, 0, 0, vu, vu, 0, 0, vu, 0")))]- "TARGET_VECTOR"+ "TARGET_VECTOR && !TARGET_XTHEADVECTOR" "v<sz>ext.vf2\t%0,%3%p1" [(set_attr "type" "vext") (set_attr "mode" "<MODE>")@@ -3713,7 +3744,7 @@ (any_extend:VQEXTI (match_operand:<V_QUAD_TRUNC> 3 "register_operand" "W43,W43,W43,W43,W86,W86,W86,W86, vr, vr")) (match_operand:VQEXTI 2 "vector_merge_operand" " vu, vu, 0, 0, vu, vu, 0, 0, vu, 0")))]- "TARGET_VECTOR"+ "TARGET_VECTOR && !TARGET_XTHEADVECTOR" "v<sz>ext.vf4\t%0,%3%p1" [(set_attr "type" "vext") (set_attr "mode" "<MODE>")@@ -3734,7 +3765,7 @@ (any_extend:VOEXTI (match_operand:<V_OCT_TRUNC> 3 "register_operand" "W87,W87,W87,W87, vr, vr")) (match_operand:VOEXTI 2 "vector_merge_operand" " vu, vu, 0, 0, vu, 0")))]- "TARGET_VECTOR"+ "TARGET_VECTOR && !TARGET_XTHEADVECTOR" "v<sz>ext.vf8\t%0,%3%p1" [(set_attr "type" "vext") (set_attr "mode" "<MODE>")
Why do you add these !TARGERT_XTHEADVECRTOR ?
juzhe.zhong@rivai.ai
From: Jun Sha (Joshua)
Date: 2024-01-11 16:46
To: gcc-patches
CC: jim.wilson.gcc; palmer; andrew; philipp.tomsich; jeffreyalaw; christoph.muellner; juzhe.zhong; Jun Sha (Joshua); Jin Ma; Xianmiao Qu
Subject: [PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
This patch only involves the generation of xtheadvector
special load/store instructions and vext instructions.
gcc/ChangeLog:
* config/riscv/riscv-vector-builtins-bases.cc
(class th_loadstore_width): Define new builtin bases.
(class th_extract): Define new builtin bases.
(BASE): Define new builtin bases.
* config/riscv/riscv-vector-builtins-bases.h:
Define new builtin class.
* config/riscv/riscv-vector-builtins-shapes.cc
(struct th_loadstore_width_def): Define new builtin shapes.
(struct th_indexed_loadstore_width_def):
Define new builtin shapes.
(struct th_extract_def): Define new builtin shapes.
(SHAPE): Define new builtin shapes.
* config/riscv/riscv-vector-builtins-shapes.h:
Define new builtin shapes.
* config/riscv/riscv-vector-builtins.cc (DEF_RVV_FUNCTION):
* config/riscv/riscv-vector-builtins.h (enum required_ext):
(struct function_group_info):
* config/riscv/t-riscv: Add thead-vector-builtins-functions.def
* config/riscv/thead-vector.md
(@pred_mov_width<vlmem_op_attr><mode>): Add new patterns.
(*pred_mov_width<vlmem_op_attr><mode>): Likewise.
(@pred_store_width<vlmem_op_attr><mode>): Likewise.
(@pred_strided_load_width<vlmem_op_attr><mode>): Likewise.
(@pred_strided_store_width<vlmem_op_attr><mode>): Likewise.
(@pred_indexed_load_width<vlmem_op_attr><mode>): Likewise.
(@pred_indexed_<vlmem_order_attr>store_width<vlmem_op_attr><mode>):
(@pred_th_extract<mode>): Likewise.
(*pred_th_extract<mode>): Likewise.
* config/riscv/thead-vector-builtins-functions.def: New file.
gcc/testsuite/ChangeLog:
* gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c: New test.
Co-authored-by: Jin Ma <jinma@linux.alibaba.com>
Co-authored-by: Xianmiao Qu <cooper.qu@linux.alibaba.com>
Co-authored-by: Christoph Müllner <christoph.muellner@vrull.eu>
---
.../riscv/riscv-vector-builtins-bases.cc | 139 ++++++++++
.../riscv/riscv-vector-builtins-bases.h | 31 +++
.../riscv/riscv-vector-builtins-shapes.cc | 160 +++++++++++
.../riscv/riscv-vector-builtins-shapes.h | 3 +
gcc/config/riscv/riscv-vector-builtins.cc | 70 +++++
gcc/config/riscv/riscv-vector-builtins.h | 3 +
gcc/config/riscv/t-riscv | 1 +
.../riscv/thead-vector-builtins-functions.def | 39 +++
gcc/config/riscv/thead-vector.md | 253 ++++++++++++++++++
.../riscv/rvv/xtheadvector/vlb-vsb.c | 68 +++++
.../riscv/rvv/xtheadvector/vlbu-vsb.c | 68 +++++
.../riscv/rvv/xtheadvector/vlh-vsh.c | 68 +++++
.../riscv/rvv/xtheadvector/vlhu-vsh.c | 68 +++++
.../riscv/rvv/xtheadvector/vlw-vsw.c | 68 +++++
.../riscv/rvv/xtheadvector/vlwu-vsw.c | 68 +++++
15 files changed, 1107 insertions(+)
create mode 100644 gcc/config/riscv/thead-vector-builtins-functions.def
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c
diff --git a/gcc/config/riscv/riscv-vector-builtins-bases.cc b/gcc/config/riscv/riscv-vector-builtins-bases.cc
index 46f1a1da33e..3eba7943757 100644
--- a/gcc/config/riscv/riscv-vector-builtins-bases.cc
+++ b/gcc/config/riscv/riscv-vector-builtins-bases.cc
@@ -2125,6 +2125,83 @@ public:
}
};
+/* Implements
+ * th.vl(b/h/w)[u].v/th.vs(b/h/w)[u].v/th.vls(b/h/w)[u].v/th.vss(b/h/w)[u].v/
+ * th.vlx(b/h/w)[u].v/th.vs[u]x(b/h/w).v
+ * codegen. */
+template<bool STORE_P, lst_type LST_TYPE, int UNSPEC>
+class th_loadstore_width : public function_base
+{
+public:
+ bool apply_tail_policy_p () const override { return !STORE_P; }
+ bool apply_mask_policy_p () const override { return !STORE_P; }
+
+ unsigned int call_properties (const function_instance &) const override
+ {
+ if (STORE_P)
+ return CP_WRITE_MEMORY;
+ else
+ return CP_READ_MEMORY;
+ }
+
+ bool can_be_overloaded_p (enum predication_type_index pred) const override
+ {
+ if (STORE_P || LST_TYPE == LST_INDEXED)
+ return true;
+ return pred != PRED_TYPE_none;
+ }
+
+ rtx expand (function_expander &e) const override
+ {
+ gcc_assert (TARGET_XTHEADVECTOR);
+ if (LST_TYPE == LST_INDEXED)
+ {
+ if (STORE_P)
+ return e.use_exact_insn (
+ code_for_pred_indexed_store_width (UNSPEC, UNSPEC,
+ e.vector_mode ()));
+ else
+ return e.use_exact_insn (
+ code_for_pred_indexed_load_width (UNSPEC, e.vector_mode ()));
+ }
+ else if (LST_TYPE == LST_STRIDED)
+ {
+ if (STORE_P)
+ return e.use_contiguous_store_insn (
+ code_for_pred_strided_store_width (UNSPEC, e.vector_mode ()));
+ else
+ return e.use_contiguous_load_insn (
+ code_for_pred_strided_load_width (UNSPEC, e.vector_mode ()));
+ }
+ else
+ {
+ if (STORE_P)
+ return e.use_contiguous_store_insn (
+ code_for_pred_store_width (UNSPEC, e.vector_mode ()));
+ else
+ return e.use_contiguous_load_insn (
+ code_for_pred_mov_width (UNSPEC, e.vector_mode ()));
+ }
+ }
+};
+
+/* Implements vext.x.v. */
+class th_extract : public function_base
+{
+public:
+ bool apply_vl_p () const override { return false; }
+ bool apply_tail_policy_p () const override { return false; }
+ bool apply_mask_policy_p () const override { return false; }
+ bool use_mask_predication_p () const override { return false; }
+ bool has_merge_operand_p () const override { return false; }
+
+ rtx expand (function_expander &e) const override
+ {
+ gcc_assert (TARGET_XTHEADVECTOR);
+ return e.use_exact_insn (code_for_pred_th_extract (e.vector_mode ()));
+ }
+};
+
/* Below implements are vector crypto */
/* Implements vandn.[vv,vx] */
class vandn : public function_base
@@ -2587,6 +2664,37 @@ static CONSTEXPR const seg_indexed_load<UNSPEC_ORDERED> vloxseg_obj;
static CONSTEXPR const seg_indexed_store<UNSPEC_UNORDERED> vsuxseg_obj;
static CONSTEXPR const seg_indexed_store<UNSPEC_ORDERED> vsoxseg_obj;
static CONSTEXPR const vlsegff vlsegff_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLB> vlb_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLBU> vlbu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLH> vlh_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLHU> vlhu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLW> vlw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLWU> vlwu_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_UNIT_STRIDE, UNSPEC_TH_VLB> vsb_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_UNIT_STRIDE, UNSPEC_TH_VLH> vsh_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_UNIT_STRIDE, UNSPEC_TH_VLW> vsw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSB> vlsb_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSBU> vlsbu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSH> vlsh_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSHU> vlshu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSW> vlsw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSWU> vlswu_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_STRIDED, UNSPEC_TH_VLSB> vssb_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_STRIDED, UNSPEC_TH_VLSH> vssh_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_STRIDED, UNSPEC_TH_VLSW> vssw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXB> vlxb_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXBU> vlxbu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXH> vlxh_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXHU> vlxhu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXW> vlxw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXWU> vlxwu_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VLXB> vsxb_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VLXH> vsxh_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VLXW> vsxw_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VSUXB> vsuxb_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VSUXH> vsuxh_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VSUXW> vsuxw_obj;
+static CONSTEXPR const th_extract vext_x_v_obj;
/* Crypto Vector */
static CONSTEXPR const vandn vandn_obj;
@@ -2878,6 +2986,37 @@ BASE (vloxseg)
BASE (vsuxseg)
BASE (vsoxseg)
BASE (vlsegff)
+BASE (vlb)
+BASE (vlh)
+BASE (vlw)
+BASE (vlbu)
+BASE (vlhu)
+BASE (vlwu)
+BASE (vsb)
+BASE (vsh)
+BASE (vsw)
+BASE (vlsb)
+BASE (vlsh)
+BASE (vlsw)
+BASE (vlsbu)
+BASE (vlshu)
+BASE (vlswu)
+BASE (vssb)
+BASE (vssh)
+BASE (vssw)
+BASE (vlxb)
+BASE (vlxh)
+BASE (vlxw)
+BASE (vlxbu)
+BASE (vlxhu)
+BASE (vlxwu)
+BASE (vsxb)
+BASE (vsxh)
+BASE (vsxw)
+BASE (vsuxb)
+BASE (vsuxh)
+BASE (vsuxw)
+BASE (vext_x_v)
/* Crypto vector */
BASE (vandn)
BASE (vbrev)
diff --git a/gcc/config/riscv/riscv-vector-builtins-bases.h b/gcc/config/riscv/riscv-vector-builtins-bases.h
index 1122e3801a7..565a0311d2b 100644
--- a/gcc/config/riscv/riscv-vector-builtins-bases.h
+++ b/gcc/config/riscv/riscv-vector-builtins-bases.h
@@ -299,6 +299,37 @@ extern const function_base *const vloxseg;
extern const function_base *const vsuxseg;
extern const function_base *const vsoxseg;
extern const function_base *const vlsegff;
+extern const function_base *const vlb;
+extern const function_base *const vlh;
+extern const function_base *const vlw;
+extern const function_base *const vlbu;
+extern const function_base *const vlhu;
+extern const function_base *const vlwu;
+extern const function_base *const vsb;
+extern const function_base *const vsh;
+extern const function_base *const vsw;
+extern const function_base *const vlsb;
+extern const function_base *const vlsh;
+extern const function_base *const vlsw;
+extern const function_base *const vlsbu;
+extern const function_base *const vlshu;
+extern const function_base *const vlswu;
+extern const function_base *const vssb;
+extern const function_base *const vssh;
+extern const function_base *const vssw;
+extern const function_base *const vlxb;
+extern const function_base *const vlxh;
+extern const function_base *const vlxw;
+extern const function_base *const vlxbu;
+extern const function_base *const vlxhu;
+extern const function_base *const vlxwu;
+extern const function_base *const vsxb;
+extern const function_base *const vsxh;
+extern const function_base *const vsxw;
+extern const function_base *const vsuxb;
+extern const function_base *const vsuxh;
+extern const function_base *const vsuxw;
+extern const function_base *const vext_x_v;
/* Below function_base are Vectro Crypto*/
extern const function_base *const vandn;
extern const function_base *const vbrev;
diff --git a/gcc/config/riscv/riscv-vector-builtins-shapes.cc b/gcc/config/riscv/riscv-vector-builtins-shapes.cc
index 1e4f4d53de6..8e90b17a94b 100644
--- a/gcc/config/riscv/riscv-vector-builtins-shapes.cc
+++ b/gcc/config/riscv/riscv-vector-builtins-shapes.cc
@@ -211,6 +211,146 @@ struct indexed_loadstore_def : public function_shape
}
};
+/* Add one function instance for GROUP, using operand suffix at index OI,
+ mode suffix at index PAIR && bi and predication suffix at index pred_idx. */
+static void
+build_th_loadstore (function_builder &b, const function_group_info &group,
+ unsigned int pred_idx, unsigned int vec_type_idx)
+{
+ auto_vec<tree, 5> argument_types;
+ function_instance function_instance (group.base_name, *group.base,
+ *group.shape,
+ group.ops_infos.types[vec_type_idx],
+ group.preds[pred_idx], &group.ops_infos);
+ tree return_type = group.ops_infos.ret.get_tree_type (
+ group.ops_infos.types[vec_type_idx].index);
+ b.allocate_argument_types (function_instance, argument_types);
+ b.apply_predication (function_instance, return_type, argument_types);
+
+ if (TARGET_XTHEADVECTOR && !check_type (return_type, argument_types))
+ return;
+
+ tree type = builtin_types[group.ops_infos.types[vec_type_idx].index].vector;
+ if (strstr (group.base_name, "l")
+ && strstr (group.base_name, "u")
+ && !TYPE_UNSIGNED (TREE_TYPE (type)))
+ return;
+
+ if (strstr (group.base_name, "l")
+ && !strstr (group.base_name, "u")
+ && TYPE_UNSIGNED (TREE_TYPE (type)))
+ return;
+
+ machine_mode mode = TYPE_MODE (type);
+ int sew = GET_MODE_BITSIZE (GET_MODE_INNER (mode));
+ if (strstr (group.base_name, "h") && sew == 8)
+ return;
+
+ if (strstr (group.base_name, "w") && (sew == 8 || sew ==16))
+ return;
+
+ b.add_overloaded_function (function_instance, *group.shape);
+ b.add_unique_function (function_instance, (*group.shape), return_type,
+ argument_types);
+}
+
+/* th_loadstore_width_def class. */
+struct th_loadstore_width_def : public build_base
+{
+ void build (function_builder &b,
+ const function_group_info &group) const override
+ {
+ for (unsigned int pred_idx = 0; group.preds[pred_idx] != NUM_PRED_TYPES;
+ ++pred_idx)
+ {
+ for (unsigned int vec_type_idx = 0;
+ group.ops_infos.types[vec_type_idx].index != NUM_VECTOR_TYPES;
+ ++vec_type_idx)
+ {
+ build_th_loadstore (b, group, pred_idx, vec_type_idx);
+ }
+ }
+ }
+
+ char *get_name (function_builder &b, const function_instance &instance,
+ bool overloaded_p) const override
+ {
+ /* Return nullptr if it can not be overloaded. */
+ if (overloaded_p && !instance.base->can_be_overloaded_p (instance.pred))
+ return nullptr;
+
+ b.append_name ("__riscv_th_");
+ b.append_name (instance.base_name);
+
+ /* vop_v --> vop_v_<type>. */
+ if (!overloaded_p)
+ {
+ /* vop --> vop_v. */
+ b.append_name (operand_suffixes[instance.op_info->op]);
+ /* vop_v --> vop_v_<type>. */
+ b.append_name (type_suffixes[instance.type.index].vector);
+ }
+
+ /* According to rvv-intrinsic-doc, it does not add "_m" suffix
+ for vop_m C++ overloaded API. */
+ if (overloaded_p && instance.pred == PRED_TYPE_m)
+ return b.finish_name ();
+ b.append_name (predication_suffixes[instance.pred]);
+ return b.finish_name ();
+ }
+};
+
+
+/* th_indexed_loadstore_width_def class. */
+struct th_indexed_loadstore_width_def : public function_shape
+{
+ void build (function_builder &b,
+ const function_group_info &group) const override
+ {
+ for (unsigned int pred_idx = 0; group.preds[pred_idx] != NUM_PRED_TYPES;
+ ++pred_idx)
+ {
+ for (unsigned int vec_type_idx = 0;
+ group.ops_infos.types[vec_type_idx].index != NUM_VECTOR_TYPES;
+ ++vec_type_idx)
+ {
+ tree index_type = group.ops_infos.args[1].get_tree_type (
+ group.ops_infos.types[vec_type_idx].index);
+ if (!index_type)
+ continue;
+ build_th_loadstore (b, group, pred_idx, vec_type_idx);
+ }
+ }
+ }
+
+ char *get_name (function_builder &b, const function_instance &instance,
+ bool overloaded_p) const override
+ {
+
+ /* Return nullptr if it can not be overloaded. */
+ if (overloaded_p && !instance.base->can_be_overloaded_p (instance.pred))
+ return nullptr;
+
+ b.append_name ("__riscv_th_");
+ b.append_name (instance.base_name);
+ /* vop_v --> vop_v_<type>. */
+ if (!overloaded_p)
+ {
+ /* vop --> vop_v. */
+ b.append_name (operand_suffixes[instance.op_info->op]);
+ /* vop_v --> vop_v_<type>. */
+ b.append_name (type_suffixes[instance.type.index].vector);
+ }
+
+ /* According to rvv-intrinsic-doc, it does not add "_m" suffix
+ for vop_m C++ overloaded API. */
+ if (overloaded_p && instance.pred == PRED_TYPE_m)
+ return b.finish_name ();
+ b.append_name (predication_suffixes[instance.pred]);
+ return b.finish_name ();
+ }
+};
+
/* alu_def class. */
struct alu_def : public build_base
{
@@ -632,6 +772,23 @@ struct reduc_alu_def : public build_base
}
};
+/* th_extract_def class. */
+struct th_extract_def : public build_base
+{
+ char *get_name (function_builder &b, const function_instance &instance,
+ bool overloaded_p) const override
+ {
+ b.append_name ("__riscv_th_");
+ b.append_name (instance.base_name);
+
+ if (overloaded_p)
+ return b.finish_name ();
+ b.append_name (type_suffixes[instance.type.index].vector);
+ b.append_name (type_suffixes[instance.type.index].scalar);
+ return b.finish_name ();
+ }
+};
+
/* scalar_move_def class. */
struct scalar_move_def : public build_base
{
@@ -1094,6 +1251,8 @@ SHAPE(vsetvl, vsetvl)
SHAPE(vsetvl, vsetvlmax)
SHAPE(loadstore, loadstore)
SHAPE(indexed_loadstore, indexed_loadstore)
+SHAPE(th_loadstore_width, th_loadstore_width)
+SHAPE(th_indexed_loadstore_width, th_indexed_loadstore_width)
SHAPE(alu, alu)
SHAPE(alu_frm, alu_frm)
SHAPE(widen_alu, widen_alu)
@@ -1106,6 +1265,7 @@ SHAPE(move, move)
SHAPE(mask_alu, mask_alu)
SHAPE(reduc_alu, reduc_alu)
SHAPE(reduc_alu_frm, reduc_alu_frm)
+SHAPE(th_extract, th_extract)
SHAPE(scalar_move, scalar_move)
SHAPE(vundefined, vundefined)
SHAPE(misc, misc)
diff --git a/gcc/config/riscv/riscv-vector-builtins-shapes.h b/gcc/config/riscv/riscv-vector-builtins-shapes.h
index ac2a28ce017..a7624d0fabd 100644
--- a/gcc/config/riscv/riscv-vector-builtins-shapes.h
+++ b/gcc/config/riscv/riscv-vector-builtins-shapes.h
@@ -28,6 +28,8 @@ extern const function_shape *const vsetvl;
extern const function_shape *const vsetvlmax;
extern const function_shape *const loadstore;
extern const function_shape *const indexed_loadstore;
+extern const function_shape *const th_loadstore_width;
+extern const function_shape *const th_indexed_loadstore_width;
extern const function_shape *const alu;
extern const function_shape *const alu_frm;
extern const function_shape *const widen_alu;
@@ -41,6 +43,7 @@ extern const function_shape *const mask_alu;
extern const function_shape *const reduc_alu;
extern const function_shape *const reduc_alu_frm;
extern const function_shape *const scalar_move;
+extern const function_shape *const th_extract;
extern const function_shape *const vundefined;
extern const function_shape *const misc;
extern const function_shape *const vset;
diff --git a/gcc/config/riscv/riscv-vector-builtins.cc b/gcc/config/riscv/riscv-vector-builtins.cc
index 25e0b6e56de..44b9fec1898 100644
--- a/gcc/config/riscv/riscv-vector-builtins.cc
+++ b/gcc/config/riscv/riscv-vector-builtins.cc
@@ -934,6 +934,32 @@ static CONSTEXPR const rvv_arg_type_info ext_vcreate_args[]
= {rvv_arg_type_info (RVV_BASE_vector),
rvv_arg_type_info_end};
+/* A list of args for vector_type func (const scalar_type *, size_t)
+ * function. */
+static CONSTEXPR const rvv_arg_type_info scalar_const_ptr_size_args[]
+ = {rvv_arg_type_info (RVV_BASE_scalar_const_ptr),
+ rvv_arg_type_info (RVV_BASE_size), rvv_arg_type_info_end};
+
+/* A list of args for vector_type func (const scalar_type *, eew8_index_type)
+ * function. */
+static CONSTEXPR const rvv_arg_type_info scalar_const_ptr_index_args[]
+ = {rvv_arg_type_info (RVV_BASE_scalar_const_ptr),
+ rvv_arg_type_info (RVV_BASE_unsigned_vector), rvv_arg_type_info_end};
+
+/* A list of args for void func (scalar_type *, eew8_index_type, vector_type)
+ * function. */
+static CONSTEXPR const rvv_arg_type_info scalar_ptr_index_args[]
+ = {rvv_arg_type_info (RVV_BASE_scalar_ptr),
+ rvv_arg_type_info (RVV_BASE_unsigned_vector),
+ rvv_arg_type_info (RVV_BASE_vector), rvv_arg_type_info_end};
+
+/* A list of args for void func (scalar_type *, size_t, vector_type)
+ * function. */
+static CONSTEXPR const rvv_arg_type_info scalar_ptr_size_args[]
+ = {rvv_arg_type_info (RVV_BASE_scalar_ptr),
+ rvv_arg_type_info (RVV_BASE_size), rvv_arg_type_info (RVV_BASE_vector),
+ rvv_arg_type_info_end};
+
/* A list of none preds that will be registered for intrinsic functions. */
static CONSTEXPR const predication_type_index none_preds[]
= {PRED_TYPE_none, NUM_PRED_TYPES};
@@ -1455,6 +1481,14 @@ static CONSTEXPR const rvv_op_info iu_shift_vvv_ops
rvv_arg_type_info (RVV_BASE_vector), /* Return type */
shift_vv_args /* Args */};
+/* A static operand information for scalar_type func (vector_type, size_t)
+ * function registration. */
+static CONSTEXPR const rvv_op_info iu_x_s_u_ops
+ = {iu_ops, /* Types */
+ OP_TYPE_vx, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_scalar), /* Return type */
+ v_size_args /* Args */};
+
/* A static operand information for vector_type func (vector_type, size_t)
* function registration. */
static CONSTEXPR const rvv_op_info iu_shift_vvx_ops
@@ -2638,6 +2672,38 @@ static CONSTEXPR const rvv_op_info all_v_vcreate_lmul4_x2_ops
rvv_arg_type_info (RVV_BASE_vlmul_ext_x2), /* Return type */
ext_vcreate_args /* Args */};
+/* A static operand information for vector_type func (const scalar_type *,
+ * size_t) function registration. */
+static CONSTEXPR const rvv_op_info all_v_scalar_const_ptr_size_ops
+ = {all_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_size_args /* Args */};
+
+/* A static operand information for void func (scalar_type *, size_t,
+ * vector_type) function registration. */
+static CONSTEXPR const rvv_op_info all_v_scalar_ptr_size_ops
+ = {all_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_void), /* Return type */
+ scalar_ptr_size_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * index_type) function registration. */
+static CONSTEXPR const rvv_op_info all_v_scalar_const_ptr_index_ops
+ = {all_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_index_args /* Args */};
+
+/* A static operand information for void func (scalar_type *, index_type,
+ * vector_type) function registration. */
+static CONSTEXPR const rvv_op_info all_v_scalar_ptr_index_ops
+ = {all_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_void), /* Return type */
+ scalar_ptr_index_args /* Args */};
+
/* A static operand information for vector_type func (vector_type).
Some ins just supports SEW=32, such as crypto vectol Zvkg extension.
* function registration. */
@@ -2816,6 +2882,10 @@ static function_group_info function_groups[] = {
#define DEF_RVV_FUNCTION(NAME, SHAPE, PREDS, OPS_INFO) \
{#NAME, &bases::NAME, &shapes::SHAPE, PREDS, OPS_INFO, REQUIRED_EXTENSIONS},
#include "riscv-vector-builtins-functions.def"
+#undef DEF_RVV_FUNCTION
+#define DEF_RVV_FUNCTION(NAME, SHAPE, PREDS, OPS_INFO) \
+ {#NAME, &bases::NAME, &shapes::SHAPE, PREDS, OPS_INFO, REQUIRED_EXTENSIONS},
+#include "thead-vector-builtins-functions.def"
};
/* The RVV types, with their built-in
diff --git a/gcc/config/riscv/riscv-vector-builtins.h b/gcc/config/riscv/riscv-vector-builtins.h
index 54c8824ff92..a8ee39a3cb2 100644
--- a/gcc/config/riscv/riscv-vector-builtins.h
+++ b/gcc/config/riscv/riscv-vector-builtins.h
@@ -114,6 +114,7 @@ static const unsigned int CP_WRITE_CSR = 1U << 5;
enum required_ext
{
VECTOR_EXT, /* Vector extension */
+ XTHEADVECTOR_EXT, /* XTheadVector extension */
ZVBB_EXT, /* Cryto vector Zvbb sub-ext */
ZVBB_OR_ZVKB_EXT, /* Cryto vector Zvbb or zvkb sub-ext */
ZVBC_EXT, /* Crypto vector Zvbc sub-ext */
@@ -234,6 +235,8 @@ struct function_group_info
{
case VECTOR_EXT:
return TARGET_VECTOR;
+ case XTHEADVECTOR_EXT:
+ return TARGET_XTHEADVECTOR;
case ZVBB_EXT:
return TARGET_ZVBB;
case ZVBB_OR_ZVKB_EXT:
diff --git a/gcc/config/riscv/t-riscv b/gcc/config/riscv/t-riscv
index 32de6b851c1..38494320d8b 100644
--- a/gcc/config/riscv/t-riscv
+++ b/gcc/config/riscv/t-riscv
@@ -1,6 +1,7 @@
RISCV_BUILTINS_H = $(srcdir)/config/riscv/riscv-vector-builtins.h \
$(srcdir)/config/riscv/riscv-vector-builtins.def \
$(srcdir)/config/riscv/riscv-vector-builtins-functions.def \
+ $(srcdir)/config/riscv/thead-vector-builtins-functions.def \
riscv-vector-type-indexer.gen.def
riscv-builtins.o: $(srcdir)/config/riscv/riscv-builtins.cc $(CONFIG_H) \
diff --git a/gcc/config/riscv/thead-vector-builtins-functions.def b/gcc/config/riscv/thead-vector-builtins-functions.def
new file mode 100644
index 00000000000..fd3ba29bae9
--- /dev/null
+++ b/gcc/config/riscv/thead-vector-builtins-functions.def
@@ -0,0 +1,39 @@
+#ifndef DEF_RVV_FUNCTION
+#define DEF_RVV_FUNCTION(NAME, SHAPE, PREDS, OPS_INFO)
+#endif
+
+#define REQUIRED_EXTENSIONS XTHEADVECTOR_EXT
+DEF_RVV_FUNCTION (vlb, th_loadstore_width, full_preds, all_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (vlh, th_loadstore_width, full_preds, all_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (vlw, th_loadstore_width, full_preds, all_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (vlbu, th_loadstore_width, full_preds, all_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (vlhu, th_loadstore_width, full_preds, all_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (vlwu, th_loadstore_width, full_preds, all_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (vsb, th_loadstore_width, none_m_preds, all_v_scalar_ptr_ops)
+DEF_RVV_FUNCTION (vsh, th_loadstore_width, none_m_preds, all_v_scalar_ptr_ops)
+DEF_RVV_FUNCTION (vsw, th_loadstore_width, none_m_preds, all_v_scalar_ptr_ops)
+DEF_RVV_FUNCTION (vlsb, th_loadstore_width, full_preds, all_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (vlsh, th_loadstore_width, full_preds, all_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (vlsw, th_loadstore_width, full_preds, all_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (vlsbu, th_loadstore_width, full_preds, all_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (vlshu, th_loadstore_width, full_preds, all_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (vlswu, th_loadstore_width, full_preds, all_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (vssb, th_loadstore_width, none_m_preds, all_v_scalar_ptr_size_ops)
+DEF_RVV_FUNCTION (vssh, th_loadstore_width, none_m_preds, all_v_scalar_ptr_size_ops)
+DEF_RVV_FUNCTION (vssw, th_loadstore_width, none_m_preds, all_v_scalar_ptr_size_ops)
+DEF_RVV_FUNCTION (vlxb, th_indexed_loadstore_width, full_preds, all_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (vlxh, th_indexed_loadstore_width, full_preds, all_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (vlxw, th_indexed_loadstore_width, full_preds, all_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (vlxbu, th_indexed_loadstore_width, full_preds, all_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (vlxhu, th_indexed_loadstore_width, full_preds, all_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (vlxwu, th_indexed_loadstore_width, full_preds, all_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (vsxb, th_indexed_loadstore_width, none_m_preds, all_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (vsxh, th_indexed_loadstore_width, none_m_preds, all_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (vsxw, th_indexed_loadstore_width, none_m_preds, all_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (vsuxb, th_indexed_loadstore_width, none_m_preds, all_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (vsuxh, th_indexed_loadstore_width, none_m_preds, all_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (vsuxw, th_indexed_loadstore_width, none_m_preds, all_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (vext_x_v, th_extract, none_preds, iu_x_s_u_ops)
+#undef REQUIRED_EXTENSIONS
+
+#undef DEF_RVV_FUNCTION
diff --git a/gcc/config/riscv/thead-vector.md b/gcc/config/riscv/thead-vector.md
index 696b815252d..0f3700d9269 100644
--- a/gcc/config/riscv/thead-vector.md
+++ b/gcc/config/riscv/thead-vector.md
@@ -1,7 +1,95 @@
(define_c_enum "unspec" [
+ UNSPEC_TH_VLB
+ UNSPEC_TH_VLBU
+ UNSPEC_TH_VLH
+ UNSPEC_TH_VLHU
+ UNSPEC_TH_VLW
+ UNSPEC_TH_VLWU
+
+ UNSPEC_TH_VLSB
+ UNSPEC_TH_VLSBU
+ UNSPEC_TH_VLSH
+ UNSPEC_TH_VLSHU
+ UNSPEC_TH_VLSW
+ UNSPEC_TH_VLSWU
+
+ UNSPEC_TH_VLXB
+ UNSPEC_TH_VLXBU
+ UNSPEC_TH_VLXH
+ UNSPEC_TH_VLXHU
+ UNSPEC_TH_VLXW
+ UNSPEC_TH_VLXWU
+
+ UNSPEC_TH_VSUXB
+ UNSPEC_TH_VSUXH
+ UNSPEC_TH_VSUXW
+
UNSPEC_TH_VWLDST
])
+(define_int_iterator UNSPEC_TH_VLMEM_OP [
+ UNSPEC_TH_VLB UNSPEC_TH_VLBU
+ UNSPEC_TH_VLH UNSPEC_TH_VLHU
+ UNSPEC_TH_VLW UNSPEC_TH_VLWU
+])
+
+(define_int_iterator UNSPEC_TH_VLSMEM_OP [
+ UNSPEC_TH_VLSB UNSPEC_TH_VLSBU
+ UNSPEC_TH_VLSH UNSPEC_TH_VLSHU
+ UNSPEC_TH_VLSW UNSPEC_TH_VLSWU
+])
+
+(define_int_iterator UNSPEC_TH_VLXMEM_OP [
+ UNSPEC_TH_VLXB UNSPEC_TH_VLXBU
+ UNSPEC_TH_VLXH UNSPEC_TH_VLXHU
+ UNSPEC_TH_VLXW UNSPEC_TH_VLXWU
+])
+
+(define_int_attr vlmem_op_attr [
+ (UNSPEC_TH_VLB "b") (UNSPEC_TH_VLBU "bu")
+ (UNSPEC_TH_VLH "h") (UNSPEC_TH_VLHU "hu")
+ (UNSPEC_TH_VLW "w") (UNSPEC_TH_VLWU "wu")
+ (UNSPEC_TH_VLSB "b") (UNSPEC_TH_VLSBU "bu")
+ (UNSPEC_TH_VLSH "h") (UNSPEC_TH_VLSHU "hu")
+ (UNSPEC_TH_VLSW "w") (UNSPEC_TH_VLSWU "wu")
+ (UNSPEC_TH_VLXB "b") (UNSPEC_TH_VLXBU "bu")
+ (UNSPEC_TH_VLXH "h") (UNSPEC_TH_VLXHU "hu")
+ (UNSPEC_TH_VLXW "w") (UNSPEC_TH_VLXWU "wu")
+ (UNSPEC_TH_VSUXB "b")
+ (UNSPEC_TH_VSUXH "h")
+ (UNSPEC_TH_VSUXW "w")
+])
+
+(define_int_attr vlmem_order_attr [
+ (UNSPEC_TH_VLXB "")
+ (UNSPEC_TH_VLXH "")
+ (UNSPEC_TH_VLXW "")
+ (UNSPEC_TH_VSUXB "u")
+ (UNSPEC_TH_VSUXH "u")
+ (UNSPEC_TH_VSUXW "u")
+])
+
+(define_int_iterator UNSPEC_TH_VSMEM_OP [
+ UNSPEC_TH_VLB
+ UNSPEC_TH_VLH
+ UNSPEC_TH_VLW
+])
+
+(define_int_iterator UNSPEC_TH_VSSMEM_OP [
+ UNSPEC_TH_VLSB
+ UNSPEC_TH_VLSH
+ UNSPEC_TH_VLSW
+])
+
+(define_int_iterator UNSPEC_TH_VSXMEM_OP [
+ UNSPEC_TH_VLXB
+ UNSPEC_TH_VLXH
+ UNSPEC_TH_VLXW
+ UNSPEC_TH_VSUXB
+ UNSPEC_TH_VSUXH
+ UNSPEC_TH_VSUXW
+])
+
(define_mode_iterator V_VLS_VT [V VLS VT])
(define_mode_iterator V_VB_VLS_VT [V VB VLS VT])
@@ -100,3 +188,168 @@
}
[(set_attr "type" "vldm,vstm,vmalu,vmalu,vmalu")
(set_attr "mode" "<MODE>")])
+
+(define_expand "@pred_mov_width<vlmem_op_attr><mode>"
+ [(set (match_operand:V_VLS 0 "nonimmediate_operand")
+ (if_then_else:V_VLS
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand")
+ (match_operand 4 "vector_length_operand")
+ (match_operand 5 "const_int_operand")
+ (match_operand 6 "const_int_operand")
+ (match_operand 7 "const_int_operand")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLMEM_OP)
+ (match_operand:V_VLS 3 "vector_move_operand")
+ (match_operand:V_VLS 2 "vector_merge_operand")))]
+ "TARGET_XTHEADVECTOR"
+ {})
+
+(define_insn_and_split "*pred_mov_width<vlmem_op_attr><mode>"
+ [(set (match_operand:V_VLS 0 "nonimmediate_operand" "=vr, vr, vd, m, vr, vr")
+ (if_then_else:V_VLS
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1, Wc1, vm, vmWc1, Wc1, Wc1")
+ (match_operand 4 "vector_length_operand" " rK, rK, rK, rK, rK, rK")
+ (match_operand 5 "const_int_operand" " i, i, i, i, i, i")
+ (match_operand 6 "const_int_operand" " i, i, i, i, i, i")
+ (match_operand 7 "const_int_operand" " i, i, i, i, i, i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLMEM_OP)
+ (match_operand:V_VLS 3 "reg_or_mem_operand" " m, m, m, vr, vr, vr")
+ (match_operand:V_VLS 2 "vector_merge_operand" " 0, vu, vu, vu, vu, 0")))]
+ "(TARGET_XTHEADVECTOR
+ && (register_operand (operands[0], <MODE>mode)
+ || register_operand (operands[3], <MODE>mode)))"
+ "@
+ vl<vlmem_op_attr>.v\t%0,%3%p1
+ vl<vlmem_op_attr>.v\t%0,%3
+ vl<vlmem_op_attr>.v\t%0,%3,%1.t
+ vs<vlmem_op_attr>.v\t%3,%0%p1
+ vmv.v.v\t%0,%3
+ vmv.v.v\t%0,%3"
+ "&& register_operand (operands[0], <MODE>mode)
+ && register_operand (operands[3], <MODE>mode)
+ && satisfies_constraint_vu (operands[2])
+ && INTVAL (operands[7]) == riscv_vector::VLMAX"
+ [(set (match_dup 0) (match_dup 3))]
+ ""
+ [(set_attr "type" "vlde,vlde,vlde,vste,vimov,vimov")
+ (set_attr "mode" "<MODE>")])
+
+(define_insn "@pred_store_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "memory_operand" "+m")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1")
+ (match_operand 3 "vector_length_operand" " rK")
+ (match_operand 4 "const_int_operand" " i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VSMEM_OP)
+ (match_operand:VI 2 "register_operand" " vr")
+ (match_dup 0)))]
+ "TARGET_XTHEADVECTOR"
+ "vs<vlmem_op_attr>.v\t%2,%0%p1"
+ [(set_attr "type" "vste")
+ (set_attr "mode" "<MODE>")
+ (set (attr "avl_type_idx") (const_int 4))
+ (set_attr "vl_op_idx" "3")])
+
+(define_insn "@pred_strided_load_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "register_operand" "=vr, vr, vd")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1, Wc1, vm")
+ (match_operand 5 "vector_length_operand" " rK, rK, rK")
+ (match_operand 6 "const_int_operand" " i, i, i")
+ (match_operand 7 "const_int_operand" " i, i, i")
+ (match_operand 8 "const_int_operand" " i, i, i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLSMEM_OP)
+ (unspec:VI
+ [(match_operand:VI 3 "memory_operand" " m, m, m")
+ (match_operand 4 "pmode_reg_or_0_operand" " rJ, rJ, rJ")] UNSPEC_TH_VLSMEM_OP)
+ (match_operand:VI 2 "vector_merge_operand" " 0, vu, vu")))]
+ "TARGET_XTHEADVECTOR"
+ "vls<vlmem_op_attr>.v\t%0,%3,%z4%p1"
+ [(set_attr "type" "vlds")
+ (set_attr "mode" "<MODE>")])
+
+(define_insn "@pred_strided_store_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "memory_operand" "+m")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1")
+ (match_operand 4 "vector_length_operand" " rK")
+ (match_operand 5 "const_int_operand" " i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VSSMEM_OP)
+ (unspec:VI
+ [(match_operand 2 "pmode_reg_or_0_operand" " rJ")
+ (match_operand:VI 3 "register_operand" " vr")] UNSPEC_TH_VSSMEM_OP)
+ (match_dup 0)))]
+ "TARGET_XTHEADVECTOR"
+ "vss<vlmem_op_attr>.v\t%3,%0,%z2%p1"
+ [(set_attr "type" "vsts")
+ (set_attr "mode" "<MODE>")
+ (set (attr "avl_type_idx") (const_int 5))])
+
+(define_insn "@pred_indexed_load_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "register_operand" "=vd, vr,vd, vr")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" " vm,Wc1,vm,Wc1")
+ (match_operand 5 "vector_length_operand" " rK, rK,rK, rK")
+ (match_operand 6 "const_int_operand" " i, i, i, i")
+ (match_operand 7 "const_int_operand" " i, i, i, i")
+ (match_operand 8 "const_int_operand" " i, i, i, i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLXMEM_OP)
+ (unspec:VI
+ [(match_operand 3 "pmode_reg_or_0_operand" " rJ, rJ,rJ, rJ")
+ (mem:BLK (scratch))
+ (match_operand:VI 4 "register_operand" " vr, vr,vr, vr")] UNSPEC_TH_VLXMEM_OP)
+ (match_operand:VI 2 "vector_merge_operand" " vu, vu, 0, 0")))]
+ "TARGET_XTHEADVECTOR"
+ "vlx<vlmem_op_attr>.v\t%0,(%z3),%4%p1"
+ [(set_attr "type" "vldux")
+ (set_attr "mode" "<MODE>")])
+
+(define_insn "@pred_indexed_<vlmem_order_attr>store_width<vlmem_op_attr><mode>"
+ [(set (mem:BLK (scratch))
+ (unspec:BLK
+ [(unspec:<VM>
+ [(match_operand:<VM> 0 "vector_mask_operand" "vmWc1")
+ (match_operand 4 "vector_length_operand" " rK")
+ (match_operand 5 "const_int_operand" " i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VSXMEM_OP)
+ (match_operand 1 "pmode_reg_or_0_operand" " rJ")
+ (match_operand:VI 2 "register_operand" " vr")
+ (match_operand:VI 3 "register_operand" " vr")] UNSPEC_TH_VSXMEM_OP))]
+ "TARGET_XTHEADVECTOR"
+ "vs<vlmem_order_attr>x<vlmem_op_attr>.v\t%3,(%z1),%2%p0"
+ [(set_attr "type" "vstux")
+ (set_attr "mode" "<MODE>")])
+
+(define_expand "@pred_th_extract<mode>"
+ [(set (match_operand:<VEL> 0 "register_operand")
+ (unspec:<VEL>
+ [(vec_select:<VEL>
+ (match_operand:V_VLSI 1 "register_operand")
+ (parallel [(match_operand:DI 2 "register_operand" "r")]))
+ (reg:SI VTYPE_REGNUM)] UNSPEC_VPREDICATE))]
+ "TARGET_XTHEADVECTOR"
+{})
+
+(define_insn "*pred_th_extract<mode>"
+ [(set (match_operand:<VEL> 0 "register_operand" "=r")
+ (unspec:<VEL>
+ [(vec_select:<VEL>
+ (match_operand:V_VLSI 1 "register_operand" "vr")
+ (parallel [(match_operand:DI 2 "register_operand" "r")]))
+ (reg:SI VTYPE_REGNUM)] UNSPEC_VPREDICATE))]
+ "TARGET_XTHEADVECTOR"
+ "vext.x.v\t%0,%1,%2"
+ [(set_attr "type" "vimovvx")
+ (set_attr "mode" "<MODE>")])
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c
new file mode 100644
index 00000000000..3c12c124597
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_th_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+
+** th.vadd\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+
+** th.vsb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out)
+{
+ vint32m1_t v = __riscv_th_vlb_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlb_v_i32m1_tu (v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vv_i32m1 (v2, v2, 4);
+ vint32m1_t v4 = __riscv_vadd_vv_i32m1_tu (v3, v2, v2, 4);
+ __riscv_th_vsb_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlb.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+
+** th.vadd\.vv\tv[1-9][0-9]?,\s*v[0-9]+,\s*v[0-9]+,\s*v0.t
+** th.vsb.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlb_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlb_v_i32m1_m (mask, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vv_i32m1 (v2, v2, 4);
+ vint32m1_t v4 = __riscv_vadd_vv_i32m1_m (mask, v3, v3, 4);
+ __riscv_th_vsb_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlb.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+
+** th.vadd\.vv\tv[1-9][0-9]?,\s*v[0-9]+,\s*v[0-9]+,\s*v0.t
+** th.vsb.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlb_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlb_v_i32m1_tumu (mask, v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vv_i32m1 (v2, v2, 4);
+ vint32m1_t v4 = __riscv_vadd_vv_i32m1_tumu (mask, v3, v2, v2, 4);
+ __riscv_th_vsb_v_i32m1 (out, v4, 4);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c
new file mode 100644
index 00000000000..30bef369375
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_th_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlbu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlbu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vsb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, uint32_t x)
+{
+ vuint32m1_t v = __riscv_th_vlbu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlbu_v_u32m1_tu (v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tu (v3, v2, -16, 4);
+ __riscv_th_vsb_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlbu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsb.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlbu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlbu_v_u32m1_m (mask, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_m (mask, v3, -16, 4);
+ __riscv_th_vsb_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlbu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlbu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsb.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlbu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlbu_v_u32m1_tumu (mask, v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tumu (mask, v3, v2, -16, 4);
+ __riscv_th_vsb_v_u32m1 (out, v4, 4);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c
new file mode 100644
index 00000000000..3c8b5ccc16b
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_th_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vsh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, int32_t x)
+{
+ vint32m1_t v = __riscv_th_vlh_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlh_v_i32m1_tu (v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, -16, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_tu (v3, v2, -16, 4);
+ __riscv_th_vsh_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlh.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsh.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, int32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlh_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlh_v_i32m1_m (mask, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, -16, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_m (mask, v3, -16, 4);
+ __riscv_th_vsh_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlh.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsh.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, int32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlh_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlh_v_i32m1_tumu (mask, v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, -16, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_tumu (mask, v3, v2, -16, 4);
+ __riscv_th_vsh_v_i32m1 (out, v4, 4);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c
new file mode 100644
index 00000000000..b7c00404f18
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_th_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlhu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlhu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vsh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, uint32_t x)
+{
+ vuint32m1_t v = __riscv_th_vlhu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlhu_v_u32m1_tu (v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tu (v3, v2, -16, 4);
+ __riscv_th_vsh_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlhu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsh.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlhu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlhu_v_u32m1_m (mask, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_m (mask, v3, -16, 4);
+ __riscv_th_vsh_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlhu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlhu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsh.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlhu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlhu_v_u32m1_tumu (mask, v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tumu (mask, v3, v2, -16, 4);
+ __riscv_th_vsh_v_u32m1 (out, v4, 4);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c
new file mode 100644
index 00000000000..17a53012acf
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_th_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vx\tv[0-9]+,\s*v[0-9]+,\s*[a-x0-9]+
+** th.vadd\.vx\tv[0-9]+,\s*v[0-9]+,\s*[a-x0-9]+
+** th.vsw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, int32_t x)
+{
+ vint32m1_t v = __riscv_th_vlw_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlw_v_i32m1_tu (v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, x, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_tu (v3, v2, x, 4);
+ __riscv_th_vsw_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlw.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vx\tv[0-9]+,\s*v[0-9]+,\s*[a-x0-9]+
+** th.vadd\.vx\tv[1-9][0-9]?,\s*v[0-9]+,\s*[a-x0-9]+,\s*v0.t
+** th.vsw.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, int32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlw_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlw_v_i32m1_m (mask, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, x, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_m (mask, v3, x, 4);
+ __riscv_th_vsw_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlw.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vx\tv[0-9]+,\s*v[0-9]+,\s*[a-x0-9]+
+** th.vadd\.vx\tv[1-9][0-9]?,\s*v[0-9]+,\s*[a-x0-9]+,\s*v0.t
+** th.vsw.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, int32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlw_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlw_v_i32m1_tumu (mask, v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, x, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_tumu (mask, v3, v2, x, 4);
+ __riscv_th_vsw_v_i32m1 (out, v4, 4);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c
new file mode 100644
index 00000000000..b187cfc852b
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_th_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlwu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlwu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vsw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, uint32_t x)
+{
+ vuint32m1_t v = __riscv_th_vlwu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlwu_v_u32m1_tu (v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tu (v3, v2, -16, 4);
+ __riscv_th_vsw_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlwu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsw.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlwu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlwu_v_u32m1_m (mask, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_m (mask, v3, -16, 4);
+ __riscv_th_vsw_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlwu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlwu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsw.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlwu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlwu_v_u32m1_tumu (mask, v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tumu (mask, v3, v2, -16, 4);
+ __riscv_th_vsw_v_u32m1 (out, v4, 4);
+}
--
2.17.1
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re：Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
2024-01-11 9:32 Re：[PATCH " juzhe.zhong
2024-01-11 9:38 ` Re：Re：[PATCH " joshua
@ 2024-01-11 12:05 ` joshua
1 sibling, 0 replies; 17+ messages in thread
From: joshua @ 2024-01-11 12:05 UTC (permalink / raw)
To: juzhe.zhong, gcc-patches
Cc: Jim Wilson, palmer, andrew, philipp.tomsich, jeffreyalaw,
christoph.muellner, jinma, cooper.qu
without CSE:
(insn 16 15 17 2 (set (reg/v:RVVM2HI 137 [ output_var_0 ])
(if_then_else:RVVM2HI (unspec:RVVMF8BI [
(const_vector:RVVMF8BI repeat [
(const_int 1 [0x1])
])
(reg:DI 146)
(const_int 2 [0x2]) repeated x2
(const_int 0 [0])
(reg:SI 66 vl)
(reg:SI 67 vtype)
] UNSPEC_VPREDICATE)
(mult:RVVM2HI (sign_extend:RVVM2HI (reg/v:RVVM1QI 136 [ op1 ]))
(sign_extend:RVVM2HI (vec_duplicate:RVVM1QI (reg:QI 147))))
(unspec:RVVM2HI [
(reg:SI 0 zero)
] UNSPEC_VUNDEF))) {pred_dual_widen_mulsrvvm2hi_scalar}
with CSE:
(insn 16 15 17 2 (set (reg/v:RVVM2HI 137 [ output_var_0 ])
(if_then_else:RVVM2HI (unspec:RVVMF8BI [
(const_vector:RVVMF8BI repeat [
(const_int 1 [0x1])
])
(reg:DI 146)
(const_int 2 [0x2]) repeated x2
(const_int 0 [0])
(reg:SI 66 vl)
(reg:SI 67 vtype)
] UNSPEC_VPREDICATE)
(sign_extend:RVVM2HI (reg/v:RVVM1QI 136 [ op1 ]))
(unspec:RVVM2HI [
(reg:SI 0 zero)
] UNSPEC_VUNDEF))) {pred_extendrvvm2hi_vf2}
------------------------------------------------------------------
发件人：juzhe.zhong@rivai.ai <juzhe.zhong@rivai.ai>
发送时间：2024年1月11日(星期四) 17:32
收件人："cooper.joshua"<cooper.joshua@linux.alibaba.com>; "gcc-patches"<gcc-patches@gcc.gnu.org>
抄 送：Jim Wilson<jim.wilson.gcc@gmail.com>; palmer<palmer@dabbelt.com>; andrew<andrew@sifive.com>; "philipp.tomsich"<philipp.tomsich@vrull.eu>; jeffreyalaw<jeffreyalaw@gmail.com>; "christoph.muellner"<christoph.muellner@vrull.eu>; jinma<jinma@linux.alibaba.com>; "cooper.qu"<cooper.qu@linux.alibaba.com>
主 题：Re: Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
Ok. Let's hold on "RISC-V: Handle differences between XTheadvector and Vector" patch
until you can reproduce the issue for me.
juzhe.zhong@rivai.ai
发件人： joshua
发送时间： 2024-01-11 17:29
收件人： juzhe.zhong@rivai.ai; gcc-patches
抄送： Jim Wilson; palmer; andrew; philipp.tomsich; jeffreyalaw; christoph.muellner; jinma; cooper.qu
主题： Re：Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
The sext/zext issue is not related to xtheadvector-special patterns.
I added !TARGET_XTHEADVECTOR to sext/zext patterns in
"RISC-V: Handle differences between XTheadvector and Vector"
That is caused by the vwmul pattern, but I cannot reproduce it right now.
------------------------------------------------------------------
发件人：juzhe.zhong@rivai.ai <juzhe.zhong@rivai.ai>
发送时间：2024年1月11日(星期四) 17:24
收件人："cooper.joshua"<cooper.joshua@linux.alibaba.com>; "gcc-patches"<gcc-patches@gcc.gnu.org>
抄 送：Jim Wilson<jim.wilson.gcc@gmail.com>; palmer<palmer@dabbelt.com>; andrew<andrew@sifive.com>; "philipp.tomsich"<philipp.tomsich@vrull.eu>; jeffreyalaw<jeffreyalaw@gmail.com>; "christoph.muellner"<christoph.muellner@vrull.eu>; jinma<jinma@linux.alibaba.com>; "cooper.qu"<cooper.qu@linux.alibaba.com>
主 题：Re: Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
I don't see any patterns are possible CSE into sext/zext patterns:
+(define_expand "@pred_mov_width<vlmem_op_attr><mode>"
+ [(set (match_operand:V_VLS 0 "nonimmediate_operand")
+ (if_then_else:V_VLS
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand")
+ (match_operand 4 "vector_length_operand")
+ (match_operand 5 "const_int_operand")
+ (match_operand 6 "const_int_operand")
+ (match_operand 7 "const_int_operand")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLMEM_OP)
+ (match_operand:V_VLS 3 "vector_move_operand")
+ (match_operand:V_VLS 2 "vector_merge_operand")))]
+ "TARGET_XTHEADVECTOR"
+ {})
+
+(define_insn_and_split "*pred_mov_width<vlmem_op_attr><mode>"
+ [(set (match_operand:V_VLS 0 "nonimmediate_operand" "=vr, vr, vd, m, vr, vr")
+ (if_then_else:V_VLS
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1, Wc1, vm, vmWc1, Wc1, Wc1")
+ (match_operand 4 "vector_length_operand" " rK, rK, rK, rK, rK, rK")
+ (match_operand 5 "const_int_operand" " i, i, i, i, i, i")
+ (match_operand 6 "const_int_operand" " i, i, i, i, i, i")
+ (match_operand 7 "const_int_operand" " i, i, i, i, i, i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLMEM_OP)
+ (match_operand:V_VLS 3 "reg_or_mem_operand" " m, m, m, vr, vr, vr")
+ (match_operand:V_VLS 2 "vector_merge_operand" " 0, vu, vu, vu, vu, 0")))]
+ "(TARGET_XTHEADVECTOR
+ && (register_operand (operands[0], <MODE>mode)
+ || register_operand (operands[3], <MODE>mode)))"
+ "@
+ vl<vlmem_op_attr>.v\t%0,%3%p1
+ vl<vlmem_op_attr>.v\t%0,%3
+ vl<vlmem_op_attr>.v\t%0,%3,%1.t
+ vs<vlmem_op_attr>.v\t%3,%0%p1
+ vmv.v.v\t%0,%3
+ vmv.v.v\t%0,%3"
+ "&& register_operand (operands[0], <MODE>mode)
+ && register_operand (operands[3], <MODE>mode)
+ && satisfies_constraint_vu (operands[2])
+ && INTVAL (operands[7]) == riscv_vector::VLMAX"
+ [(set (match_dup 0) (match_dup 3))]
+ ""
+ [(set_attr "type" "vlde,vlde,vlde,vste,vimov,vimov")
+ (set_attr "mode" "<MODE>")])
+
+(define_insn "@pred_store_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "memory_operand" "+m")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1")
+ (match_operand 3 "vector_length_operand" " rK")
+ (match_operand 4 "const_int_operand" " i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VSMEM_OP)
+ (match_operand:VI 2 "register_operand" " vr")
+ (match_dup 0)))]
+ "TARGET_XTHEADVECTOR"
+ "vs<vlmem_op_attr>.v\t%2,%0%p1"
+ [(set_attr "type" "vste")
+ (set_attr "mode" "<MODE>")
+ (set (attr "avl_type_idx") (const_int 4))
+ (set_attr "vl_op_idx" "3")])
+
+(define_insn "@pred_strided_load_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "register_operand" "=vr, vr, vd")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1, Wc1, vm")
+ (match_operand 5 "vector_length_operand" " rK, rK, rK")
+ (match_operand 6 "const_int_operand" " i, i, i")
+ (match_operand 7 "const_int_operand" " i, i, i")
+ (match_operand 8 "const_int_operand" " i, i, i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLSMEM_OP)
+ (unspec:VI
+ [(match_operand:VI 3 "memory_operand" " m, m, m")
+ (match_operand 4 "pmode_reg_or_0_operand" " rJ, rJ, rJ")] UNSPEC_TH_VLSMEM_OP)
+ (match_operand:VI 2 "vector_merge_operand" " 0, vu, vu")))]
+ "TARGET_XTHEADVECTOR"
+ "vls<vlmem_op_attr>.v\t%0,%3,%z4%p1"
+ [(set_attr "type" "vlds")
+ (set_attr "mode" "<MODE>")])
+
+(define_insn "@pred_strided_store_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "memory_operand" "+m")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1")
+ (match_operand 4 "vector_length_operand" " rK")
+ (match_operand 5 "const_int_operand" " i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VSSMEM_OP)
+ (unspec:VI
+ [(match_operand 2 "pmode_reg_or_0_operand" " rJ")
+ (match_operand:VI 3 "register_operand" " vr")] UNSPEC_TH_VSSMEM_OP)
+ (match_dup 0)))]
+ "TARGET_XTHEADVECTOR"
+ "vss<vlmem_op_attr>.v\t%3,%0,%z2%p1"
+ [(set_attr "type" "vsts")
+ (set_attr "mode" "<MODE>")
+ (set (attr "avl_type_idx") (const_int 5))])
+
+(define_insn "@pred_indexed_load_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "register_operand" "=vd, vr,vd, vr")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" " vm,Wc1,vm,Wc1")
+ (match_operand 5 "vector_length_operand" " rK, rK,rK, rK")
+ (match_operand 6 "const_int_operand" " i, i, i, i")
+ (match_operand 7 "const_int_operand" " i, i, i, i")
+ (match_operand 8 "const_int_operand" " i, i, i, i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLXMEM_OP)
+ (unspec:VI
+ [(match_operand 3 "pmode_reg_or_0_operand" " rJ, rJ,rJ, rJ")
+ (mem:BLK (scratch))
+ (match_operand:VI 4 "register_operand" " vr, vr,vr, vr")] UNSPEC_TH_VLXMEM_OP)
+ (match_operand:VI 2 "vector_merge_operand" " vu, vu, 0, 0")))]
+ "TARGET_XTHEADVECTOR"
+ "vlx<vlmem_op_attr>.v\t%0,(%z3),%4%p1"
+ [(set_attr "type" "vldux")
+ (set_attr "mode" "<MODE>")])
+
+(define_insn "@pred_indexed_<vlmem_order_attr>store_width<vlmem_op_attr><mode>"
+ [(set (mem:BLK (scratch))
+ (unspec:BLK
+ [(unspec:<VM>
+ [(match_operand:<VM> 0 "vector_mask_operand" "vmWc1")
+ (match_operand 4 "vector_length_operand" " rK")
+ (match_operand 5 "const_int_operand" " i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VSXMEM_OP)
+ (match_operand 1 "pmode_reg_or_0_operand" " rJ")
+ (match_operand:VI 2 "register_operand" " vr")
+ (match_operand:VI 3 "register_operand" " vr")] UNSPEC_TH_VSXMEM_OP))]
+ "TARGET_XTHEADVECTOR"
+ "vs<vlmem_order_attr>x<vlmem_op_attr>.v\t%3,(%z1),%2%p0"
+ [(set_attr "type" "vstux")
+ (set_attr "mode" "<MODE>")])
+
+(define_expand "@pred_th_extract<mode>"
+ [(set (match_operand:<VEL> 0 "register_operand")
+ (unspec:<VEL>
+ [(vec_select:<VEL>
+ (match_operand:V_VLSI 1 "register_operand")
+ (parallel [(match_operand:DI 2 "register_operand" "r")]))
+ (reg:SI VTYPE_REGNUM)] UNSPEC_VPREDICATE))]
+ "TARGET_XTHEADVECTOR"
+{})
+
+(define_insn "*pred_th_extract<mode>"
+ [(set (match_operand:<VEL> 0 "register_operand" "=r")
+ (unspec:<VEL>
+ [(vec_select:<VEL>
+ (match_operand:V_VLSI 1 "register_operand" "vr")
+ (parallel [(match_operand:DI 2 "register_operand" "r")]))
+ (reg:SI VTYPE_REGNUM)] UNSPEC_VPREDICATE))]
+ "TARGET_XTHEADVECTOR"
+ "vext.x.v\t%0,%1,%2"
+ [(set_attr "type" "vimovvx")
+ (set_attr "mode" "<MODE>")])
juzhe.zhong@rivai.ai
发件人： joshua
发送时间： 2024-01-11 17:21
收件人： juzhe.zhong@rivai.ai; gcc-patches
抄送： Jim Wilson; palmer; andrew; philipp.tomsich; jeffreyalaw; christoph.muellner; jinma; cooper.qu
主题： Re：Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
"I didn't see theadvector-specific extension patterns. Could you show me?"
They are all in the file thead-vector.md.
For the sext/zext issue, perhaps I need some time to reproduce that optimization,
but I can clearly remember it is related to vwmul.
------------------------------------------------------------------
发件人：juzhe.zhong@rivai.ai <juzhe.zhong@rivai.ai>
发送时间：2024年1月11日(星期四) 17:17
收件人："cooper.joshua"<cooper.joshua@linux.alibaba.com>; "gcc-patches"<gcc-patches@gcc.gnu.org>
抄 送：Jim Wilson<jim.wilson.gcc@gmail.com>; palmer<palmer@dabbelt.com>; andrew<andrew@sifive.com>; "philipp.tomsich"<philipp.tomsich@vrull.eu>; jeffreyalaw<jeffreyalaw@gmail.com>; "christoph.muellner"<christoph.muellner@vrull.eu>; jinma<jinma@linux.alibaba.com>; "cooper.qu"<cooper.qu@linux.alibaba.com>
主 题：Re: Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
You mean which pattern optimized sext/vzext pattern?
I didn't see theadvector-specific extension patterns. Could you show me?
juzhe.zhong@rivai.ai
发件人： joshua
发送时间： 2024-01-11 17:14
收件人： juzhe.zhong@rivai.ai; gcc-patches
抄送： Jim Wilson; palmer; andrew; philipp.tomsich; jeffreyalaw; christoph.muellner; jinma; cooper.qu
主题： Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
To be specific, in CSE pass, the initial pattern will be optimized into the sext/zext pattern.
------------------------------------------------------------------
发件人：joshua <cooper.joshua@linux.alibaba.com>
发送时间：2024年1月11日(星期四) 17:11
收件人："juzhe.zhong@rivai.ai"<juzhe.zhong@rivai.ai>; "gcc-patches"<gcc-patches@gcc.gnu.org>
抄 送：Jim Wilson<jim.wilson.gcc@gmail.com>; palmer<palmer@dabbelt.com>; andrew<andrew@sifive.com>; "philipp.tomsich"<philipp.tomsich@vrull.eu>; jeffreyalaw<jeffreyalaw@gmail.com>; "christoph.muellner"<christoph.muellner@vrull.eu>; jinma<jinma@linux.alibaba.com>; "cooper.qu"<cooper.qu@linux.alibaba.com>
主 题：Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
sext/zext will be generated in O2 even without corresponding intrinsics.
------------------------------------------------------------------
发件人：juzhe.zhong@rivai.ai <juzhe.zhong@rivai.ai>
发送时间：2024年1月11日(星期四) 17:07
收件人："cooper.joshua"<cooper.joshua@linux.alibaba.com>; "gcc-patches"<gcc-patches@gcc.gnu.org>
抄 送：Jim Wilson<jim.wilson.gcc@gmail.com>; palmer<palmer@dabbelt.com>; andrew<andrew@sifive.com>; "philipp.tomsich"<philipp.tomsich@vrull.eu>; jeffreyalaw<jeffreyalaw@gmail.com>; "christoph.muellner"<christoph.muellner@vrull.eu>; "cooper.joshua"<cooper.joshua@linux.alibaba.com>; jinma<jinma@linux.alibaba.com>; "cooper.qu"<cooper.qu@linux.alibaba.com>
主 题：Re: [PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
enum required_ext
{
VECTOR_EXT, /* Vector extension */
+ XTHEADVECTOR_EXT, /* XTheadVector extension */
ZVBB_EXT, /* Cryto vector Zvbb sub-ext */
ZVBB_OR_ZVKB_EXT, /* Cryto vector Zvbb or zvkb sub-ext */
ZVBC_EXT, /* Crypto vector Zvbc sub-ext */
Add theadvector to the end of the enum.
+ case XTHEADVECTOR_EXT:
+ return TARGET_XTHEADVECTOR;
Same.
+ "&& register_operand (operands[0], <MODE>mode)
+ && register_operand (operands[3], <MODE>mode)
+ && satisfies_constraint_vu (operands[2])
+ && INTVAL (operands[7]) == riscv_vector::VLMAX"
You can use whole_reg_to_reg_move_p
Btw, I review again : RISC-V: Handle differences between XTheadvector and Vector
(any_extend:VWEXTI (match_operand:<V_DOUBLE_TRUNC> 3 "register_operand" "W21,W21,W21,W21,W42,W42,W42,W42,W84,W84,W84,W84, vr, vr")) (match_operand:VWEXTI 2 "vector_merge_operand" " vu, vu, 0, 0, vu, vu, 0, 0, vu, vu, 0, 0, vu, 0")))]- "TARGET_VECTOR"+ "TARGET_VECTOR && !TARGET_XTHEADVECTOR" "v<sz>ext.vf2\t%0,%3%p1" [(set_attr "type" "vext") (set_attr "mode" "<MODE>")@@ -3713,7 +3744,7 @@ (any_extend:VQEXTI (match_operand:<V_QUAD_TRUNC> 3 "register_operand" "W43,W43,W43,W43,W86,W86,W86,W86, vr, vr")) (match_operand:VQEXTI 2 "vector_merge_operand" " vu, vu, 0, 0, vu, vu, 0, 0, vu, 0")))]- "TARGET_VECTOR"+ "TARGET_VECTOR && !TARGET_XTHEADVECTOR" "v<sz>ext.vf4\t%0,%3%p1" [(set_attr "type" "vext") (set_attr "mode" "<MODE>")@@ -3734,7 +3765,7 @@ (any_extend:VOEXTI (match_operand:<V_OCT_TRUNC> 3 "register_operand" "W87,W87,W87,W87, vr, vr")) (match_operand:VOEXTI 2 "vector_merge_operand" " vu, vu, 0, 0, vu, 0")))]- "TARGET_VECTOR"+ "TARGET_VECTOR && !TARGET_XTHEADVECTOR" "v<sz>ext.vf8\t%0,%3%p1" [(set_attr "type" "vext") (set_attr "mode" "<MODE>")
Why do you add these !TARGERT_XTHEADVECRTOR ?
juzhe.zhong@rivai.ai
From: Jun Sha (Joshua)
Date: 2024-01-11 16:46
To: gcc-patches
CC: jim.wilson.gcc; palmer; andrew; philipp.tomsich; jeffreyalaw; christoph.muellner; juzhe.zhong; Jun Sha (Joshua); Jin Ma; Xianmiao Qu
Subject: [PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
This patch only involves the generation of xtheadvector
special load/store instructions and vext instructions.
gcc/ChangeLog:
* config/riscv/riscv-vector-builtins-bases.cc
(class th_loadstore_width): Define new builtin bases.
(class th_extract): Define new builtin bases.
(BASE): Define new builtin bases.
* config/riscv/riscv-vector-builtins-bases.h:
Define new builtin class.
* config/riscv/riscv-vector-builtins-shapes.cc
(struct th_loadstore_width_def): Define new builtin shapes.
(struct th_indexed_loadstore_width_def):
Define new builtin shapes.
(struct th_extract_def): Define new builtin shapes.
(SHAPE): Define new builtin shapes.
* config/riscv/riscv-vector-builtins-shapes.h:
Define new builtin shapes.
* config/riscv/riscv-vector-builtins.cc (DEF_RVV_FUNCTION):
* config/riscv/riscv-vector-builtins.h (enum required_ext):
(struct function_group_info):
* config/riscv/t-riscv: Add thead-vector-builtins-functions.def
* config/riscv/thead-vector.md
(@pred_mov_width<vlmem_op_attr><mode>): Add new patterns.
(*pred_mov_width<vlmem_op_attr><mode>): Likewise.
(@pred_store_width<vlmem_op_attr><mode>): Likewise.
(@pred_strided_load_width<vlmem_op_attr><mode>): Likewise.
(@pred_strided_store_width<vlmem_op_attr><mode>): Likewise.
(@pred_indexed_load_width<vlmem_op_attr><mode>): Likewise.
(@pred_indexed_<vlmem_order_attr>store_width<vlmem_op_attr><mode>):
(@pred_th_extract<mode>): Likewise.
(*pred_th_extract<mode>): Likewise.
* config/riscv/thead-vector-builtins-functions.def: New file.
gcc/testsuite/ChangeLog:
* gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c: New test.
Co-authored-by: Jin Ma <jinma@linux.alibaba.com>
Co-authored-by: Xianmiao Qu <cooper.qu@linux.alibaba.com>
Co-authored-by: Christoph Müllner <christoph.muellner@vrull.eu>
---
.../riscv/riscv-vector-builtins-bases.cc | 139 ++++++++++
.../riscv/riscv-vector-builtins-bases.h | 31 +++
.../riscv/riscv-vector-builtins-shapes.cc | 160 +++++++++++
.../riscv/riscv-vector-builtins-shapes.h | 3 +
gcc/config/riscv/riscv-vector-builtins.cc | 70 +++++
gcc/config/riscv/riscv-vector-builtins.h | 3 +
gcc/config/riscv/t-riscv | 1 +
.../riscv/thead-vector-builtins-functions.def | 39 +++
gcc/config/riscv/thead-vector.md | 253 ++++++++++++++++++
.../riscv/rvv/xtheadvector/vlb-vsb.c | 68 +++++
.../riscv/rvv/xtheadvector/vlbu-vsb.c | 68 +++++
.../riscv/rvv/xtheadvector/vlh-vsh.c | 68 +++++
.../riscv/rvv/xtheadvector/vlhu-vsh.c | 68 +++++
.../riscv/rvv/xtheadvector/vlw-vsw.c | 68 +++++
.../riscv/rvv/xtheadvector/vlwu-vsw.c | 68 +++++
15 files changed, 1107 insertions(+)
create mode 100644 gcc/config/riscv/thead-vector-builtins-functions.def
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c
diff --git a/gcc/config/riscv/riscv-vector-builtins-bases.cc b/gcc/config/riscv/riscv-vector-builtins-bases.cc
index 46f1a1da33e..3eba7943757 100644
--- a/gcc/config/riscv/riscv-vector-builtins-bases.cc
+++ b/gcc/config/riscv/riscv-vector-builtins-bases.cc
@@ -2125,6 +2125,83 @@ public:
}
};
+/* Implements
+ * th.vl(b/h/w)[u].v/th.vs(b/h/w)[u].v/th.vls(b/h/w)[u].v/th.vss(b/h/w)[u].v/
+ * th.vlx(b/h/w)[u].v/th.vs[u]x(b/h/w).v
+ * codegen. */
+template<bool STORE_P, lst_type LST_TYPE, int UNSPEC>
+class th_loadstore_width : public function_base
+{
+public:
+ bool apply_tail_policy_p () const override { return !STORE_P; }
+ bool apply_mask_policy_p () const override { return !STORE_P; }
+
+ unsigned int call_properties (const function_instance &) const override
+ {
+ if (STORE_P)
+ return CP_WRITE_MEMORY;
+ else
+ return CP_READ_MEMORY;
+ }
+
+ bool can_be_overloaded_p (enum predication_type_index pred) const override
+ {
+ if (STORE_P || LST_TYPE == LST_INDEXED)
+ return true;
+ return pred != PRED_TYPE_none;
+ }
+
+ rtx expand (function_expander &e) const override
+ {
+ gcc_assert (TARGET_XTHEADVECTOR);
+ if (LST_TYPE == LST_INDEXED)
+ {
+ if (STORE_P)
+ return e.use_exact_insn (
+ code_for_pred_indexed_store_width (UNSPEC, UNSPEC,
+ e.vector_mode ()));
+ else
+ return e.use_exact_insn (
+ code_for_pred_indexed_load_width (UNSPEC, e.vector_mode ()));
+ }
+ else if (LST_TYPE == LST_STRIDED)
+ {
+ if (STORE_P)
+ return e.use_contiguous_store_insn (
+ code_for_pred_strided_store_width (UNSPEC, e.vector_mode ()));
+ else
+ return e.use_contiguous_load_insn (
+ code_for_pred_strided_load_width (UNSPEC, e.vector_mode ()));
+ }
+ else
+ {
+ if (STORE_P)
+ return e.use_contiguous_store_insn (
+ code_for_pred_store_width (UNSPEC, e.vector_mode ()));
+ else
+ return e.use_contiguous_load_insn (
+ code_for_pred_mov_width (UNSPEC, e.vector_mode ()));
+ }
+ }
+};
+
+/* Implements vext.x.v. */
+class th_extract : public function_base
+{
+public:
+ bool apply_vl_p () const override { return false; }
+ bool apply_tail_policy_p () const override { return false; }
+ bool apply_mask_policy_p () const override { return false; }
+ bool use_mask_predication_p () const override { return false; }
+ bool has_merge_operand_p () const override { return false; }
+
+ rtx expand (function_expander &e) const override
+ {
+ gcc_assert (TARGET_XTHEADVECTOR);
+ return e.use_exact_insn (code_for_pred_th_extract (e.vector_mode ()));
+ }
+};
+
/* Below implements are vector crypto */
/* Implements vandn.[vv,vx] */
class vandn : public function_base
@@ -2587,6 +2664,37 @@ static CONSTEXPR const seg_indexed_load<UNSPEC_ORDERED> vloxseg_obj;
static CONSTEXPR const seg_indexed_store<UNSPEC_UNORDERED> vsuxseg_obj;
static CONSTEXPR const seg_indexed_store<UNSPEC_ORDERED> vsoxseg_obj;
static CONSTEXPR const vlsegff vlsegff_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLB> vlb_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLBU> vlbu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLH> vlh_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLHU> vlhu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLW> vlw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLWU> vlwu_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_UNIT_STRIDE, UNSPEC_TH_VLB> vsb_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_UNIT_STRIDE, UNSPEC_TH_VLH> vsh_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_UNIT_STRIDE, UNSPEC_TH_VLW> vsw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSB> vlsb_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSBU> vlsbu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSH> vlsh_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSHU> vlshu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSW> vlsw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSWU> vlswu_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_STRIDED, UNSPEC_TH_VLSB> vssb_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_STRIDED, UNSPEC_TH_VLSH> vssh_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_STRIDED, UNSPEC_TH_VLSW> vssw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXB> vlxb_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXBU> vlxbu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXH> vlxh_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXHU> vlxhu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXW> vlxw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXWU> vlxwu_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VLXB> vsxb_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VLXH> vsxh_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VLXW> vsxw_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VSUXB> vsuxb_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VSUXH> vsuxh_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VSUXW> vsuxw_obj;
+static CONSTEXPR const th_extract vext_x_v_obj;
/* Crypto Vector */
static CONSTEXPR const vandn vandn_obj;
@@ -2878,6 +2986,37 @@ BASE (vloxseg)
BASE (vsuxseg)
BASE (vsoxseg)
BASE (vlsegff)
+BASE (vlb)
+BASE (vlh)
+BASE (vlw)
+BASE (vlbu)
+BASE (vlhu)
+BASE (vlwu)
+BASE (vsb)
+BASE (vsh)
+BASE (vsw)
+BASE (vlsb)
+BASE (vlsh)
+BASE (vlsw)
+BASE (vlsbu)
+BASE (vlshu)
+BASE (vlswu)
+BASE (vssb)
+BASE (vssh)
+BASE (vssw)
+BASE (vlxb)
+BASE (vlxh)
+BASE (vlxw)
+BASE (vlxbu)
+BASE (vlxhu)
+BASE (vlxwu)
+BASE (vsxb)
+BASE (vsxh)
+BASE (vsxw)
+BASE (vsuxb)
+BASE (vsuxh)
+BASE (vsuxw)
+BASE (vext_x_v)
/* Crypto vector */
BASE (vandn)
BASE (vbrev)
diff --git a/gcc/config/riscv/riscv-vector-builtins-bases.h b/gcc/config/riscv/riscv-vector-builtins-bases.h
index 1122e3801a7..565a0311d2b 100644
--- a/gcc/config/riscv/riscv-vector-builtins-bases.h
+++ b/gcc/config/riscv/riscv-vector-builtins-bases.h
@@ -299,6 +299,37 @@ extern const function_base *const vloxseg;
extern const function_base *const vsuxseg;
extern const function_base *const vsoxseg;
extern const function_base *const vlsegff;
+extern const function_base *const vlb;
+extern const function_base *const vlh;
+extern const function_base *const vlw;
+extern const function_base *const vlbu;
+extern const function_base *const vlhu;
+extern const function_base *const vlwu;
+extern const function_base *const vsb;
+extern const function_base *const vsh;
+extern const function_base *const vsw;
+extern const function_base *const vlsb;
+extern const function_base *const vlsh;
+extern const function_base *const vlsw;
+extern const function_base *const vlsbu;
+extern const function_base *const vlshu;
+extern const function_base *const vlswu;
+extern const function_base *const vssb;
+extern const function_base *const vssh;
+extern const function_base *const vssw;
+extern const function_base *const vlxb;
+extern const function_base *const vlxh;
+extern const function_base *const vlxw;
+extern const function_base *const vlxbu;
+extern const function_base *const vlxhu;
+extern const function_base *const vlxwu;
+extern const function_base *const vsxb;
+extern const function_base *const vsxh;
+extern const function_base *const vsxw;
+extern const function_base *const vsuxb;
+extern const function_base *const vsuxh;
+extern const function_base *const vsuxw;
+extern const function_base *const vext_x_v;
/* Below function_base are Vectro Crypto*/
extern const function_base *const vandn;
extern const function_base *const vbrev;
diff --git a/gcc/config/riscv/riscv-vector-builtins-shapes.cc b/gcc/config/riscv/riscv-vector-builtins-shapes.cc
index 1e4f4d53de6..8e90b17a94b 100644
--- a/gcc/config/riscv/riscv-vector-builtins-shapes.cc
+++ b/gcc/config/riscv/riscv-vector-builtins-shapes.cc
@@ -211,6 +211,146 @@ struct indexed_loadstore_def : public function_shape
}
};
+/* Add one function instance for GROUP, using operand suffix at index OI,
+ mode suffix at index PAIR && bi and predication suffix at index pred_idx. */
+static void
+build_th_loadstore (function_builder &b, const function_group_info &group,
+ unsigned int pred_idx, unsigned int vec_type_idx)
+{
+ auto_vec<tree, 5> argument_types;
+ function_instance function_instance (group.base_name, *group.base,
+ *group.shape,
+ group.ops_infos.types[vec_type_idx],
+ group.preds[pred_idx], &group.ops_infos);
+ tree return_type = group.ops_infos.ret.get_tree_type (
+ group.ops_infos.types[vec_type_idx].index);
+ b.allocate_argument_types (function_instance, argument_types);
+ b.apply_predication (function_instance, return_type, argument_types);
+
+ if (TARGET_XTHEADVECTOR && !check_type (return_type, argument_types))
+ return;
+
+ tree type = builtin_types[group.ops_infos.types[vec_type_idx].index].vector;
+ if (strstr (group.base_name, "l")
+ && strstr (group.base_name, "u")
+ && !TYPE_UNSIGNED (TREE_TYPE (type)))
+ return;
+
+ if (strstr (group.base_name, "l")
+ && !strstr (group.base_name, "u")
+ && TYPE_UNSIGNED (TREE_TYPE (type)))
+ return;
+
+ machine_mode mode = TYPE_MODE (type);
+ int sew = GET_MODE_BITSIZE (GET_MODE_INNER (mode));
+ if (strstr (group.base_name, "h") && sew == 8)
+ return;
+
+ if (strstr (group.base_name, "w") && (sew == 8 || sew ==16))
+ return;
+
+ b.add_overloaded_function (function_instance, *group.shape);
+ b.add_unique_function (function_instance, (*group.shape), return_type,
+ argument_types);
+}
+
+/* th_loadstore_width_def class. */
+struct th_loadstore_width_def : public build_base
+{
+ void build (function_builder &b,
+ const function_group_info &group) const override
+ {
+ for (unsigned int pred_idx = 0; group.preds[pred_idx] != NUM_PRED_TYPES;
+ ++pred_idx)
+ {
+ for (unsigned int vec_type_idx = 0;
+ group.ops_infos.types[vec_type_idx].index != NUM_VECTOR_TYPES;
+ ++vec_type_idx)
+ {
+ build_th_loadstore (b, group, pred_idx, vec_type_idx);
+ }
+ }
+ }
+
+ char *get_name (function_builder &b, const function_instance &instance,
+ bool overloaded_p) const override
+ {
+ /* Return nullptr if it can not be overloaded. */
+ if (overloaded_p && !instance.base->can_be_overloaded_p (instance.pred))
+ return nullptr;
+
+ b.append_name ("__riscv_th_");
+ b.append_name (instance.base_name);
+
+ /* vop_v --> vop_v_<type>. */
+ if (!overloaded_p)
+ {
+ /* vop --> vop_v. */
+ b.append_name (operand_suffixes[instance.op_info->op]);
+ /* vop_v --> vop_v_<type>. */
+ b.append_name (type_suffixes[instance.type.index].vector);
+ }
+
+ /* According to rvv-intrinsic-doc, it does not add "_m" suffix
+ for vop_m C++ overloaded API. */
+ if (overloaded_p && instance.pred == PRED_TYPE_m)
+ return b.finish_name ();
+ b.append_name (predication_suffixes[instance.pred]);
+ return b.finish_name ();
+ }
+};
+
+
+/* th_indexed_loadstore_width_def class. */
+struct th_indexed_loadstore_width_def : public function_shape
+{
+ void build (function_builder &b,
+ const function_group_info &group) const override
+ {
+ for (unsigned int pred_idx = 0; group.preds[pred_idx] != NUM_PRED_TYPES;
+ ++pred_idx)
+ {
+ for (unsigned int vec_type_idx = 0;
+ group.ops_infos.types[vec_type_idx].index != NUM_VECTOR_TYPES;
+ ++vec_type_idx)
+ {
+ tree index_type = group.ops_infos.args[1].get_tree_type (
+ group.ops_infos.types[vec_type_idx].index);
+ if (!index_type)
+ continue;
+ build_th_loadstore (b, group, pred_idx, vec_type_idx);
+ }
+ }
+ }
+
+ char *get_name (function_builder &b, const function_instance &instance,
+ bool overloaded_p) const override
+ {
+
+ /* Return nullptr if it can not be overloaded. */
+ if (overloaded_p && !instance.base->can_be_overloaded_p (instance.pred))
+ return nullptr;
+
+ b.append_name ("__riscv_th_");
+ b.append_name (instance.base_name);
+ /* vop_v --> vop_v_<type>. */
+ if (!overloaded_p)
+ {
+ /* vop --> vop_v. */
+ b.append_name (operand_suffixes[instance.op_info->op]);
+ /* vop_v --> vop_v_<type>. */
+ b.append_name (type_suffixes[instance.type.index].vector);
+ }
+
+ /* According to rvv-intrinsic-doc, it does not add "_m" suffix
+ for vop_m C++ overloaded API. */
+ if (overloaded_p && instance.pred == PRED_TYPE_m)
+ return b.finish_name ();
+ b.append_name (predication_suffixes[instance.pred]);
+ return b.finish_name ();
+ }
+};
+
/* alu_def class. */
struct alu_def : public build_base
{
@@ -632,6 +772,23 @@ struct reduc_alu_def : public build_base
}
};
+/* th_extract_def class. */
+struct th_extract_def : public build_base
+{
+ char *get_name (function_builder &b, const function_instance &instance,
+ bool overloaded_p) const override
+ {
+ b.append_name ("__riscv_th_");
+ b.append_name (instance.base_name);
+
+ if (overloaded_p)
+ return b.finish_name ();
+ b.append_name (type_suffixes[instance.type.index].vector);
+ b.append_name (type_suffixes[instance.type.index].scalar);
+ return b.finish_name ();
+ }
+};
+
/* scalar_move_def class. */
struct scalar_move_def : public build_base
{
@@ -1094,6 +1251,8 @@ SHAPE(vsetvl, vsetvl)
SHAPE(vsetvl, vsetvlmax)
SHAPE(loadstore, loadstore)
SHAPE(indexed_loadstore, indexed_loadstore)
+SHAPE(th_loadstore_width, th_loadstore_width)
+SHAPE(th_indexed_loadstore_width, th_indexed_loadstore_width)
SHAPE(alu, alu)
SHAPE(alu_frm, alu_frm)
SHAPE(widen_alu, widen_alu)
@@ -1106,6 +1265,7 @@ SHAPE(move, move)
SHAPE(mask_alu, mask_alu)
SHAPE(reduc_alu, reduc_alu)
SHAPE(reduc_alu_frm, reduc_alu_frm)
+SHAPE(th_extract, th_extract)
SHAPE(scalar_move, scalar_move)
SHAPE(vundefined, vundefined)
SHAPE(misc, misc)
diff --git a/gcc/config/riscv/riscv-vector-builtins-shapes.h b/gcc/config/riscv/riscv-vector-builtins-shapes.h
index ac2a28ce017..a7624d0fabd 100644
--- a/gcc/config/riscv/riscv-vector-builtins-shapes.h
+++ b/gcc/config/riscv/riscv-vector-builtins-shapes.h
@@ -28,6 +28,8 @@ extern const function_shape *const vsetvl;
extern const function_shape *const vsetvlmax;
extern const function_shape *const loadstore;
extern const function_shape *const indexed_loadstore;
+extern const function_shape *const th_loadstore_width;
+extern const function_shape *const th_indexed_loadstore_width;
extern const function_shape *const alu;
extern const function_shape *const alu_frm;
extern const function_shape *const widen_alu;
@@ -41,6 +43,7 @@ extern const function_shape *const mask_alu;
extern const function_shape *const reduc_alu;
extern const function_shape *const reduc_alu_frm;
extern const function_shape *const scalar_move;
+extern const function_shape *const th_extract;
extern const function_shape *const vundefined;
extern const function_shape *const misc;
extern const function_shape *const vset;
diff --git a/gcc/config/riscv/riscv-vector-builtins.cc b/gcc/config/riscv/riscv-vector-builtins.cc
index 25e0b6e56de..44b9fec1898 100644
--- a/gcc/config/riscv/riscv-vector-builtins.cc
+++ b/gcc/config/riscv/riscv-vector-builtins.cc
@@ -934,6 +934,32 @@ static CONSTEXPR const rvv_arg_type_info ext_vcreate_args[]
= {rvv_arg_type_info (RVV_BASE_vector),
rvv_arg_type_info_end};
+/* A list of args for vector_type func (const scalar_type *, size_t)
+ * function. */
+static CONSTEXPR const rvv_arg_type_info scalar_const_ptr_size_args[]
+ = {rvv_arg_type_info (RVV_BASE_scalar_const_ptr),
+ rvv_arg_type_info (RVV_BASE_size), rvv_arg_type_info_end};
+
+/* A list of args for vector_type func (const scalar_type *, eew8_index_type)
+ * function. */
+static CONSTEXPR const rvv_arg_type_info scalar_const_ptr_index_args[]
+ = {rvv_arg_type_info (RVV_BASE_scalar_const_ptr),
+ rvv_arg_type_info (RVV_BASE_unsigned_vector), rvv_arg_type_info_end};
+
+/* A list of args for void func (scalar_type *, eew8_index_type, vector_type)
+ * function. */
+static CONSTEXPR const rvv_arg_type_info scalar_ptr_index_args[]
+ = {rvv_arg_type_info (RVV_BASE_scalar_ptr),
+ rvv_arg_type_info (RVV_BASE_unsigned_vector),
+ rvv_arg_type_info (RVV_BASE_vector), rvv_arg_type_info_end};
+
+/* A list of args for void func (scalar_type *, size_t, vector_type)
+ * function. */
+static CONSTEXPR const rvv_arg_type_info scalar_ptr_size_args[]
+ = {rvv_arg_type_info (RVV_BASE_scalar_ptr),
+ rvv_arg_type_info (RVV_BASE_size), rvv_arg_type_info (RVV_BASE_vector),
+ rvv_arg_type_info_end};
+
/* A list of none preds that will be registered for intrinsic functions. */
static CONSTEXPR const predication_type_index none_preds[]
= {PRED_TYPE_none, NUM_PRED_TYPES};
@@ -1455,6 +1481,14 @@ static CONSTEXPR const rvv_op_info iu_shift_vvv_ops
rvv_arg_type_info (RVV_BASE_vector), /* Return type */
shift_vv_args /* Args */};
+/* A static operand information for scalar_type func (vector_type, size_t)
+ * function registration. */
+static CONSTEXPR const rvv_op_info iu_x_s_u_ops
+ = {iu_ops, /* Types */
+ OP_TYPE_vx, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_scalar), /* Return type */
+ v_size_args /* Args */};
+
/* A static operand information for vector_type func (vector_type, size_t)
* function registration. */
static CONSTEXPR const rvv_op_info iu_shift_vvx_ops
@@ -2638,6 +2672,38 @@ static CONSTEXPR const rvv_op_info all_v_vcreate_lmul4_x2_ops
rvv_arg_type_info (RVV_BASE_vlmul_ext_x2), /* Return type */
ext_vcreate_args /* Args */};
+/* A static operand information for vector_type func (const scalar_type *,
+ * size_t) function registration. */
+static CONSTEXPR const rvv_op_info all_v_scalar_const_ptr_size_ops
+ = {all_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_size_args /* Args */};
+
+/* A static operand information for void func (scalar_type *, size_t,
+ * vector_type) function registration. */
+static CONSTEXPR const rvv_op_info all_v_scalar_ptr_size_ops
+ = {all_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_void), /* Return type */
+ scalar_ptr_size_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * index_type) function registration. */
+static CONSTEXPR const rvv_op_info all_v_scalar_const_ptr_index_ops
+ = {all_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_index_args /* Args */};
+
+/* A static operand information for void func (scalar_type *, index_type,
+ * vector_type) function registration. */
+static CONSTEXPR const rvv_op_info all_v_scalar_ptr_index_ops
+ = {all_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_void), /* Return type */
+ scalar_ptr_index_args /* Args */};
+
/* A static operand information for vector_type func (vector_type).
Some ins just supports SEW=32, such as crypto vectol Zvkg extension.
* function registration. */
@@ -2816,6 +2882,10 @@ static function_group_info function_groups[] = {
#define DEF_RVV_FUNCTION(NAME, SHAPE, PREDS, OPS_INFO) \
{#NAME, &bases::NAME, &shapes::SHAPE, PREDS, OPS_INFO, REQUIRED_EXTENSIONS},
#include "riscv-vector-builtins-functions.def"
+#undef DEF_RVV_FUNCTION
+#define DEF_RVV_FUNCTION(NAME, SHAPE, PREDS, OPS_INFO) \
+ {#NAME, &bases::NAME, &shapes::SHAPE, PREDS, OPS_INFO, REQUIRED_EXTENSIONS},
+#include "thead-vector-builtins-functions.def"
};
/* The RVV types, with their built-in
diff --git a/gcc/config/riscv/riscv-vector-builtins.h b/gcc/config/riscv/riscv-vector-builtins.h
index 54c8824ff92..a8ee39a3cb2 100644
--- a/gcc/config/riscv/riscv-vector-builtins.h
+++ b/gcc/config/riscv/riscv-vector-builtins.h
@@ -114,6 +114,7 @@ static const unsigned int CP_WRITE_CSR = 1U << 5;
enum required_ext
{
VECTOR_EXT, /* Vector extension */
+ XTHEADVECTOR_EXT, /* XTheadVector extension */
ZVBB_EXT, /* Cryto vector Zvbb sub-ext */
ZVBB_OR_ZVKB_EXT, /* Cryto vector Zvbb or zvkb sub-ext */
ZVBC_EXT, /* Crypto vector Zvbc sub-ext */
@@ -234,6 +235,8 @@ struct function_group_info
{
case VECTOR_EXT:
return TARGET_VECTOR;
+ case XTHEADVECTOR_EXT:
+ return TARGET_XTHEADVECTOR;
case ZVBB_EXT:
return TARGET_ZVBB;
case ZVBB_OR_ZVKB_EXT:
diff --git a/gcc/config/riscv/t-riscv b/gcc/config/riscv/t-riscv
index 32de6b851c1..38494320d8b 100644
--- a/gcc/config/riscv/t-riscv
+++ b/gcc/config/riscv/t-riscv
@@ -1,6 +1,7 @@
RISCV_BUILTINS_H = $(srcdir)/config/riscv/riscv-vector-builtins.h \
$(srcdir)/config/riscv/riscv-vector-builtins.def \
$(srcdir)/config/riscv/riscv-vector-builtins-functions.def \
+ $(srcdir)/config/riscv/thead-vector-builtins-functions.def \
riscv-vector-type-indexer.gen.def
riscv-builtins.o: $(srcdir)/config/riscv/riscv-builtins.cc $(CONFIG_H) \
diff --git a/gcc/config/riscv/thead-vector-builtins-functions.def b/gcc/config/riscv/thead-vector-builtins-functions.def
new file mode 100644
index 00000000000..fd3ba29bae9
--- /dev/null
+++ b/gcc/config/riscv/thead-vector-builtins-functions.def
@@ -0,0 +1,39 @@
+#ifndef DEF_RVV_FUNCTION
+#define DEF_RVV_FUNCTION(NAME, SHAPE, PREDS, OPS_INFO)
+#endif
+
+#define REQUIRED_EXTENSIONS XTHEADVECTOR_EXT
+DEF_RVV_FUNCTION (vlb, th_loadstore_width, full_preds, all_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (vlh, th_loadstore_width, full_preds, all_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (vlw, th_loadstore_width, full_preds, all_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (vlbu, th_loadstore_width, full_preds, all_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (vlhu, th_loadstore_width, full_preds, all_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (vlwu, th_loadstore_width, full_preds, all_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (vsb, th_loadstore_width, none_m_preds, all_v_scalar_ptr_ops)
+DEF_RVV_FUNCTION (vsh, th_loadstore_width, none_m_preds, all_v_scalar_ptr_ops)
+DEF_RVV_FUNCTION (vsw, th_loadstore_width, none_m_preds, all_v_scalar_ptr_ops)
+DEF_RVV_FUNCTION (vlsb, th_loadstore_width, full_preds, all_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (vlsh, th_loadstore_width, full_preds, all_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (vlsw, th_loadstore_width, full_preds, all_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (vlsbu, th_loadstore_width, full_preds, all_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (vlshu, th_loadstore_width, full_preds, all_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (vlswu, th_loadstore_width, full_preds, all_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (vssb, th_loadstore_width, none_m_preds, all_v_scalar_ptr_size_ops)
+DEF_RVV_FUNCTION (vssh, th_loadstore_width, none_m_preds, all_v_scalar_ptr_size_ops)
+DEF_RVV_FUNCTION (vssw, th_loadstore_width, none_m_preds, all_v_scalar_ptr_size_ops)
+DEF_RVV_FUNCTION (vlxb, th_indexed_loadstore_width, full_preds, all_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (vlxh, th_indexed_loadstore_width, full_preds, all_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (vlxw, th_indexed_loadstore_width, full_preds, all_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (vlxbu, th_indexed_loadstore_width, full_preds, all_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (vlxhu, th_indexed_loadstore_width, full_preds, all_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (vlxwu, th_indexed_loadstore_width, full_preds, all_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (vsxb, th_indexed_loadstore_width, none_m_preds, all_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (vsxh, th_indexed_loadstore_width, none_m_preds, all_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (vsxw, th_indexed_loadstore_width, none_m_preds, all_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (vsuxb, th_indexed_loadstore_width, none_m_preds, all_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (vsuxh, th_indexed_loadstore_width, none_m_preds, all_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (vsuxw, th_indexed_loadstore_width, none_m_preds, all_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (vext_x_v, th_extract, none_preds, iu_x_s_u_ops)
+#undef REQUIRED_EXTENSIONS
+
+#undef DEF_RVV_FUNCTION
diff --git a/gcc/config/riscv/thead-vector.md b/gcc/config/riscv/thead-vector.md
index 696b815252d..0f3700d9269 100644
--- a/gcc/config/riscv/thead-vector.md
+++ b/gcc/config/riscv/thead-vector.md
@@ -1,7 +1,95 @@
(define_c_enum "unspec" [
+ UNSPEC_TH_VLB
+ UNSPEC_TH_VLBU
+ UNSPEC_TH_VLH
+ UNSPEC_TH_VLHU
+ UNSPEC_TH_VLW
+ UNSPEC_TH_VLWU
+
+ UNSPEC_TH_VLSB
+ UNSPEC_TH_VLSBU
+ UNSPEC_TH_VLSH
+ UNSPEC_TH_VLSHU
+ UNSPEC_TH_VLSW
+ UNSPEC_TH_VLSWU
+
+ UNSPEC_TH_VLXB
+ UNSPEC_TH_VLXBU
+ UNSPEC_TH_VLXH
+ UNSPEC_TH_VLXHU
+ UNSPEC_TH_VLXW
+ UNSPEC_TH_VLXWU
+
+ UNSPEC_TH_VSUXB
+ UNSPEC_TH_VSUXH
+ UNSPEC_TH_VSUXW
+
UNSPEC_TH_VWLDST
])
+(define_int_iterator UNSPEC_TH_VLMEM_OP [
+ UNSPEC_TH_VLB UNSPEC_TH_VLBU
+ UNSPEC_TH_VLH UNSPEC_TH_VLHU
+ UNSPEC_TH_VLW UNSPEC_TH_VLWU
+])
+
+(define_int_iterator UNSPEC_TH_VLSMEM_OP [
+ UNSPEC_TH_VLSB UNSPEC_TH_VLSBU
+ UNSPEC_TH_VLSH UNSPEC_TH_VLSHU
+ UNSPEC_TH_VLSW UNSPEC_TH_VLSWU
+])
+
+(define_int_iterator UNSPEC_TH_VLXMEM_OP [
+ UNSPEC_TH_VLXB UNSPEC_TH_VLXBU
+ UNSPEC_TH_VLXH UNSPEC_TH_VLXHU
+ UNSPEC_TH_VLXW UNSPEC_TH_VLXWU
+])
+
+(define_int_attr vlmem_op_attr [
+ (UNSPEC_TH_VLB "b") (UNSPEC_TH_VLBU "bu")
+ (UNSPEC_TH_VLH "h") (UNSPEC_TH_VLHU "hu")
+ (UNSPEC_TH_VLW "w") (UNSPEC_TH_VLWU "wu")
+ (UNSPEC_TH_VLSB "b") (UNSPEC_TH_VLSBU "bu")
+ (UNSPEC_TH_VLSH "h") (UNSPEC_TH_VLSHU "hu")
+ (UNSPEC_TH_VLSW "w") (UNSPEC_TH_VLSWU "wu")
+ (UNSPEC_TH_VLXB "b") (UNSPEC_TH_VLXBU "bu")
+ (UNSPEC_TH_VLXH "h") (UNSPEC_TH_VLXHU "hu")
+ (UNSPEC_TH_VLXW "w") (UNSPEC_TH_VLXWU "wu")
+ (UNSPEC_TH_VSUXB "b")
+ (UNSPEC_TH_VSUXH "h")
+ (UNSPEC_TH_VSUXW "w")
+])
+
+(define_int_attr vlmem_order_attr [
+ (UNSPEC_TH_VLXB "")
+ (UNSPEC_TH_VLXH "")
+ (UNSPEC_TH_VLXW "")
+ (UNSPEC_TH_VSUXB "u")
+ (UNSPEC_TH_VSUXH "u")
+ (UNSPEC_TH_VSUXW "u")
+])
+
+(define_int_iterator UNSPEC_TH_VSMEM_OP [
+ UNSPEC_TH_VLB
+ UNSPEC_TH_VLH
+ UNSPEC_TH_VLW
+])
+
+(define_int_iterator UNSPEC_TH_VSSMEM_OP [
+ UNSPEC_TH_VLSB
+ UNSPEC_TH_VLSH
+ UNSPEC_TH_VLSW
+])
+
+(define_int_iterator UNSPEC_TH_VSXMEM_OP [
+ UNSPEC_TH_VLXB
+ UNSPEC_TH_VLXH
+ UNSPEC_TH_VLXW
+ UNSPEC_TH_VSUXB
+ UNSPEC_TH_VSUXH
+ UNSPEC_TH_VSUXW
+])
+
(define_mode_iterator V_VLS_VT [V VLS VT])
(define_mode_iterator V_VB_VLS_VT [V VB VLS VT])
@@ -100,3 +188,168 @@
}
[(set_attr "type" "vldm,vstm,vmalu,vmalu,vmalu")
(set_attr "mode" "<MODE>")])
+
+(define_expand "@pred_mov_width<vlmem_op_attr><mode>"
+ [(set (match_operand:V_VLS 0 "nonimmediate_operand")
+ (if_then_else:V_VLS
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand")
+ (match_operand 4 "vector_length_operand")
+ (match_operand 5 "const_int_operand")
+ (match_operand 6 "const_int_operand")
+ (match_operand 7 "const_int_operand")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLMEM_OP)
+ (match_operand:V_VLS 3 "vector_move_operand")
+ (match_operand:V_VLS 2 "vector_merge_operand")))]
+ "TARGET_XTHEADVECTOR"
+ {})
+
+(define_insn_and_split "*pred_mov_width<vlmem_op_attr><mode>"
+ [(set (match_operand:V_VLS 0 "nonimmediate_operand" "=vr, vr, vd, m, vr, vr")
+ (if_then_else:V_VLS
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1, Wc1, vm, vmWc1, Wc1, Wc1")
+ (match_operand 4 "vector_length_operand" " rK, rK, rK, rK, rK, rK")
+ (match_operand 5 "const_int_operand" " i, i, i, i, i, i")
+ (match_operand 6 "const_int_operand" " i, i, i, i, i, i")
+ (match_operand 7 "const_int_operand" " i, i, i, i, i, i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLMEM_OP)
+ (match_operand:V_VLS 3 "reg_or_mem_operand" " m, m, m, vr, vr, vr")
+ (match_operand:V_VLS 2 "vector_merge_operand" " 0, vu, vu, vu, vu, 0")))]
+ "(TARGET_XTHEADVECTOR
+ && (register_operand (operands[0], <MODE>mode)
+ || register_operand (operands[3], <MODE>mode)))"
+ "@
+ vl<vlmem_op_attr>.v\t%0,%3%p1
+ vl<vlmem_op_attr>.v\t%0,%3
+ vl<vlmem_op_attr>.v\t%0,%3,%1.t
+ vs<vlmem_op_attr>.v\t%3,%0%p1
+ vmv.v.v\t%0,%3
+ vmv.v.v\t%0,%3"
+ "&& register_operand (operands[0], <MODE>mode)
+ && register_operand (operands[3], <MODE>mode)
+ && satisfies_constraint_vu (operands[2])
+ && INTVAL (operands[7]) == riscv_vector::VLMAX"
+ [(set (match_dup 0) (match_dup 3))]
+ ""
+ [(set_attr "type" "vlde,vlde,vlde,vste,vimov,vimov")
+ (set_attr "mode" "<MODE>")])
+
+(define_insn "@pred_store_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "memory_operand" "+m")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1")
+ (match_operand 3 "vector_length_operand" " rK")
+ (match_operand 4 "const_int_operand" " i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VSMEM_OP)
+ (match_operand:VI 2 "register_operand" " vr")
+ (match_dup 0)))]
+ "TARGET_XTHEADVECTOR"
+ "vs<vlmem_op_attr>.v\t%2,%0%p1"
+ [(set_attr "type" "vste")
+ (set_attr "mode" "<MODE>")
+ (set (attr "avl_type_idx") (const_int 4))
+ (set_attr "vl_op_idx" "3")])
+
+(define_insn "@pred_strided_load_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "register_operand" "=vr, vr, vd")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1, Wc1, vm")
+ (match_operand 5 "vector_length_operand" " rK, rK, rK")
+ (match_operand 6 "const_int_operand" " i, i, i")
+ (match_operand 7 "const_int_operand" " i, i, i")
+ (match_operand 8 "const_int_operand" " i, i, i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLSMEM_OP)
+ (unspec:VI
+ [(match_operand:VI 3 "memory_operand" " m, m, m")
+ (match_operand 4 "pmode_reg_or_0_operand" " rJ, rJ, rJ")] UNSPEC_TH_VLSMEM_OP)
+ (match_operand:VI 2 "vector_merge_operand" " 0, vu, vu")))]
+ "TARGET_XTHEADVECTOR"
+ "vls<vlmem_op_attr>.v\t%0,%3,%z4%p1"
+ [(set_attr "type" "vlds")
+ (set_attr "mode" "<MODE>")])
+
+(define_insn "@pred_strided_store_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "memory_operand" "+m")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1")
+ (match_operand 4 "vector_length_operand" " rK")
+ (match_operand 5 "const_int_operand" " i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VSSMEM_OP)
+ (unspec:VI
+ [(match_operand 2 "pmode_reg_or_0_operand" " rJ")
+ (match_operand:VI 3 "register_operand" " vr")] UNSPEC_TH_VSSMEM_OP)
+ (match_dup 0)))]
+ "TARGET_XTHEADVECTOR"
+ "vss<vlmem_op_attr>.v\t%3,%0,%z2%p1"
+ [(set_attr "type" "vsts")
+ (set_attr "mode" "<MODE>")
+ (set (attr "avl_type_idx") (const_int 5))])
+
+(define_insn "@pred_indexed_load_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "register_operand" "=vd, vr,vd, vr")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" " vm,Wc1,vm,Wc1")
+ (match_operand 5 "vector_length_operand" " rK, rK,rK, rK")
+ (match_operand 6 "const_int_operand" " i, i, i, i")
+ (match_operand 7 "const_int_operand" " i, i, i, i")
+ (match_operand 8 "const_int_operand" " i, i, i, i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLXMEM_OP)
+ (unspec:VI
+ [(match_operand 3 "pmode_reg_or_0_operand" " rJ, rJ,rJ, rJ")
+ (mem:BLK (scratch))
+ (match_operand:VI 4 "register_operand" " vr, vr,vr, vr")] UNSPEC_TH_VLXMEM_OP)
+ (match_operand:VI 2 "vector_merge_operand" " vu, vu, 0, 0")))]
+ "TARGET_XTHEADVECTOR"
+ "vlx<vlmem_op_attr>.v\t%0,(%z3),%4%p1"
+ [(set_attr "type" "vldux")
+ (set_attr "mode" "<MODE>")])
+
+(define_insn "@pred_indexed_<vlmem_order_attr>store_width<vlmem_op_attr><mode>"
+ [(set (mem:BLK (scratch))
+ (unspec:BLK
+ [(unspec:<VM>
+ [(match_operand:<VM> 0 "vector_mask_operand" "vmWc1")
+ (match_operand 4 "vector_length_operand" " rK")
+ (match_operand 5 "const_int_operand" " i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VSXMEM_OP)
+ (match_operand 1 "pmode_reg_or_0_operand" " rJ")
+ (match_operand:VI 2 "register_operand" " vr")
+ (match_operand:VI 3 "register_operand" " vr")] UNSPEC_TH_VSXMEM_OP))]
+ "TARGET_XTHEADVECTOR"
+ "vs<vlmem_order_attr>x<vlmem_op_attr>.v\t%3,(%z1),%2%p0"
+ [(set_attr "type" "vstux")
+ (set_attr "mode" "<MODE>")])
+
+(define_expand "@pred_th_extract<mode>"
+ [(set (match_operand:<VEL> 0 "register_operand")
+ (unspec:<VEL>
+ [(vec_select:<VEL>
+ (match_operand:V_VLSI 1 "register_operand")
+ (parallel [(match_operand:DI 2 "register_operand" "r")]))
+ (reg:SI VTYPE_REGNUM)] UNSPEC_VPREDICATE))]
+ "TARGET_XTHEADVECTOR"
+{})
+
+(define_insn "*pred_th_extract<mode>"
+ [(set (match_operand:<VEL> 0 "register_operand" "=r")
+ (unspec:<VEL>
+ [(vec_select:<VEL>
+ (match_operand:V_VLSI 1 "register_operand" "vr")
+ (parallel [(match_operand:DI 2 "register_operand" "r")]))
+ (reg:SI VTYPE_REGNUM)] UNSPEC_VPREDICATE))]
+ "TARGET_XTHEADVECTOR"
+ "vext.x.v\t%0,%1,%2"
+ [(set_attr "type" "vimovvx")
+ (set_attr "mode" "<MODE>")])
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c
new file mode 100644
index 00000000000..3c12c124597
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_th_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+
+** th.vadd\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+
+** th.vsb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out)
+{
+ vint32m1_t v = __riscv_th_vlb_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlb_v_i32m1_tu (v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vv_i32m1 (v2, v2, 4);
+ vint32m1_t v4 = __riscv_vadd_vv_i32m1_tu (v3, v2, v2, 4);
+ __riscv_th_vsb_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlb.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+
+** th.vadd\.vv\tv[1-9][0-9]?,\s*v[0-9]+,\s*v[0-9]+,\s*v0.t
+** th.vsb.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlb_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlb_v_i32m1_m (mask, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vv_i32m1 (v2, v2, 4);
+ vint32m1_t v4 = __riscv_vadd_vv_i32m1_m (mask, v3, v3, 4);
+ __riscv_th_vsb_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlb.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+
+** th.vadd\.vv\tv[1-9][0-9]?,\s*v[0-9]+,\s*v[0-9]+,\s*v0.t
+** th.vsb.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlb_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlb_v_i32m1_tumu (mask, v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vv_i32m1 (v2, v2, 4);
+ vint32m1_t v4 = __riscv_vadd_vv_i32m1_tumu (mask, v3, v2, v2, 4);
+ __riscv_th_vsb_v_i32m1 (out, v4, 4);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c
new file mode 100644
index 00000000000..30bef369375
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_th_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlbu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlbu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vsb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, uint32_t x)
+{
+ vuint32m1_t v = __riscv_th_vlbu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlbu_v_u32m1_tu (v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tu (v3, v2, -16, 4);
+ __riscv_th_vsb_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlbu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsb.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlbu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlbu_v_u32m1_m (mask, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_m (mask, v3, -16, 4);
+ __riscv_th_vsb_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlbu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlbu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsb.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlbu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlbu_v_u32m1_tumu (mask, v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tumu (mask, v3, v2, -16, 4);
+ __riscv_th_vsb_v_u32m1 (out, v4, 4);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c
new file mode 100644
index 00000000000..3c8b5ccc16b
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_th_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vsh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, int32_t x)
+{
+ vint32m1_t v = __riscv_th_vlh_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlh_v_i32m1_tu (v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, -16, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_tu (v3, v2, -16, 4);
+ __riscv_th_vsh_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlh.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsh.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, int32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlh_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlh_v_i32m1_m (mask, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, -16, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_m (mask, v3, -16, 4);
+ __riscv_th_vsh_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlh.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsh.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, int32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlh_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlh_v_i32m1_tumu (mask, v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, -16, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_tumu (mask, v3, v2, -16, 4);
+ __riscv_th_vsh_v_i32m1 (out, v4, 4);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c
new file mode 100644
index 00000000000..b7c00404f18
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_th_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlhu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlhu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vsh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, uint32_t x)
+{
+ vuint32m1_t v = __riscv_th_vlhu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlhu_v_u32m1_tu (v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tu (v3, v2, -16, 4);
+ __riscv_th_vsh_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlhu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsh.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlhu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlhu_v_u32m1_m (mask, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_m (mask, v3, -16, 4);
+ __riscv_th_vsh_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlhu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlhu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsh.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlhu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlhu_v_u32m1_tumu (mask, v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tumu (mask, v3, v2, -16, 4);
+ __riscv_th_vsh_v_u32m1 (out, v4, 4);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c
new file mode 100644
index 00000000000..17a53012acf
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_th_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vx\tv[0-9]+,\s*v[0-9]+,\s*[a-x0-9]+
+** th.vadd\.vx\tv[0-9]+,\s*v[0-9]+,\s*[a-x0-9]+
+** th.vsw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, int32_t x)
+{
+ vint32m1_t v = __riscv_th_vlw_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlw_v_i32m1_tu (v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, x, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_tu (v3, v2, x, 4);
+ __riscv_th_vsw_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlw.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vx\tv[0-9]+,\s*v[0-9]+,\s*[a-x0-9]+
+** th.vadd\.vx\tv[1-9][0-9]?,\s*v[0-9]+,\s*[a-x0-9]+,\s*v0.t
+** th.vsw.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, int32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlw_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlw_v_i32m1_m (mask, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, x, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_m (mask, v3, x, 4);
+ __riscv_th_vsw_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlw.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vx\tv[0-9]+,\s*v[0-9]+,\s*[a-x0-9]+
+** th.vadd\.vx\tv[1-9][0-9]?,\s*v[0-9]+,\s*[a-x0-9]+,\s*v0.t
+** th.vsw.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, int32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlw_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlw_v_i32m1_tumu (mask, v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, x, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_tumu (mask, v3, v2, x, 4);
+ __riscv_th_vsw_v_i32m1 (out, v4, 4);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c
new file mode 100644
index 00000000000..b187cfc852b
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_th_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlwu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlwu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vsw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, uint32_t x)
+{
+ vuint32m1_t v = __riscv_th_vlwu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlwu_v_u32m1_tu (v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tu (v3, v2, -16, 4);
+ __riscv_th_vsw_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlwu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsw.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlwu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlwu_v_u32m1_m (mask, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_m (mask, v3, -16, 4);
+ __riscv_th_vsw_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlwu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlwu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsw.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlwu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlwu_v_u32m1_tumu (mask, v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tumu (mask, v3, v2, -16, 4);
+ __riscv_th_vsw_v_u32m1 (out, v4, 4);
+}
--
2.17.1
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re：Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
2024-01-11 9:32 Re：[PATCH " juzhe.zhong
@ 2024-01-11 9:38 ` joshua
2024-01-11 12:05 ` joshua
1 sibling, 0 replies; 17+ messages in thread
From: joshua @ 2024-01-11 9:38 UTC (permalink / raw)
To: juzhe.zhong, gcc-patches
Cc: Jim Wilson, palmer, andrew, philipp.tomsich, jeffreyalaw,
christoph.muellner, jinma, cooper.qu
I think removing these changes for the "RISC-V: Handle differences between XTheadvector and Vector" patch is better.
This is an extra e issue that can be handled in a seperate patch.
------------------------------------------------------------------
发件人：juzhe.zhong@rivai.ai <juzhe.zhong@rivai.ai>
发送时间：2024年1月11日(星期四) 17:32
收件人："cooper.joshua"<cooper.joshua@linux.alibaba.com>; "gcc-patches"<gcc-patches@gcc.gnu.org>
抄 送：Jim Wilson<jim.wilson.gcc@gmail.com>; palmer<palmer@dabbelt.com>; andrew<andrew@sifive.com>; "philipp.tomsich"<philipp.tomsich@vrull.eu>; jeffreyalaw<jeffreyalaw@gmail.com>; "christoph.muellner"<christoph.muellner@vrull.eu>; jinma<jinma@linux.alibaba.com>; "cooper.qu"<cooper.qu@linux.alibaba.com>
主 题：Re: Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
Ok. Let's hold on "RISC-V: Handle differences between XTheadvector and Vector" patch
until you can reproduce the issue for me.
juzhe.zhong@rivai.ai
发件人： joshua
发送时间： 2024-01-11 17:29
收件人： juzhe.zhong@rivai.ai; gcc-patches
抄送： Jim Wilson; palmer; andrew; philipp.tomsich; jeffreyalaw; christoph.muellner; jinma; cooper.qu
主题： Re：Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
The sext/zext issue is not related to xtheadvector-special patterns.
I added !TARGET_XTHEADVECTOR to sext/zext patterns in
"RISC-V: Handle differences between XTheadvector and Vector"
That is caused by the vwmul pattern, but I cannot reproduce it right now.
------------------------------------------------------------------
发件人：juzhe.zhong@rivai.ai <juzhe.zhong@rivai.ai>
发送时间：2024年1月11日(星期四) 17:24
收件人："cooper.joshua"<cooper.joshua@linux.alibaba.com>; "gcc-patches"<gcc-patches@gcc.gnu.org>
抄 送：Jim Wilson<jim.wilson.gcc@gmail.com>; palmer<palmer@dabbelt.com>; andrew<andrew@sifive.com>; "philipp.tomsich"<philipp.tomsich@vrull.eu>; jeffreyalaw<jeffreyalaw@gmail.com>; "christoph.muellner"<christoph.muellner@vrull.eu>; jinma<jinma@linux.alibaba.com>; "cooper.qu"<cooper.qu@linux.alibaba.com>
主 题：Re: Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
I don't see any patterns are possible CSE into sext/zext patterns:
+(define_expand "@pred_mov_width<vlmem_op_attr><mode>"
+ [(set (match_operand:V_VLS 0 "nonimmediate_operand")
+ (if_then_else:V_VLS
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand")
+ (match_operand 4 "vector_length_operand")
+ (match_operand 5 "const_int_operand")
+ (match_operand 6 "const_int_operand")
+ (match_operand 7 "const_int_operand")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLMEM_OP)
+ (match_operand:V_VLS 3 "vector_move_operand")
+ (match_operand:V_VLS 2 "vector_merge_operand")))]
+ "TARGET_XTHEADVECTOR"
+ {})
+
+(define_insn_and_split "*pred_mov_width<vlmem_op_attr><mode>"
+ [(set (match_operand:V_VLS 0 "nonimmediate_operand" "=vr, vr, vd, m, vr, vr")
+ (if_then_else:V_VLS
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1, Wc1, vm, vmWc1, Wc1, Wc1")
+ (match_operand 4 "vector_length_operand" " rK, rK, rK, rK, rK, rK")
+ (match_operand 5 "const_int_operand" " i, i, i, i, i, i")
+ (match_operand 6 "const_int_operand" " i, i, i, i, i, i")
+ (match_operand 7 "const_int_operand" " i, i, i, i, i, i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLMEM_OP)
+ (match_operand:V_VLS 3 "reg_or_mem_operand" " m, m, m, vr, vr, vr")
+ (match_operand:V_VLS 2 "vector_merge_operand" " 0, vu, vu, vu, vu, 0")))]
+ "(TARGET_XTHEADVECTOR
+ && (register_operand (operands[0], <MODE>mode)
+ || register_operand (operands[3], <MODE>mode)))"
+ "@
+ vl<vlmem_op_attr>.v\t%0,%3%p1
+ vl<vlmem_op_attr>.v\t%0,%3
+ vl<vlmem_op_attr>.v\t%0,%3,%1.t
+ vs<vlmem_op_attr>.v\t%3,%0%p1
+ vmv.v.v\t%0,%3
+ vmv.v.v\t%0,%3"
+ "&& register_operand (operands[0], <MODE>mode)
+ && register_operand (operands[3], <MODE>mode)
+ && satisfies_constraint_vu (operands[2])
+ && INTVAL (operands[7]) == riscv_vector::VLMAX"
+ [(set (match_dup 0) (match_dup 3))]
+ ""
+ [(set_attr "type" "vlde,vlde,vlde,vste,vimov,vimov")
+ (set_attr "mode" "<MODE>")])
+
+(define_insn "@pred_store_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "memory_operand" "+m")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1")
+ (match_operand 3 "vector_length_operand" " rK")
+ (match_operand 4 "const_int_operand" " i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VSMEM_OP)
+ (match_operand:VI 2 "register_operand" " vr")
+ (match_dup 0)))]
+ "TARGET_XTHEADVECTOR"
+ "vs<vlmem_op_attr>.v\t%2,%0%p1"
+ [(set_attr "type" "vste")
+ (set_attr "mode" "<MODE>")
+ (set (attr "avl_type_idx") (const_int 4))
+ (set_attr "vl_op_idx" "3")])
+
+(define_insn "@pred_strided_load_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "register_operand" "=vr, vr, vd")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1, Wc1, vm")
+ (match_operand 5 "vector_length_operand" " rK, rK, rK")
+ (match_operand 6 "const_int_operand" " i, i, i")
+ (match_operand 7 "const_int_operand" " i, i, i")
+ (match_operand 8 "const_int_operand" " i, i, i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLSMEM_OP)
+ (unspec:VI
+ [(match_operand:VI 3 "memory_operand" " m, m, m")
+ (match_operand 4 "pmode_reg_or_0_operand" " rJ, rJ, rJ")] UNSPEC_TH_VLSMEM_OP)
+ (match_operand:VI 2 "vector_merge_operand" " 0, vu, vu")))]
+ "TARGET_XTHEADVECTOR"
+ "vls<vlmem_op_attr>.v\t%0,%3,%z4%p1"
+ [(set_attr "type" "vlds")
+ (set_attr "mode" "<MODE>")])
+
+(define_insn "@pred_strided_store_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "memory_operand" "+m")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1")
+ (match_operand 4 "vector_length_operand" " rK")
+ (match_operand 5 "const_int_operand" " i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VSSMEM_OP)
+ (unspec:VI
+ [(match_operand 2 "pmode_reg_or_0_operand" " rJ")
+ (match_operand:VI 3 "register_operand" " vr")] UNSPEC_TH_VSSMEM_OP)
+ (match_dup 0)))]
+ "TARGET_XTHEADVECTOR"
+ "vss<vlmem_op_attr>.v\t%3,%0,%z2%p1"
+ [(set_attr "type" "vsts")
+ (set_attr "mode" "<MODE>")
+ (set (attr "avl_type_idx") (const_int 5))])
+
+(define_insn "@pred_indexed_load_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "register_operand" "=vd, vr,vd, vr")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" " vm,Wc1,vm,Wc1")
+ (match_operand 5 "vector_length_operand" " rK, rK,rK, rK")
+ (match_operand 6 "const_int_operand" " i, i, i, i")
+ (match_operand 7 "const_int_operand" " i, i, i, i")
+ (match_operand 8 "const_int_operand" " i, i, i, i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLXMEM_OP)
+ (unspec:VI
+ [(match_operand 3 "pmode_reg_or_0_operand" " rJ, rJ,rJ, rJ")
+ (mem:BLK (scratch))
+ (match_operand:VI 4 "register_operand" " vr, vr,vr, vr")] UNSPEC_TH_VLXMEM_OP)
+ (match_operand:VI 2 "vector_merge_operand" " vu, vu, 0, 0")))]
+ "TARGET_XTHEADVECTOR"
+ "vlx<vlmem_op_attr>.v\t%0,(%z3),%4%p1"
+ [(set_attr "type" "vldux")
+ (set_attr "mode" "<MODE>")])
+
+(define_insn "@pred_indexed_<vlmem_order_attr>store_width<vlmem_op_attr><mode>"
+ [(set (mem:BLK (scratch))
+ (unspec:BLK
+ [(unspec:<VM>
+ [(match_operand:<VM> 0 "vector_mask_operand" "vmWc1")
+ (match_operand 4 "vector_length_operand" " rK")
+ (match_operand 5 "const_int_operand" " i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VSXMEM_OP)
+ (match_operand 1 "pmode_reg_or_0_operand" " rJ")
+ (match_operand:VI 2 "register_operand" " vr")
+ (match_operand:VI 3 "register_operand" " vr")] UNSPEC_TH_VSXMEM_OP))]
+ "TARGET_XTHEADVECTOR"
+ "vs<vlmem_order_attr>x<vlmem_op_attr>.v\t%3,(%z1),%2%p0"
+ [(set_attr "type" "vstux")
+ (set_attr "mode" "<MODE>")])
+
+(define_expand "@pred_th_extract<mode>"
+ [(set (match_operand:<VEL> 0 "register_operand")
+ (unspec:<VEL>
+ [(vec_select:<VEL>
+ (match_operand:V_VLSI 1 "register_operand")
+ (parallel [(match_operand:DI 2 "register_operand" "r")]))
+ (reg:SI VTYPE_REGNUM)] UNSPEC_VPREDICATE))]
+ "TARGET_XTHEADVECTOR"
+{})
+
+(define_insn "*pred_th_extract<mode>"
+ [(set (match_operand:<VEL> 0 "register_operand" "=r")
+ (unspec:<VEL>
+ [(vec_select:<VEL>
+ (match_operand:V_VLSI 1 "register_operand" "vr")
+ (parallel [(match_operand:DI 2 "register_operand" "r")]))
+ (reg:SI VTYPE_REGNUM)] UNSPEC_VPREDICATE))]
+ "TARGET_XTHEADVECTOR"
+ "vext.x.v\t%0,%1,%2"
+ [(set_attr "type" "vimovvx")
+ (set_attr "mode" "<MODE>")])
juzhe.zhong@rivai.ai
发件人： joshua
发送时间： 2024-01-11 17:21
收件人： juzhe.zhong@rivai.ai; gcc-patches
抄送： Jim Wilson; palmer; andrew; philipp.tomsich; jeffreyalaw; christoph.muellner; jinma; cooper.qu
主题： Re：Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
"I didn't see theadvector-specific extension patterns. Could you show me?"
They are all in the file thead-vector.md.
For the sext/zext issue, perhaps I need some time to reproduce that optimization,
but I can clearly remember it is related to vwmul.
------------------------------------------------------------------
发件人：juzhe.zhong@rivai.ai <juzhe.zhong@rivai.ai>
发送时间：2024年1月11日(星期四) 17:17
收件人："cooper.joshua"<cooper.joshua@linux.alibaba.com>; "gcc-patches"<gcc-patches@gcc.gnu.org>
抄 送：Jim Wilson<jim.wilson.gcc@gmail.com>; palmer<palmer@dabbelt.com>; andrew<andrew@sifive.com>; "philipp.tomsich"<philipp.tomsich@vrull.eu>; jeffreyalaw<jeffreyalaw@gmail.com>; "christoph.muellner"<christoph.muellner@vrull.eu>; jinma<jinma@linux.alibaba.com>; "cooper.qu"<cooper.qu@linux.alibaba.com>
主 题：Re: Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
You mean which pattern optimized sext/vzext pattern?
I didn't see theadvector-specific extension patterns. Could you show me?
juzhe.zhong@rivai.ai
发件人： joshua
发送时间： 2024-01-11 17:14
收件人： juzhe.zhong@rivai.ai; gcc-patches
抄送： Jim Wilson; palmer; andrew; philipp.tomsich; jeffreyalaw; christoph.muellner; jinma; cooper.qu
主题： Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
To be specific, in CSE pass, the initial pattern will be optimized into the sext/zext pattern.
------------------------------------------------------------------
发件人：joshua <cooper.joshua@linux.alibaba.com>
发送时间：2024年1月11日(星期四) 17:11
收件人："juzhe.zhong@rivai.ai"<juzhe.zhong@rivai.ai>; "gcc-patches"<gcc-patches@gcc.gnu.org>
抄 送：Jim Wilson<jim.wilson.gcc@gmail.com>; palmer<palmer@dabbelt.com>; andrew<andrew@sifive.com>; "philipp.tomsich"<philipp.tomsich@vrull.eu>; jeffreyalaw<jeffreyalaw@gmail.com>; "christoph.muellner"<christoph.muellner@vrull.eu>; jinma<jinma@linux.alibaba.com>; "cooper.qu"<cooper.qu@linux.alibaba.com>
主 题：Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
sext/zext will be generated in O2 even without corresponding intrinsics.
------------------------------------------------------------------
发件人：juzhe.zhong@rivai.ai <juzhe.zhong@rivai.ai>
发送时间：2024年1月11日(星期四) 17:07
收件人："cooper.joshua"<cooper.joshua@linux.alibaba.com>; "gcc-patches"<gcc-patches@gcc.gnu.org>
抄 送：Jim Wilson<jim.wilson.gcc@gmail.com>; palmer<palmer@dabbelt.com>; andrew<andrew@sifive.com>; "philipp.tomsich"<philipp.tomsich@vrull.eu>; jeffreyalaw<jeffreyalaw@gmail.com>; "christoph.muellner"<christoph.muellner@vrull.eu>; "cooper.joshua"<cooper.joshua@linux.alibaba.com>; jinma<jinma@linux.alibaba.com>; "cooper.qu"<cooper.qu@linux.alibaba.com>
主 题：Re: [PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
enum required_ext
{
VECTOR_EXT, /* Vector extension */
+ XTHEADVECTOR_EXT, /* XTheadVector extension */
ZVBB_EXT, /* Cryto vector Zvbb sub-ext */
ZVBB_OR_ZVKB_EXT, /* Cryto vector Zvbb or zvkb sub-ext */
ZVBC_EXT, /* Crypto vector Zvbc sub-ext */
Add theadvector to the end of the enum.
+ case XTHEADVECTOR_EXT:
+ return TARGET_XTHEADVECTOR;
Same.
+ "&& register_operand (operands[0], <MODE>mode)
+ && register_operand (operands[3], <MODE>mode)
+ && satisfies_constraint_vu (operands[2])
+ && INTVAL (operands[7]) == riscv_vector::VLMAX"
You can use whole_reg_to_reg_move_p
Btw, I review again : RISC-V: Handle differences between XTheadvector and Vector
(any_extend:VWEXTI (match_operand:<V_DOUBLE_TRUNC> 3 "register_operand" "W21,W21,W21,W21,W42,W42,W42,W42,W84,W84,W84,W84, vr, vr")) (match_operand:VWEXTI 2 "vector_merge_operand" " vu, vu, 0, 0, vu, vu, 0, 0, vu, vu, 0, 0, vu, 0")))]- "TARGET_VECTOR"+ "TARGET_VECTOR && !TARGET_XTHEADVECTOR" "v<sz>ext.vf2\t%0,%3%p1" [(set_attr "type" "vext") (set_attr "mode" "<MODE>")@@ -3713,7 +3744,7 @@ (any_extend:VQEXTI (match_operand:<V_QUAD_TRUNC> 3 "register_operand" "W43,W43,W43,W43,W86,W86,W86,W86, vr, vr")) (match_operand:VQEXTI 2 "vector_merge_operand" " vu, vu, 0, 0, vu, vu, 0, 0, vu, 0")))]- "TARGET_VECTOR"+ "TARGET_VECTOR && !TARGET_XTHEADVECTOR" "v<sz>ext.vf4\t%0,%3%p1" [(set_attr "type" "vext") (set_attr "mode" "<MODE>")@@ -3734,7 +3765,7 @@ (any_extend:VOEXTI (match_operand:<V_OCT_TRUNC> 3 "register_operand" "W87,W87,W87,W87, vr, vr")) (match_operand:VOEXTI 2 "vector_merge_operand" " vu, vu, 0, 0, vu, 0")))]- "TARGET_VECTOR"+ "TARGET_VECTOR && !TARGET_XTHEADVECTOR" "v<sz>ext.vf8\t%0,%3%p1" [(set_attr "type" "vext") (set_attr "mode" "<MODE>")
Why do you add these !TARGERT_XTHEADVECRTOR ?
juzhe.zhong@rivai.ai
From: Jun Sha (Joshua)
Date: 2024-01-11 16:46
To: gcc-patches
CC: jim.wilson.gcc; palmer; andrew; philipp.tomsich; jeffreyalaw; christoph.muellner; juzhe.zhong; Jun Sha (Joshua); Jin Ma; Xianmiao Qu
Subject: [PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
This patch only involves the generation of xtheadvector
special load/store instructions and vext instructions.
gcc/ChangeLog:
* config/riscv/riscv-vector-builtins-bases.cc
(class th_loadstore_width): Define new builtin bases.
(class th_extract): Define new builtin bases.
(BASE): Define new builtin bases.
* config/riscv/riscv-vector-builtins-bases.h:
Define new builtin class.
* config/riscv/riscv-vector-builtins-shapes.cc
(struct th_loadstore_width_def): Define new builtin shapes.
(struct th_indexed_loadstore_width_def):
Define new builtin shapes.
(struct th_extract_def): Define new builtin shapes.
(SHAPE): Define new builtin shapes.
* config/riscv/riscv-vector-builtins-shapes.h:
Define new builtin shapes.
* config/riscv/riscv-vector-builtins.cc (DEF_RVV_FUNCTION):
* config/riscv/riscv-vector-builtins.h (enum required_ext):
(struct function_group_info):
* config/riscv/t-riscv: Add thead-vector-builtins-functions.def
* config/riscv/thead-vector.md
(@pred_mov_width<vlmem_op_attr><mode>): Add new patterns.
(*pred_mov_width<vlmem_op_attr><mode>): Likewise.
(@pred_store_width<vlmem_op_attr><mode>): Likewise.
(@pred_strided_load_width<vlmem_op_attr><mode>): Likewise.
(@pred_strided_store_width<vlmem_op_attr><mode>): Likewise.
(@pred_indexed_load_width<vlmem_op_attr><mode>): Likewise.
(@pred_indexed_<vlmem_order_attr>store_width<vlmem_op_attr><mode>):
(@pred_th_extract<mode>): Likewise.
(*pred_th_extract<mode>): Likewise.
* config/riscv/thead-vector-builtins-functions.def: New file.
gcc/testsuite/ChangeLog:
* gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c: New test.
Co-authored-by: Jin Ma <jinma@linux.alibaba.com>
Co-authored-by: Xianmiao Qu <cooper.qu@linux.alibaba.com>
Co-authored-by: Christoph Müllner <christoph.muellner@vrull.eu>
---
.../riscv/riscv-vector-builtins-bases.cc | 139 ++++++++++
.../riscv/riscv-vector-builtins-bases.h | 31 +++
.../riscv/riscv-vector-builtins-shapes.cc | 160 +++++++++++
.../riscv/riscv-vector-builtins-shapes.h | 3 +
gcc/config/riscv/riscv-vector-builtins.cc | 70 +++++
gcc/config/riscv/riscv-vector-builtins.h | 3 +
gcc/config/riscv/t-riscv | 1 +
.../riscv/thead-vector-builtins-functions.def | 39 +++
gcc/config/riscv/thead-vector.md | 253 ++++++++++++++++++
.../riscv/rvv/xtheadvector/vlb-vsb.c | 68 +++++
.../riscv/rvv/xtheadvector/vlbu-vsb.c | 68 +++++
.../riscv/rvv/xtheadvector/vlh-vsh.c | 68 +++++
.../riscv/rvv/xtheadvector/vlhu-vsh.c | 68 +++++
.../riscv/rvv/xtheadvector/vlw-vsw.c | 68 +++++
.../riscv/rvv/xtheadvector/vlwu-vsw.c | 68 +++++
15 files changed, 1107 insertions(+)
create mode 100644 gcc/config/riscv/thead-vector-builtins-functions.def
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c
diff --git a/gcc/config/riscv/riscv-vector-builtins-bases.cc b/gcc/config/riscv/riscv-vector-builtins-bases.cc
index 46f1a1da33e..3eba7943757 100644
--- a/gcc/config/riscv/riscv-vector-builtins-bases.cc
+++ b/gcc/config/riscv/riscv-vector-builtins-bases.cc
@@ -2125,6 +2125,83 @@ public:
}
};
+/* Implements
+ * th.vl(b/h/w)[u].v/th.vs(b/h/w)[u].v/th.vls(b/h/w)[u].v/th.vss(b/h/w)[u].v/
+ * th.vlx(b/h/w)[u].v/th.vs[u]x(b/h/w).v
+ * codegen. */
+template<bool STORE_P, lst_type LST_TYPE, int UNSPEC>
+class th_loadstore_width : public function_base
+{
+public:
+ bool apply_tail_policy_p () const override { return !STORE_P; }
+ bool apply_mask_policy_p () const override { return !STORE_P; }
+
+ unsigned int call_properties (const function_instance &) const override
+ {
+ if (STORE_P)
+ return CP_WRITE_MEMORY;
+ else
+ return CP_READ_MEMORY;
+ }
+
+ bool can_be_overloaded_p (enum predication_type_index pred) const override
+ {
+ if (STORE_P || LST_TYPE == LST_INDEXED)
+ return true;
+ return pred != PRED_TYPE_none;
+ }
+
+ rtx expand (function_expander &e) const override
+ {
+ gcc_assert (TARGET_XTHEADVECTOR);
+ if (LST_TYPE == LST_INDEXED)
+ {
+ if (STORE_P)
+ return e.use_exact_insn (
+ code_for_pred_indexed_store_width (UNSPEC, UNSPEC,
+ e.vector_mode ()));
+ else
+ return e.use_exact_insn (
+ code_for_pred_indexed_load_width (UNSPEC, e.vector_mode ()));
+ }
+ else if (LST_TYPE == LST_STRIDED)
+ {
+ if (STORE_P)
+ return e.use_contiguous_store_insn (
+ code_for_pred_strided_store_width (UNSPEC, e.vector_mode ()));
+ else
+ return e.use_contiguous_load_insn (
+ code_for_pred_strided_load_width (UNSPEC, e.vector_mode ()));
+ }
+ else
+ {
+ if (STORE_P)
+ return e.use_contiguous_store_insn (
+ code_for_pred_store_width (UNSPEC, e.vector_mode ()));
+ else
+ return e.use_contiguous_load_insn (
+ code_for_pred_mov_width (UNSPEC, e.vector_mode ()));
+ }
+ }
+};
+
+/* Implements vext.x.v. */
+class th_extract : public function_base
+{
+public:
+ bool apply_vl_p () const override { return false; }
+ bool apply_tail_policy_p () const override { return false; }
+ bool apply_mask_policy_p () const override { return false; }
+ bool use_mask_predication_p () const override { return false; }
+ bool has_merge_operand_p () const override { return false; }
+
+ rtx expand (function_expander &e) const override
+ {
+ gcc_assert (TARGET_XTHEADVECTOR);
+ return e.use_exact_insn (code_for_pred_th_extract (e.vector_mode ()));
+ }
+};
+
/* Below implements are vector crypto */
/* Implements vandn.[vv,vx] */
class vandn : public function_base
@@ -2587,6 +2664,37 @@ static CONSTEXPR const seg_indexed_load<UNSPEC_ORDERED> vloxseg_obj;
static CONSTEXPR const seg_indexed_store<UNSPEC_UNORDERED> vsuxseg_obj;
static CONSTEXPR const seg_indexed_store<UNSPEC_ORDERED> vsoxseg_obj;
static CONSTEXPR const vlsegff vlsegff_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLB> vlb_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLBU> vlbu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLH> vlh_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLHU> vlhu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLW> vlw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLWU> vlwu_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_UNIT_STRIDE, UNSPEC_TH_VLB> vsb_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_UNIT_STRIDE, UNSPEC_TH_VLH> vsh_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_UNIT_STRIDE, UNSPEC_TH_VLW> vsw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSB> vlsb_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSBU> vlsbu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSH> vlsh_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSHU> vlshu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSW> vlsw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSWU> vlswu_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_STRIDED, UNSPEC_TH_VLSB> vssb_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_STRIDED, UNSPEC_TH_VLSH> vssh_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_STRIDED, UNSPEC_TH_VLSW> vssw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXB> vlxb_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXBU> vlxbu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXH> vlxh_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXHU> vlxhu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXW> vlxw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXWU> vlxwu_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VLXB> vsxb_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VLXH> vsxh_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VLXW> vsxw_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VSUXB> vsuxb_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VSUXH> vsuxh_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VSUXW> vsuxw_obj;
+static CONSTEXPR const th_extract vext_x_v_obj;
/* Crypto Vector */
static CONSTEXPR const vandn vandn_obj;
@@ -2878,6 +2986,37 @@ BASE (vloxseg)
BASE (vsuxseg)
BASE (vsoxseg)
BASE (vlsegff)
+BASE (vlb)
+BASE (vlh)
+BASE (vlw)
+BASE (vlbu)
+BASE (vlhu)
+BASE (vlwu)
+BASE (vsb)
+BASE (vsh)
+BASE (vsw)
+BASE (vlsb)
+BASE (vlsh)
+BASE (vlsw)
+BASE (vlsbu)
+BASE (vlshu)
+BASE (vlswu)
+BASE (vssb)
+BASE (vssh)
+BASE (vssw)
+BASE (vlxb)
+BASE (vlxh)
+BASE (vlxw)
+BASE (vlxbu)
+BASE (vlxhu)
+BASE (vlxwu)
+BASE (vsxb)
+BASE (vsxh)
+BASE (vsxw)
+BASE (vsuxb)
+BASE (vsuxh)
+BASE (vsuxw)
+BASE (vext_x_v)
/* Crypto vector */
BASE (vandn)
BASE (vbrev)
diff --git a/gcc/config/riscv/riscv-vector-builtins-bases.h b/gcc/config/riscv/riscv-vector-builtins-bases.h
index 1122e3801a7..565a0311d2b 100644
--- a/gcc/config/riscv/riscv-vector-builtins-bases.h
+++ b/gcc/config/riscv/riscv-vector-builtins-bases.h
@@ -299,6 +299,37 @@ extern const function_base *const vloxseg;
extern const function_base *const vsuxseg;
extern const function_base *const vsoxseg;
extern const function_base *const vlsegff;
+extern const function_base *const vlb;
+extern const function_base *const vlh;
+extern const function_base *const vlw;
+extern const function_base *const vlbu;
+extern const function_base *const vlhu;
+extern const function_base *const vlwu;
+extern const function_base *const vsb;
+extern const function_base *const vsh;
+extern const function_base *const vsw;
+extern const function_base *const vlsb;
+extern const function_base *const vlsh;
+extern const function_base *const vlsw;
+extern const function_base *const vlsbu;
+extern const function_base *const vlshu;
+extern const function_base *const vlswu;
+extern const function_base *const vssb;
+extern const function_base *const vssh;
+extern const function_base *const vssw;
+extern const function_base *const vlxb;
+extern const function_base *const vlxh;
+extern const function_base *const vlxw;
+extern const function_base *const vlxbu;
+extern const function_base *const vlxhu;
+extern const function_base *const vlxwu;
+extern const function_base *const vsxb;
+extern const function_base *const vsxh;
+extern const function_base *const vsxw;
+extern const function_base *const vsuxb;
+extern const function_base *const vsuxh;
+extern const function_base *const vsuxw;
+extern const function_base *const vext_x_v;
/* Below function_base are Vectro Crypto*/
extern const function_base *const vandn;
extern const function_base *const vbrev;
diff --git a/gcc/config/riscv/riscv-vector-builtins-shapes.cc b/gcc/config/riscv/riscv-vector-builtins-shapes.cc
index 1e4f4d53de6..8e90b17a94b 100644
--- a/gcc/config/riscv/riscv-vector-builtins-shapes.cc
+++ b/gcc/config/riscv/riscv-vector-builtins-shapes.cc
@@ -211,6 +211,146 @@ struct indexed_loadstore_def : public function_shape
}
};
+/* Add one function instance for GROUP, using operand suffix at index OI,
+ mode suffix at index PAIR && bi and predication suffix at index pred_idx. */
+static void
+build_th_loadstore (function_builder &b, const function_group_info &group,
+ unsigned int pred_idx, unsigned int vec_type_idx)
+{
+ auto_vec<tree, 5> argument_types;
+ function_instance function_instance (group.base_name, *group.base,
+ *group.shape,
+ group.ops_infos.types[vec_type_idx],
+ group.preds[pred_idx], &group.ops_infos);
+ tree return_type = group.ops_infos.ret.get_tree_type (
+ group.ops_infos.types[vec_type_idx].index);
+ b.allocate_argument_types (function_instance, argument_types);
+ b.apply_predication (function_instance, return_type, argument_types);
+
+ if (TARGET_XTHEADVECTOR && !check_type (return_type, argument_types))
+ return;
+
+ tree type = builtin_types[group.ops_infos.types[vec_type_idx].index].vector;
+ if (strstr (group.base_name, "l")
+ && strstr (group.base_name, "u")
+ && !TYPE_UNSIGNED (TREE_TYPE (type)))
+ return;
+
+ if (strstr (group.base_name, "l")
+ && !strstr (group.base_name, "u")
+ && TYPE_UNSIGNED (TREE_TYPE (type)))
+ return;
+
+ machine_mode mode = TYPE_MODE (type);
+ int sew = GET_MODE_BITSIZE (GET_MODE_INNER (mode));
+ if (strstr (group.base_name, "h") && sew == 8)
+ return;
+
+ if (strstr (group.base_name, "w") && (sew == 8 || sew ==16))
+ return;
+
+ b.add_overloaded_function (function_instance, *group.shape);
+ b.add_unique_function (function_instance, (*group.shape), return_type,
+ argument_types);
+}
+
+/* th_loadstore_width_def class. */
+struct th_loadstore_width_def : public build_base
+{
+ void build (function_builder &b,
+ const function_group_info &group) const override
+ {
+ for (unsigned int pred_idx = 0; group.preds[pred_idx] != NUM_PRED_TYPES;
+ ++pred_idx)
+ {
+ for (unsigned int vec_type_idx = 0;
+ group.ops_infos.types[vec_type_idx].index != NUM_VECTOR_TYPES;
+ ++vec_type_idx)
+ {
+ build_th_loadstore (b, group, pred_idx, vec_type_idx);
+ }
+ }
+ }
+
+ char *get_name (function_builder &b, const function_instance &instance,
+ bool overloaded_p) const override
+ {
+ /* Return nullptr if it can not be overloaded. */
+ if (overloaded_p && !instance.base->can_be_overloaded_p (instance.pred))
+ return nullptr;
+
+ b.append_name ("__riscv_th_");
+ b.append_name (instance.base_name);
+
+ /* vop_v --> vop_v_<type>. */
+ if (!overloaded_p)
+ {
+ /* vop --> vop_v. */
+ b.append_name (operand_suffixes[instance.op_info->op]);
+ /* vop_v --> vop_v_<type>. */
+ b.append_name (type_suffixes[instance.type.index].vector);
+ }
+
+ /* According to rvv-intrinsic-doc, it does not add "_m" suffix
+ for vop_m C++ overloaded API. */
+ if (overloaded_p && instance.pred == PRED_TYPE_m)
+ return b.finish_name ();
+ b.append_name (predication_suffixes[instance.pred]);
+ return b.finish_name ();
+ }
+};
+
+
+/* th_indexed_loadstore_width_def class. */
+struct th_indexed_loadstore_width_def : public function_shape
+{
+ void build (function_builder &b,
+ const function_group_info &group) const override
+ {
+ for (unsigned int pred_idx = 0; group.preds[pred_idx] != NUM_PRED_TYPES;
+ ++pred_idx)
+ {
+ for (unsigned int vec_type_idx = 0;
+ group.ops_infos.types[vec_type_idx].index != NUM_VECTOR_TYPES;
+ ++vec_type_idx)
+ {
+ tree index_type = group.ops_infos.args[1].get_tree_type (
+ group.ops_infos.types[vec_type_idx].index);
+ if (!index_type)
+ continue;
+ build_th_loadstore (b, group, pred_idx, vec_type_idx);
+ }
+ }
+ }
+
+ char *get_name (function_builder &b, const function_instance &instance,
+ bool overloaded_p) const override
+ {
+
+ /* Return nullptr if it can not be overloaded. */
+ if (overloaded_p && !instance.base->can_be_overloaded_p (instance.pred))
+ return nullptr;
+
+ b.append_name ("__riscv_th_");
+ b.append_name (instance.base_name);
+ /* vop_v --> vop_v_<type>. */
+ if (!overloaded_p)
+ {
+ /* vop --> vop_v. */
+ b.append_name (operand_suffixes[instance.op_info->op]);
+ /* vop_v --> vop_v_<type>. */
+ b.append_name (type_suffixes[instance.type.index].vector);
+ }
+
+ /* According to rvv-intrinsic-doc, it does not add "_m" suffix
+ for vop_m C++ overloaded API. */
+ if (overloaded_p && instance.pred == PRED_TYPE_m)
+ return b.finish_name ();
+ b.append_name (predication_suffixes[instance.pred]);
+ return b.finish_name ();
+ }
+};
+
/* alu_def class. */
struct alu_def : public build_base
{
@@ -632,6 +772,23 @@ struct reduc_alu_def : public build_base
}
};
+/* th_extract_def class. */
+struct th_extract_def : public build_base
+{
+ char *get_name (function_builder &b, const function_instance &instance,
+ bool overloaded_p) const override
+ {
+ b.append_name ("__riscv_th_");
+ b.append_name (instance.base_name);
+
+ if (overloaded_p)
+ return b.finish_name ();
+ b.append_name (type_suffixes[instance.type.index].vector);
+ b.append_name (type_suffixes[instance.type.index].scalar);
+ return b.finish_name ();
+ }
+};
+
/* scalar_move_def class. */
struct scalar_move_def : public build_base
{
@@ -1094,6 +1251,8 @@ SHAPE(vsetvl, vsetvl)
SHAPE(vsetvl, vsetvlmax)
SHAPE(loadstore, loadstore)
SHAPE(indexed_loadstore, indexed_loadstore)
+SHAPE(th_loadstore_width, th_loadstore_width)
+SHAPE(th_indexed_loadstore_width, th_indexed_loadstore_width)
SHAPE(alu, alu)
SHAPE(alu_frm, alu_frm)
SHAPE(widen_alu, widen_alu)
@@ -1106,6 +1265,7 @@ SHAPE(move, move)
SHAPE(mask_alu, mask_alu)
SHAPE(reduc_alu, reduc_alu)
SHAPE(reduc_alu_frm, reduc_alu_frm)
+SHAPE(th_extract, th_extract)
SHAPE(scalar_move, scalar_move)
SHAPE(vundefined, vundefined)
SHAPE(misc, misc)
diff --git a/gcc/config/riscv/riscv-vector-builtins-shapes.h b/gcc/config/riscv/riscv-vector-builtins-shapes.h
index ac2a28ce017..a7624d0fabd 100644
--- a/gcc/config/riscv/riscv-vector-builtins-shapes.h
+++ b/gcc/config/riscv/riscv-vector-builtins-shapes.h
@@ -28,6 +28,8 @@ extern const function_shape *const vsetvl;
extern const function_shape *const vsetvlmax;
extern const function_shape *const loadstore;
extern const function_shape *const indexed_loadstore;
+extern const function_shape *const th_loadstore_width;
+extern const function_shape *const th_indexed_loadstore_width;
extern const function_shape *const alu;
extern const function_shape *const alu_frm;
extern const function_shape *const widen_alu;
@@ -41,6 +43,7 @@ extern const function_shape *const mask_alu;
extern const function_shape *const reduc_alu;
extern const function_shape *const reduc_alu_frm;
extern const function_shape *const scalar_move;
+extern const function_shape *const th_extract;
extern const function_shape *const vundefined;
extern const function_shape *const misc;
extern const function_shape *const vset;
diff --git a/gcc/config/riscv/riscv-vector-builtins.cc b/gcc/config/riscv/riscv-vector-builtins.cc
index 25e0b6e56de..44b9fec1898 100644
--- a/gcc/config/riscv/riscv-vector-builtins.cc
+++ b/gcc/config/riscv/riscv-vector-builtins.cc
@@ -934,6 +934,32 @@ static CONSTEXPR const rvv_arg_type_info ext_vcreate_args[]
= {rvv_arg_type_info (RVV_BASE_vector),
rvv_arg_type_info_end};
+/* A list of args for vector_type func (const scalar_type *, size_t)
+ * function. */
+static CONSTEXPR const rvv_arg_type_info scalar_const_ptr_size_args[]
+ = {rvv_arg_type_info (RVV_BASE_scalar_const_ptr),
+ rvv_arg_type_info (RVV_BASE_size), rvv_arg_type_info_end};
+
+/* A list of args for vector_type func (const scalar_type *, eew8_index_type)
+ * function. */
+static CONSTEXPR const rvv_arg_type_info scalar_const_ptr_index_args[]
+ = {rvv_arg_type_info (RVV_BASE_scalar_const_ptr),
+ rvv_arg_type_info (RVV_BASE_unsigned_vector), rvv_arg_type_info_end};
+
+/* A list of args for void func (scalar_type *, eew8_index_type, vector_type)
+ * function. */
+static CONSTEXPR const rvv_arg_type_info scalar_ptr_index_args[]
+ = {rvv_arg_type_info (RVV_BASE_scalar_ptr),
+ rvv_arg_type_info (RVV_BASE_unsigned_vector),
+ rvv_arg_type_info (RVV_BASE_vector), rvv_arg_type_info_end};
+
+/* A list of args for void func (scalar_type *, size_t, vector_type)
+ * function. */
+static CONSTEXPR const rvv_arg_type_info scalar_ptr_size_args[]
+ = {rvv_arg_type_info (RVV_BASE_scalar_ptr),
+ rvv_arg_type_info (RVV_BASE_size), rvv_arg_type_info (RVV_BASE_vector),
+ rvv_arg_type_info_end};
+
/* A list of none preds that will be registered for intrinsic functions. */
static CONSTEXPR const predication_type_index none_preds[]
= {PRED_TYPE_none, NUM_PRED_TYPES};
@@ -1455,6 +1481,14 @@ static CONSTEXPR const rvv_op_info iu_shift_vvv_ops
rvv_arg_type_info (RVV_BASE_vector), /* Return type */
shift_vv_args /* Args */};
+/* A static operand information for scalar_type func (vector_type, size_t)
+ * function registration. */
+static CONSTEXPR const rvv_op_info iu_x_s_u_ops
+ = {iu_ops, /* Types */
+ OP_TYPE_vx, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_scalar), /* Return type */
+ v_size_args /* Args */};
+
/* A static operand information for vector_type func (vector_type, size_t)
* function registration. */
static CONSTEXPR const rvv_op_info iu_shift_vvx_ops
@@ -2638,6 +2672,38 @@ static CONSTEXPR const rvv_op_info all_v_vcreate_lmul4_x2_ops
rvv_arg_type_info (RVV_BASE_vlmul_ext_x2), /* Return type */
ext_vcreate_args /* Args */};
+/* A static operand information for vector_type func (const scalar_type *,
+ * size_t) function registration. */
+static CONSTEXPR const rvv_op_info all_v_scalar_const_ptr_size_ops
+ = {all_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_size_args /* Args */};
+
+/* A static operand information for void func (scalar_type *, size_t,
+ * vector_type) function registration. */
+static CONSTEXPR const rvv_op_info all_v_scalar_ptr_size_ops
+ = {all_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_void), /* Return type */
+ scalar_ptr_size_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * index_type) function registration. */
+static CONSTEXPR const rvv_op_info all_v_scalar_const_ptr_index_ops
+ = {all_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_index_args /* Args */};
+
+/* A static operand information for void func (scalar_type *, index_type,
+ * vector_type) function registration. */
+static CONSTEXPR const rvv_op_info all_v_scalar_ptr_index_ops
+ = {all_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_void), /* Return type */
+ scalar_ptr_index_args /* Args */};
+
/* A static operand information for vector_type func (vector_type).
Some ins just supports SEW=32, such as crypto vectol Zvkg extension.
* function registration. */
@@ -2816,6 +2882,10 @@ static function_group_info function_groups[] = {
#define DEF_RVV_FUNCTION(NAME, SHAPE, PREDS, OPS_INFO) \
{#NAME, &bases::NAME, &shapes::SHAPE, PREDS, OPS_INFO, REQUIRED_EXTENSIONS},
#include "riscv-vector-builtins-functions.def"
+#undef DEF_RVV_FUNCTION
+#define DEF_RVV_FUNCTION(NAME, SHAPE, PREDS, OPS_INFO) \
+ {#NAME, &bases::NAME, &shapes::SHAPE, PREDS, OPS_INFO, REQUIRED_EXTENSIONS},
+#include "thead-vector-builtins-functions.def"
};
/* The RVV types, with their built-in
diff --git a/gcc/config/riscv/riscv-vector-builtins.h b/gcc/config/riscv/riscv-vector-builtins.h
index 54c8824ff92..a8ee39a3cb2 100644
--- a/gcc/config/riscv/riscv-vector-builtins.h
+++ b/gcc/config/riscv/riscv-vector-builtins.h
@@ -114,6 +114,7 @@ static const unsigned int CP_WRITE_CSR = 1U << 5;
enum required_ext
{
VECTOR_EXT, /* Vector extension */
+ XTHEADVECTOR_EXT, /* XTheadVector extension */
ZVBB_EXT, /* Cryto vector Zvbb sub-ext */
ZVBB_OR_ZVKB_EXT, /* Cryto vector Zvbb or zvkb sub-ext */
ZVBC_EXT, /* Crypto vector Zvbc sub-ext */
@@ -234,6 +235,8 @@ struct function_group_info
{
case VECTOR_EXT:
return TARGET_VECTOR;
+ case XTHEADVECTOR_EXT:
+ return TARGET_XTHEADVECTOR;
case ZVBB_EXT:
return TARGET_ZVBB;
case ZVBB_OR_ZVKB_EXT:
diff --git a/gcc/config/riscv/t-riscv b/gcc/config/riscv/t-riscv
index 32de6b851c1..38494320d8b 100644
--- a/gcc/config/riscv/t-riscv
+++ b/gcc/config/riscv/t-riscv
@@ -1,6 +1,7 @@
RISCV_BUILTINS_H = $(srcdir)/config/riscv/riscv-vector-builtins.h \
$(srcdir)/config/riscv/riscv-vector-builtins.def \
$(srcdir)/config/riscv/riscv-vector-builtins-functions.def \
+ $(srcdir)/config/riscv/thead-vector-builtins-functions.def \
riscv-vector-type-indexer.gen.def
riscv-builtins.o: $(srcdir)/config/riscv/riscv-builtins.cc $(CONFIG_H) \
diff --git a/gcc/config/riscv/thead-vector-builtins-functions.def b/gcc/config/riscv/thead-vector-builtins-functions.def
new file mode 100644
index 00000000000..fd3ba29bae9
--- /dev/null
+++ b/gcc/config/riscv/thead-vector-builtins-functions.def
@@ -0,0 +1,39 @@
+#ifndef DEF_RVV_FUNCTION
+#define DEF_RVV_FUNCTION(NAME, SHAPE, PREDS, OPS_INFO)
+#endif
+
+#define REQUIRED_EXTENSIONS XTHEADVECTOR_EXT
+DEF_RVV_FUNCTION (vlb, th_loadstore_width, full_preds, all_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (vlh, th_loadstore_width, full_preds, all_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (vlw, th_loadstore_width, full_preds, all_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (vlbu, th_loadstore_width, full_preds, all_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (vlhu, th_loadstore_width, full_preds, all_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (vlwu, th_loadstore_width, full_preds, all_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (vsb, th_loadstore_width, none_m_preds, all_v_scalar_ptr_ops)
+DEF_RVV_FUNCTION (vsh, th_loadstore_width, none_m_preds, all_v_scalar_ptr_ops)
+DEF_RVV_FUNCTION (vsw, th_loadstore_width, none_m_preds, all_v_scalar_ptr_ops)
+DEF_RVV_FUNCTION (vlsb, th_loadstore_width, full_preds, all_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (vlsh, th_loadstore_width, full_preds, all_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (vlsw, th_loadstore_width, full_preds, all_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (vlsbu, th_loadstore_width, full_preds, all_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (vlshu, th_loadstore_width, full_preds, all_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (vlswu, th_loadstore_width, full_preds, all_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (vssb, th_loadstore_width, none_m_preds, all_v_scalar_ptr_size_ops)
+DEF_RVV_FUNCTION (vssh, th_loadstore_width, none_m_preds, all_v_scalar_ptr_size_ops)
+DEF_RVV_FUNCTION (vssw, th_loadstore_width, none_m_preds, all_v_scalar_ptr_size_ops)
+DEF_RVV_FUNCTION (vlxb, th_indexed_loadstore_width, full_preds, all_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (vlxh, th_indexed_loadstore_width, full_preds, all_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (vlxw, th_indexed_loadstore_width, full_preds, all_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (vlxbu, th_indexed_loadstore_width, full_preds, all_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (vlxhu, th_indexed_loadstore_width, full_preds, all_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (vlxwu, th_indexed_loadstore_width, full_preds, all_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (vsxb, th_indexed_loadstore_width, none_m_preds, all_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (vsxh, th_indexed_loadstore_width, none_m_preds, all_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (vsxw, th_indexed_loadstore_width, none_m_preds, all_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (vsuxb, th_indexed_loadstore_width, none_m_preds, all_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (vsuxh, th_indexed_loadstore_width, none_m_preds, all_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (vsuxw, th_indexed_loadstore_width, none_m_preds, all_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (vext_x_v, th_extract, none_preds, iu_x_s_u_ops)
+#undef REQUIRED_EXTENSIONS
+
+#undef DEF_RVV_FUNCTION
diff --git a/gcc/config/riscv/thead-vector.md b/gcc/config/riscv/thead-vector.md
index 696b815252d..0f3700d9269 100644
--- a/gcc/config/riscv/thead-vector.md
+++ b/gcc/config/riscv/thead-vector.md
@@ -1,7 +1,95 @@
(define_c_enum "unspec" [
+ UNSPEC_TH_VLB
+ UNSPEC_TH_VLBU
+ UNSPEC_TH_VLH
+ UNSPEC_TH_VLHU
+ UNSPEC_TH_VLW
+ UNSPEC_TH_VLWU
+
+ UNSPEC_TH_VLSB
+ UNSPEC_TH_VLSBU
+ UNSPEC_TH_VLSH
+ UNSPEC_TH_VLSHU
+ UNSPEC_TH_VLSW
+ UNSPEC_TH_VLSWU
+
+ UNSPEC_TH_VLXB
+ UNSPEC_TH_VLXBU
+ UNSPEC_TH_VLXH
+ UNSPEC_TH_VLXHU
+ UNSPEC_TH_VLXW
+ UNSPEC_TH_VLXWU
+
+ UNSPEC_TH_VSUXB
+ UNSPEC_TH_VSUXH
+ UNSPEC_TH_VSUXW
+
UNSPEC_TH_VWLDST
])
+(define_int_iterator UNSPEC_TH_VLMEM_OP [
+ UNSPEC_TH_VLB UNSPEC_TH_VLBU
+ UNSPEC_TH_VLH UNSPEC_TH_VLHU
+ UNSPEC_TH_VLW UNSPEC_TH_VLWU
+])
+
+(define_int_iterator UNSPEC_TH_VLSMEM_OP [
+ UNSPEC_TH_VLSB UNSPEC_TH_VLSBU
+ UNSPEC_TH_VLSH UNSPEC_TH_VLSHU
+ UNSPEC_TH_VLSW UNSPEC_TH_VLSWU
+])
+
+(define_int_iterator UNSPEC_TH_VLXMEM_OP [
+ UNSPEC_TH_VLXB UNSPEC_TH_VLXBU
+ UNSPEC_TH_VLXH UNSPEC_TH_VLXHU
+ UNSPEC_TH_VLXW UNSPEC_TH_VLXWU
+])
+
+(define_int_attr vlmem_op_attr [
+ (UNSPEC_TH_VLB "b") (UNSPEC_TH_VLBU "bu")
+ (UNSPEC_TH_VLH "h") (UNSPEC_TH_VLHU "hu")
+ (UNSPEC_TH_VLW "w") (UNSPEC_TH_VLWU "wu")
+ (UNSPEC_TH_VLSB "b") (UNSPEC_TH_VLSBU "bu")
+ (UNSPEC_TH_VLSH "h") (UNSPEC_TH_VLSHU "hu")
+ (UNSPEC_TH_VLSW "w") (UNSPEC_TH_VLSWU "wu")
+ (UNSPEC_TH_VLXB "b") (UNSPEC_TH_VLXBU "bu")
+ (UNSPEC_TH_VLXH "h") (UNSPEC_TH_VLXHU "hu")
+ (UNSPEC_TH_VLXW "w") (UNSPEC_TH_VLXWU "wu")
+ (UNSPEC_TH_VSUXB "b")
+ (UNSPEC_TH_VSUXH "h")
+ (UNSPEC_TH_VSUXW "w")
+])
+
+(define_int_attr vlmem_order_attr [
+ (UNSPEC_TH_VLXB "")
+ (UNSPEC_TH_VLXH "")
+ (UNSPEC_TH_VLXW "")
+ (UNSPEC_TH_VSUXB "u")
+ (UNSPEC_TH_VSUXH "u")
+ (UNSPEC_TH_VSUXW "u")
+])
+
+(define_int_iterator UNSPEC_TH_VSMEM_OP [
+ UNSPEC_TH_VLB
+ UNSPEC_TH_VLH
+ UNSPEC_TH_VLW
+])
+
+(define_int_iterator UNSPEC_TH_VSSMEM_OP [
+ UNSPEC_TH_VLSB
+ UNSPEC_TH_VLSH
+ UNSPEC_TH_VLSW
+])
+
+(define_int_iterator UNSPEC_TH_VSXMEM_OP [
+ UNSPEC_TH_VLXB
+ UNSPEC_TH_VLXH
+ UNSPEC_TH_VLXW
+ UNSPEC_TH_VSUXB
+ UNSPEC_TH_VSUXH
+ UNSPEC_TH_VSUXW
+])
+
(define_mode_iterator V_VLS_VT [V VLS VT])
(define_mode_iterator V_VB_VLS_VT [V VB VLS VT])
@@ -100,3 +188,168 @@
}
[(set_attr "type" "vldm,vstm,vmalu,vmalu,vmalu")
(set_attr "mode" "<MODE>")])
+
+(define_expand "@pred_mov_width<vlmem_op_attr><mode>"
+ [(set (match_operand:V_VLS 0 "nonimmediate_operand")
+ (if_then_else:V_VLS
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand")
+ (match_operand 4 "vector_length_operand")
+ (match_operand 5 "const_int_operand")
+ (match_operand 6 "const_int_operand")
+ (match_operand 7 "const_int_operand")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLMEM_OP)
+ (match_operand:V_VLS 3 "vector_move_operand")
+ (match_operand:V_VLS 2 "vector_merge_operand")))]
+ "TARGET_XTHEADVECTOR"
+ {})
+
+(define_insn_and_split "*pred_mov_width<vlmem_op_attr><mode>"
+ [(set (match_operand:V_VLS 0 "nonimmediate_operand" "=vr, vr, vd, m, vr, vr")
+ (if_then_else:V_VLS
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1, Wc1, vm, vmWc1, Wc1, Wc1")
+ (match_operand 4 "vector_length_operand" " rK, rK, rK, rK, rK, rK")
+ (match_operand 5 "const_int_operand" " i, i, i, i, i, i")
+ (match_operand 6 "const_int_operand" " i, i, i, i, i, i")
+ (match_operand 7 "const_int_operand" " i, i, i, i, i, i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLMEM_OP)
+ (match_operand:V_VLS 3 "reg_or_mem_operand" " m, m, m, vr, vr, vr")
+ (match_operand:V_VLS 2 "vector_merge_operand" " 0, vu, vu, vu, vu, 0")))]
+ "(TARGET_XTHEADVECTOR
+ && (register_operand (operands[0], <MODE>mode)
+ || register_operand (operands[3], <MODE>mode)))"
+ "@
+ vl<vlmem_op_attr>.v\t%0,%3%p1
+ vl<vlmem_op_attr>.v\t%0,%3
+ vl<vlmem_op_attr>.v\t%0,%3,%1.t
+ vs<vlmem_op_attr>.v\t%3,%0%p1
+ vmv.v.v\t%0,%3
+ vmv.v.v\t%0,%3"
+ "&& register_operand (operands[0], <MODE>mode)
+ && register_operand (operands[3], <MODE>mode)
+ && satisfies_constraint_vu (operands[2])
+ && INTVAL (operands[7]) == riscv_vector::VLMAX"
+ [(set (match_dup 0) (match_dup 3))]
+ ""
+ [(set_attr "type" "vlde,vlde,vlde,vste,vimov,vimov")
+ (set_attr "mode" "<MODE>")])
+
+(define_insn "@pred_store_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "memory_operand" "+m")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1")
+ (match_operand 3 "vector_length_operand" " rK")
+ (match_operand 4 "const_int_operand" " i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VSMEM_OP)
+ (match_operand:VI 2 "register_operand" " vr")
+ (match_dup 0)))]
+ "TARGET_XTHEADVECTOR"
+ "vs<vlmem_op_attr>.v\t%2,%0%p1"
+ [(set_attr "type" "vste")
+ (set_attr "mode" "<MODE>")
+ (set (attr "avl_type_idx") (const_int 4))
+ (set_attr "vl_op_idx" "3")])
+
+(define_insn "@pred_strided_load_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "register_operand" "=vr, vr, vd")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1, Wc1, vm")
+ (match_operand 5 "vector_length_operand" " rK, rK, rK")
+ (match_operand 6 "const_int_operand" " i, i, i")
+ (match_operand 7 "const_int_operand" " i, i, i")
+ (match_operand 8 "const_int_operand" " i, i, i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLSMEM_OP)
+ (unspec:VI
+ [(match_operand:VI 3 "memory_operand" " m, m, m")
+ (match_operand 4 "pmode_reg_or_0_operand" " rJ, rJ, rJ")] UNSPEC_TH_VLSMEM_OP)
+ (match_operand:VI 2 "vector_merge_operand" " 0, vu, vu")))]
+ "TARGET_XTHEADVECTOR"
+ "vls<vlmem_op_attr>.v\t%0,%3,%z4%p1"
+ [(set_attr "type" "vlds")
+ (set_attr "mode" "<MODE>")])
+
+(define_insn "@pred_strided_store_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "memory_operand" "+m")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1")
+ (match_operand 4 "vector_length_operand" " rK")
+ (match_operand 5 "const_int_operand" " i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VSSMEM_OP)
+ (unspec:VI
+ [(match_operand 2 "pmode_reg_or_0_operand" " rJ")
+ (match_operand:VI 3 "register_operand" " vr")] UNSPEC_TH_VSSMEM_OP)
+ (match_dup 0)))]
+ "TARGET_XTHEADVECTOR"
+ "vss<vlmem_op_attr>.v\t%3,%0,%z2%p1"
+ [(set_attr "type" "vsts")
+ (set_attr "mode" "<MODE>")
+ (set (attr "avl_type_idx") (const_int 5))])
+
+(define_insn "@pred_indexed_load_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "register_operand" "=vd, vr,vd, vr")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" " vm,Wc1,vm,Wc1")
+ (match_operand 5 "vector_length_operand" " rK, rK,rK, rK")
+ (match_operand 6 "const_int_operand" " i, i, i, i")
+ (match_operand 7 "const_int_operand" " i, i, i, i")
+ (match_operand 8 "const_int_operand" " i, i, i, i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLXMEM_OP)
+ (unspec:VI
+ [(match_operand 3 "pmode_reg_or_0_operand" " rJ, rJ,rJ, rJ")
+ (mem:BLK (scratch))
+ (match_operand:VI 4 "register_operand" " vr, vr,vr, vr")] UNSPEC_TH_VLXMEM_OP)
+ (match_operand:VI 2 "vector_merge_operand" " vu, vu, 0, 0")))]
+ "TARGET_XTHEADVECTOR"
+ "vlx<vlmem_op_attr>.v\t%0,(%z3),%4%p1"
+ [(set_attr "type" "vldux")
+ (set_attr "mode" "<MODE>")])
+
+(define_insn "@pred_indexed_<vlmem_order_attr>store_width<vlmem_op_attr><mode>"
+ [(set (mem:BLK (scratch))
+ (unspec:BLK
+ [(unspec:<VM>
+ [(match_operand:<VM> 0 "vector_mask_operand" "vmWc1")
+ (match_operand 4 "vector_length_operand" " rK")
+ (match_operand 5 "const_int_operand" " i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VSXMEM_OP)
+ (match_operand 1 "pmode_reg_or_0_operand" " rJ")
+ (match_operand:VI 2 "register_operand" " vr")
+ (match_operand:VI 3 "register_operand" " vr")] UNSPEC_TH_VSXMEM_OP))]
+ "TARGET_XTHEADVECTOR"
+ "vs<vlmem_order_attr>x<vlmem_op_attr>.v\t%3,(%z1),%2%p0"
+ [(set_attr "type" "vstux")
+ (set_attr "mode" "<MODE>")])
+
+(define_expand "@pred_th_extract<mode>"
+ [(set (match_operand:<VEL> 0 "register_operand")
+ (unspec:<VEL>
+ [(vec_select:<VEL>
+ (match_operand:V_VLSI 1 "register_operand")
+ (parallel [(match_operand:DI 2 "register_operand" "r")]))
+ (reg:SI VTYPE_REGNUM)] UNSPEC_VPREDICATE))]
+ "TARGET_XTHEADVECTOR"
+{})
+
+(define_insn "*pred_th_extract<mode>"
+ [(set (match_operand:<VEL> 0 "register_operand" "=r")
+ (unspec:<VEL>
+ [(vec_select:<VEL>
+ (match_operand:V_VLSI 1 "register_operand" "vr")
+ (parallel [(match_operand:DI 2 "register_operand" "r")]))
+ (reg:SI VTYPE_REGNUM)] UNSPEC_VPREDICATE))]
+ "TARGET_XTHEADVECTOR"
+ "vext.x.v\t%0,%1,%2"
+ [(set_attr "type" "vimovvx")
+ (set_attr "mode" "<MODE>")])
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c
new file mode 100644
index 00000000000..3c12c124597
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_th_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+
+** th.vadd\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+
+** th.vsb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out)
+{
+ vint32m1_t v = __riscv_th_vlb_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlb_v_i32m1_tu (v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vv_i32m1 (v2, v2, 4);
+ vint32m1_t v4 = __riscv_vadd_vv_i32m1_tu (v3, v2, v2, 4);
+ __riscv_th_vsb_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlb.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+
+** th.vadd\.vv\tv[1-9][0-9]?,\s*v[0-9]+,\s*v[0-9]+,\s*v0.t
+** th.vsb.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlb_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlb_v_i32m1_m (mask, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vv_i32m1 (v2, v2, 4);
+ vint32m1_t v4 = __riscv_vadd_vv_i32m1_m (mask, v3, v3, 4);
+ __riscv_th_vsb_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlb.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+
+** th.vadd\.vv\tv[1-9][0-9]?,\s*v[0-9]+,\s*v[0-9]+,\s*v0.t
+** th.vsb.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlb_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlb_v_i32m1_tumu (mask, v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vv_i32m1 (v2, v2, 4);
+ vint32m1_t v4 = __riscv_vadd_vv_i32m1_tumu (mask, v3, v2, v2, 4);
+ __riscv_th_vsb_v_i32m1 (out, v4, 4);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c
new file mode 100644
index 00000000000..30bef369375
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_th_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlbu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlbu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vsb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, uint32_t x)
+{
+ vuint32m1_t v = __riscv_th_vlbu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlbu_v_u32m1_tu (v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tu (v3, v2, -16, 4);
+ __riscv_th_vsb_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlbu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsb.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlbu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlbu_v_u32m1_m (mask, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_m (mask, v3, -16, 4);
+ __riscv_th_vsb_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlbu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlbu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsb.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlbu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlbu_v_u32m1_tumu (mask, v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tumu (mask, v3, v2, -16, 4);
+ __riscv_th_vsb_v_u32m1 (out, v4, 4);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c
new file mode 100644
index 00000000000..3c8b5ccc16b
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_th_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vsh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, int32_t x)
+{
+ vint32m1_t v = __riscv_th_vlh_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlh_v_i32m1_tu (v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, -16, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_tu (v3, v2, -16, 4);
+ __riscv_th_vsh_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlh.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsh.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, int32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlh_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlh_v_i32m1_m (mask, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, -16, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_m (mask, v3, -16, 4);
+ __riscv_th_vsh_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlh.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsh.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, int32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlh_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlh_v_i32m1_tumu (mask, v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, -16, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_tumu (mask, v3, v2, -16, 4);
+ __riscv_th_vsh_v_i32m1 (out, v4, 4);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c
new file mode 100644
index 00000000000..b7c00404f18
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_th_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlhu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlhu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vsh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, uint32_t x)
+{
+ vuint32m1_t v = __riscv_th_vlhu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlhu_v_u32m1_tu (v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tu (v3, v2, -16, 4);
+ __riscv_th_vsh_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlhu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsh.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlhu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlhu_v_u32m1_m (mask, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_m (mask, v3, -16, 4);
+ __riscv_th_vsh_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlhu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlhu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsh.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlhu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlhu_v_u32m1_tumu (mask, v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tumu (mask, v3, v2, -16, 4);
+ __riscv_th_vsh_v_u32m1 (out, v4, 4);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c
new file mode 100644
index 00000000000..17a53012acf
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_th_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vx\tv[0-9]+,\s*v[0-9]+,\s*[a-x0-9]+
+** th.vadd\.vx\tv[0-9]+,\s*v[0-9]+,\s*[a-x0-9]+
+** th.vsw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, int32_t x)
+{
+ vint32m1_t v = __riscv_th_vlw_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlw_v_i32m1_tu (v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, x, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_tu (v3, v2, x, 4);
+ __riscv_th_vsw_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlw.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vx\tv[0-9]+,\s*v[0-9]+,\s*[a-x0-9]+
+** th.vadd\.vx\tv[1-9][0-9]?,\s*v[0-9]+,\s*[a-x0-9]+,\s*v0.t
+** th.vsw.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, int32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlw_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlw_v_i32m1_m (mask, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, x, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_m (mask, v3, x, 4);
+ __riscv_th_vsw_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlw.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vx\tv[0-9]+,\s*v[0-9]+,\s*[a-x0-9]+
+** th.vadd\.vx\tv[1-9][0-9]?,\s*v[0-9]+,\s*[a-x0-9]+,\s*v0.t
+** th.vsw.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, int32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlw_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlw_v_i32m1_tumu (mask, v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, x, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_tumu (mask, v3, v2, x, 4);
+ __riscv_th_vsw_v_i32m1 (out, v4, 4);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c
new file mode 100644
index 00000000000..b187cfc852b
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_th_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlwu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlwu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vsw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, uint32_t x)
+{
+ vuint32m1_t v = __riscv_th_vlwu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlwu_v_u32m1_tu (v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tu (v3, v2, -16, 4);
+ __riscv_th_vsw_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlwu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsw.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlwu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlwu_v_u32m1_m (mask, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_m (mask, v3, -16, 4);
+ __riscv_th_vsw_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlwu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlwu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsw.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlwu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlwu_v_u32m1_tumu (mask, v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tumu (mask, v3, v2, -16, 4);
+ __riscv_th_vsw_v_u32m1 (out, v4, 4);
+}
--
2.17.1
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re：Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
2024-01-11 9:28 Re：[PATCH " juzhe.zhong
@ 2024-01-11 9:35 ` joshua
0 siblings, 0 replies; 17+ messages in thread
From: joshua @ 2024-01-11 9:35 UTC (permalink / raw)
To: juzhe.zhong, gcc-patches
Cc: Jim Wilson, palmer, andrew, philipp.tomsich, jeffreyalaw,
christoph.muellner, jinma, cooper.qu
I see. We added ! TARGET_THEADVECTOR initally becasue
we want to provide a patchset version that can work without any
errors both in O0 and O2. Without these changes, we will get "
unrecognized opcode" in O2 during assembly stage.
------------------------------------------------------------------
发件人：juzhe.zhong@rivai.ai <juzhe.zhong@rivai.ai>
发送时间：2024年1月11日(星期四) 17:28
收件人："cooper.joshua"<cooper.joshua@linux.alibaba.com>; "gcc-patches"<gcc-patches@gcc.gnu.org>
抄 送：Jim Wilson<jim.wilson.gcc@gmail.com>; palmer<palmer@dabbelt.com>; andrew<andrew@sifive.com>; "philipp.tomsich"<philipp.tomsich@vrull.eu>; jeffreyalaw<jeffreyalaw@gmail.com>; "christoph.muellner"<christoph.muellner@vrull.eu>; jinma<jinma@linux.alibaba.com>; "cooper.qu"<cooper.qu@linux.alibaba.com>
主 题：Re: Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
I prefer you remove those TARGET_THEADVECTOR for now.
And file PR let me see the real problem.
I don't believe this should not fixed by this way.
juzhe.zhong@rivai.ai
发件人： joshua
发送时间： 2024-01-11 17:26
收件人： juzhe.zhong@rivai.ai; gcc-patches
抄送： Jim Wilson; palmer; andrew; philipp.tomsich; jeffreyalaw; christoph.muellner; jinma; cooper.qu
主题： Re：Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
Maybe the optimization cannot be done in simple cases. We run some complex cases
in O2 and dsicovered it.
------------------------------------------------------------------
发件人：juzhe.zhong@rivai.ai <juzhe.zhong@rivai.ai>
发送时间：2024年1月11日(星期四) 17:17
收件人："cooper.joshua"<cooper.joshua@linux.alibaba.com>; "gcc-patches"<gcc-patches@gcc.gnu.org>
抄 送：Jim Wilson<jim.wilson.gcc@gmail.com>; palmer<palmer@dabbelt.com>; andrew<andrew@sifive.com>; "philipp.tomsich"<philipp.tomsich@vrull.eu>; jeffreyalaw<jeffreyalaw@gmail.com>; "christoph.muellner"<christoph.muellner@vrull.eu>; jinma<jinma@linux.alibaba.com>; "cooper.qu"<cooper.qu@linux.alibaba.com>
主 题：Re: Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
You mean which pattern optimized sext/vzext pattern?
I didn't see theadvector-specific extension patterns. Could you show me?
juzhe.zhong@rivai.ai
发件人： joshua
发送时间： 2024-01-11 17:14
收件人： juzhe.zhong@rivai.ai; gcc-patches
抄送： Jim Wilson; palmer; andrew; philipp.tomsich; jeffreyalaw; christoph.muellner; jinma; cooper.qu
主题： Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
To be specific, in CSE pass, the initial pattern will be optimized into the sext/zext pattern.
------------------------------------------------------------------
发件人：joshua <cooper.joshua@linux.alibaba.com>
发送时间：2024年1月11日(星期四) 17:11
收件人："juzhe.zhong@rivai.ai"<juzhe.zhong@rivai.ai>; "gcc-patches"<gcc-patches@gcc.gnu.org>
抄 送：Jim Wilson<jim.wilson.gcc@gmail.com>; palmer<palmer@dabbelt.com>; andrew<andrew@sifive.com>; "philipp.tomsich"<philipp.tomsich@vrull.eu>; jeffreyalaw<jeffreyalaw@gmail.com>; "christoph.muellner"<christoph.muellner@vrull.eu>; jinma<jinma@linux.alibaba.com>; "cooper.qu"<cooper.qu@linux.alibaba.com>
主 题：Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
sext/zext will be generated in O2 even without corresponding intrinsics.
------------------------------------------------------------------
发件人：juzhe.zhong@rivai.ai <juzhe.zhong@rivai.ai>
发送时间：2024年1月11日(星期四) 17:07
收件人："cooper.joshua"<cooper.joshua@linux.alibaba.com>; "gcc-patches"<gcc-patches@gcc.gnu.org>
抄 送：Jim Wilson<jim.wilson.gcc@gmail.com>; palmer<palmer@dabbelt.com>; andrew<andrew@sifive.com>; "philipp.tomsich"<philipp.tomsich@vrull.eu>; jeffreyalaw<jeffreyalaw@gmail.com>; "christoph.muellner"<christoph.muellner@vrull.eu>; "cooper.joshua"<cooper.joshua@linux.alibaba.com>; jinma<jinma@linux.alibaba.com>; "cooper.qu"<cooper.qu@linux.alibaba.com>
主 题：Re: [PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
enum required_ext
{
VECTOR_EXT, /* Vector extension */
+ XTHEADVECTOR_EXT, /* XTheadVector extension */
ZVBB_EXT, /* Cryto vector Zvbb sub-ext */
ZVBB_OR_ZVKB_EXT, /* Cryto vector Zvbb or zvkb sub-ext */
ZVBC_EXT, /* Crypto vector Zvbc sub-ext */
Add theadvector to the end of the enum.
+ case XTHEADVECTOR_EXT:
+ return TARGET_XTHEADVECTOR;
Same.
+ "&& register_operand (operands[0], <MODE>mode)
+ && register_operand (operands[3], <MODE>mode)
+ && satisfies_constraint_vu (operands[2])
+ && INTVAL (operands[7]) == riscv_vector::VLMAX"
You can use whole_reg_to_reg_move_p
Btw, I review again : RISC-V: Handle differences between XTheadvector and Vector
(any_extend:VWEXTI (match_operand:<V_DOUBLE_TRUNC> 3 "register_operand" "W21,W21,W21,W21,W42,W42,W42,W42,W84,W84,W84,W84, vr, vr")) (match_operand:VWEXTI 2 "vector_merge_operand" " vu, vu, 0, 0, vu, vu, 0, 0, vu, vu, 0, 0, vu, 0")))]- "TARGET_VECTOR"+ "TARGET_VECTOR && !TARGET_XTHEADVECTOR" "v<sz>ext.vf2\t%0,%3%p1" [(set_attr "type" "vext") (set_attr "mode" "<MODE>")@@ -3713,7 +3744,7 @@ (any_extend:VQEXTI (match_operand:<V_QUAD_TRUNC> 3 "register_operand" "W43,W43,W43,W43,W86,W86,W86,W86, vr, vr")) (match_operand:VQEXTI 2 "vector_merge_operand" " vu, vu, 0, 0, vu, vu, 0, 0, vu, 0")))]- "TARGET_VECTOR"+ "TARGET_VECTOR && !TARGET_XTHEADVECTOR" "v<sz>ext.vf4\t%0,%3%p1" [(set_attr "type" "vext") (set_attr "mode" "<MODE>")@@ -3734,7 +3765,7 @@ (any_extend:VOEXTI (match_operand:<V_OCT_TRUNC> 3 "register_operand" "W87,W87,W87,W87, vr, vr")) (match_operand:VOEXTI 2 "vector_merge_operand" " vu, vu, 0, 0, vu, 0")))]- "TARGET_VECTOR"+ "TARGET_VECTOR && !TARGET_XTHEADVECTOR" "v<sz>ext.vf8\t%0,%3%p1" [(set_attr "type" "vext") (set_attr "mode" "<MODE>")
Why do you add these !TARGERT_XTHEADVECRTOR ?
juzhe.zhong@rivai.ai
From: Jun Sha (Joshua)
Date: 2024-01-11 16:46
To: gcc-patches
CC: jim.wilson.gcc; palmer; andrew; philipp.tomsich; jeffreyalaw; christoph.muellner; juzhe.zhong; Jun Sha (Joshua); Jin Ma; Xianmiao Qu
Subject: [PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
This patch only involves the generation of xtheadvector
special load/store instructions and vext instructions.
gcc/ChangeLog:
* config/riscv/riscv-vector-builtins-bases.cc
(class th_loadstore_width): Define new builtin bases.
(class th_extract): Define new builtin bases.
(BASE): Define new builtin bases.
* config/riscv/riscv-vector-builtins-bases.h:
Define new builtin class.
* config/riscv/riscv-vector-builtins-shapes.cc
(struct th_loadstore_width_def): Define new builtin shapes.
(struct th_indexed_loadstore_width_def):
Define new builtin shapes.
(struct th_extract_def): Define new builtin shapes.
(SHAPE): Define new builtin shapes.
* config/riscv/riscv-vector-builtins-shapes.h:
Define new builtin shapes.
* config/riscv/riscv-vector-builtins.cc (DEF_RVV_FUNCTION):
* config/riscv/riscv-vector-builtins.h (enum required_ext):
(struct function_group_info):
* config/riscv/t-riscv: Add thead-vector-builtins-functions.def
* config/riscv/thead-vector.md
(@pred_mov_width<vlmem_op_attr><mode>): Add new patterns.
(*pred_mov_width<vlmem_op_attr><mode>): Likewise.
(@pred_store_width<vlmem_op_attr><mode>): Likewise.
(@pred_strided_load_width<vlmem_op_attr><mode>): Likewise.
(@pred_strided_store_width<vlmem_op_attr><mode>): Likewise.
(@pred_indexed_load_width<vlmem_op_attr><mode>): Likewise.
(@pred_indexed_<vlmem_order_attr>store_width<vlmem_op_attr><mode>):
(@pred_th_extract<mode>): Likewise.
(*pred_th_extract<mode>): Likewise.
* config/riscv/thead-vector-builtins-functions.def: New file.
gcc/testsuite/ChangeLog:
* gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c: New test.
Co-authored-by: Jin Ma <jinma@linux.alibaba.com>
Co-authored-by: Xianmiao Qu <cooper.qu@linux.alibaba.com>
Co-authored-by: Christoph Müllner <christoph.muellner@vrull.eu>
---
.../riscv/riscv-vector-builtins-bases.cc | 139 ++++++++++
.../riscv/riscv-vector-builtins-bases.h | 31 +++
.../riscv/riscv-vector-builtins-shapes.cc | 160 +++++++++++
.../riscv/riscv-vector-builtins-shapes.h | 3 +
gcc/config/riscv/riscv-vector-builtins.cc | 70 +++++
gcc/config/riscv/riscv-vector-builtins.h | 3 +
gcc/config/riscv/t-riscv | 1 +
.../riscv/thead-vector-builtins-functions.def | 39 +++
gcc/config/riscv/thead-vector.md | 253 ++++++++++++++++++
.../riscv/rvv/xtheadvector/vlb-vsb.c | 68 +++++
.../riscv/rvv/xtheadvector/vlbu-vsb.c | 68 +++++
.../riscv/rvv/xtheadvector/vlh-vsh.c | 68 +++++
.../riscv/rvv/xtheadvector/vlhu-vsh.c | 68 +++++
.../riscv/rvv/xtheadvector/vlw-vsw.c | 68 +++++
.../riscv/rvv/xtheadvector/vlwu-vsw.c | 68 +++++
15 files changed, 1107 insertions(+)
create mode 100644 gcc/config/riscv/thead-vector-builtins-functions.def
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c
diff --git a/gcc/config/riscv/riscv-vector-builtins-bases.cc b/gcc/config/riscv/riscv-vector-builtins-bases.cc
index 46f1a1da33e..3eba7943757 100644
--- a/gcc/config/riscv/riscv-vector-builtins-bases.cc
+++ b/gcc/config/riscv/riscv-vector-builtins-bases.cc
@@ -2125,6 +2125,83 @@ public:
}
};
+/* Implements
+ * th.vl(b/h/w)[u].v/th.vs(b/h/w)[u].v/th.vls(b/h/w)[u].v/th.vss(b/h/w)[u].v/
+ * th.vlx(b/h/w)[u].v/th.vs[u]x(b/h/w).v
+ * codegen. */
+template<bool STORE_P, lst_type LST_TYPE, int UNSPEC>
+class th_loadstore_width : public function_base
+{
+public:
+ bool apply_tail_policy_p () const override { return !STORE_P; }
+ bool apply_mask_policy_p () const override { return !STORE_P; }
+
+ unsigned int call_properties (const function_instance &) const override
+ {
+ if (STORE_P)
+ return CP_WRITE_MEMORY;
+ else
+ return CP_READ_MEMORY;
+ }
+
+ bool can_be_overloaded_p (enum predication_type_index pred) const override
+ {
+ if (STORE_P || LST_TYPE == LST_INDEXED)
+ return true;
+ return pred != PRED_TYPE_none;
+ }
+
+ rtx expand (function_expander &e) const override
+ {
+ gcc_assert (TARGET_XTHEADVECTOR);
+ if (LST_TYPE == LST_INDEXED)
+ {
+ if (STORE_P)
+ return e.use_exact_insn (
+ code_for_pred_indexed_store_width (UNSPEC, UNSPEC,
+ e.vector_mode ()));
+ else
+ return e.use_exact_insn (
+ code_for_pred_indexed_load_width (UNSPEC, e.vector_mode ()));
+ }
+ else if (LST_TYPE == LST_STRIDED)
+ {
+ if (STORE_P)
+ return e.use_contiguous_store_insn (
+ code_for_pred_strided_store_width (UNSPEC, e.vector_mode ()));
+ else
+ return e.use_contiguous_load_insn (
+ code_for_pred_strided_load_width (UNSPEC, e.vector_mode ()));
+ }
+ else
+ {
+ if (STORE_P)
+ return e.use_contiguous_store_insn (
+ code_for_pred_store_width (UNSPEC, e.vector_mode ()));
+ else
+ return e.use_contiguous_load_insn (
+ code_for_pred_mov_width (UNSPEC, e.vector_mode ()));
+ }
+ }
+};
+
+/* Implements vext.x.v. */
+class th_extract : public function_base
+{
+public:
+ bool apply_vl_p () const override { return false; }
+ bool apply_tail_policy_p () const override { return false; }
+ bool apply_mask_policy_p () const override { return false; }
+ bool use_mask_predication_p () const override { return false; }
+ bool has_merge_operand_p () const override { return false; }
+
+ rtx expand (function_expander &e) const override
+ {
+ gcc_assert (TARGET_XTHEADVECTOR);
+ return e.use_exact_insn (code_for_pred_th_extract (e.vector_mode ()));
+ }
+};
+
/* Below implements are vector crypto */
/* Implements vandn.[vv,vx] */
class vandn : public function_base
@@ -2587,6 +2664,37 @@ static CONSTEXPR const seg_indexed_load<UNSPEC_ORDERED> vloxseg_obj;
static CONSTEXPR const seg_indexed_store<UNSPEC_UNORDERED> vsuxseg_obj;
static CONSTEXPR const seg_indexed_store<UNSPEC_ORDERED> vsoxseg_obj;
static CONSTEXPR const vlsegff vlsegff_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLB> vlb_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLBU> vlbu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLH> vlh_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLHU> vlhu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLW> vlw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLWU> vlwu_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_UNIT_STRIDE, UNSPEC_TH_VLB> vsb_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_UNIT_STRIDE, UNSPEC_TH_VLH> vsh_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_UNIT_STRIDE, UNSPEC_TH_VLW> vsw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSB> vlsb_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSBU> vlsbu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSH> vlsh_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSHU> vlshu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSW> vlsw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSWU> vlswu_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_STRIDED, UNSPEC_TH_VLSB> vssb_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_STRIDED, UNSPEC_TH_VLSH> vssh_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_STRIDED, UNSPEC_TH_VLSW> vssw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXB> vlxb_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXBU> vlxbu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXH> vlxh_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXHU> vlxhu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXW> vlxw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXWU> vlxwu_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VLXB> vsxb_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VLXH> vsxh_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VLXW> vsxw_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VSUXB> vsuxb_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VSUXH> vsuxh_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VSUXW> vsuxw_obj;
+static CONSTEXPR const th_extract vext_x_v_obj;
/* Crypto Vector */
static CONSTEXPR const vandn vandn_obj;
@@ -2878,6 +2986,37 @@ BASE (vloxseg)
BASE (vsuxseg)
BASE (vsoxseg)
BASE (vlsegff)
+BASE (vlb)
+BASE (vlh)
+BASE (vlw)
+BASE (vlbu)
+BASE (vlhu)
+BASE (vlwu)
+BASE (vsb)
+BASE (vsh)
+BASE (vsw)
+BASE (vlsb)
+BASE (vlsh)
+BASE (vlsw)
+BASE (vlsbu)
+BASE (vlshu)
+BASE (vlswu)
+BASE (vssb)
+BASE (vssh)
+BASE (vssw)
+BASE (vlxb)
+BASE (vlxh)
+BASE (vlxw)
+BASE (vlxbu)
+BASE (vlxhu)
+BASE (vlxwu)
+BASE (vsxb)
+BASE (vsxh)
+BASE (vsxw)
+BASE (vsuxb)
+BASE (vsuxh)
+BASE (vsuxw)
+BASE (vext_x_v)
/* Crypto vector */
BASE (vandn)
BASE (vbrev)
diff --git a/gcc/config/riscv/riscv-vector-builtins-bases.h b/gcc/config/riscv/riscv-vector-builtins-bases.h
index 1122e3801a7..565a0311d2b 100644
--- a/gcc/config/riscv/riscv-vector-builtins-bases.h
+++ b/gcc/config/riscv/riscv-vector-builtins-bases.h
@@ -299,6 +299,37 @@ extern const function_base *const vloxseg;
extern const function_base *const vsuxseg;
extern const function_base *const vsoxseg;
extern const function_base *const vlsegff;
+extern const function_base *const vlb;
+extern const function_base *const vlh;
+extern const function_base *const vlw;
+extern const function_base *const vlbu;
+extern const function_base *const vlhu;
+extern const function_base *const vlwu;
+extern const function_base *const vsb;
+extern const function_base *const vsh;
+extern const function_base *const vsw;
+extern const function_base *const vlsb;
+extern const function_base *const vlsh;
+extern const function_base *const vlsw;
+extern const function_base *const vlsbu;
+extern const function_base *const vlshu;
+extern const function_base *const vlswu;
+extern const function_base *const vssb;
+extern const function_base *const vssh;
+extern const function_base *const vssw;
+extern const function_base *const vlxb;
+extern const function_base *const vlxh;
+extern const function_base *const vlxw;
+extern const function_base *const vlxbu;
+extern const function_base *const vlxhu;
+extern const function_base *const vlxwu;
+extern const function_base *const vsxb;
+extern const function_base *const vsxh;
+extern const function_base *const vsxw;
+extern const function_base *const vsuxb;
+extern const function_base *const vsuxh;
+extern const function_base *const vsuxw;
+extern const function_base *const vext_x_v;
/* Below function_base are Vectro Crypto*/
extern const function_base *const vandn;
extern const function_base *const vbrev;
diff --git a/gcc/config/riscv/riscv-vector-builtins-shapes.cc b/gcc/config/riscv/riscv-vector-builtins-shapes.cc
index 1e4f4d53de6..8e90b17a94b 100644
--- a/gcc/config/riscv/riscv-vector-builtins-shapes.cc
+++ b/gcc/config/riscv/riscv-vector-builtins-shapes.cc
@@ -211,6 +211,146 @@ struct indexed_loadstore_def : public function_shape
}
};
+/* Add one function instance for GROUP, using operand suffix at index OI,
+ mode suffix at index PAIR && bi and predication suffix at index pred_idx. */
+static void
+build_th_loadstore (function_builder &b, const function_group_info &group,
+ unsigned int pred_idx, unsigned int vec_type_idx)
+{
+ auto_vec<tree, 5> argument_types;
+ function_instance function_instance (group.base_name, *group.base,
+ *group.shape,
+ group.ops_infos.types[vec_type_idx],
+ group.preds[pred_idx], &group.ops_infos);
+ tree return_type = group.ops_infos.ret.get_tree_type (
+ group.ops_infos.types[vec_type_idx].index);
+ b.allocate_argument_types (function_instance, argument_types);
+ b.apply_predication (function_instance, return_type, argument_types);
+
+ if (TARGET_XTHEADVECTOR && !check_type (return_type, argument_types))
+ return;
+
+ tree type = builtin_types[group.ops_infos.types[vec_type_idx].index].vector;
+ if (strstr (group.base_name, "l")
+ && strstr (group.base_name, "u")
+ && !TYPE_UNSIGNED (TREE_TYPE (type)))
+ return;
+
+ if (strstr (group.base_name, "l")
+ && !strstr (group.base_name, "u")
+ && TYPE_UNSIGNED (TREE_TYPE (type)))
+ return;
+
+ machine_mode mode = TYPE_MODE (type);
+ int sew = GET_MODE_BITSIZE (GET_MODE_INNER (mode));
+ if (strstr (group.base_name, "h") && sew == 8)
+ return;
+
+ if (strstr (group.base_name, "w") && (sew == 8 || sew ==16))
+ return;
+
+ b.add_overloaded_function (function_instance, *group.shape);
+ b.add_unique_function (function_instance, (*group.shape), return_type,
+ argument_types);
+}
+
+/* th_loadstore_width_def class. */
+struct th_loadstore_width_def : public build_base
+{
+ void build (function_builder &b,
+ const function_group_info &group) const override
+ {
+ for (unsigned int pred_idx = 0; group.preds[pred_idx] != NUM_PRED_TYPES;
+ ++pred_idx)
+ {
+ for (unsigned int vec_type_idx = 0;
+ group.ops_infos.types[vec_type_idx].index != NUM_VECTOR_TYPES;
+ ++vec_type_idx)
+ {
+ build_th_loadstore (b, group, pred_idx, vec_type_idx);
+ }
+ }
+ }
+
+ char *get_name (function_builder &b, const function_instance &instance,
+ bool overloaded_p) const override
+ {
+ /* Return nullptr if it can not be overloaded. */
+ if (overloaded_p && !instance.base->can_be_overloaded_p (instance.pred))
+ return nullptr;
+
+ b.append_name ("__riscv_th_");
+ b.append_name (instance.base_name);
+
+ /* vop_v --> vop_v_<type>. */
+ if (!overloaded_p)
+ {
+ /* vop --> vop_v. */
+ b.append_name (operand_suffixes[instance.op_info->op]);
+ /* vop_v --> vop_v_<type>. */
+ b.append_name (type_suffixes[instance.type.index].vector);
+ }
+
+ /* According to rvv-intrinsic-doc, it does not add "_m" suffix
+ for vop_m C++ overloaded API. */
+ if (overloaded_p && instance.pred == PRED_TYPE_m)
+ return b.finish_name ();
+ b.append_name (predication_suffixes[instance.pred]);
+ return b.finish_name ();
+ }
+};
+
+
+/* th_indexed_loadstore_width_def class. */
+struct th_indexed_loadstore_width_def : public function_shape
+{
+ void build (function_builder &b,
+ const function_group_info &group) const override
+ {
+ for (unsigned int pred_idx = 0; group.preds[pred_idx] != NUM_PRED_TYPES;
+ ++pred_idx)
+ {
+ for (unsigned int vec_type_idx = 0;
+ group.ops_infos.types[vec_type_idx].index != NUM_VECTOR_TYPES;
+ ++vec_type_idx)
+ {
+ tree index_type = group.ops_infos.args[1].get_tree_type (
+ group.ops_infos.types[vec_type_idx].index);
+ if (!index_type)
+ continue;
+ build_th_loadstore (b, group, pred_idx, vec_type_idx);
+ }
+ }
+ }
+
+ char *get_name (function_builder &b, const function_instance &instance,
+ bool overloaded_p) const override
+ {
+
+ /* Return nullptr if it can not be overloaded. */
+ if (overloaded_p && !instance.base->can_be_overloaded_p (instance.pred))
+ return nullptr;
+
+ b.append_name ("__riscv_th_");
+ b.append_name (instance.base_name);
+ /* vop_v --> vop_v_<type>. */
+ if (!overloaded_p)
+ {
+ /* vop --> vop_v. */
+ b.append_name (operand_suffixes[instance.op_info->op]);
+ /* vop_v --> vop_v_<type>. */
+ b.append_name (type_suffixes[instance.type.index].vector);
+ }
+
+ /* According to rvv-intrinsic-doc, it does not add "_m" suffix
+ for vop_m C++ overloaded API. */
+ if (overloaded_p && instance.pred == PRED_TYPE_m)
+ return b.finish_name ();
+ b.append_name (predication_suffixes[instance.pred]);
+ return b.finish_name ();
+ }
+};
+
/* alu_def class. */
struct alu_def : public build_base
{
@@ -632,6 +772,23 @@ struct reduc_alu_def : public build_base
}
};
+/* th_extract_def class. */
+struct th_extract_def : public build_base
+{
+ char *get_name (function_builder &b, const function_instance &instance,
+ bool overloaded_p) const override
+ {
+ b.append_name ("__riscv_th_");
+ b.append_name (instance.base_name);
+
+ if (overloaded_p)
+ return b.finish_name ();
+ b.append_name (type_suffixes[instance.type.index].vector);
+ b.append_name (type_suffixes[instance.type.index].scalar);
+ return b.finish_name ();
+ }
+};
+
/* scalar_move_def class. */
struct scalar_move_def : public build_base
{
@@ -1094,6 +1251,8 @@ SHAPE(vsetvl, vsetvl)
SHAPE(vsetvl, vsetvlmax)
SHAPE(loadstore, loadstore)
SHAPE(indexed_loadstore, indexed_loadstore)
+SHAPE(th_loadstore_width, th_loadstore_width)
+SHAPE(th_indexed_loadstore_width, th_indexed_loadstore_width)
SHAPE(alu, alu)
SHAPE(alu_frm, alu_frm)
SHAPE(widen_alu, widen_alu)
@@ -1106,6 +1265,7 @@ SHAPE(move, move)
SHAPE(mask_alu, mask_alu)
SHAPE(reduc_alu, reduc_alu)
SHAPE(reduc_alu_frm, reduc_alu_frm)
+SHAPE(th_extract, th_extract)
SHAPE(scalar_move, scalar_move)
SHAPE(vundefined, vundefined)
SHAPE(misc, misc)
diff --git a/gcc/config/riscv/riscv-vector-builtins-shapes.h b/gcc/config/riscv/riscv-vector-builtins-shapes.h
index ac2a28ce017..a7624d0fabd 100644
--- a/gcc/config/riscv/riscv-vector-builtins-shapes.h
+++ b/gcc/config/riscv/riscv-vector-builtins-shapes.h
@@ -28,6 +28,8 @@ extern const function_shape *const vsetvl;
extern const function_shape *const vsetvlmax;
extern const function_shape *const loadstore;
extern const function_shape *const indexed_loadstore;
+extern const function_shape *const th_loadstore_width;
+extern const function_shape *const th_indexed_loadstore_width;
extern const function_shape *const alu;
extern const function_shape *const alu_frm;
extern const function_shape *const widen_alu;
@@ -41,6 +43,7 @@ extern const function_shape *const mask_alu;
extern const function_shape *const reduc_alu;
extern const function_shape *const reduc_alu_frm;
extern const function_shape *const scalar_move;
+extern const function_shape *const th_extract;
extern const function_shape *const vundefined;
extern const function_shape *const misc;
extern const function_shape *const vset;
diff --git a/gcc/config/riscv/riscv-vector-builtins.cc b/gcc/config/riscv/riscv-vector-builtins.cc
index 25e0b6e56de..44b9fec1898 100644
--- a/gcc/config/riscv/riscv-vector-builtins.cc
+++ b/gcc/config/riscv/riscv-vector-builtins.cc
@@ -934,6 +934,32 @@ static CONSTEXPR const rvv_arg_type_info ext_vcreate_args[]
= {rvv_arg_type_info (RVV_BASE_vector),
rvv_arg_type_info_end};
+/* A list of args for vector_type func (const scalar_type *, size_t)
+ * function. */
+static CONSTEXPR const rvv_arg_type_info scalar_const_ptr_size_args[]
+ = {rvv_arg_type_info (RVV_BASE_scalar_const_ptr),
+ rvv_arg_type_info (RVV_BASE_size), rvv_arg_type_info_end};
+
+/* A list of args for vector_type func (const scalar_type *, eew8_index_type)
+ * function. */
+static CONSTEXPR const rvv_arg_type_info scalar_const_ptr_index_args[]
+ = {rvv_arg_type_info (RVV_BASE_scalar_const_ptr),
+ rvv_arg_type_info (RVV_BASE_unsigned_vector), rvv_arg_type_info_end};
+
+/* A list of args for void func (scalar_type *, eew8_index_type, vector_type)
+ * function. */
+static CONSTEXPR const rvv_arg_type_info scalar_ptr_index_args[]
+ = {rvv_arg_type_info (RVV_BASE_scalar_ptr),
+ rvv_arg_type_info (RVV_BASE_unsigned_vector),
+ rvv_arg_type_info (RVV_BASE_vector), rvv_arg_type_info_end};
+
+/* A list of args for void func (scalar_type *, size_t, vector_type)
+ * function. */
+static CONSTEXPR const rvv_arg_type_info scalar_ptr_size_args[]
+ = {rvv_arg_type_info (RVV_BASE_scalar_ptr),
+ rvv_arg_type_info (RVV_BASE_size), rvv_arg_type_info (RVV_BASE_vector),
+ rvv_arg_type_info_end};
+
/* A list of none preds that will be registered for intrinsic functions. */
static CONSTEXPR const predication_type_index none_preds[]
= {PRED_TYPE_none, NUM_PRED_TYPES};
@@ -1455,6 +1481,14 @@ static CONSTEXPR const rvv_op_info iu_shift_vvv_ops
rvv_arg_type_info (RVV_BASE_vector), /* Return type */
shift_vv_args /* Args */};
+/* A static operand information for scalar_type func (vector_type, size_t)
+ * function registration. */
+static CONSTEXPR const rvv_op_info iu_x_s_u_ops
+ = {iu_ops, /* Types */
+ OP_TYPE_vx, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_scalar), /* Return type */
+ v_size_args /* Args */};
+
/* A static operand information for vector_type func (vector_type, size_t)
* function registration. */
static CONSTEXPR const rvv_op_info iu_shift_vvx_ops
@@ -2638,6 +2672,38 @@ static CONSTEXPR const rvv_op_info all_v_vcreate_lmul4_x2_ops
rvv_arg_type_info (RVV_BASE_vlmul_ext_x2), /* Return type */
ext_vcreate_args /* Args */};
+/* A static operand information for vector_type func (const scalar_type *,
+ * size_t) function registration. */
+static CONSTEXPR const rvv_op_info all_v_scalar_const_ptr_size_ops
+ = {all_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_size_args /* Args */};
+
+/* A static operand information for void func (scalar_type *, size_t,
+ * vector_type) function registration. */
+static CONSTEXPR const rvv_op_info all_v_scalar_ptr_size_ops
+ = {all_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_void), /* Return type */
+ scalar_ptr_size_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * index_type) function registration. */
+static CONSTEXPR const rvv_op_info all_v_scalar_const_ptr_index_ops
+ = {all_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_index_args /* Args */};
+
+/* A static operand information for void func (scalar_type *, index_type,
+ * vector_type) function registration. */
+static CONSTEXPR const rvv_op_info all_v_scalar_ptr_index_ops
+ = {all_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_void), /* Return type */
+ scalar_ptr_index_args /* Args */};
+
/* A static operand information for vector_type func (vector_type).
Some ins just supports SEW=32, such as crypto vectol Zvkg extension.
* function registration. */
@@ -2816,6 +2882,10 @@ static function_group_info function_groups[] = {
#define DEF_RVV_FUNCTION(NAME, SHAPE, PREDS, OPS_INFO) \
{#NAME, &bases::NAME, &shapes::SHAPE, PREDS, OPS_INFO, REQUIRED_EXTENSIONS},
#include "riscv-vector-builtins-functions.def"
+#undef DEF_RVV_FUNCTION
+#define DEF_RVV_FUNCTION(NAME, SHAPE, PREDS, OPS_INFO) \
+ {#NAME, &bases::NAME, &shapes::SHAPE, PREDS, OPS_INFO, REQUIRED_EXTENSIONS},
+#include "thead-vector-builtins-functions.def"
};
/* The RVV types, with their built-in
diff --git a/gcc/config/riscv/riscv-vector-builtins.h b/gcc/config/riscv/riscv-vector-builtins.h
index 54c8824ff92..a8ee39a3cb2 100644
--- a/gcc/config/riscv/riscv-vector-builtins.h
+++ b/gcc/config/riscv/riscv-vector-builtins.h
@@ -114,6 +114,7 @@ static const unsigned int CP_WRITE_CSR = 1U << 5;
enum required_ext
{
VECTOR_EXT, /* Vector extension */
+ XTHEADVECTOR_EXT, /* XTheadVector extension */
ZVBB_EXT, /* Cryto vector Zvbb sub-ext */
ZVBB_OR_ZVKB_EXT, /* Cryto vector Zvbb or zvkb sub-ext */
ZVBC_EXT, /* Crypto vector Zvbc sub-ext */
@@ -234,6 +235,8 @@ struct function_group_info
{
case VECTOR_EXT:
return TARGET_VECTOR;
+ case XTHEADVECTOR_EXT:
+ return TARGET_XTHEADVECTOR;
case ZVBB_EXT:
return TARGET_ZVBB;
case ZVBB_OR_ZVKB_EXT:
diff --git a/gcc/config/riscv/t-riscv b/gcc/config/riscv/t-riscv
index 32de6b851c1..38494320d8b 100644
--- a/gcc/config/riscv/t-riscv
+++ b/gcc/config/riscv/t-riscv
@@ -1,6 +1,7 @@
RISCV_BUILTINS_H = $(srcdir)/config/riscv/riscv-vector-builtins.h \
$(srcdir)/config/riscv/riscv-vector-builtins.def \
$(srcdir)/config/riscv/riscv-vector-builtins-functions.def \
+ $(srcdir)/config/riscv/thead-vector-builtins-functions.def \
riscv-vector-type-indexer.gen.def
riscv-builtins.o: $(srcdir)/config/riscv/riscv-builtins.cc $(CONFIG_H) \
diff --git a/gcc/config/riscv/thead-vector-builtins-functions.def b/gcc/config/riscv/thead-vector-builtins-functions.def
new file mode 100644
index 00000000000..fd3ba29bae9
--- /dev/null
+++ b/gcc/config/riscv/thead-vector-builtins-functions.def
@@ -0,0 +1,39 @@
+#ifndef DEF_RVV_FUNCTION
+#define DEF_RVV_FUNCTION(NAME, SHAPE, PREDS, OPS_INFO)
+#endif
+
+#define REQUIRED_EXTENSIONS XTHEADVECTOR_EXT
+DEF_RVV_FUNCTION (vlb, th_loadstore_width, full_preds, all_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (vlh, th_loadstore_width, full_preds, all_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (vlw, th_loadstore_width, full_preds, all_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (vlbu, th_loadstore_width, full_preds, all_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (vlhu, th_loadstore_width, full_preds, all_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (vlwu, th_loadstore_width, full_preds, all_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (vsb, th_loadstore_width, none_m_preds, all_v_scalar_ptr_ops)
+DEF_RVV_FUNCTION (vsh, th_loadstore_width, none_m_preds, all_v_scalar_ptr_ops)
+DEF_RVV_FUNCTION (vsw, th_loadstore_width, none_m_preds, all_v_scalar_ptr_ops)
+DEF_RVV_FUNCTION (vlsb, th_loadstore_width, full_preds, all_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (vlsh, th_loadstore_width, full_preds, all_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (vlsw, th_loadstore_width, full_preds, all_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (vlsbu, th_loadstore_width, full_preds, all_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (vlshu, th_loadstore_width, full_preds, all_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (vlswu, th_loadstore_width, full_preds, all_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (vssb, th_loadstore_width, none_m_preds, all_v_scalar_ptr_size_ops)
+DEF_RVV_FUNCTION (vssh, th_loadstore_width, none_m_preds, all_v_scalar_ptr_size_ops)
+DEF_RVV_FUNCTION (vssw, th_loadstore_width, none_m_preds, all_v_scalar_ptr_size_ops)
+DEF_RVV_FUNCTION (vlxb, th_indexed_loadstore_width, full_preds, all_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (vlxh, th_indexed_loadstore_width, full_preds, all_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (vlxw, th_indexed_loadstore_width, full_preds, all_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (vlxbu, th_indexed_loadstore_width, full_preds, all_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (vlxhu, th_indexed_loadstore_width, full_preds, all_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (vlxwu, th_indexed_loadstore_width, full_preds, all_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (vsxb, th_indexed_loadstore_width, none_m_preds, all_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (vsxh, th_indexed_loadstore_width, none_m_preds, all_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (vsxw, th_indexed_loadstore_width, none_m_preds, all_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (vsuxb, th_indexed_loadstore_width, none_m_preds, all_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (vsuxh, th_indexed_loadstore_width, none_m_preds, all_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (vsuxw, th_indexed_loadstore_width, none_m_preds, all_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (vext_x_v, th_extract, none_preds, iu_x_s_u_ops)
+#undef REQUIRED_EXTENSIONS
+
+#undef DEF_RVV_FUNCTION
diff --git a/gcc/config/riscv/thead-vector.md b/gcc/config/riscv/thead-vector.md
index 696b815252d..0f3700d9269 100644
--- a/gcc/config/riscv/thead-vector.md
+++ b/gcc/config/riscv/thead-vector.md
@@ -1,7 +1,95 @@
(define_c_enum "unspec" [
+ UNSPEC_TH_VLB
+ UNSPEC_TH_VLBU
+ UNSPEC_TH_VLH
+ UNSPEC_TH_VLHU
+ UNSPEC_TH_VLW
+ UNSPEC_TH_VLWU
+
+ UNSPEC_TH_VLSB
+ UNSPEC_TH_VLSBU
+ UNSPEC_TH_VLSH
+ UNSPEC_TH_VLSHU
+ UNSPEC_TH_VLSW
+ UNSPEC_TH_VLSWU
+
+ UNSPEC_TH_VLXB
+ UNSPEC_TH_VLXBU
+ UNSPEC_TH_VLXH
+ UNSPEC_TH_VLXHU
+ UNSPEC_TH_VLXW
+ UNSPEC_TH_VLXWU
+
+ UNSPEC_TH_VSUXB
+ UNSPEC_TH_VSUXH
+ UNSPEC_TH_VSUXW
+
UNSPEC_TH_VWLDST
])
+(define_int_iterator UNSPEC_TH_VLMEM_OP [
+ UNSPEC_TH_VLB UNSPEC_TH_VLBU
+ UNSPEC_TH_VLH UNSPEC_TH_VLHU
+ UNSPEC_TH_VLW UNSPEC_TH_VLWU
+])
+
+(define_int_iterator UNSPEC_TH_VLSMEM_OP [
+ UNSPEC_TH_VLSB UNSPEC_TH_VLSBU
+ UNSPEC_TH_VLSH UNSPEC_TH_VLSHU
+ UNSPEC_TH_VLSW UNSPEC_TH_VLSWU
+])
+
+(define_int_iterator UNSPEC_TH_VLXMEM_OP [
+ UNSPEC_TH_VLXB UNSPEC_TH_VLXBU
+ UNSPEC_TH_VLXH UNSPEC_TH_VLXHU
+ UNSPEC_TH_VLXW UNSPEC_TH_VLXWU
+])
+
+(define_int_attr vlmem_op_attr [
+ (UNSPEC_TH_VLB "b") (UNSPEC_TH_VLBU "bu")
+ (UNSPEC_TH_VLH "h") (UNSPEC_TH_VLHU "hu")
+ (UNSPEC_TH_VLW "w") (UNSPEC_TH_VLWU "wu")
+ (UNSPEC_TH_VLSB "b") (UNSPEC_TH_VLSBU "bu")
+ (UNSPEC_TH_VLSH "h") (UNSPEC_TH_VLSHU "hu")
+ (UNSPEC_TH_VLSW "w") (UNSPEC_TH_VLSWU "wu")
+ (UNSPEC_TH_VLXB "b") (UNSPEC_TH_VLXBU "bu")
+ (UNSPEC_TH_VLXH "h") (UNSPEC_TH_VLXHU "hu")
+ (UNSPEC_TH_VLXW "w") (UNSPEC_TH_VLXWU "wu")
+ (UNSPEC_TH_VSUXB "b")
+ (UNSPEC_TH_VSUXH "h")
+ (UNSPEC_TH_VSUXW "w")
+])
+
+(define_int_attr vlmem_order_attr [
+ (UNSPEC_TH_VLXB "")
+ (UNSPEC_TH_VLXH "")
+ (UNSPEC_TH_VLXW "")
+ (UNSPEC_TH_VSUXB "u")
+ (UNSPEC_TH_VSUXH "u")
+ (UNSPEC_TH_VSUXW "u")
+])
+
+(define_int_iterator UNSPEC_TH_VSMEM_OP [
+ UNSPEC_TH_VLB
+ UNSPEC_TH_VLH
+ UNSPEC_TH_VLW
+])
+
+(define_int_iterator UNSPEC_TH_VSSMEM_OP [
+ UNSPEC_TH_VLSB
+ UNSPEC_TH_VLSH
+ UNSPEC_TH_VLSW
+])
+
+(define_int_iterator UNSPEC_TH_VSXMEM_OP [
+ UNSPEC_TH_VLXB
+ UNSPEC_TH_VLXH
+ UNSPEC_TH_VLXW
+ UNSPEC_TH_VSUXB
+ UNSPEC_TH_VSUXH
+ UNSPEC_TH_VSUXW
+])
+
(define_mode_iterator V_VLS_VT [V VLS VT])
(define_mode_iterator V_VB_VLS_VT [V VB VLS VT])
@@ -100,3 +188,168 @@
}
[(set_attr "type" "vldm,vstm,vmalu,vmalu,vmalu")
(set_attr "mode" "<MODE>")])
+
+(define_expand "@pred_mov_width<vlmem_op_attr><mode>"
+ [(set (match_operand:V_VLS 0 "nonimmediate_operand")
+ (if_then_else:V_VLS
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand")
+ (match_operand 4 "vector_length_operand")
+ (match_operand 5 "const_int_operand")
+ (match_operand 6 "const_int_operand")
+ (match_operand 7 "const_int_operand")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLMEM_OP)
+ (match_operand:V_VLS 3 "vector_move_operand")
+ (match_operand:V_VLS 2 "vector_merge_operand")))]
+ "TARGET_XTHEADVECTOR"
+ {})
+
+(define_insn_and_split "*pred_mov_width<vlmem_op_attr><mode>"
+ [(set (match_operand:V_VLS 0 "nonimmediate_operand" "=vr, vr, vd, m, vr, vr")
+ (if_then_else:V_VLS
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1, Wc1, vm, vmWc1, Wc1, Wc1")
+ (match_operand 4 "vector_length_operand" " rK, rK, rK, rK, rK, rK")
+ (match_operand 5 "const_int_operand" " i, i, i, i, i, i")
+ (match_operand 6 "const_int_operand" " i, i, i, i, i, i")
+ (match_operand 7 "const_int_operand" " i, i, i, i, i, i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLMEM_OP)
+ (match_operand:V_VLS 3 "reg_or_mem_operand" " m, m, m, vr, vr, vr")
+ (match_operand:V_VLS 2 "vector_merge_operand" " 0, vu, vu, vu, vu, 0")))]
+ "(TARGET_XTHEADVECTOR
+ && (register_operand (operands[0], <MODE>mode)
+ || register_operand (operands[3], <MODE>mode)))"
+ "@
+ vl<vlmem_op_attr>.v\t%0,%3%p1
+ vl<vlmem_op_attr>.v\t%0,%3
+ vl<vlmem_op_attr>.v\t%0,%3,%1.t
+ vs<vlmem_op_attr>.v\t%3,%0%p1
+ vmv.v.v\t%0,%3
+ vmv.v.v\t%0,%3"
+ "&& register_operand (operands[0], <MODE>mode)
+ && register_operand (operands[3], <MODE>mode)
+ && satisfies_constraint_vu (operands[2])
+ && INTVAL (operands[7]) == riscv_vector::VLMAX"
+ [(set (match_dup 0) (match_dup 3))]
+ ""
+ [(set_attr "type" "vlde,vlde,vlde,vste,vimov,vimov")
+ (set_attr "mode" "<MODE>")])
+
+(define_insn "@pred_store_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "memory_operand" "+m")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1")
+ (match_operand 3 "vector_length_operand" " rK")
+ (match_operand 4 "const_int_operand" " i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VSMEM_OP)
+ (match_operand:VI 2 "register_operand" " vr")
+ (match_dup 0)))]
+ "TARGET_XTHEADVECTOR"
+ "vs<vlmem_op_attr>.v\t%2,%0%p1"
+ [(set_attr "type" "vste")
+ (set_attr "mode" "<MODE>")
+ (set (attr "avl_type_idx") (const_int 4))
+ (set_attr "vl_op_idx" "3")])
+
+(define_insn "@pred_strided_load_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "register_operand" "=vr, vr, vd")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1, Wc1, vm")
+ (match_operand 5 "vector_length_operand" " rK, rK, rK")
+ (match_operand 6 "const_int_operand" " i, i, i")
+ (match_operand 7 "const_int_operand" " i, i, i")
+ (match_operand 8 "const_int_operand" " i, i, i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLSMEM_OP)
+ (unspec:VI
+ [(match_operand:VI 3 "memory_operand" " m, m, m")
+ (match_operand 4 "pmode_reg_or_0_operand" " rJ, rJ, rJ")] UNSPEC_TH_VLSMEM_OP)
+ (match_operand:VI 2 "vector_merge_operand" " 0, vu, vu")))]
+ "TARGET_XTHEADVECTOR"
+ "vls<vlmem_op_attr>.v\t%0,%3,%z4%p1"
+ [(set_attr "type" "vlds")
+ (set_attr "mode" "<MODE>")])
+
+(define_insn "@pred_strided_store_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "memory_operand" "+m")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1")
+ (match_operand 4 "vector_length_operand" " rK")
+ (match_operand 5 "const_int_operand" " i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VSSMEM_OP)
+ (unspec:VI
+ [(match_operand 2 "pmode_reg_or_0_operand" " rJ")
+ (match_operand:VI 3 "register_operand" " vr")] UNSPEC_TH_VSSMEM_OP)
+ (match_dup 0)))]
+ "TARGET_XTHEADVECTOR"
+ "vss<vlmem_op_attr>.v\t%3,%0,%z2%p1"
+ [(set_attr "type" "vsts")
+ (set_attr "mode" "<MODE>")
+ (set (attr "avl_type_idx") (const_int 5))])
+
+(define_insn "@pred_indexed_load_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "register_operand" "=vd, vr,vd, vr")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" " vm,Wc1,vm,Wc1")
+ (match_operand 5 "vector_length_operand" " rK, rK,rK, rK")
+ (match_operand 6 "const_int_operand" " i, i, i, i")
+ (match_operand 7 "const_int_operand" " i, i, i, i")
+ (match_operand 8 "const_int_operand" " i, i, i, i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLXMEM_OP)
+ (unspec:VI
+ [(match_operand 3 "pmode_reg_or_0_operand" " rJ, rJ,rJ, rJ")
+ (mem:BLK (scratch))
+ (match_operand:VI 4 "register_operand" " vr, vr,vr, vr")] UNSPEC_TH_VLXMEM_OP)
+ (match_operand:VI 2 "vector_merge_operand" " vu, vu, 0, 0")))]
+ "TARGET_XTHEADVECTOR"
+ "vlx<vlmem_op_attr>.v\t%0,(%z3),%4%p1"
+ [(set_attr "type" "vldux")
+ (set_attr "mode" "<MODE>")])
+
+(define_insn "@pred_indexed_<vlmem_order_attr>store_width<vlmem_op_attr><mode>"
+ [(set (mem:BLK (scratch))
+ (unspec:BLK
+ [(unspec:<VM>
+ [(match_operand:<VM> 0 "vector_mask_operand" "vmWc1")
+ (match_operand 4 "vector_length_operand" " rK")
+ (match_operand 5 "const_int_operand" " i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VSXMEM_OP)
+ (match_operand 1 "pmode_reg_or_0_operand" " rJ")
+ (match_operand:VI 2 "register_operand" " vr")
+ (match_operand:VI 3 "register_operand" " vr")] UNSPEC_TH_VSXMEM_OP))]
+ "TARGET_XTHEADVECTOR"
+ "vs<vlmem_order_attr>x<vlmem_op_attr>.v\t%3,(%z1),%2%p0"
+ [(set_attr "type" "vstux")
+ (set_attr "mode" "<MODE>")])
+
+(define_expand "@pred_th_extract<mode>"
+ [(set (match_operand:<VEL> 0 "register_operand")
+ (unspec:<VEL>
+ [(vec_select:<VEL>
+ (match_operand:V_VLSI 1 "register_operand")
+ (parallel [(match_operand:DI 2 "register_operand" "r")]))
+ (reg:SI VTYPE_REGNUM)] UNSPEC_VPREDICATE))]
+ "TARGET_XTHEADVECTOR"
+{})
+
+(define_insn "*pred_th_extract<mode>"
+ [(set (match_operand:<VEL> 0 "register_operand" "=r")
+ (unspec:<VEL>
+ [(vec_select:<VEL>
+ (match_operand:V_VLSI 1 "register_operand" "vr")
+ (parallel [(match_operand:DI 2 "register_operand" "r")]))
+ (reg:SI VTYPE_REGNUM)] UNSPEC_VPREDICATE))]
+ "TARGET_XTHEADVECTOR"
+ "vext.x.v\t%0,%1,%2"
+ [(set_attr "type" "vimovvx")
+ (set_attr "mode" "<MODE>")])
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c
new file mode 100644
index 00000000000..3c12c124597
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_th_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+
+** th.vadd\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+
+** th.vsb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out)
+{
+ vint32m1_t v = __riscv_th_vlb_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlb_v_i32m1_tu (v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vv_i32m1 (v2, v2, 4);
+ vint32m1_t v4 = __riscv_vadd_vv_i32m1_tu (v3, v2, v2, 4);
+ __riscv_th_vsb_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlb.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+
+** th.vadd\.vv\tv[1-9][0-9]?,\s*v[0-9]+,\s*v[0-9]+,\s*v0.t
+** th.vsb.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlb_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlb_v_i32m1_m (mask, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vv_i32m1 (v2, v2, 4);
+ vint32m1_t v4 = __riscv_vadd_vv_i32m1_m (mask, v3, v3, 4);
+ __riscv_th_vsb_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlb.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+
+** th.vadd\.vv\tv[1-9][0-9]?,\s*v[0-9]+,\s*v[0-9]+,\s*v0.t
+** th.vsb.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlb_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlb_v_i32m1_tumu (mask, v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vv_i32m1 (v2, v2, 4);
+ vint32m1_t v4 = __riscv_vadd_vv_i32m1_tumu (mask, v3, v2, v2, 4);
+ __riscv_th_vsb_v_i32m1 (out, v4, 4);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c
new file mode 100644
index 00000000000..30bef369375
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_th_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlbu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlbu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vsb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, uint32_t x)
+{
+ vuint32m1_t v = __riscv_th_vlbu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlbu_v_u32m1_tu (v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tu (v3, v2, -16, 4);
+ __riscv_th_vsb_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlbu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsb.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlbu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlbu_v_u32m1_m (mask, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_m (mask, v3, -16, 4);
+ __riscv_th_vsb_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlbu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlbu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsb.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlbu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlbu_v_u32m1_tumu (mask, v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tumu (mask, v3, v2, -16, 4);
+ __riscv_th_vsb_v_u32m1 (out, v4, 4);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c
new file mode 100644
index 00000000000..3c8b5ccc16b
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_th_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vsh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, int32_t x)
+{
+ vint32m1_t v = __riscv_th_vlh_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlh_v_i32m1_tu (v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, -16, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_tu (v3, v2, -16, 4);
+ __riscv_th_vsh_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlh.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsh.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, int32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlh_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlh_v_i32m1_m (mask, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, -16, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_m (mask, v3, -16, 4);
+ __riscv_th_vsh_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlh.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsh.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, int32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlh_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlh_v_i32m1_tumu (mask, v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, -16, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_tumu (mask, v3, v2, -16, 4);
+ __riscv_th_vsh_v_i32m1 (out, v4, 4);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c
new file mode 100644
index 00000000000..b7c00404f18
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_th_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlhu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlhu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vsh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, uint32_t x)
+{
+ vuint32m1_t v = __riscv_th_vlhu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlhu_v_u32m1_tu (v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tu (v3, v2, -16, 4);
+ __riscv_th_vsh_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlhu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsh.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlhu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlhu_v_u32m1_m (mask, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_m (mask, v3, -16, 4);
+ __riscv_th_vsh_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlhu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlhu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsh.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlhu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlhu_v_u32m1_tumu (mask, v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tumu (mask, v3, v2, -16, 4);
+ __riscv_th_vsh_v_u32m1 (out, v4, 4);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c
new file mode 100644
index 00000000000..17a53012acf
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_th_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vx\tv[0-9]+,\s*v[0-9]+,\s*[a-x0-9]+
+** th.vadd\.vx\tv[0-9]+,\s*v[0-9]+,\s*[a-x0-9]+
+** th.vsw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, int32_t x)
+{
+ vint32m1_t v = __riscv_th_vlw_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlw_v_i32m1_tu (v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, x, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_tu (v3, v2, x, 4);
+ __riscv_th_vsw_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlw.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vx\tv[0-9]+,\s*v[0-9]+,\s*[a-x0-9]+
+** th.vadd\.vx\tv[1-9][0-9]?,\s*v[0-9]+,\s*[a-x0-9]+,\s*v0.t
+** th.vsw.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, int32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlw_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlw_v_i32m1_m (mask, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, x, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_m (mask, v3, x, 4);
+ __riscv_th_vsw_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlw.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vx\tv[0-9]+,\s*v[0-9]+,\s*[a-x0-9]+
+** th.vadd\.vx\tv[1-9][0-9]?,\s*v[0-9]+,\s*[a-x0-9]+,\s*v0.t
+** th.vsw.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, int32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlw_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlw_v_i32m1_tumu (mask, v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, x, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_tumu (mask, v3, v2, x, 4);
+ __riscv_th_vsw_v_i32m1 (out, v4, 4);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c
new file mode 100644
index 00000000000..b187cfc852b
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_th_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlwu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlwu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vsw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, uint32_t x)
+{
+ vuint32m1_t v = __riscv_th_vlwu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlwu_v_u32m1_tu (v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tu (v3, v2, -16, 4);
+ __riscv_th_vsw_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlwu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsw.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlwu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlwu_v_u32m1_m (mask, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_m (mask, v3, -16, 4);
+ __riscv_th_vsw_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlwu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlwu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsw.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlwu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlwu_v_u32m1_tumu (mask, v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tumu (mask, v3, v2, -16, 4);
+ __riscv_th_vsw_v_u32m1 (out, v4, 4);
+}
--
2.17.1
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re：Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
2024-01-11 9:24 Re：[PATCH " juzhe.zhong
@ 2024-01-11 9:29 ` joshua
0 siblings, 0 replies; 17+ messages in thread
From: joshua @ 2024-01-11 9:29 UTC (permalink / raw)
To: juzhe.zhong, gcc-patches
Cc: Jim Wilson, palmer, andrew, philipp.tomsich, jeffreyalaw,
christoph.muellner, jinma, cooper.qu
The sext/zext issue is not related to xtheadvector-special patterns.
I added !TARGET_XTHEADVECTOR to sext/zext patterns in
"RISC-V: Handle differences between XTheadvector and Vector"
That is caused by the vwmul pattern, but I cannot reproduce it right now.
------------------------------------------------------------------
发件人：juzhe.zhong@rivai.ai <juzhe.zhong@rivai.ai>
发送时间：2024年1月11日(星期四) 17:24
收件人："cooper.joshua"<cooper.joshua@linux.alibaba.com>; "gcc-patches"<gcc-patches@gcc.gnu.org>
抄 送：Jim Wilson<jim.wilson.gcc@gmail.com>; palmer<palmer@dabbelt.com>; andrew<andrew@sifive.com>; "philipp.tomsich"<philipp.tomsich@vrull.eu>; jeffreyalaw<jeffreyalaw@gmail.com>; "christoph.muellner"<christoph.muellner@vrull.eu>; jinma<jinma@linux.alibaba.com>; "cooper.qu"<cooper.qu@linux.alibaba.com>
主 题：Re: Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
I don't see any patterns are possible CSE into sext/zext patterns:
+(define_expand "@pred_mov_width<vlmem_op_attr><mode>"
+ [(set (match_operand:V_VLS 0 "nonimmediate_operand")
+ (if_then_else:V_VLS
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand")
+ (match_operand 4 "vector_length_operand")
+ (match_operand 5 "const_int_operand")
+ (match_operand 6 "const_int_operand")
+ (match_operand 7 "const_int_operand")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLMEM_OP)
+ (match_operand:V_VLS 3 "vector_move_operand")
+ (match_operand:V_VLS 2 "vector_merge_operand")))]
+ "TARGET_XTHEADVECTOR"
+ {})
+
+(define_insn_and_split "*pred_mov_width<vlmem_op_attr><mode>"
+ [(set (match_operand:V_VLS 0 "nonimmediate_operand" "=vr, vr, vd, m, vr, vr")
+ (if_then_else:V_VLS
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1, Wc1, vm, vmWc1, Wc1, Wc1")
+ (match_operand 4 "vector_length_operand" " rK, rK, rK, rK, rK, rK")
+ (match_operand 5 "const_int_operand" " i, i, i, i, i, i")
+ (match_operand 6 "const_int_operand" " i, i, i, i, i, i")
+ (match_operand 7 "const_int_operand" " i, i, i, i, i, i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLMEM_OP)
+ (match_operand:V_VLS 3 "reg_or_mem_operand" " m, m, m, vr, vr, vr")
+ (match_operand:V_VLS 2 "vector_merge_operand" " 0, vu, vu, vu, vu, 0")))]
+ "(TARGET_XTHEADVECTOR
+ && (register_operand (operands[0], <MODE>mode)
+ || register_operand (operands[3], <MODE>mode)))"
+ "@
+ vl<vlmem_op_attr>.v\t%0,%3%p1
+ vl<vlmem_op_attr>.v\t%0,%3
+ vl<vlmem_op_attr>.v\t%0,%3,%1.t
+ vs<vlmem_op_attr>.v\t%3,%0%p1
+ vmv.v.v\t%0,%3
+ vmv.v.v\t%0,%3"
+ "&& register_operand (operands[0], <MODE>mode)
+ && register_operand (operands[3], <MODE>mode)
+ && satisfies_constraint_vu (operands[2])
+ && INTVAL (operands[7]) == riscv_vector::VLMAX"
+ [(set (match_dup 0) (match_dup 3))]
+ ""
+ [(set_attr "type" "vlde,vlde,vlde,vste,vimov,vimov")
+ (set_attr "mode" "<MODE>")])
+
+(define_insn "@pred_store_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "memory_operand" "+m")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1")
+ (match_operand 3 "vector_length_operand" " rK")
+ (match_operand 4 "const_int_operand" " i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VSMEM_OP)
+ (match_operand:VI 2 "register_operand" " vr")
+ (match_dup 0)))]
+ "TARGET_XTHEADVECTOR"
+ "vs<vlmem_op_attr>.v\t%2,%0%p1"
+ [(set_attr "type" "vste")
+ (set_attr "mode" "<MODE>")
+ (set (attr "avl_type_idx") (const_int 4))
+ (set_attr "vl_op_idx" "3")])
+
+(define_insn "@pred_strided_load_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "register_operand" "=vr, vr, vd")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1, Wc1, vm")
+ (match_operand 5 "vector_length_operand" " rK, rK, rK")
+ (match_operand 6 "const_int_operand" " i, i, i")
+ (match_operand 7 "const_int_operand" " i, i, i")
+ (match_operand 8 "const_int_operand" " i, i, i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLSMEM_OP)
+ (unspec:VI
+ [(match_operand:VI 3 "memory_operand" " m, m, m")
+ (match_operand 4 "pmode_reg_or_0_operand" " rJ, rJ, rJ")] UNSPEC_TH_VLSMEM_OP)
+ (match_operand:VI 2 "vector_merge_operand" " 0, vu, vu")))]
+ "TARGET_XTHEADVECTOR"
+ "vls<vlmem_op_attr>.v\t%0,%3,%z4%p1"
+ [(set_attr "type" "vlds")
+ (set_attr "mode" "<MODE>")])
+
+(define_insn "@pred_strided_store_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "memory_operand" "+m")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1")
+ (match_operand 4 "vector_length_operand" " rK")
+ (match_operand 5 "const_int_operand" " i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VSSMEM_OP)
+ (unspec:VI
+ [(match_operand 2 "pmode_reg_or_0_operand" " rJ")
+ (match_operand:VI 3 "register_operand" " vr")] UNSPEC_TH_VSSMEM_OP)
+ (match_dup 0)))]
+ "TARGET_XTHEADVECTOR"
+ "vss<vlmem_op_attr>.v\t%3,%0,%z2%p1"
+ [(set_attr "type" "vsts")
+ (set_attr "mode" "<MODE>")
+ (set (attr "avl_type_idx") (const_int 5))])
+
+(define_insn "@pred_indexed_load_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "register_operand" "=vd, vr,vd, vr")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" " vm,Wc1,vm,Wc1")
+ (match_operand 5 "vector_length_operand" " rK, rK,rK, rK")
+ (match_operand 6 "const_int_operand" " i, i, i, i")
+ (match_operand 7 "const_int_operand" " i, i, i, i")
+ (match_operand 8 "const_int_operand" " i, i, i, i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLXMEM_OP)
+ (unspec:VI
+ [(match_operand 3 "pmode_reg_or_0_operand" " rJ, rJ,rJ, rJ")
+ (mem:BLK (scratch))
+ (match_operand:VI 4 "register_operand" " vr, vr,vr, vr")] UNSPEC_TH_VLXMEM_OP)
+ (match_operand:VI 2 "vector_merge_operand" " vu, vu, 0, 0")))]
+ "TARGET_XTHEADVECTOR"
+ "vlx<vlmem_op_attr>.v\t%0,(%z3),%4%p1"
+ [(set_attr "type" "vldux")
+ (set_attr "mode" "<MODE>")])
+
+(define_insn "@pred_indexed_<vlmem_order_attr>store_width<vlmem_op_attr><mode>"
+ [(set (mem:BLK (scratch))
+ (unspec:BLK
+ [(unspec:<VM>
+ [(match_operand:<VM> 0 "vector_mask_operand" "vmWc1")
+ (match_operand 4 "vector_length_operand" " rK")
+ (match_operand 5 "const_int_operand" " i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VSXMEM_OP)
+ (match_operand 1 "pmode_reg_or_0_operand" " rJ")
+ (match_operand:VI 2 "register_operand" " vr")
+ (match_operand:VI 3 "register_operand" " vr")] UNSPEC_TH_VSXMEM_OP))]
+ "TARGET_XTHEADVECTOR"
+ "vs<vlmem_order_attr>x<vlmem_op_attr>.v\t%3,(%z1),%2%p0"
+ [(set_attr "type" "vstux")
+ (set_attr "mode" "<MODE>")])
+
+(define_expand "@pred_th_extract<mode>"
+ [(set (match_operand:<VEL> 0 "register_operand")
+ (unspec:<VEL>
+ [(vec_select:<VEL>
+ (match_operand:V_VLSI 1 "register_operand")
+ (parallel [(match_operand:DI 2 "register_operand" "r")]))
+ (reg:SI VTYPE_REGNUM)] UNSPEC_VPREDICATE))]
+ "TARGET_XTHEADVECTOR"
+{})
+
+(define_insn "*pred_th_extract<mode>"
+ [(set (match_operand:<VEL> 0 "register_operand" "=r")
+ (unspec:<VEL>
+ [(vec_select:<VEL>
+ (match_operand:V_VLSI 1 "register_operand" "vr")
+ (parallel [(match_operand:DI 2 "register_operand" "r")]))
+ (reg:SI VTYPE_REGNUM)] UNSPEC_VPREDICATE))]
+ "TARGET_XTHEADVECTOR"
+ "vext.x.v\t%0,%1,%2"
+ [(set_attr "type" "vimovvx")
+ (set_attr "mode" "<MODE>")])
juzhe.zhong@rivai.ai
发件人： joshua
发送时间： 2024-01-11 17:21
收件人： juzhe.zhong@rivai.ai; gcc-patches
抄送： Jim Wilson; palmer; andrew; philipp.tomsich; jeffreyalaw; christoph.muellner; jinma; cooper.qu
主题： Re：Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
"I didn't see theadvector-specific extension patterns. Could you show me?"
They are all in the file thead-vector.md.
For the sext/zext issue, perhaps I need some time to reproduce that optimization,
but I can clearly remember it is related to vwmul.
------------------------------------------------------------------
发件人：juzhe.zhong@rivai.ai <juzhe.zhong@rivai.ai>
发送时间：2024年1月11日(星期四) 17:17
收件人："cooper.joshua"<cooper.joshua@linux.alibaba.com>; "gcc-patches"<gcc-patches@gcc.gnu.org>
抄 送：Jim Wilson<jim.wilson.gcc@gmail.com>; palmer<palmer@dabbelt.com>; andrew<andrew@sifive.com>; "philipp.tomsich"<philipp.tomsich@vrull.eu>; jeffreyalaw<jeffreyalaw@gmail.com>; "christoph.muellner"<christoph.muellner@vrull.eu>; jinma<jinma@linux.alibaba.com>; "cooper.qu"<cooper.qu@linux.alibaba.com>
主 题：Re: Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
You mean which pattern optimized sext/vzext pattern?
I didn't see theadvector-specific extension patterns. Could you show me?
juzhe.zhong@rivai.ai
发件人： joshua
发送时间： 2024-01-11 17:14
收件人： juzhe.zhong@rivai.ai; gcc-patches
抄送： Jim Wilson; palmer; andrew; philipp.tomsich; jeffreyalaw; christoph.muellner; jinma; cooper.qu
主题： Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
To be specific, in CSE pass, the initial pattern will be optimized into the sext/zext pattern.
------------------------------------------------------------------
发件人：joshua <cooper.joshua@linux.alibaba.com>
发送时间：2024年1月11日(星期四) 17:11
收件人："juzhe.zhong@rivai.ai"<juzhe.zhong@rivai.ai>; "gcc-patches"<gcc-patches@gcc.gnu.org>
抄 送：Jim Wilson<jim.wilson.gcc@gmail.com>; palmer<palmer@dabbelt.com>; andrew<andrew@sifive.com>; "philipp.tomsich"<philipp.tomsich@vrull.eu>; jeffreyalaw<jeffreyalaw@gmail.com>; "christoph.muellner"<christoph.muellner@vrull.eu>; jinma<jinma@linux.alibaba.com>; "cooper.qu"<cooper.qu@linux.alibaba.com>
主 题：Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
sext/zext will be generated in O2 even without corresponding intrinsics.
------------------------------------------------------------------
发件人：juzhe.zhong@rivai.ai <juzhe.zhong@rivai.ai>
发送时间：2024年1月11日(星期四) 17:07
收件人："cooper.joshua"<cooper.joshua@linux.alibaba.com>; "gcc-patches"<gcc-patches@gcc.gnu.org>
抄 送：Jim Wilson<jim.wilson.gcc@gmail.com>; palmer<palmer@dabbelt.com>; andrew<andrew@sifive.com>; "philipp.tomsich"<philipp.tomsich@vrull.eu>; jeffreyalaw<jeffreyalaw@gmail.com>; "christoph.muellner"<christoph.muellner@vrull.eu>; "cooper.joshua"<cooper.joshua@linux.alibaba.com>; jinma<jinma@linux.alibaba.com>; "cooper.qu"<cooper.qu@linux.alibaba.com>
主 题：Re: [PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
enum required_ext
{
VECTOR_EXT, /* Vector extension */
+ XTHEADVECTOR_EXT, /* XTheadVector extension */
ZVBB_EXT, /* Cryto vector Zvbb sub-ext */
ZVBB_OR_ZVKB_EXT, /* Cryto vector Zvbb or zvkb sub-ext */
ZVBC_EXT, /* Crypto vector Zvbc sub-ext */
Add theadvector to the end of the enum.
+ case XTHEADVECTOR_EXT:
+ return TARGET_XTHEADVECTOR;
Same.
+ "&& register_operand (operands[0], <MODE>mode)
+ && register_operand (operands[3], <MODE>mode)
+ && satisfies_constraint_vu (operands[2])
+ && INTVAL (operands[7]) == riscv_vector::VLMAX"
You can use whole_reg_to_reg_move_p
Btw, I review again : RISC-V: Handle differences between XTheadvector and Vector
(any_extend:VWEXTI (match_operand:<V_DOUBLE_TRUNC> 3 "register_operand" "W21,W21,W21,W21,W42,W42,W42,W42,W84,W84,W84,W84, vr, vr")) (match_operand:VWEXTI 2 "vector_merge_operand" " vu, vu, 0, 0, vu, vu, 0, 0, vu, vu, 0, 0, vu, 0")))]- "TARGET_VECTOR"+ "TARGET_VECTOR && !TARGET_XTHEADVECTOR" "v<sz>ext.vf2\t%0,%3%p1" [(set_attr "type" "vext") (set_attr "mode" "<MODE>")@@ -3713,7 +3744,7 @@ (any_extend:VQEXTI (match_operand:<V_QUAD_TRUNC> 3 "register_operand" "W43,W43,W43,W43,W86,W86,W86,W86, vr, vr")) (match_operand:VQEXTI 2 "vector_merge_operand" " vu, vu, 0, 0, vu, vu, 0, 0, vu, 0")))]- "TARGET_VECTOR"+ "TARGET_VECTOR && !TARGET_XTHEADVECTOR" "v<sz>ext.vf4\t%0,%3%p1" [(set_attr "type" "vext") (set_attr "mode" "<MODE>")@@ -3734,7 +3765,7 @@ (any_extend:VOEXTI (match_operand:<V_OCT_TRUNC> 3 "register_operand" "W87,W87,W87,W87, vr, vr")) (match_operand:VOEXTI 2 "vector_merge_operand" " vu, vu, 0, 0, vu, 0")))]- "TARGET_VECTOR"+ "TARGET_VECTOR && !TARGET_XTHEADVECTOR" "v<sz>ext.vf8\t%0,%3%p1" [(set_attr "type" "vext") (set_attr "mode" "<MODE>")
Why do you add these !TARGERT_XTHEADVECRTOR ?
juzhe.zhong@rivai.ai
From: Jun Sha (Joshua)
Date: 2024-01-11 16:46
To: gcc-patches
CC: jim.wilson.gcc; palmer; andrew; philipp.tomsich; jeffreyalaw; christoph.muellner; juzhe.zhong; Jun Sha (Joshua); Jin Ma; Xianmiao Qu
Subject: [PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
This patch only involves the generation of xtheadvector
special load/store instructions and vext instructions.
gcc/ChangeLog:
* config/riscv/riscv-vector-builtins-bases.cc
(class th_loadstore_width): Define new builtin bases.
(class th_extract): Define new builtin bases.
(BASE): Define new builtin bases.
* config/riscv/riscv-vector-builtins-bases.h:
Define new builtin class.
* config/riscv/riscv-vector-builtins-shapes.cc
(struct th_loadstore_width_def): Define new builtin shapes.
(struct th_indexed_loadstore_width_def):
Define new builtin shapes.
(struct th_extract_def): Define new builtin shapes.
(SHAPE): Define new builtin shapes.
* config/riscv/riscv-vector-builtins-shapes.h:
Define new builtin shapes.
* config/riscv/riscv-vector-builtins.cc (DEF_RVV_FUNCTION):
* config/riscv/riscv-vector-builtins.h (enum required_ext):
(struct function_group_info):
* config/riscv/t-riscv: Add thead-vector-builtins-functions.def
* config/riscv/thead-vector.md
(@pred_mov_width<vlmem_op_attr><mode>): Add new patterns.
(*pred_mov_width<vlmem_op_attr><mode>): Likewise.
(@pred_store_width<vlmem_op_attr><mode>): Likewise.
(@pred_strided_load_width<vlmem_op_attr><mode>): Likewise.
(@pred_strided_store_width<vlmem_op_attr><mode>): Likewise.
(@pred_indexed_load_width<vlmem_op_attr><mode>): Likewise.
(@pred_indexed_<vlmem_order_attr>store_width<vlmem_op_attr><mode>):
(@pred_th_extract<mode>): Likewise.
(*pred_th_extract<mode>): Likewise.
* config/riscv/thead-vector-builtins-functions.def: New file.
gcc/testsuite/ChangeLog:
* gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c: New test.
Co-authored-by: Jin Ma <jinma@linux.alibaba.com>
Co-authored-by: Xianmiao Qu <cooper.qu@linux.alibaba.com>
Co-authored-by: Christoph Müllner <christoph.muellner@vrull.eu>
---
.../riscv/riscv-vector-builtins-bases.cc | 139 ++++++++++
.../riscv/riscv-vector-builtins-bases.h | 31 +++
.../riscv/riscv-vector-builtins-shapes.cc | 160 +++++++++++
.../riscv/riscv-vector-builtins-shapes.h | 3 +
gcc/config/riscv/riscv-vector-builtins.cc | 70 +++++
gcc/config/riscv/riscv-vector-builtins.h | 3 +
gcc/config/riscv/t-riscv | 1 +
.../riscv/thead-vector-builtins-functions.def | 39 +++
gcc/config/riscv/thead-vector.md | 253 ++++++++++++++++++
.../riscv/rvv/xtheadvector/vlb-vsb.c | 68 +++++
.../riscv/rvv/xtheadvector/vlbu-vsb.c | 68 +++++
.../riscv/rvv/xtheadvector/vlh-vsh.c | 68 +++++
.../riscv/rvv/xtheadvector/vlhu-vsh.c | 68 +++++
.../riscv/rvv/xtheadvector/vlw-vsw.c | 68 +++++
.../riscv/rvv/xtheadvector/vlwu-vsw.c | 68 +++++
15 files changed, 1107 insertions(+)
create mode 100644 gcc/config/riscv/thead-vector-builtins-functions.def
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c
diff --git a/gcc/config/riscv/riscv-vector-builtins-bases.cc b/gcc/config/riscv/riscv-vector-builtins-bases.cc
index 46f1a1da33e..3eba7943757 100644
--- a/gcc/config/riscv/riscv-vector-builtins-bases.cc
+++ b/gcc/config/riscv/riscv-vector-builtins-bases.cc
@@ -2125,6 +2125,83 @@ public:
}
};
+/* Implements
+ * th.vl(b/h/w)[u].v/th.vs(b/h/w)[u].v/th.vls(b/h/w)[u].v/th.vss(b/h/w)[u].v/
+ * th.vlx(b/h/w)[u].v/th.vs[u]x(b/h/w).v
+ * codegen. */
+template<bool STORE_P, lst_type LST_TYPE, int UNSPEC>
+class th_loadstore_width : public function_base
+{
+public:
+ bool apply_tail_policy_p () const override { return !STORE_P; }
+ bool apply_mask_policy_p () const override { return !STORE_P; }
+
+ unsigned int call_properties (const function_instance &) const override
+ {
+ if (STORE_P)
+ return CP_WRITE_MEMORY;
+ else
+ return CP_READ_MEMORY;
+ }
+
+ bool can_be_overloaded_p (enum predication_type_index pred) const override
+ {
+ if (STORE_P || LST_TYPE == LST_INDEXED)
+ return true;
+ return pred != PRED_TYPE_none;
+ }
+
+ rtx expand (function_expander &e) const override
+ {
+ gcc_assert (TARGET_XTHEADVECTOR);
+ if (LST_TYPE == LST_INDEXED)
+ {
+ if (STORE_P)
+ return e.use_exact_insn (
+ code_for_pred_indexed_store_width (UNSPEC, UNSPEC,
+ e.vector_mode ()));
+ else
+ return e.use_exact_insn (
+ code_for_pred_indexed_load_width (UNSPEC, e.vector_mode ()));
+ }
+ else if (LST_TYPE == LST_STRIDED)
+ {
+ if (STORE_P)
+ return e.use_contiguous_store_insn (
+ code_for_pred_strided_store_width (UNSPEC, e.vector_mode ()));
+ else
+ return e.use_contiguous_load_insn (
+ code_for_pred_strided_load_width (UNSPEC, e.vector_mode ()));
+ }
+ else
+ {
+ if (STORE_P)
+ return e.use_contiguous_store_insn (
+ code_for_pred_store_width (UNSPEC, e.vector_mode ()));
+ else
+ return e.use_contiguous_load_insn (
+ code_for_pred_mov_width (UNSPEC, e.vector_mode ()));
+ }
+ }
+};
+
+/* Implements vext.x.v. */
+class th_extract : public function_base
+{
+public:
+ bool apply_vl_p () const override { return false; }
+ bool apply_tail_policy_p () const override { return false; }
+ bool apply_mask_policy_p () const override { return false; }
+ bool use_mask_predication_p () const override { return false; }
+ bool has_merge_operand_p () const override { return false; }
+
+ rtx expand (function_expander &e) const override
+ {
+ gcc_assert (TARGET_XTHEADVECTOR);
+ return e.use_exact_insn (code_for_pred_th_extract (e.vector_mode ()));
+ }
+};
+
/* Below implements are vector crypto */
/* Implements vandn.[vv,vx] */
class vandn : public function_base
@@ -2587,6 +2664,37 @@ static CONSTEXPR const seg_indexed_load<UNSPEC_ORDERED> vloxseg_obj;
static CONSTEXPR const seg_indexed_store<UNSPEC_UNORDERED> vsuxseg_obj;
static CONSTEXPR const seg_indexed_store<UNSPEC_ORDERED> vsoxseg_obj;
static CONSTEXPR const vlsegff vlsegff_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLB> vlb_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLBU> vlbu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLH> vlh_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLHU> vlhu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLW> vlw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLWU> vlwu_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_UNIT_STRIDE, UNSPEC_TH_VLB> vsb_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_UNIT_STRIDE, UNSPEC_TH_VLH> vsh_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_UNIT_STRIDE, UNSPEC_TH_VLW> vsw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSB> vlsb_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSBU> vlsbu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSH> vlsh_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSHU> vlshu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSW> vlsw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSWU> vlswu_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_STRIDED, UNSPEC_TH_VLSB> vssb_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_STRIDED, UNSPEC_TH_VLSH> vssh_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_STRIDED, UNSPEC_TH_VLSW> vssw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXB> vlxb_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXBU> vlxbu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXH> vlxh_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXHU> vlxhu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXW> vlxw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXWU> vlxwu_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VLXB> vsxb_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VLXH> vsxh_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VLXW> vsxw_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VSUXB> vsuxb_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VSUXH> vsuxh_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VSUXW> vsuxw_obj;
+static CONSTEXPR const th_extract vext_x_v_obj;
/* Crypto Vector */
static CONSTEXPR const vandn vandn_obj;
@@ -2878,6 +2986,37 @@ BASE (vloxseg)
BASE (vsuxseg)
BASE (vsoxseg)
BASE (vlsegff)
+BASE (vlb)
+BASE (vlh)
+BASE (vlw)
+BASE (vlbu)
+BASE (vlhu)
+BASE (vlwu)
+BASE (vsb)
+BASE (vsh)
+BASE (vsw)
+BASE (vlsb)
+BASE (vlsh)
+BASE (vlsw)
+BASE (vlsbu)
+BASE (vlshu)
+BASE (vlswu)
+BASE (vssb)
+BASE (vssh)
+BASE (vssw)
+BASE (vlxb)
+BASE (vlxh)
+BASE (vlxw)
+BASE (vlxbu)
+BASE (vlxhu)
+BASE (vlxwu)
+BASE (vsxb)
+BASE (vsxh)
+BASE (vsxw)
+BASE (vsuxb)
+BASE (vsuxh)
+BASE (vsuxw)
+BASE (vext_x_v)
/* Crypto vector */
BASE (vandn)
BASE (vbrev)
diff --git a/gcc/config/riscv/riscv-vector-builtins-bases.h b/gcc/config/riscv/riscv-vector-builtins-bases.h
index 1122e3801a7..565a0311d2b 100644
--- a/gcc/config/riscv/riscv-vector-builtins-bases.h
+++ b/gcc/config/riscv/riscv-vector-builtins-bases.h
@@ -299,6 +299,37 @@ extern const function_base *const vloxseg;
extern const function_base *const vsuxseg;
extern const function_base *const vsoxseg;
extern const function_base *const vlsegff;
+extern const function_base *const vlb;
+extern const function_base *const vlh;
+extern const function_base *const vlw;
+extern const function_base *const vlbu;
+extern const function_base *const vlhu;
+extern const function_base *const vlwu;
+extern const function_base *const vsb;
+extern const function_base *const vsh;
+extern const function_base *const vsw;
+extern const function_base *const vlsb;
+extern const function_base *const vlsh;
+extern const function_base *const vlsw;
+extern const function_base *const vlsbu;
+extern const function_base *const vlshu;
+extern const function_base *const vlswu;
+extern const function_base *const vssb;
+extern const function_base *const vssh;
+extern const function_base *const vssw;
+extern const function_base *const vlxb;
+extern const function_base *const vlxh;
+extern const function_base *const vlxw;
+extern const function_base *const vlxbu;
+extern const function_base *const vlxhu;
+extern const function_base *const vlxwu;
+extern const function_base *const vsxb;
+extern const function_base *const vsxh;
+extern const function_base *const vsxw;
+extern const function_base *const vsuxb;
+extern const function_base *const vsuxh;
+extern const function_base *const vsuxw;
+extern const function_base *const vext_x_v;
/* Below function_base are Vectro Crypto*/
extern const function_base *const vandn;
extern const function_base *const vbrev;
diff --git a/gcc/config/riscv/riscv-vector-builtins-shapes.cc b/gcc/config/riscv/riscv-vector-builtins-shapes.cc
index 1e4f4d53de6..8e90b17a94b 100644
--- a/gcc/config/riscv/riscv-vector-builtins-shapes.cc
+++ b/gcc/config/riscv/riscv-vector-builtins-shapes.cc
@@ -211,6 +211,146 @@ struct indexed_loadstore_def : public function_shape
}
};
+/* Add one function instance for GROUP, using operand suffix at index OI,
+ mode suffix at index PAIR && bi and predication suffix at index pred_idx. */
+static void
+build_th_loadstore (function_builder &b, const function_group_info &group,
+ unsigned int pred_idx, unsigned int vec_type_idx)
+{
+ auto_vec<tree, 5> argument_types;
+ function_instance function_instance (group.base_name, *group.base,
+ *group.shape,
+ group.ops_infos.types[vec_type_idx],
+ group.preds[pred_idx], &group.ops_infos);
+ tree return_type = group.ops_infos.ret.get_tree_type (
+ group.ops_infos.types[vec_type_idx].index);
+ b.allocate_argument_types (function_instance, argument_types);
+ b.apply_predication (function_instance, return_type, argument_types);
+
+ if (TARGET_XTHEADVECTOR && !check_type (return_type, argument_types))
+ return;
+
+ tree type = builtin_types[group.ops_infos.types[vec_type_idx].index].vector;
+ if (strstr (group.base_name, "l")
+ && strstr (group.base_name, "u")
+ && !TYPE_UNSIGNED (TREE_TYPE (type)))
+ return;
+
+ if (strstr (group.base_name, "l")
+ && !strstr (group.base_name, "u")
+ && TYPE_UNSIGNED (TREE_TYPE (type)))
+ return;
+
+ machine_mode mode = TYPE_MODE (type);
+ int sew = GET_MODE_BITSIZE (GET_MODE_INNER (mode));
+ if (strstr (group.base_name, "h") && sew == 8)
+ return;
+
+ if (strstr (group.base_name, "w") && (sew == 8 || sew ==16))
+ return;
+
+ b.add_overloaded_function (function_instance, *group.shape);
+ b.add_unique_function (function_instance, (*group.shape), return_type,
+ argument_types);
+}
+
+/* th_loadstore_width_def class. */
+struct th_loadstore_width_def : public build_base
+{
+ void build (function_builder &b,
+ const function_group_info &group) const override
+ {
+ for (unsigned int pred_idx = 0; group.preds[pred_idx] != NUM_PRED_TYPES;
+ ++pred_idx)
+ {
+ for (unsigned int vec_type_idx = 0;
+ group.ops_infos.types[vec_type_idx].index != NUM_VECTOR_TYPES;
+ ++vec_type_idx)
+ {
+ build_th_loadstore (b, group, pred_idx, vec_type_idx);
+ }
+ }
+ }
+
+ char *get_name (function_builder &b, const function_instance &instance,
+ bool overloaded_p) const override
+ {
+ /* Return nullptr if it can not be overloaded. */
+ if (overloaded_p && !instance.base->can_be_overloaded_p (instance.pred))
+ return nullptr;
+
+ b.append_name ("__riscv_th_");
+ b.append_name (instance.base_name);
+
+ /* vop_v --> vop_v_<type>. */
+ if (!overloaded_p)
+ {
+ /* vop --> vop_v. */
+ b.append_name (operand_suffixes[instance.op_info->op]);
+ /* vop_v --> vop_v_<type>. */
+ b.append_name (type_suffixes[instance.type.index].vector);
+ }
+
+ /* According to rvv-intrinsic-doc, it does not add "_m" suffix
+ for vop_m C++ overloaded API. */
+ if (overloaded_p && instance.pred == PRED_TYPE_m)
+ return b.finish_name ();
+ b.append_name (predication_suffixes[instance.pred]);
+ return b.finish_name ();
+ }
+};
+
+
+/* th_indexed_loadstore_width_def class. */
+struct th_indexed_loadstore_width_def : public function_shape
+{
+ void build (function_builder &b,
+ const function_group_info &group) const override
+ {
+ for (unsigned int pred_idx = 0; group.preds[pred_idx] != NUM_PRED_TYPES;
+ ++pred_idx)
+ {
+ for (unsigned int vec_type_idx = 0;
+ group.ops_infos.types[vec_type_idx].index != NUM_VECTOR_TYPES;
+ ++vec_type_idx)
+ {
+ tree index_type = group.ops_infos.args[1].get_tree_type (
+ group.ops_infos.types[vec_type_idx].index);
+ if (!index_type)
+ continue;
+ build_th_loadstore (b, group, pred_idx, vec_type_idx);
+ }
+ }
+ }
+
+ char *get_name (function_builder &b, const function_instance &instance,
+ bool overloaded_p) const override
+ {
+
+ /* Return nullptr if it can not be overloaded. */
+ if (overloaded_p && !instance.base->can_be_overloaded_p (instance.pred))
+ return nullptr;
+
+ b.append_name ("__riscv_th_");
+ b.append_name (instance.base_name);
+ /* vop_v --> vop_v_<type>. */
+ if (!overloaded_p)
+ {
+ /* vop --> vop_v. */
+ b.append_name (operand_suffixes[instance.op_info->op]);
+ /* vop_v --> vop_v_<type>. */
+ b.append_name (type_suffixes[instance.type.index].vector);
+ }
+
+ /* According to rvv-intrinsic-doc, it does not add "_m" suffix
+ for vop_m C++ overloaded API. */
+ if (overloaded_p && instance.pred == PRED_TYPE_m)
+ return b.finish_name ();
+ b.append_name (predication_suffixes[instance.pred]);
+ return b.finish_name ();
+ }
+};
+
/* alu_def class. */
struct alu_def : public build_base
{
@@ -632,6 +772,23 @@ struct reduc_alu_def : public build_base
}
};
+/* th_extract_def class. */
+struct th_extract_def : public build_base
+{
+ char *get_name (function_builder &b, const function_instance &instance,
+ bool overloaded_p) const override
+ {
+ b.append_name ("__riscv_th_");
+ b.append_name (instance.base_name);
+
+ if (overloaded_p)
+ return b.finish_name ();
+ b.append_name (type_suffixes[instance.type.index].vector);
+ b.append_name (type_suffixes[instance.type.index].scalar);
+ return b.finish_name ();
+ }
+};
+
/* scalar_move_def class. */
struct scalar_move_def : public build_base
{
@@ -1094,6 +1251,8 @@ SHAPE(vsetvl, vsetvl)
SHAPE(vsetvl, vsetvlmax)
SHAPE(loadstore, loadstore)
SHAPE(indexed_loadstore, indexed_loadstore)
+SHAPE(th_loadstore_width, th_loadstore_width)
+SHAPE(th_indexed_loadstore_width, th_indexed_loadstore_width)
SHAPE(alu, alu)
SHAPE(alu_frm, alu_frm)
SHAPE(widen_alu, widen_alu)
@@ -1106,6 +1265,7 @@ SHAPE(move, move)
SHAPE(mask_alu, mask_alu)
SHAPE(reduc_alu, reduc_alu)
SHAPE(reduc_alu_frm, reduc_alu_frm)
+SHAPE(th_extract, th_extract)
SHAPE(scalar_move, scalar_move)
SHAPE(vundefined, vundefined)
SHAPE(misc, misc)
diff --git a/gcc/config/riscv/riscv-vector-builtins-shapes.h b/gcc/config/riscv/riscv-vector-builtins-shapes.h
index ac2a28ce017..a7624d0fabd 100644
--- a/gcc/config/riscv/riscv-vector-builtins-shapes.h
+++ b/gcc/config/riscv/riscv-vector-builtins-shapes.h
@@ -28,6 +28,8 @@ extern const function_shape *const vsetvl;
extern const function_shape *const vsetvlmax;
extern const function_shape *const loadstore;
extern const function_shape *const indexed_loadstore;
+extern const function_shape *const th_loadstore_width;
+extern const function_shape *const th_indexed_loadstore_width;
extern const function_shape *const alu;
extern const function_shape *const alu_frm;
extern const function_shape *const widen_alu;
@@ -41,6 +43,7 @@ extern const function_shape *const mask_alu;
extern const function_shape *const reduc_alu;
extern const function_shape *const reduc_alu_frm;
extern const function_shape *const scalar_move;
+extern const function_shape *const th_extract;
extern const function_shape *const vundefined;
extern const function_shape *const misc;
extern const function_shape *const vset;
diff --git a/gcc/config/riscv/riscv-vector-builtins.cc b/gcc/config/riscv/riscv-vector-builtins.cc
index 25e0b6e56de..44b9fec1898 100644
--- a/gcc/config/riscv/riscv-vector-builtins.cc
+++ b/gcc/config/riscv/riscv-vector-builtins.cc
@@ -934,6 +934,32 @@ static CONSTEXPR const rvv_arg_type_info ext_vcreate_args[]
= {rvv_arg_type_info (RVV_BASE_vector),
rvv_arg_type_info_end};
+/* A list of args for vector_type func (const scalar_type *, size_t)
+ * function. */
+static CONSTEXPR const rvv_arg_type_info scalar_const_ptr_size_args[]
+ = {rvv_arg_type_info (RVV_BASE_scalar_const_ptr),
+ rvv_arg_type_info (RVV_BASE_size), rvv_arg_type_info_end};
+
+/* A list of args for vector_type func (const scalar_type *, eew8_index_type)
+ * function. */
+static CONSTEXPR const rvv_arg_type_info scalar_const_ptr_index_args[]
+ = {rvv_arg_type_info (RVV_BASE_scalar_const_ptr),
+ rvv_arg_type_info (RVV_BASE_unsigned_vector), rvv_arg_type_info_end};
+
+/* A list of args for void func (scalar_type *, eew8_index_type, vector_type)
+ * function. */
+static CONSTEXPR const rvv_arg_type_info scalar_ptr_index_args[]
+ = {rvv_arg_type_info (RVV_BASE_scalar_ptr),
+ rvv_arg_type_info (RVV_BASE_unsigned_vector),
+ rvv_arg_type_info (RVV_BASE_vector), rvv_arg_type_info_end};
+
+/* A list of args for void func (scalar_type *, size_t, vector_type)
+ * function. */
+static CONSTEXPR const rvv_arg_type_info scalar_ptr_size_args[]
+ = {rvv_arg_type_info (RVV_BASE_scalar_ptr),
+ rvv_arg_type_info (RVV_BASE_size), rvv_arg_type_info (RVV_BASE_vector),
+ rvv_arg_type_info_end};
+
/* A list of none preds that will be registered for intrinsic functions. */
static CONSTEXPR const predication_type_index none_preds[]
= {PRED_TYPE_none, NUM_PRED_TYPES};
@@ -1455,6 +1481,14 @@ static CONSTEXPR const rvv_op_info iu_shift_vvv_ops
rvv_arg_type_info (RVV_BASE_vector), /* Return type */
shift_vv_args /* Args */};
+/* A static operand information for scalar_type func (vector_type, size_t)
+ * function registration. */
+static CONSTEXPR const rvv_op_info iu_x_s_u_ops
+ = {iu_ops, /* Types */
+ OP_TYPE_vx, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_scalar), /* Return type */
+ v_size_args /* Args */};
+
/* A static operand information for vector_type func (vector_type, size_t)
* function registration. */
static CONSTEXPR const rvv_op_info iu_shift_vvx_ops
@@ -2638,6 +2672,38 @@ static CONSTEXPR const rvv_op_info all_v_vcreate_lmul4_x2_ops
rvv_arg_type_info (RVV_BASE_vlmul_ext_x2), /* Return type */
ext_vcreate_args /* Args */};
+/* A static operand information for vector_type func (const scalar_type *,
+ * size_t) function registration. */
+static CONSTEXPR const rvv_op_info all_v_scalar_const_ptr_size_ops
+ = {all_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_size_args /* Args */};
+
+/* A static operand information for void func (scalar_type *, size_t,
+ * vector_type) function registration. */
+static CONSTEXPR const rvv_op_info all_v_scalar_ptr_size_ops
+ = {all_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_void), /* Return type */
+ scalar_ptr_size_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * index_type) function registration. */
+static CONSTEXPR const rvv_op_info all_v_scalar_const_ptr_index_ops
+ = {all_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_index_args /* Args */};
+
+/* A static operand information for void func (scalar_type *, index_type,
+ * vector_type) function registration. */
+static CONSTEXPR const rvv_op_info all_v_scalar_ptr_index_ops
+ = {all_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_void), /* Return type */
+ scalar_ptr_index_args /* Args */};
+
/* A static operand information for vector_type func (vector_type).
Some ins just supports SEW=32, such as crypto vectol Zvkg extension.
* function registration. */
@@ -2816,6 +2882,10 @@ static function_group_info function_groups[] = {
#define DEF_RVV_FUNCTION(NAME, SHAPE, PREDS, OPS_INFO) \
{#NAME, &bases::NAME, &shapes::SHAPE, PREDS, OPS_INFO, REQUIRED_EXTENSIONS},
#include "riscv-vector-builtins-functions.def"
+#undef DEF_RVV_FUNCTION
+#define DEF_RVV_FUNCTION(NAME, SHAPE, PREDS, OPS_INFO) \
+ {#NAME, &bases::NAME, &shapes::SHAPE, PREDS, OPS_INFO, REQUIRED_EXTENSIONS},
+#include "thead-vector-builtins-functions.def"
};
/* The RVV types, with their built-in
diff --git a/gcc/config/riscv/riscv-vector-builtins.h b/gcc/config/riscv/riscv-vector-builtins.h
index 54c8824ff92..a8ee39a3cb2 100644
--- a/gcc/config/riscv/riscv-vector-builtins.h
+++ b/gcc/config/riscv/riscv-vector-builtins.h
@@ -114,6 +114,7 @@ static const unsigned int CP_WRITE_CSR = 1U << 5;
enum required_ext
{
VECTOR_EXT, /* Vector extension */
+ XTHEADVECTOR_EXT, /* XTheadVector extension */
ZVBB_EXT, /* Cryto vector Zvbb sub-ext */
ZVBB_OR_ZVKB_EXT, /* Cryto vector Zvbb or zvkb sub-ext */
ZVBC_EXT, /* Crypto vector Zvbc sub-ext */
@@ -234,6 +235,8 @@ struct function_group_info
{
case VECTOR_EXT:
return TARGET_VECTOR;
+ case XTHEADVECTOR_EXT:
+ return TARGET_XTHEADVECTOR;
case ZVBB_EXT:
return TARGET_ZVBB;
case ZVBB_OR_ZVKB_EXT:
diff --git a/gcc/config/riscv/t-riscv b/gcc/config/riscv/t-riscv
index 32de6b851c1..38494320d8b 100644
--- a/gcc/config/riscv/t-riscv
+++ b/gcc/config/riscv/t-riscv
@@ -1,6 +1,7 @@
RISCV_BUILTINS_H = $(srcdir)/config/riscv/riscv-vector-builtins.h \
$(srcdir)/config/riscv/riscv-vector-builtins.def \
$(srcdir)/config/riscv/riscv-vector-builtins-functions.def \
+ $(srcdir)/config/riscv/thead-vector-builtins-functions.def \
riscv-vector-type-indexer.gen.def
riscv-builtins.o: $(srcdir)/config/riscv/riscv-builtins.cc $(CONFIG_H) \
diff --git a/gcc/config/riscv/thead-vector-builtins-functions.def b/gcc/config/riscv/thead-vector-builtins-functions.def
new file mode 100644
index 00000000000..fd3ba29bae9
--- /dev/null
+++ b/gcc/config/riscv/thead-vector-builtins-functions.def
@@ -0,0 +1,39 @@
+#ifndef DEF_RVV_FUNCTION
+#define DEF_RVV_FUNCTION(NAME, SHAPE, PREDS, OPS_INFO)
+#endif
+
+#define REQUIRED_EXTENSIONS XTHEADVECTOR_EXT
+DEF_RVV_FUNCTION (vlb, th_loadstore_width, full_preds, all_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (vlh, th_loadstore_width, full_preds, all_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (vlw, th_loadstore_width, full_preds, all_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (vlbu, th_loadstore_width, full_preds, all_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (vlhu, th_loadstore_width, full_preds, all_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (vlwu, th_loadstore_width, full_preds, all_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (vsb, th_loadstore_width, none_m_preds, all_v_scalar_ptr_ops)
+DEF_RVV_FUNCTION (vsh, th_loadstore_width, none_m_preds, all_v_scalar_ptr_ops)
+DEF_RVV_FUNCTION (vsw, th_loadstore_width, none_m_preds, all_v_scalar_ptr_ops)
+DEF_RVV_FUNCTION (vlsb, th_loadstore_width, full_preds, all_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (vlsh, th_loadstore_width, full_preds, all_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (vlsw, th_loadstore_width, full_preds, all_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (vlsbu, th_loadstore_width, full_preds, all_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (vlshu, th_loadstore_width, full_preds, all_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (vlswu, th_loadstore_width, full_preds, all_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (vssb, th_loadstore_width, none_m_preds, all_v_scalar_ptr_size_ops)
+DEF_RVV_FUNCTION (vssh, th_loadstore_width, none_m_preds, all_v_scalar_ptr_size_ops)
+DEF_RVV_FUNCTION (vssw, th_loadstore_width, none_m_preds, all_v_scalar_ptr_size_ops)
+DEF_RVV_FUNCTION (vlxb, th_indexed_loadstore_width, full_preds, all_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (vlxh, th_indexed_loadstore_width, full_preds, all_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (vlxw, th_indexed_loadstore_width, full_preds, all_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (vlxbu, th_indexed_loadstore_width, full_preds, all_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (vlxhu, th_indexed_loadstore_width, full_preds, all_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (vlxwu, th_indexed_loadstore_width, full_preds, all_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (vsxb, th_indexed_loadstore_width, none_m_preds, all_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (vsxh, th_indexed_loadstore_width, none_m_preds, all_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (vsxw, th_indexed_loadstore_width, none_m_preds, all_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (vsuxb, th_indexed_loadstore_width, none_m_preds, all_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (vsuxh, th_indexed_loadstore_width, none_m_preds, all_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (vsuxw, th_indexed_loadstore_width, none_m_preds, all_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (vext_x_v, th_extract, none_preds, iu_x_s_u_ops)
+#undef REQUIRED_EXTENSIONS
+
+#undef DEF_RVV_FUNCTION
diff --git a/gcc/config/riscv/thead-vector.md b/gcc/config/riscv/thead-vector.md
index 696b815252d..0f3700d9269 100644
--- a/gcc/config/riscv/thead-vector.md
+++ b/gcc/config/riscv/thead-vector.md
@@ -1,7 +1,95 @@
(define_c_enum "unspec" [
+ UNSPEC_TH_VLB
+ UNSPEC_TH_VLBU
+ UNSPEC_TH_VLH
+ UNSPEC_TH_VLHU
+ UNSPEC_TH_VLW
+ UNSPEC_TH_VLWU
+
+ UNSPEC_TH_VLSB
+ UNSPEC_TH_VLSBU
+ UNSPEC_TH_VLSH
+ UNSPEC_TH_VLSHU
+ UNSPEC_TH_VLSW
+ UNSPEC_TH_VLSWU
+
+ UNSPEC_TH_VLXB
+ UNSPEC_TH_VLXBU
+ UNSPEC_TH_VLXH
+ UNSPEC_TH_VLXHU
+ UNSPEC_TH_VLXW
+ UNSPEC_TH_VLXWU
+
+ UNSPEC_TH_VSUXB
+ UNSPEC_TH_VSUXH
+ UNSPEC_TH_VSUXW
+
UNSPEC_TH_VWLDST
])
+(define_int_iterator UNSPEC_TH_VLMEM_OP [
+ UNSPEC_TH_VLB UNSPEC_TH_VLBU
+ UNSPEC_TH_VLH UNSPEC_TH_VLHU
+ UNSPEC_TH_VLW UNSPEC_TH_VLWU
+])
+
+(define_int_iterator UNSPEC_TH_VLSMEM_OP [
+ UNSPEC_TH_VLSB UNSPEC_TH_VLSBU
+ UNSPEC_TH_VLSH UNSPEC_TH_VLSHU
+ UNSPEC_TH_VLSW UNSPEC_TH_VLSWU
+])
+
+(define_int_iterator UNSPEC_TH_VLXMEM_OP [
+ UNSPEC_TH_VLXB UNSPEC_TH_VLXBU
+ UNSPEC_TH_VLXH UNSPEC_TH_VLXHU
+ UNSPEC_TH_VLXW UNSPEC_TH_VLXWU
+])
+
+(define_int_attr vlmem_op_attr [
+ (UNSPEC_TH_VLB "b") (UNSPEC_TH_VLBU "bu")
+ (UNSPEC_TH_VLH "h") (UNSPEC_TH_VLHU "hu")
+ (UNSPEC_TH_VLW "w") (UNSPEC_TH_VLWU "wu")
+ (UNSPEC_TH_VLSB "b") (UNSPEC_TH_VLSBU "bu")
+ (UNSPEC_TH_VLSH "h") (UNSPEC_TH_VLSHU "hu")
+ (UNSPEC_TH_VLSW "w") (UNSPEC_TH_VLSWU "wu")
+ (UNSPEC_TH_VLXB "b") (UNSPEC_TH_VLXBU "bu")
+ (UNSPEC_TH_VLXH "h") (UNSPEC_TH_VLXHU "hu")
+ (UNSPEC_TH_VLXW "w") (UNSPEC_TH_VLXWU "wu")
+ (UNSPEC_TH_VSUXB "b")
+ (UNSPEC_TH_VSUXH "h")
+ (UNSPEC_TH_VSUXW "w")
+])
+
+(define_int_attr vlmem_order_attr [
+ (UNSPEC_TH_VLXB "")
+ (UNSPEC_TH_VLXH "")
+ (UNSPEC_TH_VLXW "")
+ (UNSPEC_TH_VSUXB "u")
+ (UNSPEC_TH_VSUXH "u")
+ (UNSPEC_TH_VSUXW "u")
+])
+
+(define_int_iterator UNSPEC_TH_VSMEM_OP [
+ UNSPEC_TH_VLB
+ UNSPEC_TH_VLH
+ UNSPEC_TH_VLW
+])
+
+(define_int_iterator UNSPEC_TH_VSSMEM_OP [
+ UNSPEC_TH_VLSB
+ UNSPEC_TH_VLSH
+ UNSPEC_TH_VLSW
+])
+
+(define_int_iterator UNSPEC_TH_VSXMEM_OP [
+ UNSPEC_TH_VLXB
+ UNSPEC_TH_VLXH
+ UNSPEC_TH_VLXW
+ UNSPEC_TH_VSUXB
+ UNSPEC_TH_VSUXH
+ UNSPEC_TH_VSUXW
+])
+
(define_mode_iterator V_VLS_VT [V VLS VT])
(define_mode_iterator V_VB_VLS_VT [V VB VLS VT])
@@ -100,3 +188,168 @@
}
[(set_attr "type" "vldm,vstm,vmalu,vmalu,vmalu")
(set_attr "mode" "<MODE>")])
+
+(define_expand "@pred_mov_width<vlmem_op_attr><mode>"
+ [(set (match_operand:V_VLS 0 "nonimmediate_operand")
+ (if_then_else:V_VLS
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand")
+ (match_operand 4 "vector_length_operand")
+ (match_operand 5 "const_int_operand")
+ (match_operand 6 "const_int_operand")
+ (match_operand 7 "const_int_operand")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLMEM_OP)
+ (match_operand:V_VLS 3 "vector_move_operand")
+ (match_operand:V_VLS 2 "vector_merge_operand")))]
+ "TARGET_XTHEADVECTOR"
+ {})
+
+(define_insn_and_split "*pred_mov_width<vlmem_op_attr><mode>"
+ [(set (match_operand:V_VLS 0 "nonimmediate_operand" "=vr, vr, vd, m, vr, vr")
+ (if_then_else:V_VLS
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1, Wc1, vm, vmWc1, Wc1, Wc1")
+ (match_operand 4 "vector_length_operand" " rK, rK, rK, rK, rK, rK")
+ (match_operand 5 "const_int_operand" " i, i, i, i, i, i")
+ (match_operand 6 "const_int_operand" " i, i, i, i, i, i")
+ (match_operand 7 "const_int_operand" " i, i, i, i, i, i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLMEM_OP)
+ (match_operand:V_VLS 3 "reg_or_mem_operand" " m, m, m, vr, vr, vr")
+ (match_operand:V_VLS 2 "vector_merge_operand" " 0, vu, vu, vu, vu, 0")))]
+ "(TARGET_XTHEADVECTOR
+ && (register_operand (operands[0], <MODE>mode)
+ || register_operand (operands[3], <MODE>mode)))"
+ "@
+ vl<vlmem_op_attr>.v\t%0,%3%p1
+ vl<vlmem_op_attr>.v\t%0,%3
+ vl<vlmem_op_attr>.v\t%0,%3,%1.t
+ vs<vlmem_op_attr>.v\t%3,%0%p1
+ vmv.v.v\t%0,%3
+ vmv.v.v\t%0,%3"
+ "&& register_operand (operands[0], <MODE>mode)
+ && register_operand (operands[3], <MODE>mode)
+ && satisfies_constraint_vu (operands[2])
+ && INTVAL (operands[7]) == riscv_vector::VLMAX"
+ [(set (match_dup 0) (match_dup 3))]
+ ""
+ [(set_attr "type" "vlde,vlde,vlde,vste,vimov,vimov")
+ (set_attr "mode" "<MODE>")])
+
+(define_insn "@pred_store_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "memory_operand" "+m")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1")
+ (match_operand 3 "vector_length_operand" " rK")
+ (match_operand 4 "const_int_operand" " i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VSMEM_OP)
+ (match_operand:VI 2 "register_operand" " vr")
+ (match_dup 0)))]
+ "TARGET_XTHEADVECTOR"
+ "vs<vlmem_op_attr>.v\t%2,%0%p1"
+ [(set_attr "type" "vste")
+ (set_attr "mode" "<MODE>")
+ (set (attr "avl_type_idx") (const_int 4))
+ (set_attr "vl_op_idx" "3")])
+
+(define_insn "@pred_strided_load_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "register_operand" "=vr, vr, vd")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1, Wc1, vm")
+ (match_operand 5 "vector_length_operand" " rK, rK, rK")
+ (match_operand 6 "const_int_operand" " i, i, i")
+ (match_operand 7 "const_int_operand" " i, i, i")
+ (match_operand 8 "const_int_operand" " i, i, i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLSMEM_OP)
+ (unspec:VI
+ [(match_operand:VI 3 "memory_operand" " m, m, m")
+ (match_operand 4 "pmode_reg_or_0_operand" " rJ, rJ, rJ")] UNSPEC_TH_VLSMEM_OP)
+ (match_operand:VI 2 "vector_merge_operand" " 0, vu, vu")))]
+ "TARGET_XTHEADVECTOR"
+ "vls<vlmem_op_attr>.v\t%0,%3,%z4%p1"
+ [(set_attr "type" "vlds")
+ (set_attr "mode" "<MODE>")])
+
+(define_insn "@pred_strided_store_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "memory_operand" "+m")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1")
+ (match_operand 4 "vector_length_operand" " rK")
+ (match_operand 5 "const_int_operand" " i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VSSMEM_OP)
+ (unspec:VI
+ [(match_operand 2 "pmode_reg_or_0_operand" " rJ")
+ (match_operand:VI 3 "register_operand" " vr")] UNSPEC_TH_VSSMEM_OP)
+ (match_dup 0)))]
+ "TARGET_XTHEADVECTOR"
+ "vss<vlmem_op_attr>.v\t%3,%0,%z2%p1"
+ [(set_attr "type" "vsts")
+ (set_attr "mode" "<MODE>")
+ (set (attr "avl_type_idx") (const_int 5))])
+
+(define_insn "@pred_indexed_load_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "register_operand" "=vd, vr,vd, vr")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" " vm,Wc1,vm,Wc1")
+ (match_operand 5 "vector_length_operand" " rK, rK,rK, rK")
+ (match_operand 6 "const_int_operand" " i, i, i, i")
+ (match_operand 7 "const_int_operand" " i, i, i, i")
+ (match_operand 8 "const_int_operand" " i, i, i, i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLXMEM_OP)
+ (unspec:VI
+ [(match_operand 3 "pmode_reg_or_0_operand" " rJ, rJ,rJ, rJ")
+ (mem:BLK (scratch))
+ (match_operand:VI 4 "register_operand" " vr, vr,vr, vr")] UNSPEC_TH_VLXMEM_OP)
+ (match_operand:VI 2 "vector_merge_operand" " vu, vu, 0, 0")))]
+ "TARGET_XTHEADVECTOR"
+ "vlx<vlmem_op_attr>.v\t%0,(%z3),%4%p1"
+ [(set_attr "type" "vldux")
+ (set_attr "mode" "<MODE>")])
+
+(define_insn "@pred_indexed_<vlmem_order_attr>store_width<vlmem_op_attr><mode>"
+ [(set (mem:BLK (scratch))
+ (unspec:BLK
+ [(unspec:<VM>
+ [(match_operand:<VM> 0 "vector_mask_operand" "vmWc1")
+ (match_operand 4 "vector_length_operand" " rK")
+ (match_operand 5 "const_int_operand" " i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VSXMEM_OP)
+ (match_operand 1 "pmode_reg_or_0_operand" " rJ")
+ (match_operand:VI 2 "register_operand" " vr")
+ (match_operand:VI 3 "register_operand" " vr")] UNSPEC_TH_VSXMEM_OP))]
+ "TARGET_XTHEADVECTOR"
+ "vs<vlmem_order_attr>x<vlmem_op_attr>.v\t%3,(%z1),%2%p0"
+ [(set_attr "type" "vstux")
+ (set_attr "mode" "<MODE>")])
+
+(define_expand "@pred_th_extract<mode>"
+ [(set (match_operand:<VEL> 0 "register_operand")
+ (unspec:<VEL>
+ [(vec_select:<VEL>
+ (match_operand:V_VLSI 1 "register_operand")
+ (parallel [(match_operand:DI 2 "register_operand" "r")]))
+ (reg:SI VTYPE_REGNUM)] UNSPEC_VPREDICATE))]
+ "TARGET_XTHEADVECTOR"
+{})
+
+(define_insn "*pred_th_extract<mode>"
+ [(set (match_operand:<VEL> 0 "register_operand" "=r")
+ (unspec:<VEL>
+ [(vec_select:<VEL>
+ (match_operand:V_VLSI 1 "register_operand" "vr")
+ (parallel [(match_operand:DI 2 "register_operand" "r")]))
+ (reg:SI VTYPE_REGNUM)] UNSPEC_VPREDICATE))]
+ "TARGET_XTHEADVECTOR"
+ "vext.x.v\t%0,%1,%2"
+ [(set_attr "type" "vimovvx")
+ (set_attr "mode" "<MODE>")])
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c
new file mode 100644
index 00000000000..3c12c124597
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_th_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+
+** th.vadd\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+
+** th.vsb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out)
+{
+ vint32m1_t v = __riscv_th_vlb_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlb_v_i32m1_tu (v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vv_i32m1 (v2, v2, 4);
+ vint32m1_t v4 = __riscv_vadd_vv_i32m1_tu (v3, v2, v2, 4);
+ __riscv_th_vsb_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlb.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+
+** th.vadd\.vv\tv[1-9][0-9]?,\s*v[0-9]+,\s*v[0-9]+,\s*v0.t
+** th.vsb.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlb_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlb_v_i32m1_m (mask, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vv_i32m1 (v2, v2, 4);
+ vint32m1_t v4 = __riscv_vadd_vv_i32m1_m (mask, v3, v3, 4);
+ __riscv_th_vsb_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlb.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+
+** th.vadd\.vv\tv[1-9][0-9]?,\s*v[0-9]+,\s*v[0-9]+,\s*v0.t
+** th.vsb.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlb_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlb_v_i32m1_tumu (mask, v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vv_i32m1 (v2, v2, 4);
+ vint32m1_t v4 = __riscv_vadd_vv_i32m1_tumu (mask, v3, v2, v2, 4);
+ __riscv_th_vsb_v_i32m1 (out, v4, 4);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c
new file mode 100644
index 00000000000..30bef369375
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_th_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlbu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlbu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vsb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, uint32_t x)
+{
+ vuint32m1_t v = __riscv_th_vlbu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlbu_v_u32m1_tu (v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tu (v3, v2, -16, 4);
+ __riscv_th_vsb_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlbu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsb.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlbu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlbu_v_u32m1_m (mask, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_m (mask, v3, -16, 4);
+ __riscv_th_vsb_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlbu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlbu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsb.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlbu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlbu_v_u32m1_tumu (mask, v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tumu (mask, v3, v2, -16, 4);
+ __riscv_th_vsb_v_u32m1 (out, v4, 4);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c
new file mode 100644
index 00000000000..3c8b5ccc16b
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_th_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vsh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, int32_t x)
+{
+ vint32m1_t v = __riscv_th_vlh_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlh_v_i32m1_tu (v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, -16, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_tu (v3, v2, -16, 4);
+ __riscv_th_vsh_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlh.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsh.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, int32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlh_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlh_v_i32m1_m (mask, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, -16, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_m (mask, v3, -16, 4);
+ __riscv_th_vsh_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlh.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsh.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, int32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlh_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlh_v_i32m1_tumu (mask, v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, -16, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_tumu (mask, v3, v2, -16, 4);
+ __riscv_th_vsh_v_i32m1 (out, v4, 4);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c
new file mode 100644
index 00000000000..b7c00404f18
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_th_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlhu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlhu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vsh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, uint32_t x)
+{
+ vuint32m1_t v = __riscv_th_vlhu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlhu_v_u32m1_tu (v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tu (v3, v2, -16, 4);
+ __riscv_th_vsh_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlhu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsh.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlhu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlhu_v_u32m1_m (mask, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_m (mask, v3, -16, 4);
+ __riscv_th_vsh_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlhu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlhu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsh.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlhu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlhu_v_u32m1_tumu (mask, v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tumu (mask, v3, v2, -16, 4);
+ __riscv_th_vsh_v_u32m1 (out, v4, 4);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c
new file mode 100644
index 00000000000..17a53012acf
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_th_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vx\tv[0-9]+,\s*v[0-9]+,\s*[a-x0-9]+
+** th.vadd\.vx\tv[0-9]+,\s*v[0-9]+,\s*[a-x0-9]+
+** th.vsw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, int32_t x)
+{
+ vint32m1_t v = __riscv_th_vlw_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlw_v_i32m1_tu (v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, x, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_tu (v3, v2, x, 4);
+ __riscv_th_vsw_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlw.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vx\tv[0-9]+,\s*v[0-9]+,\s*[a-x0-9]+
+** th.vadd\.vx\tv[1-9][0-9]?,\s*v[0-9]+,\s*[a-x0-9]+,\s*v0.t
+** th.vsw.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, int32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlw_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlw_v_i32m1_m (mask, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, x, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_m (mask, v3, x, 4);
+ __riscv_th_vsw_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlw.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vx\tv[0-9]+,\s*v[0-9]+,\s*[a-x0-9]+
+** th.vadd\.vx\tv[1-9][0-9]?,\s*v[0-9]+,\s*[a-x0-9]+,\s*v0.t
+** th.vsw.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, int32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlw_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlw_v_i32m1_tumu (mask, v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, x, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_tumu (mask, v3, v2, x, 4);
+ __riscv_th_vsw_v_i32m1 (out, v4, 4);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c
new file mode 100644
index 00000000000..b187cfc852b
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_th_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlwu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlwu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vsw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, uint32_t x)
+{
+ vuint32m1_t v = __riscv_th_vlwu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlwu_v_u32m1_tu (v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tu (v3, v2, -16, 4);
+ __riscv_th_vsw_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlwu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsw.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlwu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlwu_v_u32m1_m (mask, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_m (mask, v3, -16, 4);
+ __riscv_th_vsw_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlwu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlwu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsw.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlwu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlwu_v_u32m1_tumu (mask, v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tumu (mask, v3, v2, -16, 4);
+ __riscv_th_vsw_v_u32m1 (out, v4, 4);
+}
--
2.17.1
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re：Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
2024-01-11 9:17 Re：[PATCH " juzhe.zhong
2024-01-11 9:21 ` Re：Re：[PATCH " joshua
@ 2024-01-11 9:26 ` joshua
1 sibling, 0 replies; 17+ messages in thread
From: joshua @ 2024-01-11 9:26 UTC (permalink / raw)
To: juzhe.zhong, gcc-patches
Cc: Jim Wilson, palmer, andrew, philipp.tomsich, jeffreyalaw,
christoph.muellner, jinma, cooper.qu
Maybe the optimization cannot be done in simple cases. We run some complex cases
in O2 and dsicovered it.
------------------------------------------------------------------
发件人：juzhe.zhong@rivai.ai <juzhe.zhong@rivai.ai>
发送时间：2024年1月11日(星期四) 17:17
收件人："cooper.joshua"<cooper.joshua@linux.alibaba.com>; "gcc-patches"<gcc-patches@gcc.gnu.org>
抄 送：Jim Wilson<jim.wilson.gcc@gmail.com>; palmer<palmer@dabbelt.com>; andrew<andrew@sifive.com>; "philipp.tomsich"<philipp.tomsich@vrull.eu>; jeffreyalaw<jeffreyalaw@gmail.com>; "christoph.muellner"<christoph.muellner@vrull.eu>; jinma<jinma@linux.alibaba.com>; "cooper.qu"<cooper.qu@linux.alibaba.com>
主 题：Re: Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
You mean which pattern optimized sext/vzext pattern?
I didn't see theadvector-specific extension patterns. Could you show me?
juzhe.zhong@rivai.ai
发件人： joshua
发送时间： 2024-01-11 17:14
收件人： juzhe.zhong@rivai.ai; gcc-patches
抄送： Jim Wilson; palmer; andrew; philipp.tomsich; jeffreyalaw; christoph.muellner; jinma; cooper.qu
主题： Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
To be specific, in CSE pass, the initial pattern will be optimized into the sext/zext pattern.
------------------------------------------------------------------
发件人：joshua <cooper.joshua@linux.alibaba.com>
发送时间：2024年1月11日(星期四) 17:11
收件人："juzhe.zhong@rivai.ai"<juzhe.zhong@rivai.ai>; "gcc-patches"<gcc-patches@gcc.gnu.org>
抄 送：Jim Wilson<jim.wilson.gcc@gmail.com>; palmer<palmer@dabbelt.com>; andrew<andrew@sifive.com>; "philipp.tomsich"<philipp.tomsich@vrull.eu>; jeffreyalaw<jeffreyalaw@gmail.com>; "christoph.muellner"<christoph.muellner@vrull.eu>; jinma<jinma@linux.alibaba.com>; "cooper.qu"<cooper.qu@linux.alibaba.com>
主 题：Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
sext/zext will be generated in O2 even without corresponding intrinsics.
------------------------------------------------------------------
发件人：juzhe.zhong@rivai.ai <juzhe.zhong@rivai.ai>
发送时间：2024年1月11日(星期四) 17:07
收件人："cooper.joshua"<cooper.joshua@linux.alibaba.com>; "gcc-patches"<gcc-patches@gcc.gnu.org>
抄 送：Jim Wilson<jim.wilson.gcc@gmail.com>; palmer<palmer@dabbelt.com>; andrew<andrew@sifive.com>; "philipp.tomsich"<philipp.tomsich@vrull.eu>; jeffreyalaw<jeffreyalaw@gmail.com>; "christoph.muellner"<christoph.muellner@vrull.eu>; "cooper.joshua"<cooper.joshua@linux.alibaba.com>; jinma<jinma@linux.alibaba.com>; "cooper.qu"<cooper.qu@linux.alibaba.com>
主 题：Re: [PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
enum required_ext
{
VECTOR_EXT, /* Vector extension */
+ XTHEADVECTOR_EXT, /* XTheadVector extension */
ZVBB_EXT, /* Cryto vector Zvbb sub-ext */
ZVBB_OR_ZVKB_EXT, /* Cryto vector Zvbb or zvkb sub-ext */
ZVBC_EXT, /* Crypto vector Zvbc sub-ext */
Add theadvector to the end of the enum.
+ case XTHEADVECTOR_EXT:
+ return TARGET_XTHEADVECTOR;
Same.
+ "&& register_operand (operands[0], <MODE>mode)
+ && register_operand (operands[3], <MODE>mode)
+ && satisfies_constraint_vu (operands[2])
+ && INTVAL (operands[7]) == riscv_vector::VLMAX"
You can use whole_reg_to_reg_move_p
Btw, I review again : RISC-V: Handle differences between XTheadvector and Vector
(any_extend:VWEXTI (match_operand:<V_DOUBLE_TRUNC> 3 "register_operand" "W21,W21,W21,W21,W42,W42,W42,W42,W84,W84,W84,W84, vr, vr")) (match_operand:VWEXTI 2 "vector_merge_operand" " vu, vu, 0, 0, vu, vu, 0, 0, vu, vu, 0, 0, vu, 0")))]- "TARGET_VECTOR"+ "TARGET_VECTOR && !TARGET_XTHEADVECTOR" "v<sz>ext.vf2\t%0,%3%p1" [(set_attr "type" "vext") (set_attr "mode" "<MODE>")@@ -3713,7 +3744,7 @@ (any_extend:VQEXTI (match_operand:<V_QUAD_TRUNC> 3 "register_operand" "W43,W43,W43,W43,W86,W86,W86,W86, vr, vr")) (match_operand:VQEXTI 2 "vector_merge_operand" " vu, vu, 0, 0, vu, vu, 0, 0, vu, 0")))]- "TARGET_VECTOR"+ "TARGET_VECTOR && !TARGET_XTHEADVECTOR" "v<sz>ext.vf4\t%0,%3%p1" [(set_attr "type" "vext") (set_attr "mode" "<MODE>")@@ -3734,7 +3765,7 @@ (any_extend:VOEXTI (match_operand:<V_OCT_TRUNC> 3 "register_operand" "W87,W87,W87,W87, vr, vr")) (match_operand:VOEXTI 2 "vector_merge_operand" " vu, vu, 0, 0, vu, 0")))]- "TARGET_VECTOR"+ "TARGET_VECTOR && !TARGET_XTHEADVECTOR" "v<sz>ext.vf8\t%0,%3%p1" [(set_attr "type" "vext") (set_attr "mode" "<MODE>")
Why do you add these !TARGERT_XTHEADVECRTOR ?
juzhe.zhong@rivai.ai
From: Jun Sha (Joshua)
Date: 2024-01-11 16:46
To: gcc-patches
CC: jim.wilson.gcc; palmer; andrew; philipp.tomsich; jeffreyalaw; christoph.muellner; juzhe.zhong; Jun Sha (Joshua); Jin Ma; Xianmiao Qu
Subject: [PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
This patch only involves the generation of xtheadvector
special load/store instructions and vext instructions.
gcc/ChangeLog:
* config/riscv/riscv-vector-builtins-bases.cc
(class th_loadstore_width): Define new builtin bases.
(class th_extract): Define new builtin bases.
(BASE): Define new builtin bases.
* config/riscv/riscv-vector-builtins-bases.h:
Define new builtin class.
* config/riscv/riscv-vector-builtins-shapes.cc
(struct th_loadstore_width_def): Define new builtin shapes.
(struct th_indexed_loadstore_width_def):
Define new builtin shapes.
(struct th_extract_def): Define new builtin shapes.
(SHAPE): Define new builtin shapes.
* config/riscv/riscv-vector-builtins-shapes.h:
Define new builtin shapes.
* config/riscv/riscv-vector-builtins.cc (DEF_RVV_FUNCTION):
* config/riscv/riscv-vector-builtins.h (enum required_ext):
(struct function_group_info):
* config/riscv/t-riscv: Add thead-vector-builtins-functions.def
* config/riscv/thead-vector.md
(@pred_mov_width<vlmem_op_attr><mode>): Add new patterns.
(*pred_mov_width<vlmem_op_attr><mode>): Likewise.
(@pred_store_width<vlmem_op_attr><mode>): Likewise.
(@pred_strided_load_width<vlmem_op_attr><mode>): Likewise.
(@pred_strided_store_width<vlmem_op_attr><mode>): Likewise.
(@pred_indexed_load_width<vlmem_op_attr><mode>): Likewise.
(@pred_indexed_<vlmem_order_attr>store_width<vlmem_op_attr><mode>):
(@pred_th_extract<mode>): Likewise.
(*pred_th_extract<mode>): Likewise.
* config/riscv/thead-vector-builtins-functions.def: New file.
gcc/testsuite/ChangeLog:
* gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c: New test.
Co-authored-by: Jin Ma <jinma@linux.alibaba.com>
Co-authored-by: Xianmiao Qu <cooper.qu@linux.alibaba.com>
Co-authored-by: Christoph Müllner <christoph.muellner@vrull.eu>
---
.../riscv/riscv-vector-builtins-bases.cc | 139 ++++++++++
.../riscv/riscv-vector-builtins-bases.h | 31 +++
.../riscv/riscv-vector-builtins-shapes.cc | 160 +++++++++++
.../riscv/riscv-vector-builtins-shapes.h | 3 +
gcc/config/riscv/riscv-vector-builtins.cc | 70 +++++
gcc/config/riscv/riscv-vector-builtins.h | 3 +
gcc/config/riscv/t-riscv | 1 +
.../riscv/thead-vector-builtins-functions.def | 39 +++
gcc/config/riscv/thead-vector.md | 253 ++++++++++++++++++
.../riscv/rvv/xtheadvector/vlb-vsb.c | 68 +++++
.../riscv/rvv/xtheadvector/vlbu-vsb.c | 68 +++++
.../riscv/rvv/xtheadvector/vlh-vsh.c | 68 +++++
.../riscv/rvv/xtheadvector/vlhu-vsh.c | 68 +++++
.../riscv/rvv/xtheadvector/vlw-vsw.c | 68 +++++
.../riscv/rvv/xtheadvector/vlwu-vsw.c | 68 +++++
15 files changed, 1107 insertions(+)
create mode 100644 gcc/config/riscv/thead-vector-builtins-functions.def
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c
diff --git a/gcc/config/riscv/riscv-vector-builtins-bases.cc b/gcc/config/riscv/riscv-vector-builtins-bases.cc
index 46f1a1da33e..3eba7943757 100644
--- a/gcc/config/riscv/riscv-vector-builtins-bases.cc
+++ b/gcc/config/riscv/riscv-vector-builtins-bases.cc
@@ -2125,6 +2125,83 @@ public:
}
};
+/* Implements
+ * th.vl(b/h/w)[u].v/th.vs(b/h/w)[u].v/th.vls(b/h/w)[u].v/th.vss(b/h/w)[u].v/
+ * th.vlx(b/h/w)[u].v/th.vs[u]x(b/h/w).v
+ * codegen. */
+template<bool STORE_P, lst_type LST_TYPE, int UNSPEC>
+class th_loadstore_width : public function_base
+{
+public:
+ bool apply_tail_policy_p () const override { return !STORE_P; }
+ bool apply_mask_policy_p () const override { return !STORE_P; }
+
+ unsigned int call_properties (const function_instance &) const override
+ {
+ if (STORE_P)
+ return CP_WRITE_MEMORY;
+ else
+ return CP_READ_MEMORY;
+ }
+
+ bool can_be_overloaded_p (enum predication_type_index pred) const override
+ {
+ if (STORE_P || LST_TYPE == LST_INDEXED)
+ return true;
+ return pred != PRED_TYPE_none;
+ }
+
+ rtx expand (function_expander &e) const override
+ {
+ gcc_assert (TARGET_XTHEADVECTOR);
+ if (LST_TYPE == LST_INDEXED)
+ {
+ if (STORE_P)
+ return e.use_exact_insn (
+ code_for_pred_indexed_store_width (UNSPEC, UNSPEC,
+ e.vector_mode ()));
+ else
+ return e.use_exact_insn (
+ code_for_pred_indexed_load_width (UNSPEC, e.vector_mode ()));
+ }
+ else if (LST_TYPE == LST_STRIDED)
+ {
+ if (STORE_P)
+ return e.use_contiguous_store_insn (
+ code_for_pred_strided_store_width (UNSPEC, e.vector_mode ()));
+ else
+ return e.use_contiguous_load_insn (
+ code_for_pred_strided_load_width (UNSPEC, e.vector_mode ()));
+ }
+ else
+ {
+ if (STORE_P)
+ return e.use_contiguous_store_insn (
+ code_for_pred_store_width (UNSPEC, e.vector_mode ()));
+ else
+ return e.use_contiguous_load_insn (
+ code_for_pred_mov_width (UNSPEC, e.vector_mode ()));
+ }
+ }
+};
+
+/* Implements vext.x.v. */
+class th_extract : public function_base
+{
+public:
+ bool apply_vl_p () const override { return false; }
+ bool apply_tail_policy_p () const override { return false; }
+ bool apply_mask_policy_p () const override { return false; }
+ bool use_mask_predication_p () const override { return false; }
+ bool has_merge_operand_p () const override { return false; }
+
+ rtx expand (function_expander &e) const override
+ {
+ gcc_assert (TARGET_XTHEADVECTOR);
+ return e.use_exact_insn (code_for_pred_th_extract (e.vector_mode ()));
+ }
+};
+
/* Below implements are vector crypto */
/* Implements vandn.[vv,vx] */
class vandn : public function_base
@@ -2587,6 +2664,37 @@ static CONSTEXPR const seg_indexed_load<UNSPEC_ORDERED> vloxseg_obj;
static CONSTEXPR const seg_indexed_store<UNSPEC_UNORDERED> vsuxseg_obj;
static CONSTEXPR const seg_indexed_store<UNSPEC_ORDERED> vsoxseg_obj;
static CONSTEXPR const vlsegff vlsegff_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLB> vlb_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLBU> vlbu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLH> vlh_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLHU> vlhu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLW> vlw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLWU> vlwu_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_UNIT_STRIDE, UNSPEC_TH_VLB> vsb_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_UNIT_STRIDE, UNSPEC_TH_VLH> vsh_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_UNIT_STRIDE, UNSPEC_TH_VLW> vsw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSB> vlsb_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSBU> vlsbu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSH> vlsh_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSHU> vlshu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSW> vlsw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSWU> vlswu_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_STRIDED, UNSPEC_TH_VLSB> vssb_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_STRIDED, UNSPEC_TH_VLSH> vssh_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_STRIDED, UNSPEC_TH_VLSW> vssw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXB> vlxb_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXBU> vlxbu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXH> vlxh_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXHU> vlxhu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXW> vlxw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXWU> vlxwu_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VLXB> vsxb_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VLXH> vsxh_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VLXW> vsxw_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VSUXB> vsuxb_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VSUXH> vsuxh_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VSUXW> vsuxw_obj;
+static CONSTEXPR const th_extract vext_x_v_obj;
/* Crypto Vector */
static CONSTEXPR const vandn vandn_obj;
@@ -2878,6 +2986,37 @@ BASE (vloxseg)
BASE (vsuxseg)
BASE (vsoxseg)
BASE (vlsegff)
+BASE (vlb)
+BASE (vlh)
+BASE (vlw)
+BASE (vlbu)
+BASE (vlhu)
+BASE (vlwu)
+BASE (vsb)
+BASE (vsh)
+BASE (vsw)
+BASE (vlsb)
+BASE (vlsh)
+BASE (vlsw)
+BASE (vlsbu)
+BASE (vlshu)
+BASE (vlswu)
+BASE (vssb)
+BASE (vssh)
+BASE (vssw)
+BASE (vlxb)
+BASE (vlxh)
+BASE (vlxw)
+BASE (vlxbu)
+BASE (vlxhu)
+BASE (vlxwu)
+BASE (vsxb)
+BASE (vsxh)
+BASE (vsxw)
+BASE (vsuxb)
+BASE (vsuxh)
+BASE (vsuxw)
+BASE (vext_x_v)
/* Crypto vector */
BASE (vandn)
BASE (vbrev)
diff --git a/gcc/config/riscv/riscv-vector-builtins-bases.h b/gcc/config/riscv/riscv-vector-builtins-bases.h
index 1122e3801a7..565a0311d2b 100644
--- a/gcc/config/riscv/riscv-vector-builtins-bases.h
+++ b/gcc/config/riscv/riscv-vector-builtins-bases.h
@@ -299,6 +299,37 @@ extern const function_base *const vloxseg;
extern const function_base *const vsuxseg;
extern const function_base *const vsoxseg;
extern const function_base *const vlsegff;
+extern const function_base *const vlb;
+extern const function_base *const vlh;
+extern const function_base *const vlw;
+extern const function_base *const vlbu;
+extern const function_base *const vlhu;
+extern const function_base *const vlwu;
+extern const function_base *const vsb;
+extern const function_base *const vsh;
+extern const function_base *const vsw;
+extern const function_base *const vlsb;
+extern const function_base *const vlsh;
+extern const function_base *const vlsw;
+extern const function_base *const vlsbu;
+extern const function_base *const vlshu;
+extern const function_base *const vlswu;
+extern const function_base *const vssb;
+extern const function_base *const vssh;
+extern const function_base *const vssw;
+extern const function_base *const vlxb;
+extern const function_base *const vlxh;
+extern const function_base *const vlxw;
+extern const function_base *const vlxbu;
+extern const function_base *const vlxhu;
+extern const function_base *const vlxwu;
+extern const function_base *const vsxb;
+extern const function_base *const vsxh;
+extern const function_base *const vsxw;
+extern const function_base *const vsuxb;
+extern const function_base *const vsuxh;
+extern const function_base *const vsuxw;
+extern const function_base *const vext_x_v;
/* Below function_base are Vectro Crypto*/
extern const function_base *const vandn;
extern const function_base *const vbrev;
diff --git a/gcc/config/riscv/riscv-vector-builtins-shapes.cc b/gcc/config/riscv/riscv-vector-builtins-shapes.cc
index 1e4f4d53de6..8e90b17a94b 100644
--- a/gcc/config/riscv/riscv-vector-builtins-shapes.cc
+++ b/gcc/config/riscv/riscv-vector-builtins-shapes.cc
@@ -211,6 +211,146 @@ struct indexed_loadstore_def : public function_shape
}
};
+/* Add one function instance for GROUP, using operand suffix at index OI,
+ mode suffix at index PAIR && bi and predication suffix at index pred_idx. */
+static void
+build_th_loadstore (function_builder &b, const function_group_info &group,
+ unsigned int pred_idx, unsigned int vec_type_idx)
+{
+ auto_vec<tree, 5> argument_types;
+ function_instance function_instance (group.base_name, *group.base,
+ *group.shape,
+ group.ops_infos.types[vec_type_idx],
+ group.preds[pred_idx], &group.ops_infos);
+ tree return_type = group.ops_infos.ret.get_tree_type (
+ group.ops_infos.types[vec_type_idx].index);
+ b.allocate_argument_types (function_instance, argument_types);
+ b.apply_predication (function_instance, return_type, argument_types);
+
+ if (TARGET_XTHEADVECTOR && !check_type (return_type, argument_types))
+ return;
+
+ tree type = builtin_types[group.ops_infos.types[vec_type_idx].index].vector;
+ if (strstr (group.base_name, "l")
+ && strstr (group.base_name, "u")
+ && !TYPE_UNSIGNED (TREE_TYPE (type)))
+ return;
+
+ if (strstr (group.base_name, "l")
+ && !strstr (group.base_name, "u")
+ && TYPE_UNSIGNED (TREE_TYPE (type)))
+ return;
+
+ machine_mode mode = TYPE_MODE (type);
+ int sew = GET_MODE_BITSIZE (GET_MODE_INNER (mode));
+ if (strstr (group.base_name, "h") && sew == 8)
+ return;
+
+ if (strstr (group.base_name, "w") && (sew == 8 || sew ==16))
+ return;
+
+ b.add_overloaded_function (function_instance, *group.shape);
+ b.add_unique_function (function_instance, (*group.shape), return_type,
+ argument_types);
+}
+
+/* th_loadstore_width_def class. */
+struct th_loadstore_width_def : public build_base
+{
+ void build (function_builder &b,
+ const function_group_info &group) const override
+ {
+ for (unsigned int pred_idx = 0; group.preds[pred_idx] != NUM_PRED_TYPES;
+ ++pred_idx)
+ {
+ for (unsigned int vec_type_idx = 0;
+ group.ops_infos.types[vec_type_idx].index != NUM_VECTOR_TYPES;
+ ++vec_type_idx)
+ {
+ build_th_loadstore (b, group, pred_idx, vec_type_idx);
+ }
+ }
+ }
+
+ char *get_name (function_builder &b, const function_instance &instance,
+ bool overloaded_p) const override
+ {
+ /* Return nullptr if it can not be overloaded. */
+ if (overloaded_p && !instance.base->can_be_overloaded_p (instance.pred))
+ return nullptr;
+
+ b.append_name ("__riscv_th_");
+ b.append_name (instance.base_name);
+
+ /* vop_v --> vop_v_<type>. */
+ if (!overloaded_p)
+ {
+ /* vop --> vop_v. */
+ b.append_name (operand_suffixes[instance.op_info->op]);
+ /* vop_v --> vop_v_<type>. */
+ b.append_name (type_suffixes[instance.type.index].vector);
+ }
+
+ /* According to rvv-intrinsic-doc, it does not add "_m" suffix
+ for vop_m C++ overloaded API. */
+ if (overloaded_p && instance.pred == PRED_TYPE_m)
+ return b.finish_name ();
+ b.append_name (predication_suffixes[instance.pred]);
+ return b.finish_name ();
+ }
+};
+
+
+/* th_indexed_loadstore_width_def class. */
+struct th_indexed_loadstore_width_def : public function_shape
+{
+ void build (function_builder &b,
+ const function_group_info &group) const override
+ {
+ for (unsigned int pred_idx = 0; group.preds[pred_idx] != NUM_PRED_TYPES;
+ ++pred_idx)
+ {
+ for (unsigned int vec_type_idx = 0;
+ group.ops_infos.types[vec_type_idx].index != NUM_VECTOR_TYPES;
+ ++vec_type_idx)
+ {
+ tree index_type = group.ops_infos.args[1].get_tree_type (
+ group.ops_infos.types[vec_type_idx].index);
+ if (!index_type)
+ continue;
+ build_th_loadstore (b, group, pred_idx, vec_type_idx);
+ }
+ }
+ }
+
+ char *get_name (function_builder &b, const function_instance &instance,
+ bool overloaded_p) const override
+ {
+
+ /* Return nullptr if it can not be overloaded. */
+ if (overloaded_p && !instance.base->can_be_overloaded_p (instance.pred))
+ return nullptr;
+
+ b.append_name ("__riscv_th_");
+ b.append_name (instance.base_name);
+ /* vop_v --> vop_v_<type>. */
+ if (!overloaded_p)
+ {
+ /* vop --> vop_v. */
+ b.append_name (operand_suffixes[instance.op_info->op]);
+ /* vop_v --> vop_v_<type>. */
+ b.append_name (type_suffixes[instance.type.index].vector);
+ }
+
+ /* According to rvv-intrinsic-doc, it does not add "_m" suffix
+ for vop_m C++ overloaded API. */
+ if (overloaded_p && instance.pred == PRED_TYPE_m)
+ return b.finish_name ();
+ b.append_name (predication_suffixes[instance.pred]);
+ return b.finish_name ();
+ }
+};
+
/* alu_def class. */
struct alu_def : public build_base
{
@@ -632,6 +772,23 @@ struct reduc_alu_def : public build_base
}
};
+/* th_extract_def class. */
+struct th_extract_def : public build_base
+{
+ char *get_name (function_builder &b, const function_instance &instance,
+ bool overloaded_p) const override
+ {
+ b.append_name ("__riscv_th_");
+ b.append_name (instance.base_name);
+
+ if (overloaded_p)
+ return b.finish_name ();
+ b.append_name (type_suffixes[instance.type.index].vector);
+ b.append_name (type_suffixes[instance.type.index].scalar);
+ return b.finish_name ();
+ }
+};
+
/* scalar_move_def class. */
struct scalar_move_def : public build_base
{
@@ -1094,6 +1251,8 @@ SHAPE(vsetvl, vsetvl)
SHAPE(vsetvl, vsetvlmax)
SHAPE(loadstore, loadstore)
SHAPE(indexed_loadstore, indexed_loadstore)
+SHAPE(th_loadstore_width, th_loadstore_width)
+SHAPE(th_indexed_loadstore_width, th_indexed_loadstore_width)
SHAPE(alu, alu)
SHAPE(alu_frm, alu_frm)
SHAPE(widen_alu, widen_alu)
@@ -1106,6 +1265,7 @@ SHAPE(move, move)
SHAPE(mask_alu, mask_alu)
SHAPE(reduc_alu, reduc_alu)
SHAPE(reduc_alu_frm, reduc_alu_frm)
+SHAPE(th_extract, th_extract)
SHAPE(scalar_move, scalar_move)
SHAPE(vundefined, vundefined)
SHAPE(misc, misc)
diff --git a/gcc/config/riscv/riscv-vector-builtins-shapes.h b/gcc/config/riscv/riscv-vector-builtins-shapes.h
index ac2a28ce017..a7624d0fabd 100644
--- a/gcc/config/riscv/riscv-vector-builtins-shapes.h
+++ b/gcc/config/riscv/riscv-vector-builtins-shapes.h
@@ -28,6 +28,8 @@ extern const function_shape *const vsetvl;
extern const function_shape *const vsetvlmax;
extern const function_shape *const loadstore;
extern const function_shape *const indexed_loadstore;
+extern const function_shape *const th_loadstore_width;
+extern const function_shape *const th_indexed_loadstore_width;
extern const function_shape *const alu;
extern const function_shape *const alu_frm;
extern const function_shape *const widen_alu;
@@ -41,6 +43,7 @@ extern const function_shape *const mask_alu;
extern const function_shape *const reduc_alu;
extern const function_shape *const reduc_alu_frm;
extern const function_shape *const scalar_move;
+extern const function_shape *const th_extract;
extern const function_shape *const vundefined;
extern const function_shape *const misc;
extern const function_shape *const vset;
diff --git a/gcc/config/riscv/riscv-vector-builtins.cc b/gcc/config/riscv/riscv-vector-builtins.cc
index 25e0b6e56de..44b9fec1898 100644
--- a/gcc/config/riscv/riscv-vector-builtins.cc
+++ b/gcc/config/riscv/riscv-vector-builtins.cc
@@ -934,6 +934,32 @@ static CONSTEXPR const rvv_arg_type_info ext_vcreate_args[]
= {rvv_arg_type_info (RVV_BASE_vector),
rvv_arg_type_info_end};
+/* A list of args for vector_type func (const scalar_type *, size_t)
+ * function. */
+static CONSTEXPR const rvv_arg_type_info scalar_const_ptr_size_args[]
+ = {rvv_arg_type_info (RVV_BASE_scalar_const_ptr),
+ rvv_arg_type_info (RVV_BASE_size), rvv_arg_type_info_end};
+
+/* A list of args for vector_type func (const scalar_type *, eew8_index_type)
+ * function. */
+static CONSTEXPR const rvv_arg_type_info scalar_const_ptr_index_args[]
+ = {rvv_arg_type_info (RVV_BASE_scalar_const_ptr),
+ rvv_arg_type_info (RVV_BASE_unsigned_vector), rvv_arg_type_info_end};
+
+/* A list of args for void func (scalar_type *, eew8_index_type, vector_type)
+ * function. */
+static CONSTEXPR const rvv_arg_type_info scalar_ptr_index_args[]
+ = {rvv_arg_type_info (RVV_BASE_scalar_ptr),
+ rvv_arg_type_info (RVV_BASE_unsigned_vector),
+ rvv_arg_type_info (RVV_BASE_vector), rvv_arg_type_info_end};
+
+/* A list of args for void func (scalar_type *, size_t, vector_type)
+ * function. */
+static CONSTEXPR const rvv_arg_type_info scalar_ptr_size_args[]
+ = {rvv_arg_type_info (RVV_BASE_scalar_ptr),
+ rvv_arg_type_info (RVV_BASE_size), rvv_arg_type_info (RVV_BASE_vector),
+ rvv_arg_type_info_end};
+
/* A list of none preds that will be registered for intrinsic functions. */
static CONSTEXPR const predication_type_index none_preds[]
= {PRED_TYPE_none, NUM_PRED_TYPES};
@@ -1455,6 +1481,14 @@ static CONSTEXPR const rvv_op_info iu_shift_vvv_ops
rvv_arg_type_info (RVV_BASE_vector), /* Return type */
shift_vv_args /* Args */};
+/* A static operand information for scalar_type func (vector_type, size_t)
+ * function registration. */
+static CONSTEXPR const rvv_op_info iu_x_s_u_ops
+ = {iu_ops, /* Types */
+ OP_TYPE_vx, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_scalar), /* Return type */
+ v_size_args /* Args */};
+
/* A static operand information for vector_type func (vector_type, size_t)
* function registration. */
static CONSTEXPR const rvv_op_info iu_shift_vvx_ops
@@ -2638,6 +2672,38 @@ static CONSTEXPR const rvv_op_info all_v_vcreate_lmul4_x2_ops
rvv_arg_type_info (RVV_BASE_vlmul_ext_x2), /* Return type */
ext_vcreate_args /* Args */};
+/* A static operand information for vector_type func (const scalar_type *,
+ * size_t) function registration. */
+static CONSTEXPR const rvv_op_info all_v_scalar_const_ptr_size_ops
+ = {all_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_size_args /* Args */};
+
+/* A static operand information for void func (scalar_type *, size_t,
+ * vector_type) function registration. */
+static CONSTEXPR const rvv_op_info all_v_scalar_ptr_size_ops
+ = {all_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_void), /* Return type */
+ scalar_ptr_size_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * index_type) function registration. */
+static CONSTEXPR const rvv_op_info all_v_scalar_const_ptr_index_ops
+ = {all_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_index_args /* Args */};
+
+/* A static operand information for void func (scalar_type *, index_type,
+ * vector_type) function registration. */
+static CONSTEXPR const rvv_op_info all_v_scalar_ptr_index_ops
+ = {all_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_void), /* Return type */
+ scalar_ptr_index_args /* Args */};
+
/* A static operand information for vector_type func (vector_type).
Some ins just supports SEW=32, such as crypto vectol Zvkg extension.
* function registration. */
@@ -2816,6 +2882,10 @@ static function_group_info function_groups[] = {
#define DEF_RVV_FUNCTION(NAME, SHAPE, PREDS, OPS_INFO) \
{#NAME, &bases::NAME, &shapes::SHAPE, PREDS, OPS_INFO, REQUIRED_EXTENSIONS},
#include "riscv-vector-builtins-functions.def"
+#undef DEF_RVV_FUNCTION
+#define DEF_RVV_FUNCTION(NAME, SHAPE, PREDS, OPS_INFO) \
+ {#NAME, &bases::NAME, &shapes::SHAPE, PREDS, OPS_INFO, REQUIRED_EXTENSIONS},
+#include "thead-vector-builtins-functions.def"
};
/* The RVV types, with their built-in
diff --git a/gcc/config/riscv/riscv-vector-builtins.h b/gcc/config/riscv/riscv-vector-builtins.h
index 54c8824ff92..a8ee39a3cb2 100644
--- a/gcc/config/riscv/riscv-vector-builtins.h
+++ b/gcc/config/riscv/riscv-vector-builtins.h
@@ -114,6 +114,7 @@ static const unsigned int CP_WRITE_CSR = 1U << 5;
enum required_ext
{
VECTOR_EXT, /* Vector extension */
+ XTHEADVECTOR_EXT, /* XTheadVector extension */
ZVBB_EXT, /* Cryto vector Zvbb sub-ext */
ZVBB_OR_ZVKB_EXT, /* Cryto vector Zvbb or zvkb sub-ext */
ZVBC_EXT, /* Crypto vector Zvbc sub-ext */
@@ -234,6 +235,8 @@ struct function_group_info
{
case VECTOR_EXT:
return TARGET_VECTOR;
+ case XTHEADVECTOR_EXT:
+ return TARGET_XTHEADVECTOR;
case ZVBB_EXT:
return TARGET_ZVBB;
case ZVBB_OR_ZVKB_EXT:
diff --git a/gcc/config/riscv/t-riscv b/gcc/config/riscv/t-riscv
index 32de6b851c1..38494320d8b 100644
--- a/gcc/config/riscv/t-riscv
+++ b/gcc/config/riscv/t-riscv
@@ -1,6 +1,7 @@
RISCV_BUILTINS_H = $(srcdir)/config/riscv/riscv-vector-builtins.h \
$(srcdir)/config/riscv/riscv-vector-builtins.def \
$(srcdir)/config/riscv/riscv-vector-builtins-functions.def \
+ $(srcdir)/config/riscv/thead-vector-builtins-functions.def \
riscv-vector-type-indexer.gen.def
riscv-builtins.o: $(srcdir)/config/riscv/riscv-builtins.cc $(CONFIG_H) \
diff --git a/gcc/config/riscv/thead-vector-builtins-functions.def b/gcc/config/riscv/thead-vector-builtins-functions.def
new file mode 100644
index 00000000000..fd3ba29bae9
--- /dev/null
+++ b/gcc/config/riscv/thead-vector-builtins-functions.def
@@ -0,0 +1,39 @@
+#ifndef DEF_RVV_FUNCTION
+#define DEF_RVV_FUNCTION(NAME, SHAPE, PREDS, OPS_INFO)
+#endif
+
+#define REQUIRED_EXTENSIONS XTHEADVECTOR_EXT
+DEF_RVV_FUNCTION (vlb, th_loadstore_width, full_preds, all_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (vlh, th_loadstore_width, full_preds, all_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (vlw, th_loadstore_width, full_preds, all_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (vlbu, th_loadstore_width, full_preds, all_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (vlhu, th_loadstore_width, full_preds, all_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (vlwu, th_loadstore_width, full_preds, all_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (vsb, th_loadstore_width, none_m_preds, all_v_scalar_ptr_ops)
+DEF_RVV_FUNCTION (vsh, th_loadstore_width, none_m_preds, all_v_scalar_ptr_ops)
+DEF_RVV_FUNCTION (vsw, th_loadstore_width, none_m_preds, all_v_scalar_ptr_ops)
+DEF_RVV_FUNCTION (vlsb, th_loadstore_width, full_preds, all_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (vlsh, th_loadstore_width, full_preds, all_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (vlsw, th_loadstore_width, full_preds, all_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (vlsbu, th_loadstore_width, full_preds, all_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (vlshu, th_loadstore_width, full_preds, all_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (vlswu, th_loadstore_width, full_preds, all_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (vssb, th_loadstore_width, none_m_preds, all_v_scalar_ptr_size_ops)
+DEF_RVV_FUNCTION (vssh, th_loadstore_width, none_m_preds, all_v_scalar_ptr_size_ops)
+DEF_RVV_FUNCTION (vssw, th_loadstore_width, none_m_preds, all_v_scalar_ptr_size_ops)
+DEF_RVV_FUNCTION (vlxb, th_indexed_loadstore_width, full_preds, all_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (vlxh, th_indexed_loadstore_width, full_preds, all_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (vlxw, th_indexed_loadstore_width, full_preds, all_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (vlxbu, th_indexed_loadstore_width, full_preds, all_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (vlxhu, th_indexed_loadstore_width, full_preds, all_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (vlxwu, th_indexed_loadstore_width, full_preds, all_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (vsxb, th_indexed_loadstore_width, none_m_preds, all_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (vsxh, th_indexed_loadstore_width, none_m_preds, all_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (vsxw, th_indexed_loadstore_width, none_m_preds, all_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (vsuxb, th_indexed_loadstore_width, none_m_preds, all_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (vsuxh, th_indexed_loadstore_width, none_m_preds, all_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (vsuxw, th_indexed_loadstore_width, none_m_preds, all_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (vext_x_v, th_extract, none_preds, iu_x_s_u_ops)
+#undef REQUIRED_EXTENSIONS
+
+#undef DEF_RVV_FUNCTION
diff --git a/gcc/config/riscv/thead-vector.md b/gcc/config/riscv/thead-vector.md
index 696b815252d..0f3700d9269 100644
--- a/gcc/config/riscv/thead-vector.md
+++ b/gcc/config/riscv/thead-vector.md
@@ -1,7 +1,95 @@
(define_c_enum "unspec" [
+ UNSPEC_TH_VLB
+ UNSPEC_TH_VLBU
+ UNSPEC_TH_VLH
+ UNSPEC_TH_VLHU
+ UNSPEC_TH_VLW
+ UNSPEC_TH_VLWU
+
+ UNSPEC_TH_VLSB
+ UNSPEC_TH_VLSBU
+ UNSPEC_TH_VLSH
+ UNSPEC_TH_VLSHU
+ UNSPEC_TH_VLSW
+ UNSPEC_TH_VLSWU
+
+ UNSPEC_TH_VLXB
+ UNSPEC_TH_VLXBU
+ UNSPEC_TH_VLXH
+ UNSPEC_TH_VLXHU
+ UNSPEC_TH_VLXW
+ UNSPEC_TH_VLXWU
+
+ UNSPEC_TH_VSUXB
+ UNSPEC_TH_VSUXH
+ UNSPEC_TH_VSUXW
+
UNSPEC_TH_VWLDST
])
+(define_int_iterator UNSPEC_TH_VLMEM_OP [
+ UNSPEC_TH_VLB UNSPEC_TH_VLBU
+ UNSPEC_TH_VLH UNSPEC_TH_VLHU
+ UNSPEC_TH_VLW UNSPEC_TH_VLWU
+])
+
+(define_int_iterator UNSPEC_TH_VLSMEM_OP [
+ UNSPEC_TH_VLSB UNSPEC_TH_VLSBU
+ UNSPEC_TH_VLSH UNSPEC_TH_VLSHU
+ UNSPEC_TH_VLSW UNSPEC_TH_VLSWU
+])
+
+(define_int_iterator UNSPEC_TH_VLXMEM_OP [
+ UNSPEC_TH_VLXB UNSPEC_TH_VLXBU
+ UNSPEC_TH_VLXH UNSPEC_TH_VLXHU
+ UNSPEC_TH_VLXW UNSPEC_TH_VLXWU
+])
+
+(define_int_attr vlmem_op_attr [
+ (UNSPEC_TH_VLB "b") (UNSPEC_TH_VLBU "bu")
+ (UNSPEC_TH_VLH "h") (UNSPEC_TH_VLHU "hu")
+ (UNSPEC_TH_VLW "w") (UNSPEC_TH_VLWU "wu")
+ (UNSPEC_TH_VLSB "b") (UNSPEC_TH_VLSBU "bu")
+ (UNSPEC_TH_VLSH "h") (UNSPEC_TH_VLSHU "hu")
+ (UNSPEC_TH_VLSW "w") (UNSPEC_TH_VLSWU "wu")
+ (UNSPEC_TH_VLXB "b") (UNSPEC_TH_VLXBU "bu")
+ (UNSPEC_TH_VLXH "h") (UNSPEC_TH_VLXHU "hu")
+ (UNSPEC_TH_VLXW "w") (UNSPEC_TH_VLXWU "wu")
+ (UNSPEC_TH_VSUXB "b")
+ (UNSPEC_TH_VSUXH "h")
+ (UNSPEC_TH_VSUXW "w")
+])
+
+(define_int_attr vlmem_order_attr [
+ (UNSPEC_TH_VLXB "")
+ (UNSPEC_TH_VLXH "")
+ (UNSPEC_TH_VLXW "")
+ (UNSPEC_TH_VSUXB "u")
+ (UNSPEC_TH_VSUXH "u")
+ (UNSPEC_TH_VSUXW "u")
+])
+
+(define_int_iterator UNSPEC_TH_VSMEM_OP [
+ UNSPEC_TH_VLB
+ UNSPEC_TH_VLH
+ UNSPEC_TH_VLW
+])
+
+(define_int_iterator UNSPEC_TH_VSSMEM_OP [
+ UNSPEC_TH_VLSB
+ UNSPEC_TH_VLSH
+ UNSPEC_TH_VLSW
+])
+
+(define_int_iterator UNSPEC_TH_VSXMEM_OP [
+ UNSPEC_TH_VLXB
+ UNSPEC_TH_VLXH
+ UNSPEC_TH_VLXW
+ UNSPEC_TH_VSUXB
+ UNSPEC_TH_VSUXH
+ UNSPEC_TH_VSUXW
+])
+
(define_mode_iterator V_VLS_VT [V VLS VT])
(define_mode_iterator V_VB_VLS_VT [V VB VLS VT])
@@ -100,3 +188,168 @@
}
[(set_attr "type" "vldm,vstm,vmalu,vmalu,vmalu")
(set_attr "mode" "<MODE>")])
+
+(define_expand "@pred_mov_width<vlmem_op_attr><mode>"
+ [(set (match_operand:V_VLS 0 "nonimmediate_operand")
+ (if_then_else:V_VLS
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand")
+ (match_operand 4 "vector_length_operand")
+ (match_operand 5 "const_int_operand")
+ (match_operand 6 "const_int_operand")
+ (match_operand 7 "const_int_operand")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLMEM_OP)
+ (match_operand:V_VLS 3 "vector_move_operand")
+ (match_operand:V_VLS 2 "vector_merge_operand")))]
+ "TARGET_XTHEADVECTOR"
+ {})
+
+(define_insn_and_split "*pred_mov_width<vlmem_op_attr><mode>"
+ [(set (match_operand:V_VLS 0 "nonimmediate_operand" "=vr, vr, vd, m, vr, vr")
+ (if_then_else:V_VLS
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1, Wc1, vm, vmWc1, Wc1, Wc1")
+ (match_operand 4 "vector_length_operand" " rK, rK, rK, rK, rK, rK")
+ (match_operand 5 "const_int_operand" " i, i, i, i, i, i")
+ (match_operand 6 "const_int_operand" " i, i, i, i, i, i")
+ (match_operand 7 "const_int_operand" " i, i, i, i, i, i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLMEM_OP)
+ (match_operand:V_VLS 3 "reg_or_mem_operand" " m, m, m, vr, vr, vr")
+ (match_operand:V_VLS 2 "vector_merge_operand" " 0, vu, vu, vu, vu, 0")))]
+ "(TARGET_XTHEADVECTOR
+ && (register_operand (operands[0], <MODE>mode)
+ || register_operand (operands[3], <MODE>mode)))"
+ "@
+ vl<vlmem_op_attr>.v\t%0,%3%p1
+ vl<vlmem_op_attr>.v\t%0,%3
+ vl<vlmem_op_attr>.v\t%0,%3,%1.t
+ vs<vlmem_op_attr>.v\t%3,%0%p1
+ vmv.v.v\t%0,%3
+ vmv.v.v\t%0,%3"
+ "&& register_operand (operands[0], <MODE>mode)
+ && register_operand (operands[3], <MODE>mode)
+ && satisfies_constraint_vu (operands[2])
+ && INTVAL (operands[7]) == riscv_vector::VLMAX"
+ [(set (match_dup 0) (match_dup 3))]
+ ""
+ [(set_attr "type" "vlde,vlde,vlde,vste,vimov,vimov")
+ (set_attr "mode" "<MODE>")])
+
+(define_insn "@pred_store_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "memory_operand" "+m")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1")
+ (match_operand 3 "vector_length_operand" " rK")
+ (match_operand 4 "const_int_operand" " i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VSMEM_OP)
+ (match_operand:VI 2 "register_operand" " vr")
+ (match_dup 0)))]
+ "TARGET_XTHEADVECTOR"
+ "vs<vlmem_op_attr>.v\t%2,%0%p1"
+ [(set_attr "type" "vste")
+ (set_attr "mode" "<MODE>")
+ (set (attr "avl_type_idx") (const_int 4))
+ (set_attr "vl_op_idx" "3")])
+
+(define_insn "@pred_strided_load_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "register_operand" "=vr, vr, vd")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1, Wc1, vm")
+ (match_operand 5 "vector_length_operand" " rK, rK, rK")
+ (match_operand 6 "const_int_operand" " i, i, i")
+ (match_operand 7 "const_int_operand" " i, i, i")
+ (match_operand 8 "const_int_operand" " i, i, i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLSMEM_OP)
+ (unspec:VI
+ [(match_operand:VI 3 "memory_operand" " m, m, m")
+ (match_operand 4 "pmode_reg_or_0_operand" " rJ, rJ, rJ")] UNSPEC_TH_VLSMEM_OP)
+ (match_operand:VI 2 "vector_merge_operand" " 0, vu, vu")))]
+ "TARGET_XTHEADVECTOR"
+ "vls<vlmem_op_attr>.v\t%0,%3,%z4%p1"
+ [(set_attr "type" "vlds")
+ (set_attr "mode" "<MODE>")])
+
+(define_insn "@pred_strided_store_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "memory_operand" "+m")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1")
+ (match_operand 4 "vector_length_operand" " rK")
+ (match_operand 5 "const_int_operand" " i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VSSMEM_OP)
+ (unspec:VI
+ [(match_operand 2 "pmode_reg_or_0_operand" " rJ")
+ (match_operand:VI 3 "register_operand" " vr")] UNSPEC_TH_VSSMEM_OP)
+ (match_dup 0)))]
+ "TARGET_XTHEADVECTOR"
+ "vss<vlmem_op_attr>.v\t%3,%0,%z2%p1"
+ [(set_attr "type" "vsts")
+ (set_attr "mode" "<MODE>")
+ (set (attr "avl_type_idx") (const_int 5))])
+
+(define_insn "@pred_indexed_load_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "register_operand" "=vd, vr,vd, vr")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" " vm,Wc1,vm,Wc1")
+ (match_operand 5 "vector_length_operand" " rK, rK,rK, rK")
+ (match_operand 6 "const_int_operand" " i, i, i, i")
+ (match_operand 7 "const_int_operand" " i, i, i, i")
+ (match_operand 8 "const_int_operand" " i, i, i, i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLXMEM_OP)
+ (unspec:VI
+ [(match_operand 3 "pmode_reg_or_0_operand" " rJ, rJ,rJ, rJ")
+ (mem:BLK (scratch))
+ (match_operand:VI 4 "register_operand" " vr, vr,vr, vr")] UNSPEC_TH_VLXMEM_OP)
+ (match_operand:VI 2 "vector_merge_operand" " vu, vu, 0, 0")))]
+ "TARGET_XTHEADVECTOR"
+ "vlx<vlmem_op_attr>.v\t%0,(%z3),%4%p1"
+ [(set_attr "type" "vldux")
+ (set_attr "mode" "<MODE>")])
+
+(define_insn "@pred_indexed_<vlmem_order_attr>store_width<vlmem_op_attr><mode>"
+ [(set (mem:BLK (scratch))
+ (unspec:BLK
+ [(unspec:<VM>
+ [(match_operand:<VM> 0 "vector_mask_operand" "vmWc1")
+ (match_operand 4 "vector_length_operand" " rK")
+ (match_operand 5 "const_int_operand" " i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VSXMEM_OP)
+ (match_operand 1 "pmode_reg_or_0_operand" " rJ")
+ (match_operand:VI 2 "register_operand" " vr")
+ (match_operand:VI 3 "register_operand" " vr")] UNSPEC_TH_VSXMEM_OP))]
+ "TARGET_XTHEADVECTOR"
+ "vs<vlmem_order_attr>x<vlmem_op_attr>.v\t%3,(%z1),%2%p0"
+ [(set_attr "type" "vstux")
+ (set_attr "mode" "<MODE>")])
+
+(define_expand "@pred_th_extract<mode>"
+ [(set (match_operand:<VEL> 0 "register_operand")
+ (unspec:<VEL>
+ [(vec_select:<VEL>
+ (match_operand:V_VLSI 1 "register_operand")
+ (parallel [(match_operand:DI 2 "register_operand" "r")]))
+ (reg:SI VTYPE_REGNUM)] UNSPEC_VPREDICATE))]
+ "TARGET_XTHEADVECTOR"
+{})
+
+(define_insn "*pred_th_extract<mode>"
+ [(set (match_operand:<VEL> 0 "register_operand" "=r")
+ (unspec:<VEL>
+ [(vec_select:<VEL>
+ (match_operand:V_VLSI 1 "register_operand" "vr")
+ (parallel [(match_operand:DI 2 "register_operand" "r")]))
+ (reg:SI VTYPE_REGNUM)] UNSPEC_VPREDICATE))]
+ "TARGET_XTHEADVECTOR"
+ "vext.x.v\t%0,%1,%2"
+ [(set_attr "type" "vimovvx")
+ (set_attr "mode" "<MODE>")])
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c
new file mode 100644
index 00000000000..3c12c124597
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_th_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+
+** th.vadd\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+
+** th.vsb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out)
+{
+ vint32m1_t v = __riscv_th_vlb_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlb_v_i32m1_tu (v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vv_i32m1 (v2, v2, 4);
+ vint32m1_t v4 = __riscv_vadd_vv_i32m1_tu (v3, v2, v2, 4);
+ __riscv_th_vsb_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlb.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+
+** th.vadd\.vv\tv[1-9][0-9]?,\s*v[0-9]+,\s*v[0-9]+,\s*v0.t
+** th.vsb.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlb_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlb_v_i32m1_m (mask, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vv_i32m1 (v2, v2, 4);
+ vint32m1_t v4 = __riscv_vadd_vv_i32m1_m (mask, v3, v3, 4);
+ __riscv_th_vsb_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlb.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+
+** th.vadd\.vv\tv[1-9][0-9]?,\s*v[0-9]+,\s*v[0-9]+,\s*v0.t
+** th.vsb.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlb_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlb_v_i32m1_tumu (mask, v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vv_i32m1 (v2, v2, 4);
+ vint32m1_t v4 = __riscv_vadd_vv_i32m1_tumu (mask, v3, v2, v2, 4);
+ __riscv_th_vsb_v_i32m1 (out, v4, 4);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c
new file mode 100644
index 00000000000..30bef369375
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_th_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlbu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlbu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vsb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, uint32_t x)
+{
+ vuint32m1_t v = __riscv_th_vlbu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlbu_v_u32m1_tu (v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tu (v3, v2, -16, 4);
+ __riscv_th_vsb_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlbu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsb.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlbu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlbu_v_u32m1_m (mask, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_m (mask, v3, -16, 4);
+ __riscv_th_vsb_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlbu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlbu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsb.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlbu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlbu_v_u32m1_tumu (mask, v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tumu (mask, v3, v2, -16, 4);
+ __riscv_th_vsb_v_u32m1 (out, v4, 4);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c
new file mode 100644
index 00000000000..3c8b5ccc16b
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_th_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vsh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, int32_t x)
+{
+ vint32m1_t v = __riscv_th_vlh_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlh_v_i32m1_tu (v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, -16, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_tu (v3, v2, -16, 4);
+ __riscv_th_vsh_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlh.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsh.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, int32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlh_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlh_v_i32m1_m (mask, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, -16, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_m (mask, v3, -16, 4);
+ __riscv_th_vsh_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlh.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsh.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, int32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlh_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlh_v_i32m1_tumu (mask, v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, -16, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_tumu (mask, v3, v2, -16, 4);
+ __riscv_th_vsh_v_i32m1 (out, v4, 4);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c
new file mode 100644
index 00000000000..b7c00404f18
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_th_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlhu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlhu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vsh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, uint32_t x)
+{
+ vuint32m1_t v = __riscv_th_vlhu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlhu_v_u32m1_tu (v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tu (v3, v2, -16, 4);
+ __riscv_th_vsh_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlhu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsh.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlhu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlhu_v_u32m1_m (mask, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_m (mask, v3, -16, 4);
+ __riscv_th_vsh_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlhu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlhu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsh.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlhu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlhu_v_u32m1_tumu (mask, v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tumu (mask, v3, v2, -16, 4);
+ __riscv_th_vsh_v_u32m1 (out, v4, 4);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c
new file mode 100644
index 00000000000..17a53012acf
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_th_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vx\tv[0-9]+,\s*v[0-9]+,\s*[a-x0-9]+
+** th.vadd\.vx\tv[0-9]+,\s*v[0-9]+,\s*[a-x0-9]+
+** th.vsw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, int32_t x)
+{
+ vint32m1_t v = __riscv_th_vlw_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlw_v_i32m1_tu (v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, x, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_tu (v3, v2, x, 4);
+ __riscv_th_vsw_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlw.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vx\tv[0-9]+,\s*v[0-9]+,\s*[a-x0-9]+
+** th.vadd\.vx\tv[1-9][0-9]?,\s*v[0-9]+,\s*[a-x0-9]+,\s*v0.t
+** th.vsw.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, int32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlw_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlw_v_i32m1_m (mask, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, x, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_m (mask, v3, x, 4);
+ __riscv_th_vsw_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlw.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vx\tv[0-9]+,\s*v[0-9]+,\s*[a-x0-9]+
+** th.vadd\.vx\tv[1-9][0-9]?,\s*v[0-9]+,\s*[a-x0-9]+,\s*v0.t
+** th.vsw.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, int32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlw_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlw_v_i32m1_tumu (mask, v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, x, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_tumu (mask, v3, v2, x, 4);
+ __riscv_th_vsw_v_i32m1 (out, v4, 4);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c
new file mode 100644
index 00000000000..b187cfc852b
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_th_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlwu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlwu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vsw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, uint32_t x)
+{
+ vuint32m1_t v = __riscv_th_vlwu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlwu_v_u32m1_tu (v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tu (v3, v2, -16, 4);
+ __riscv_th_vsw_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlwu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsw.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlwu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlwu_v_u32m1_m (mask, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_m (mask, v3, -16, 4);
+ __riscv_th_vsw_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlwu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlwu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsw.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlwu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlwu_v_u32m1_tumu (mask, v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tumu (mask, v3, v2, -16, 4);
+ __riscv_th_vsw_v_u32m1 (out, v4, 4);
+}
--
2.17.1
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re：Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
2024-01-11 9:17 Re：[PATCH " juzhe.zhong
@ 2024-01-11 9:21 ` joshua
2024-01-11 9:26 ` joshua
1 sibling, 0 replies; 17+ messages in thread
From: joshua @ 2024-01-11 9:21 UTC (permalink / raw)
To: juzhe.zhong, gcc-patches
Cc: Jim Wilson, palmer, andrew, philipp.tomsich, jeffreyalaw,
christoph.muellner, jinma, cooper.qu
"I didn't see theadvector-specific extension patterns. Could you show me?"
They are all in the file thead-vector.md.
For the sext/zext issue, perhaps I need some time to reproduce that optimization,
but I can clearly remember it is related to vwmul.
------------------------------------------------------------------
发件人：juzhe.zhong@rivai.ai <juzhe.zhong@rivai.ai>
发送时间：2024年1月11日(星期四) 17:17
收件人："cooper.joshua"<cooper.joshua@linux.alibaba.com>; "gcc-patches"<gcc-patches@gcc.gnu.org>
抄 送：Jim Wilson<jim.wilson.gcc@gmail.com>; palmer<palmer@dabbelt.com>; andrew<andrew@sifive.com>; "philipp.tomsich"<philipp.tomsich@vrull.eu>; jeffreyalaw<jeffreyalaw@gmail.com>; "christoph.muellner"<christoph.muellner@vrull.eu>; jinma<jinma@linux.alibaba.com>; "cooper.qu"<cooper.qu@linux.alibaba.com>
主 题：Re: Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
You mean which pattern optimized sext/vzext pattern?
I didn't see theadvector-specific extension patterns. Could you show me?
juzhe.zhong@rivai.ai
发件人： joshua
发送时间： 2024-01-11 17:14
收件人： juzhe.zhong@rivai.ai; gcc-patches
抄送： Jim Wilson; palmer; andrew; philipp.tomsich; jeffreyalaw; christoph.muellner; jinma; cooper.qu
主题： Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
To be specific, in CSE pass, the initial pattern will be optimized into the sext/zext pattern.
------------------------------------------------------------------
发件人：joshua <cooper.joshua@linux.alibaba.com>
发送时间：2024年1月11日(星期四) 17:11
收件人："juzhe.zhong@rivai.ai"<juzhe.zhong@rivai.ai>; "gcc-patches"<gcc-patches@gcc.gnu.org>
抄 送：Jim Wilson<jim.wilson.gcc@gmail.com>; palmer<palmer@dabbelt.com>; andrew<andrew@sifive.com>; "philipp.tomsich"<philipp.tomsich@vrull.eu>; jeffreyalaw<jeffreyalaw@gmail.com>; "christoph.muellner"<christoph.muellner@vrull.eu>; jinma<jinma@linux.alibaba.com>; "cooper.qu"<cooper.qu@linux.alibaba.com>
主 题：Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
sext/zext will be generated in O2 even without corresponding intrinsics.
------------------------------------------------------------------
发件人：juzhe.zhong@rivai.ai <juzhe.zhong@rivai.ai>
发送时间：2024年1月11日(星期四) 17:07
收件人："cooper.joshua"<cooper.joshua@linux.alibaba.com>; "gcc-patches"<gcc-patches@gcc.gnu.org>
抄 送：Jim Wilson<jim.wilson.gcc@gmail.com>; palmer<palmer@dabbelt.com>; andrew<andrew@sifive.com>; "philipp.tomsich"<philipp.tomsich@vrull.eu>; jeffreyalaw<jeffreyalaw@gmail.com>; "christoph.muellner"<christoph.muellner@vrull.eu>; "cooper.joshua"<cooper.joshua@linux.alibaba.com>; jinma<jinma@linux.alibaba.com>; "cooper.qu"<cooper.qu@linux.alibaba.com>
主 题：Re: [PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
enum required_ext
{
VECTOR_EXT, /* Vector extension */
+ XTHEADVECTOR_EXT, /* XTheadVector extension */
ZVBB_EXT, /* Cryto vector Zvbb sub-ext */
ZVBB_OR_ZVKB_EXT, /* Cryto vector Zvbb or zvkb sub-ext */
ZVBC_EXT, /* Crypto vector Zvbc sub-ext */
Add theadvector to the end of the enum.
+ case XTHEADVECTOR_EXT:
+ return TARGET_XTHEADVECTOR;
Same.
+ "&& register_operand (operands[0], <MODE>mode)
+ && register_operand (operands[3], <MODE>mode)
+ && satisfies_constraint_vu (operands[2])
+ && INTVAL (operands[7]) == riscv_vector::VLMAX"
You can use whole_reg_to_reg_move_p
Btw, I review again : RISC-V: Handle differences between XTheadvector and Vector
(any_extend:VWEXTI (match_operand:<V_DOUBLE_TRUNC> 3 "register_operand" "W21,W21,W21,W21,W42,W42,W42,W42,W84,W84,W84,W84, vr, vr")) (match_operand:VWEXTI 2 "vector_merge_operand" " vu, vu, 0, 0, vu, vu, 0, 0, vu, vu, 0, 0, vu, 0")))]- "TARGET_VECTOR"+ "TARGET_VECTOR && !TARGET_XTHEADVECTOR" "v<sz>ext.vf2\t%0,%3%p1" [(set_attr "type" "vext") (set_attr "mode" "<MODE>")@@ -3713,7 +3744,7 @@ (any_extend:VQEXTI (match_operand:<V_QUAD_TRUNC> 3 "register_operand" "W43,W43,W43,W43,W86,W86,W86,W86, vr, vr")) (match_operand:VQEXTI 2 "vector_merge_operand" " vu, vu, 0, 0, vu, vu, 0, 0, vu, 0")))]- "TARGET_VECTOR"+ "TARGET_VECTOR && !TARGET_XTHEADVECTOR" "v<sz>ext.vf4\t%0,%3%p1" [(set_attr "type" "vext") (set_attr "mode" "<MODE>")@@ -3734,7 +3765,7 @@ (any_extend:VOEXTI (match_operand:<V_OCT_TRUNC> 3 "register_operand" "W87,W87,W87,W87, vr, vr")) (match_operand:VOEXTI 2 "vector_merge_operand" " vu, vu, 0, 0, vu, 0")))]- "TARGET_VECTOR"+ "TARGET_VECTOR && !TARGET_XTHEADVECTOR" "v<sz>ext.vf8\t%0,%3%p1" [(set_attr "type" "vext") (set_attr "mode" "<MODE>")
Why do you add these !TARGERT_XTHEADVECRTOR ?
juzhe.zhong@rivai.ai
From: Jun Sha (Joshua)
Date: 2024-01-11 16:46
To: gcc-patches
CC: jim.wilson.gcc; palmer; andrew; philipp.tomsich; jeffreyalaw; christoph.muellner; juzhe.zhong; Jun Sha (Joshua); Jin Ma; Xianmiao Qu
Subject: [PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
This patch only involves the generation of xtheadvector
special load/store instructions and vext instructions.
gcc/ChangeLog:
* config/riscv/riscv-vector-builtins-bases.cc
(class th_loadstore_width): Define new builtin bases.
(class th_extract): Define new builtin bases.
(BASE): Define new builtin bases.
* config/riscv/riscv-vector-builtins-bases.h:
Define new builtin class.
* config/riscv/riscv-vector-builtins-shapes.cc
(struct th_loadstore_width_def): Define new builtin shapes.
(struct th_indexed_loadstore_width_def):
Define new builtin shapes.
(struct th_extract_def): Define new builtin shapes.
(SHAPE): Define new builtin shapes.
* config/riscv/riscv-vector-builtins-shapes.h:
Define new builtin shapes.
* config/riscv/riscv-vector-builtins.cc (DEF_RVV_FUNCTION):
* config/riscv/riscv-vector-builtins.h (enum required_ext):
(struct function_group_info):
* config/riscv/t-riscv: Add thead-vector-builtins-functions.def
* config/riscv/thead-vector.md
(@pred_mov_width<vlmem_op_attr><mode>): Add new patterns.
(*pred_mov_width<vlmem_op_attr><mode>): Likewise.
(@pred_store_width<vlmem_op_attr><mode>): Likewise.
(@pred_strided_load_width<vlmem_op_attr><mode>): Likewise.
(@pred_strided_store_width<vlmem_op_attr><mode>): Likewise.
(@pred_indexed_load_width<vlmem_op_attr><mode>): Likewise.
(@pred_indexed_<vlmem_order_attr>store_width<vlmem_op_attr><mode>):
(@pred_th_extract<mode>): Likewise.
(*pred_th_extract<mode>): Likewise.
* config/riscv/thead-vector-builtins-functions.def: New file.
gcc/testsuite/ChangeLog:
* gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c: New test.
Co-authored-by: Jin Ma <jinma@linux.alibaba.com>
Co-authored-by: Xianmiao Qu <cooper.qu@linux.alibaba.com>
Co-authored-by: Christoph Müllner <christoph.muellner@vrull.eu>
---
.../riscv/riscv-vector-builtins-bases.cc | 139 ++++++++++
.../riscv/riscv-vector-builtins-bases.h | 31 +++
.../riscv/riscv-vector-builtins-shapes.cc | 160 +++++++++++
.../riscv/riscv-vector-builtins-shapes.h | 3 +
gcc/config/riscv/riscv-vector-builtins.cc | 70 +++++
gcc/config/riscv/riscv-vector-builtins.h | 3 +
gcc/config/riscv/t-riscv | 1 +
.../riscv/thead-vector-builtins-functions.def | 39 +++
gcc/config/riscv/thead-vector.md | 253 ++++++++++++++++++
.../riscv/rvv/xtheadvector/vlb-vsb.c | 68 +++++
.../riscv/rvv/xtheadvector/vlbu-vsb.c | 68 +++++
.../riscv/rvv/xtheadvector/vlh-vsh.c | 68 +++++
.../riscv/rvv/xtheadvector/vlhu-vsh.c | 68 +++++
.../riscv/rvv/xtheadvector/vlw-vsw.c | 68 +++++
.../riscv/rvv/xtheadvector/vlwu-vsw.c | 68 +++++
15 files changed, 1107 insertions(+)
create mode 100644 gcc/config/riscv/thead-vector-builtins-functions.def
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c
diff --git a/gcc/config/riscv/riscv-vector-builtins-bases.cc b/gcc/config/riscv/riscv-vector-builtins-bases.cc
index 46f1a1da33e..3eba7943757 100644
--- a/gcc/config/riscv/riscv-vector-builtins-bases.cc
+++ b/gcc/config/riscv/riscv-vector-builtins-bases.cc
@@ -2125,6 +2125,83 @@ public:
}
};
+/* Implements
+ * th.vl(b/h/w)[u].v/th.vs(b/h/w)[u].v/th.vls(b/h/w)[u].v/th.vss(b/h/w)[u].v/
+ * th.vlx(b/h/w)[u].v/th.vs[u]x(b/h/w).v
+ * codegen. */
+template<bool STORE_P, lst_type LST_TYPE, int UNSPEC>
+class th_loadstore_width : public function_base
+{
+public:
+ bool apply_tail_policy_p () const override { return !STORE_P; }
+ bool apply_mask_policy_p () const override { return !STORE_P; }
+
+ unsigned int call_properties (const function_instance &) const override
+ {
+ if (STORE_P)
+ return CP_WRITE_MEMORY;
+ else
+ return CP_READ_MEMORY;
+ }
+
+ bool can_be_overloaded_p (enum predication_type_index pred) const override
+ {
+ if (STORE_P || LST_TYPE == LST_INDEXED)
+ return true;
+ return pred != PRED_TYPE_none;
+ }
+
+ rtx expand (function_expander &e) const override
+ {
+ gcc_assert (TARGET_XTHEADVECTOR);
+ if (LST_TYPE == LST_INDEXED)
+ {
+ if (STORE_P)
+ return e.use_exact_insn (
+ code_for_pred_indexed_store_width (UNSPEC, UNSPEC,
+ e.vector_mode ()));
+ else
+ return e.use_exact_insn (
+ code_for_pred_indexed_load_width (UNSPEC, e.vector_mode ()));
+ }
+ else if (LST_TYPE == LST_STRIDED)
+ {
+ if (STORE_P)
+ return e.use_contiguous_store_insn (
+ code_for_pred_strided_store_width (UNSPEC, e.vector_mode ()));
+ else
+ return e.use_contiguous_load_insn (
+ code_for_pred_strided_load_width (UNSPEC, e.vector_mode ()));
+ }
+ else
+ {
+ if (STORE_P)
+ return e.use_contiguous_store_insn (
+ code_for_pred_store_width (UNSPEC, e.vector_mode ()));
+ else
+ return e.use_contiguous_load_insn (
+ code_for_pred_mov_width (UNSPEC, e.vector_mode ()));
+ }
+ }
+};
+
+/* Implements vext.x.v. */
+class th_extract : public function_base
+{
+public:
+ bool apply_vl_p () const override { return false; }
+ bool apply_tail_policy_p () const override { return false; }
+ bool apply_mask_policy_p () const override { return false; }
+ bool use_mask_predication_p () const override { return false; }
+ bool has_merge_operand_p () const override { return false; }
+
+ rtx expand (function_expander &e) const override
+ {
+ gcc_assert (TARGET_XTHEADVECTOR);
+ return e.use_exact_insn (code_for_pred_th_extract (e.vector_mode ()));
+ }
+};
+
/* Below implements are vector crypto */
/* Implements vandn.[vv,vx] */
class vandn : public function_base
@@ -2587,6 +2664,37 @@ static CONSTEXPR const seg_indexed_load<UNSPEC_ORDERED> vloxseg_obj;
static CONSTEXPR const seg_indexed_store<UNSPEC_UNORDERED> vsuxseg_obj;
static CONSTEXPR const seg_indexed_store<UNSPEC_ORDERED> vsoxseg_obj;
static CONSTEXPR const vlsegff vlsegff_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLB> vlb_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLBU> vlbu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLH> vlh_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLHU> vlhu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLW> vlw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLWU> vlwu_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_UNIT_STRIDE, UNSPEC_TH_VLB> vsb_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_UNIT_STRIDE, UNSPEC_TH_VLH> vsh_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_UNIT_STRIDE, UNSPEC_TH_VLW> vsw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSB> vlsb_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSBU> vlsbu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSH> vlsh_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSHU> vlshu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSW> vlsw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSWU> vlswu_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_STRIDED, UNSPEC_TH_VLSB> vssb_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_STRIDED, UNSPEC_TH_VLSH> vssh_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_STRIDED, UNSPEC_TH_VLSW> vssw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXB> vlxb_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXBU> vlxbu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXH> vlxh_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXHU> vlxhu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXW> vlxw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXWU> vlxwu_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VLXB> vsxb_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VLXH> vsxh_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VLXW> vsxw_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VSUXB> vsuxb_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VSUXH> vsuxh_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VSUXW> vsuxw_obj;
+static CONSTEXPR const th_extract vext_x_v_obj;
/* Crypto Vector */
static CONSTEXPR const vandn vandn_obj;
@@ -2878,6 +2986,37 @@ BASE (vloxseg)
BASE (vsuxseg)
BASE (vsoxseg)
BASE (vlsegff)
+BASE (vlb)
+BASE (vlh)
+BASE (vlw)
+BASE (vlbu)
+BASE (vlhu)
+BASE (vlwu)
+BASE (vsb)
+BASE (vsh)
+BASE (vsw)
+BASE (vlsb)
+BASE (vlsh)
+BASE (vlsw)
+BASE (vlsbu)
+BASE (vlshu)
+BASE (vlswu)
+BASE (vssb)
+BASE (vssh)
+BASE (vssw)
+BASE (vlxb)
+BASE (vlxh)
+BASE (vlxw)
+BASE (vlxbu)
+BASE (vlxhu)
+BASE (vlxwu)
+BASE (vsxb)
+BASE (vsxh)
+BASE (vsxw)
+BASE (vsuxb)
+BASE (vsuxh)
+BASE (vsuxw)
+BASE (vext_x_v)
/* Crypto vector */
BASE (vandn)
BASE (vbrev)
diff --git a/gcc/config/riscv/riscv-vector-builtins-bases.h b/gcc/config/riscv/riscv-vector-builtins-bases.h
index 1122e3801a7..565a0311d2b 100644
--- a/gcc/config/riscv/riscv-vector-builtins-bases.h
+++ b/gcc/config/riscv/riscv-vector-builtins-bases.h
@@ -299,6 +299,37 @@ extern const function_base *const vloxseg;
extern const function_base *const vsuxseg;
extern const function_base *const vsoxseg;
extern const function_base *const vlsegff;
+extern const function_base *const vlb;
+extern const function_base *const vlh;
+extern const function_base *const vlw;
+extern const function_base *const vlbu;
+extern const function_base *const vlhu;
+extern const function_base *const vlwu;
+extern const function_base *const vsb;
+extern const function_base *const vsh;
+extern const function_base *const vsw;
+extern const function_base *const vlsb;
+extern const function_base *const vlsh;
+extern const function_base *const vlsw;
+extern const function_base *const vlsbu;
+extern const function_base *const vlshu;
+extern const function_base *const vlswu;
+extern const function_base *const vssb;
+extern const function_base *const vssh;
+extern const function_base *const vssw;
+extern const function_base *const vlxb;
+extern const function_base *const vlxh;
+extern const function_base *const vlxw;
+extern const function_base *const vlxbu;
+extern const function_base *const vlxhu;
+extern const function_base *const vlxwu;
+extern const function_base *const vsxb;
+extern const function_base *const vsxh;
+extern const function_base *const vsxw;
+extern const function_base *const vsuxb;
+extern const function_base *const vsuxh;
+extern const function_base *const vsuxw;
+extern const function_base *const vext_x_v;
/* Below function_base are Vectro Crypto*/
extern const function_base *const vandn;
extern const function_base *const vbrev;
diff --git a/gcc/config/riscv/riscv-vector-builtins-shapes.cc b/gcc/config/riscv/riscv-vector-builtins-shapes.cc
index 1e4f4d53de6..8e90b17a94b 100644
--- a/gcc/config/riscv/riscv-vector-builtins-shapes.cc
+++ b/gcc/config/riscv/riscv-vector-builtins-shapes.cc
@@ -211,6 +211,146 @@ struct indexed_loadstore_def : public function_shape
}
};
+/* Add one function instance for GROUP, using operand suffix at index OI,
+ mode suffix at index PAIR && bi and predication suffix at index pred_idx. */
+static void
+build_th_loadstore (function_builder &b, const function_group_info &group,
+ unsigned int pred_idx, unsigned int vec_type_idx)
+{
+ auto_vec<tree, 5> argument_types;
+ function_instance function_instance (group.base_name, *group.base,
+ *group.shape,
+ group.ops_infos.types[vec_type_idx],
+ group.preds[pred_idx], &group.ops_infos);
+ tree return_type = group.ops_infos.ret.get_tree_type (
+ group.ops_infos.types[vec_type_idx].index);
+ b.allocate_argument_types (function_instance, argument_types);
+ b.apply_predication (function_instance, return_type, argument_types);
+
+ if (TARGET_XTHEADVECTOR && !check_type (return_type, argument_types))
+ return;
+
+ tree type = builtin_types[group.ops_infos.types[vec_type_idx].index].vector;
+ if (strstr (group.base_name, "l")
+ && strstr (group.base_name, "u")
+ && !TYPE_UNSIGNED (TREE_TYPE (type)))
+ return;
+
+ if (strstr (group.base_name, "l")
+ && !strstr (group.base_name, "u")
+ && TYPE_UNSIGNED (TREE_TYPE (type)))
+ return;
+
+ machine_mode mode = TYPE_MODE (type);
+ int sew = GET_MODE_BITSIZE (GET_MODE_INNER (mode));
+ if (strstr (group.base_name, "h") && sew == 8)
+ return;
+
+ if (strstr (group.base_name, "w") && (sew == 8 || sew ==16))
+ return;
+
+ b.add_overloaded_function (function_instance, *group.shape);
+ b.add_unique_function (function_instance, (*group.shape), return_type,
+ argument_types);
+}
+
+/* th_loadstore_width_def class. */
+struct th_loadstore_width_def : public build_base
+{
+ void build (function_builder &b,
+ const function_group_info &group) const override
+ {
+ for (unsigned int pred_idx = 0; group.preds[pred_idx] != NUM_PRED_TYPES;
+ ++pred_idx)
+ {
+ for (unsigned int vec_type_idx = 0;
+ group.ops_infos.types[vec_type_idx].index != NUM_VECTOR_TYPES;
+ ++vec_type_idx)
+ {
+ build_th_loadstore (b, group, pred_idx, vec_type_idx);
+ }
+ }
+ }
+
+ char *get_name (function_builder &b, const function_instance &instance,
+ bool overloaded_p) const override
+ {
+ /* Return nullptr if it can not be overloaded. */
+ if (overloaded_p && !instance.base->can_be_overloaded_p (instance.pred))
+ return nullptr;
+
+ b.append_name ("__riscv_th_");
+ b.append_name (instance.base_name);
+
+ /* vop_v --> vop_v_<type>. */
+ if (!overloaded_p)
+ {
+ /* vop --> vop_v. */
+ b.append_name (operand_suffixes[instance.op_info->op]);
+ /* vop_v --> vop_v_<type>. */
+ b.append_name (type_suffixes[instance.type.index].vector);
+ }
+
+ /* According to rvv-intrinsic-doc, it does not add "_m" suffix
+ for vop_m C++ overloaded API. */
+ if (overloaded_p && instance.pred == PRED_TYPE_m)
+ return b.finish_name ();
+ b.append_name (predication_suffixes[instance.pred]);
+ return b.finish_name ();
+ }
+};
+
+
+/* th_indexed_loadstore_width_def class. */
+struct th_indexed_loadstore_width_def : public function_shape
+{
+ void build (function_builder &b,
+ const function_group_info &group) const override
+ {
+ for (unsigned int pred_idx = 0; group.preds[pred_idx] != NUM_PRED_TYPES;
+ ++pred_idx)
+ {
+ for (unsigned int vec_type_idx = 0;
+ group.ops_infos.types[vec_type_idx].index != NUM_VECTOR_TYPES;
+ ++vec_type_idx)
+ {
+ tree index_type = group.ops_infos.args[1].get_tree_type (
+ group.ops_infos.types[vec_type_idx].index);
+ if (!index_type)
+ continue;
+ build_th_loadstore (b, group, pred_idx, vec_type_idx);
+ }
+ }
+ }
+
+ char *get_name (function_builder &b, const function_instance &instance,
+ bool overloaded_p) const override
+ {
+
+ /* Return nullptr if it can not be overloaded. */
+ if (overloaded_p && !instance.base->can_be_overloaded_p (instance.pred))
+ return nullptr;
+
+ b.append_name ("__riscv_th_");
+ b.append_name (instance.base_name);
+ /* vop_v --> vop_v_<type>. */
+ if (!overloaded_p)
+ {
+ /* vop --> vop_v. */
+ b.append_name (operand_suffixes[instance.op_info->op]);
+ /* vop_v --> vop_v_<type>. */
+ b.append_name (type_suffixes[instance.type.index].vector);
+ }
+
+ /* According to rvv-intrinsic-doc, it does not add "_m" suffix
+ for vop_m C++ overloaded API. */
+ if (overloaded_p && instance.pred == PRED_TYPE_m)
+ return b.finish_name ();
+ b.append_name (predication_suffixes[instance.pred]);
+ return b.finish_name ();
+ }
+};
+
/* alu_def class. */
struct alu_def : public build_base
{
@@ -632,6 +772,23 @@ struct reduc_alu_def : public build_base
}
};
+/* th_extract_def class. */
+struct th_extract_def : public build_base
+{
+ char *get_name (function_builder &b, const function_instance &instance,
+ bool overloaded_p) const override
+ {
+ b.append_name ("__riscv_th_");
+ b.append_name (instance.base_name);
+
+ if (overloaded_p)
+ return b.finish_name ();
+ b.append_name (type_suffixes[instance.type.index].vector);
+ b.append_name (type_suffixes[instance.type.index].scalar);
+ return b.finish_name ();
+ }
+};
+
/* scalar_move_def class. */
struct scalar_move_def : public build_base
{
@@ -1094,6 +1251,8 @@ SHAPE(vsetvl, vsetvl)
SHAPE(vsetvl, vsetvlmax)
SHAPE(loadstore, loadstore)
SHAPE(indexed_loadstore, indexed_loadstore)
+SHAPE(th_loadstore_width, th_loadstore_width)
+SHAPE(th_indexed_loadstore_width, th_indexed_loadstore_width)
SHAPE(alu, alu)
SHAPE(alu_frm, alu_frm)
SHAPE(widen_alu, widen_alu)
@@ -1106,6 +1265,7 @@ SHAPE(move, move)
SHAPE(mask_alu, mask_alu)
SHAPE(reduc_alu, reduc_alu)
SHAPE(reduc_alu_frm, reduc_alu_frm)
+SHAPE(th_extract, th_extract)
SHAPE(scalar_move, scalar_move)
SHAPE(vundefined, vundefined)
SHAPE(misc, misc)
diff --git a/gcc/config/riscv/riscv-vector-builtins-shapes.h b/gcc/config/riscv/riscv-vector-builtins-shapes.h
index ac2a28ce017..a7624d0fabd 100644
--- a/gcc/config/riscv/riscv-vector-builtins-shapes.h
+++ b/gcc/config/riscv/riscv-vector-builtins-shapes.h
@@ -28,6 +28,8 @@ extern const function_shape *const vsetvl;
extern const function_shape *const vsetvlmax;
extern const function_shape *const loadstore;
extern const function_shape *const indexed_loadstore;
+extern const function_shape *const th_loadstore_width;
+extern const function_shape *const th_indexed_loadstore_width;
extern const function_shape *const alu;
extern const function_shape *const alu_frm;
extern const function_shape *const widen_alu;
@@ -41,6 +43,7 @@ extern const function_shape *const mask_alu;
extern const function_shape *const reduc_alu;
extern const function_shape *const reduc_alu_frm;
extern const function_shape *const scalar_move;
+extern const function_shape *const th_extract;
extern const function_shape *const vundefined;
extern const function_shape *const misc;
extern const function_shape *const vset;
diff --git a/gcc/config/riscv/riscv-vector-builtins.cc b/gcc/config/riscv/riscv-vector-builtins.cc
index 25e0b6e56de..44b9fec1898 100644
--- a/gcc/config/riscv/riscv-vector-builtins.cc
+++ b/gcc/config/riscv/riscv-vector-builtins.cc
@@ -934,6 +934,32 @@ static CONSTEXPR const rvv_arg_type_info ext_vcreate_args[]
= {rvv_arg_type_info (RVV_BASE_vector),
rvv_arg_type_info_end};
+/* A list of args for vector_type func (const scalar_type *, size_t)
+ * function. */
+static CONSTEXPR const rvv_arg_type_info scalar_const_ptr_size_args[]
+ = {rvv_arg_type_info (RVV_BASE_scalar_const_ptr),
+ rvv_arg_type_info (RVV_BASE_size), rvv_arg_type_info_end};
+
+/* A list of args for vector_type func (const scalar_type *, eew8_index_type)
+ * function. */
+static CONSTEXPR const rvv_arg_type_info scalar_const_ptr_index_args[]
+ = {rvv_arg_type_info (RVV_BASE_scalar_const_ptr),
+ rvv_arg_type_info (RVV_BASE_unsigned_vector), rvv_arg_type_info_end};
+
+/* A list of args for void func (scalar_type *, eew8_index_type, vector_type)
+ * function. */
+static CONSTEXPR const rvv_arg_type_info scalar_ptr_index_args[]
+ = {rvv_arg_type_info (RVV_BASE_scalar_ptr),
+ rvv_arg_type_info (RVV_BASE_unsigned_vector),
+ rvv_arg_type_info (RVV_BASE_vector), rvv_arg_type_info_end};
+
+/* A list of args for void func (scalar_type *, size_t, vector_type)
+ * function. */
+static CONSTEXPR const rvv_arg_type_info scalar_ptr_size_args[]
+ = {rvv_arg_type_info (RVV_BASE_scalar_ptr),
+ rvv_arg_type_info (RVV_BASE_size), rvv_arg_type_info (RVV_BASE_vector),
+ rvv_arg_type_info_end};
+
/* A list of none preds that will be registered for intrinsic functions. */
static CONSTEXPR const predication_type_index none_preds[]
= {PRED_TYPE_none, NUM_PRED_TYPES};
@@ -1455,6 +1481,14 @@ static CONSTEXPR const rvv_op_info iu_shift_vvv_ops
rvv_arg_type_info (RVV_BASE_vector), /* Return type */
shift_vv_args /* Args */};
+/* A static operand information for scalar_type func (vector_type, size_t)
+ * function registration. */
+static CONSTEXPR const rvv_op_info iu_x_s_u_ops
+ = {iu_ops, /* Types */
+ OP_TYPE_vx, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_scalar), /* Return type */
+ v_size_args /* Args */};
+
/* A static operand information for vector_type func (vector_type, size_t)
* function registration. */
static CONSTEXPR const rvv_op_info iu_shift_vvx_ops
@@ -2638,6 +2672,38 @@ static CONSTEXPR const rvv_op_info all_v_vcreate_lmul4_x2_ops
rvv_arg_type_info (RVV_BASE_vlmul_ext_x2), /* Return type */
ext_vcreate_args /* Args */};
+/* A static operand information for vector_type func (const scalar_type *,
+ * size_t) function registration. */
+static CONSTEXPR const rvv_op_info all_v_scalar_const_ptr_size_ops
+ = {all_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_size_args /* Args */};
+
+/* A static operand information for void func (scalar_type *, size_t,
+ * vector_type) function registration. */
+static CONSTEXPR const rvv_op_info all_v_scalar_ptr_size_ops
+ = {all_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_void), /* Return type */
+ scalar_ptr_size_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * index_type) function registration. */
+static CONSTEXPR const rvv_op_info all_v_scalar_const_ptr_index_ops
+ = {all_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_index_args /* Args */};
+
+/* A static operand information for void func (scalar_type *, index_type,
+ * vector_type) function registration. */
+static CONSTEXPR const rvv_op_info all_v_scalar_ptr_index_ops
+ = {all_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_void), /* Return type */
+ scalar_ptr_index_args /* Args */};
+
/* A static operand information for vector_type func (vector_type).
Some ins just supports SEW=32, such as crypto vectol Zvkg extension.
* function registration. */
@@ -2816,6 +2882,10 @@ static function_group_info function_groups[] = {
#define DEF_RVV_FUNCTION(NAME, SHAPE, PREDS, OPS_INFO) \
{#NAME, &bases::NAME, &shapes::SHAPE, PREDS, OPS_INFO, REQUIRED_EXTENSIONS},
#include "riscv-vector-builtins-functions.def"
+#undef DEF_RVV_FUNCTION
+#define DEF_RVV_FUNCTION(NAME, SHAPE, PREDS, OPS_INFO) \
+ {#NAME, &bases::NAME, &shapes::SHAPE, PREDS, OPS_INFO, REQUIRED_EXTENSIONS},
+#include "thead-vector-builtins-functions.def"
};
/* The RVV types, with their built-in
diff --git a/gcc/config/riscv/riscv-vector-builtins.h b/gcc/config/riscv/riscv-vector-builtins.h
index 54c8824ff92..a8ee39a3cb2 100644
--- a/gcc/config/riscv/riscv-vector-builtins.h
+++ b/gcc/config/riscv/riscv-vector-builtins.h
@@ -114,6 +114,7 @@ static const unsigned int CP_WRITE_CSR = 1U << 5;
enum required_ext
{
VECTOR_EXT, /* Vector extension */
+ XTHEADVECTOR_EXT, /* XTheadVector extension */
ZVBB_EXT, /* Cryto vector Zvbb sub-ext */
ZVBB_OR_ZVKB_EXT, /* Cryto vector Zvbb or zvkb sub-ext */
ZVBC_EXT, /* Crypto vector Zvbc sub-ext */
@@ -234,6 +235,8 @@ struct function_group_info
{
case VECTOR_EXT:
return TARGET_VECTOR;
+ case XTHEADVECTOR_EXT:
+ return TARGET_XTHEADVECTOR;
case ZVBB_EXT:
return TARGET_ZVBB;
case ZVBB_OR_ZVKB_EXT:
diff --git a/gcc/config/riscv/t-riscv b/gcc/config/riscv/t-riscv
index 32de6b851c1..38494320d8b 100644
--- a/gcc/config/riscv/t-riscv
+++ b/gcc/config/riscv/t-riscv
@@ -1,6 +1,7 @@
RISCV_BUILTINS_H = $(srcdir)/config/riscv/riscv-vector-builtins.h \
$(srcdir)/config/riscv/riscv-vector-builtins.def \
$(srcdir)/config/riscv/riscv-vector-builtins-functions.def \
+ $(srcdir)/config/riscv/thead-vector-builtins-functions.def \
riscv-vector-type-indexer.gen.def
riscv-builtins.o: $(srcdir)/config/riscv/riscv-builtins.cc $(CONFIG_H) \
diff --git a/gcc/config/riscv/thead-vector-builtins-functions.def b/gcc/config/riscv/thead-vector-builtins-functions.def
new file mode 100644
index 00000000000..fd3ba29bae9
--- /dev/null
+++ b/gcc/config/riscv/thead-vector-builtins-functions.def
@@ -0,0 +1,39 @@
+#ifndef DEF_RVV_FUNCTION
+#define DEF_RVV_FUNCTION(NAME, SHAPE, PREDS, OPS_INFO)
+#endif
+
+#define REQUIRED_EXTENSIONS XTHEADVECTOR_EXT
+DEF_RVV_FUNCTION (vlb, th_loadstore_width, full_preds, all_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (vlh, th_loadstore_width, full_preds, all_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (vlw, th_loadstore_width, full_preds, all_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (vlbu, th_loadstore_width, full_preds, all_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (vlhu, th_loadstore_width, full_preds, all_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (vlwu, th_loadstore_width, full_preds, all_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (vsb, th_loadstore_width, none_m_preds, all_v_scalar_ptr_ops)
+DEF_RVV_FUNCTION (vsh, th_loadstore_width, none_m_preds, all_v_scalar_ptr_ops)
+DEF_RVV_FUNCTION (vsw, th_loadstore_width, none_m_preds, all_v_scalar_ptr_ops)
+DEF_RVV_FUNCTION (vlsb, th_loadstore_width, full_preds, all_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (vlsh, th_loadstore_width, full_preds, all_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (vlsw, th_loadstore_width, full_preds, all_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (vlsbu, th_loadstore_width, full_preds, all_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (vlshu, th_loadstore_width, full_preds, all_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (vlswu, th_loadstore_width, full_preds, all_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (vssb, th_loadstore_width, none_m_preds, all_v_scalar_ptr_size_ops)
+DEF_RVV_FUNCTION (vssh, th_loadstore_width, none_m_preds, all_v_scalar_ptr_size_ops)
+DEF_RVV_FUNCTION (vssw, th_loadstore_width, none_m_preds, all_v_scalar_ptr_size_ops)
+DEF_RVV_FUNCTION (vlxb, th_indexed_loadstore_width, full_preds, all_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (vlxh, th_indexed_loadstore_width, full_preds, all_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (vlxw, th_indexed_loadstore_width, full_preds, all_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (vlxbu, th_indexed_loadstore_width, full_preds, all_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (vlxhu, th_indexed_loadstore_width, full_preds, all_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (vlxwu, th_indexed_loadstore_width, full_preds, all_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (vsxb, th_indexed_loadstore_width, none_m_preds, all_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (vsxh, th_indexed_loadstore_width, none_m_preds, all_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (vsxw, th_indexed_loadstore_width, none_m_preds, all_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (vsuxb, th_indexed_loadstore_width, none_m_preds, all_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (vsuxh, th_indexed_loadstore_width, none_m_preds, all_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (vsuxw, th_indexed_loadstore_width, none_m_preds, all_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (vext_x_v, th_extract, none_preds, iu_x_s_u_ops)
+#undef REQUIRED_EXTENSIONS
+
+#undef DEF_RVV_FUNCTION
diff --git a/gcc/config/riscv/thead-vector.md b/gcc/config/riscv/thead-vector.md
index 696b815252d..0f3700d9269 100644
--- a/gcc/config/riscv/thead-vector.md
+++ b/gcc/config/riscv/thead-vector.md
@@ -1,7 +1,95 @@
(define_c_enum "unspec" [
+ UNSPEC_TH_VLB
+ UNSPEC_TH_VLBU
+ UNSPEC_TH_VLH
+ UNSPEC_TH_VLHU
+ UNSPEC_TH_VLW
+ UNSPEC_TH_VLWU
+
+ UNSPEC_TH_VLSB
+ UNSPEC_TH_VLSBU
+ UNSPEC_TH_VLSH
+ UNSPEC_TH_VLSHU
+ UNSPEC_TH_VLSW
+ UNSPEC_TH_VLSWU
+
+ UNSPEC_TH_VLXB
+ UNSPEC_TH_VLXBU
+ UNSPEC_TH_VLXH
+ UNSPEC_TH_VLXHU
+ UNSPEC_TH_VLXW
+ UNSPEC_TH_VLXWU
+
+ UNSPEC_TH_VSUXB
+ UNSPEC_TH_VSUXH
+ UNSPEC_TH_VSUXW
+
UNSPEC_TH_VWLDST
])
+(define_int_iterator UNSPEC_TH_VLMEM_OP [
+ UNSPEC_TH_VLB UNSPEC_TH_VLBU
+ UNSPEC_TH_VLH UNSPEC_TH_VLHU
+ UNSPEC_TH_VLW UNSPEC_TH_VLWU
+])
+
+(define_int_iterator UNSPEC_TH_VLSMEM_OP [
+ UNSPEC_TH_VLSB UNSPEC_TH_VLSBU
+ UNSPEC_TH_VLSH UNSPEC_TH_VLSHU
+ UNSPEC_TH_VLSW UNSPEC_TH_VLSWU
+])
+
+(define_int_iterator UNSPEC_TH_VLXMEM_OP [
+ UNSPEC_TH_VLXB UNSPEC_TH_VLXBU
+ UNSPEC_TH_VLXH UNSPEC_TH_VLXHU
+ UNSPEC_TH_VLXW UNSPEC_TH_VLXWU
+])
+
+(define_int_attr vlmem_op_attr [
+ (UNSPEC_TH_VLB "b") (UNSPEC_TH_VLBU "bu")
+ (UNSPEC_TH_VLH "h") (UNSPEC_TH_VLHU "hu")
+ (UNSPEC_TH_VLW "w") (UNSPEC_TH_VLWU "wu")
+ (UNSPEC_TH_VLSB "b") (UNSPEC_TH_VLSBU "bu")
+ (UNSPEC_TH_VLSH "h") (UNSPEC_TH_VLSHU "hu")
+ (UNSPEC_TH_VLSW "w") (UNSPEC_TH_VLSWU "wu")
+ (UNSPEC_TH_VLXB "b") (UNSPEC_TH_VLXBU "bu")
+ (UNSPEC_TH_VLXH "h") (UNSPEC_TH_VLXHU "hu")
+ (UNSPEC_TH_VLXW "w") (UNSPEC_TH_VLXWU "wu")
+ (UNSPEC_TH_VSUXB "b")
+ (UNSPEC_TH_VSUXH "h")
+ (UNSPEC_TH_VSUXW "w")
+])
+
+(define_int_attr vlmem_order_attr [
+ (UNSPEC_TH_VLXB "")
+ (UNSPEC_TH_VLXH "")
+ (UNSPEC_TH_VLXW "")
+ (UNSPEC_TH_VSUXB "u")
+ (UNSPEC_TH_VSUXH "u")
+ (UNSPEC_TH_VSUXW "u")
+])
+
+(define_int_iterator UNSPEC_TH_VSMEM_OP [
+ UNSPEC_TH_VLB
+ UNSPEC_TH_VLH
+ UNSPEC_TH_VLW
+])
+
+(define_int_iterator UNSPEC_TH_VSSMEM_OP [
+ UNSPEC_TH_VLSB
+ UNSPEC_TH_VLSH
+ UNSPEC_TH_VLSW
+])
+
+(define_int_iterator UNSPEC_TH_VSXMEM_OP [
+ UNSPEC_TH_VLXB
+ UNSPEC_TH_VLXH
+ UNSPEC_TH_VLXW
+ UNSPEC_TH_VSUXB
+ UNSPEC_TH_VSUXH
+ UNSPEC_TH_VSUXW
+])
+
(define_mode_iterator V_VLS_VT [V VLS VT])
(define_mode_iterator V_VB_VLS_VT [V VB VLS VT])
@@ -100,3 +188,168 @@
}
[(set_attr "type" "vldm,vstm,vmalu,vmalu,vmalu")
(set_attr "mode" "<MODE>")])
+
+(define_expand "@pred_mov_width<vlmem_op_attr><mode>"
+ [(set (match_operand:V_VLS 0 "nonimmediate_operand")
+ (if_then_else:V_VLS
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand")
+ (match_operand 4 "vector_length_operand")
+ (match_operand 5 "const_int_operand")
+ (match_operand 6 "const_int_operand")
+ (match_operand 7 "const_int_operand")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLMEM_OP)
+ (match_operand:V_VLS 3 "vector_move_operand")
+ (match_operand:V_VLS 2 "vector_merge_operand")))]
+ "TARGET_XTHEADVECTOR"
+ {})
+
+(define_insn_and_split "*pred_mov_width<vlmem_op_attr><mode>"
+ [(set (match_operand:V_VLS 0 "nonimmediate_operand" "=vr, vr, vd, m, vr, vr")
+ (if_then_else:V_VLS
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1, Wc1, vm, vmWc1, Wc1, Wc1")
+ (match_operand 4 "vector_length_operand" " rK, rK, rK, rK, rK, rK")
+ (match_operand 5 "const_int_operand" " i, i, i, i, i, i")
+ (match_operand 6 "const_int_operand" " i, i, i, i, i, i")
+ (match_operand 7 "const_int_operand" " i, i, i, i, i, i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLMEM_OP)
+ (match_operand:V_VLS 3 "reg_or_mem_operand" " m, m, m, vr, vr, vr")
+ (match_operand:V_VLS 2 "vector_merge_operand" " 0, vu, vu, vu, vu, 0")))]
+ "(TARGET_XTHEADVECTOR
+ && (register_operand (operands[0], <MODE>mode)
+ || register_operand (operands[3], <MODE>mode)))"
+ "@
+ vl<vlmem_op_attr>.v\t%0,%3%p1
+ vl<vlmem_op_attr>.v\t%0,%3
+ vl<vlmem_op_attr>.v\t%0,%3,%1.t
+ vs<vlmem_op_attr>.v\t%3,%0%p1
+ vmv.v.v\t%0,%3
+ vmv.v.v\t%0,%3"
+ "&& register_operand (operands[0], <MODE>mode)
+ && register_operand (operands[3], <MODE>mode)
+ && satisfies_constraint_vu (operands[2])
+ && INTVAL (operands[7]) == riscv_vector::VLMAX"
+ [(set (match_dup 0) (match_dup 3))]
+ ""
+ [(set_attr "type" "vlde,vlde,vlde,vste,vimov,vimov")
+ (set_attr "mode" "<MODE>")])
+
+(define_insn "@pred_store_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "memory_operand" "+m")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1")
+ (match_operand 3 "vector_length_operand" " rK")
+ (match_operand 4 "const_int_operand" " i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VSMEM_OP)
+ (match_operand:VI 2 "register_operand" " vr")
+ (match_dup 0)))]
+ "TARGET_XTHEADVECTOR"
+ "vs<vlmem_op_attr>.v\t%2,%0%p1"
+ [(set_attr "type" "vste")
+ (set_attr "mode" "<MODE>")
+ (set (attr "avl_type_idx") (const_int 4))
+ (set_attr "vl_op_idx" "3")])
+
+(define_insn "@pred_strided_load_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "register_operand" "=vr, vr, vd")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1, Wc1, vm")
+ (match_operand 5 "vector_length_operand" " rK, rK, rK")
+ (match_operand 6 "const_int_operand" " i, i, i")
+ (match_operand 7 "const_int_operand" " i, i, i")
+ (match_operand 8 "const_int_operand" " i, i, i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLSMEM_OP)
+ (unspec:VI
+ [(match_operand:VI 3 "memory_operand" " m, m, m")
+ (match_operand 4 "pmode_reg_or_0_operand" " rJ, rJ, rJ")] UNSPEC_TH_VLSMEM_OP)
+ (match_operand:VI 2 "vector_merge_operand" " 0, vu, vu")))]
+ "TARGET_XTHEADVECTOR"
+ "vls<vlmem_op_attr>.v\t%0,%3,%z4%p1"
+ [(set_attr "type" "vlds")
+ (set_attr "mode" "<MODE>")])
+
+(define_insn "@pred_strided_store_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "memory_operand" "+m")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1")
+ (match_operand 4 "vector_length_operand" " rK")
+ (match_operand 5 "const_int_operand" " i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VSSMEM_OP)
+ (unspec:VI
+ [(match_operand 2 "pmode_reg_or_0_operand" " rJ")
+ (match_operand:VI 3 "register_operand" " vr")] UNSPEC_TH_VSSMEM_OP)
+ (match_dup 0)))]
+ "TARGET_XTHEADVECTOR"
+ "vss<vlmem_op_attr>.v\t%3,%0,%z2%p1"
+ [(set_attr "type" "vsts")
+ (set_attr "mode" "<MODE>")
+ (set (attr "avl_type_idx") (const_int 5))])
+
+(define_insn "@pred_indexed_load_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "register_operand" "=vd, vr,vd, vr")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" " vm,Wc1,vm,Wc1")
+ (match_operand 5 "vector_length_operand" " rK, rK,rK, rK")
+ (match_operand 6 "const_int_operand" " i, i, i, i")
+ (match_operand 7 "const_int_operand" " i, i, i, i")
+ (match_operand 8 "const_int_operand" " i, i, i, i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLXMEM_OP)
+ (unspec:VI
+ [(match_operand 3 "pmode_reg_or_0_operand" " rJ, rJ,rJ, rJ")
+ (mem:BLK (scratch))
+ (match_operand:VI 4 "register_operand" " vr, vr,vr, vr")] UNSPEC_TH_VLXMEM_OP)
+ (match_operand:VI 2 "vector_merge_operand" " vu, vu, 0, 0")))]
+ "TARGET_XTHEADVECTOR"
+ "vlx<vlmem_op_attr>.v\t%0,(%z3),%4%p1"
+ [(set_attr "type" "vldux")
+ (set_attr "mode" "<MODE>")])
+
+(define_insn "@pred_indexed_<vlmem_order_attr>store_width<vlmem_op_attr><mode>"
+ [(set (mem:BLK (scratch))
+ (unspec:BLK
+ [(unspec:<VM>
+ [(match_operand:<VM> 0 "vector_mask_operand" "vmWc1")
+ (match_operand 4 "vector_length_operand" " rK")
+ (match_operand 5 "const_int_operand" " i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VSXMEM_OP)
+ (match_operand 1 "pmode_reg_or_0_operand" " rJ")
+ (match_operand:VI 2 "register_operand" " vr")
+ (match_operand:VI 3 "register_operand" " vr")] UNSPEC_TH_VSXMEM_OP))]
+ "TARGET_XTHEADVECTOR"
+ "vs<vlmem_order_attr>x<vlmem_op_attr>.v\t%3,(%z1),%2%p0"
+ [(set_attr "type" "vstux")
+ (set_attr "mode" "<MODE>")])
+
+(define_expand "@pred_th_extract<mode>"
+ [(set (match_operand:<VEL> 0 "register_operand")
+ (unspec:<VEL>
+ [(vec_select:<VEL>
+ (match_operand:V_VLSI 1 "register_operand")
+ (parallel [(match_operand:DI 2 "register_operand" "r")]))
+ (reg:SI VTYPE_REGNUM)] UNSPEC_VPREDICATE))]
+ "TARGET_XTHEADVECTOR"
+{})
+
+(define_insn "*pred_th_extract<mode>"
+ [(set (match_operand:<VEL> 0 "register_operand" "=r")
+ (unspec:<VEL>
+ [(vec_select:<VEL>
+ (match_operand:V_VLSI 1 "register_operand" "vr")
+ (parallel [(match_operand:DI 2 "register_operand" "r")]))
+ (reg:SI VTYPE_REGNUM)] UNSPEC_VPREDICATE))]
+ "TARGET_XTHEADVECTOR"
+ "vext.x.v\t%0,%1,%2"
+ [(set_attr "type" "vimovvx")
+ (set_attr "mode" "<MODE>")])
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c
new file mode 100644
index 00000000000..3c12c124597
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_th_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+
+** th.vadd\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+
+** th.vsb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out)
+{
+ vint32m1_t v = __riscv_th_vlb_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlb_v_i32m1_tu (v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vv_i32m1 (v2, v2, 4);
+ vint32m1_t v4 = __riscv_vadd_vv_i32m1_tu (v3, v2, v2, 4);
+ __riscv_th_vsb_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlb.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+
+** th.vadd\.vv\tv[1-9][0-9]?,\s*v[0-9]+,\s*v[0-9]+,\s*v0.t
+** th.vsb.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlb_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlb_v_i32m1_m (mask, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vv_i32m1 (v2, v2, 4);
+ vint32m1_t v4 = __riscv_vadd_vv_i32m1_m (mask, v3, v3, 4);
+ __riscv_th_vsb_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlb.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+
+** th.vadd\.vv\tv[1-9][0-9]?,\s*v[0-9]+,\s*v[0-9]+,\s*v0.t
+** th.vsb.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlb_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlb_v_i32m1_tumu (mask, v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vv_i32m1 (v2, v2, 4);
+ vint32m1_t v4 = __riscv_vadd_vv_i32m1_tumu (mask, v3, v2, v2, 4);
+ __riscv_th_vsb_v_i32m1 (out, v4, 4);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c
new file mode 100644
index 00000000000..30bef369375
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_th_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlbu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlbu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vsb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, uint32_t x)
+{
+ vuint32m1_t v = __riscv_th_vlbu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlbu_v_u32m1_tu (v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tu (v3, v2, -16, 4);
+ __riscv_th_vsb_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlbu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsb.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlbu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlbu_v_u32m1_m (mask, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_m (mask, v3, -16, 4);
+ __riscv_th_vsb_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlbu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlbu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsb.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlbu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlbu_v_u32m1_tumu (mask, v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tumu (mask, v3, v2, -16, 4);
+ __riscv_th_vsb_v_u32m1 (out, v4, 4);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c
new file mode 100644
index 00000000000..3c8b5ccc16b
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_th_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vsh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, int32_t x)
+{
+ vint32m1_t v = __riscv_th_vlh_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlh_v_i32m1_tu (v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, -16, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_tu (v3, v2, -16, 4);
+ __riscv_th_vsh_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlh.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsh.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, int32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlh_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlh_v_i32m1_m (mask, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, -16, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_m (mask, v3, -16, 4);
+ __riscv_th_vsh_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlh.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsh.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, int32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlh_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlh_v_i32m1_tumu (mask, v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, -16, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_tumu (mask, v3, v2, -16, 4);
+ __riscv_th_vsh_v_i32m1 (out, v4, 4);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c
new file mode 100644
index 00000000000..b7c00404f18
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_th_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlhu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlhu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vsh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, uint32_t x)
+{
+ vuint32m1_t v = __riscv_th_vlhu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlhu_v_u32m1_tu (v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tu (v3, v2, -16, 4);
+ __riscv_th_vsh_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlhu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsh.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlhu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlhu_v_u32m1_m (mask, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_m (mask, v3, -16, 4);
+ __riscv_th_vsh_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlhu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlhu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsh.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlhu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlhu_v_u32m1_tumu (mask, v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tumu (mask, v3, v2, -16, 4);
+ __riscv_th_vsh_v_u32m1 (out, v4, 4);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c
new file mode 100644
index 00000000000..17a53012acf
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_th_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vx\tv[0-9]+,\s*v[0-9]+,\s*[a-x0-9]+
+** th.vadd\.vx\tv[0-9]+,\s*v[0-9]+,\s*[a-x0-9]+
+** th.vsw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, int32_t x)
+{
+ vint32m1_t v = __riscv_th_vlw_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlw_v_i32m1_tu (v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, x, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_tu (v3, v2, x, 4);
+ __riscv_th_vsw_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlw.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vx\tv[0-9]+,\s*v[0-9]+,\s*[a-x0-9]+
+** th.vadd\.vx\tv[1-9][0-9]?,\s*v[0-9]+,\s*[a-x0-9]+,\s*v0.t
+** th.vsw.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, int32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlw_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlw_v_i32m1_m (mask, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, x, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_m (mask, v3, x, 4);
+ __riscv_th_vsw_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlw.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vx\tv[0-9]+,\s*v[0-9]+,\s*[a-x0-9]+
+** th.vadd\.vx\tv[1-9][0-9]?,\s*v[0-9]+,\s*[a-x0-9]+,\s*v0.t
+** th.vsw.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, int32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlw_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlw_v_i32m1_tumu (mask, v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, x, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_tumu (mask, v3, v2, x, 4);
+ __riscv_th_vsw_v_i32m1 (out, v4, 4);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c
new file mode 100644
index 00000000000..b187cfc852b
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_th_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlwu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlwu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vsw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, uint32_t x)
+{
+ vuint32m1_t v = __riscv_th_vlwu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlwu_v_u32m1_tu (v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tu (v3, v2, -16, 4);
+ __riscv_th_vsw_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlwu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsw.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlwu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlwu_v_u32m1_m (mask, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_m (mask, v3, -16, 4);
+ __riscv_th_vsw_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlwu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlwu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsw.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlwu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlwu_v_u32m1_tumu (mask, v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tumu (mask, v3, v2, -16, 4);
+ __riscv_th_vsw_v_u32m1 (out, v4, 4);
+}
--
2.17.1
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re：Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
2024-01-10 10:03 Re：[PATCH " juzhe.zhong
@ 2024-01-10 10:57 ` joshua
0 siblings, 0 replies; 17+ messages in thread
From: joshua @ 2024-01-10 10:57 UTC (permalink / raw)
To: juzhe.zhong, gcc-patches
Cc: Jim Wilson, palmer, andrew, philipp.tomsich, jeffreyalaw,
christoph.muellner, jinma, cooper.qu
[-- Attachment #1.1: Type: text/plain, Size: 70720 bytes --]
Hi Juzhe,
Perhaps things are not as simple as imagined.
The differences between vlb/vlh/vlw is not the same
as vle8/vle16/vle32. "8", "16" or "32" in vle8/vle16/vle32
can be appended from "vle" according to input type.
But vlb/vlh/vlw is different not in input type.
------------------------------------------------------------------
发件人：juzhe.zhong@rivai.ai <juzhe.zhong@rivai.ai>
发送时间：2024年1月10日(星期三) 18:03
收件人："cooper.joshua"<cooper.joshua@linux.alibaba.com>; "gcc-patches"<gcc-patches@gcc.gnu.org>
抄 送：Jim Wilson<jim.wilson.gcc@gmail.com>; palmer<palmer@dabbelt.com>; andrew<andrew@sifive.com>; "philipp.tomsich"<philipp.tomsich@vrull.eu>; jeffreyalaw<jeffreyalaw@gmail.com>; "christoph.muellner"<christoph.muellner@vrull.eu>; jinma<jinma@linux.alibaba.com>; "cooper.qu"<cooper.qu@linux.alibaba.com>
主 题：Re: Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
I mean change these:
+DEF_RVV_FUNCTION (th_vlb, th_loadstore_width, full_preds, i8_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (th_vlh, th_loadstore_width, full_preds, i16_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (th_vlw, th_loadstore_width, full_preds, i32_v_scalar_const_ptr_ops)
into a single:
+DEF_RVV_FUNCTION (th_vl, th_loadstore_width, full_preds, all_v_scalar_const_ptr_ops)
and append "h", "w", or"b" according to
TYPE_UNSIGNED and
GET_MODE_BITSIZE (GET_MODE_INNER (TYPE_MODE (instance.op_info->args[i].get_tree_type (instance.type.index))))
in th_loadstore_width.
It should definitely works, I allow this flexibility in design of the framework.
juzhe.zhong@rivai.ai
发件人： joshua
发送时间： 2024-01-10 17:55
收件人： juzhe.zhong@rivai.ai; gcc-patches
抄送： Jim Wilson; palmer; andrew; philipp.tomsich; jeffreyalaw; christoph.muellner; jinma; cooper.qu
主题： Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
And revise th_loadstore_width, append the name according TYPE_UNSIGNED and
GET_MODE_BITSIZE (GET_MODE_INNER (TYPE_MODE (instance.op_info->args[i].get_tree_type (instance.type.index))))
What do you mean by it? I'm a bit confused.
Changing i8_v_scalar_const_ptr_ops into all_v_scalar_const_ptr_ops
will expand the datatypes that can be used in th_vlb. Can we restrict
again in th_loadstore_width?
------------------------------------------------------------------
发件人：juzhe.zhong@rivai.ai <juzhe.zhong@rivai.ai>
发送时间：2024年1月10日(星期三) 17:35
收件人："cooper.joshua"<cooper.joshua@linux.alibaba.com>; "gcc-patches"<gcc-patches@gcc.gnu.org>
抄 送：Jim Wilson<jim.wilson.gcc@gmail.com>; palmer<palmer@dabbelt.com>; andrew<andrew@sifive.com>; "philipp.tomsich"<philipp.tomsich@vrull.eu>; jeffreyalaw<jeffreyalaw@gmail.com>; "christoph.muellner"<christoph.muellner@vrull.eu>; "cooper.joshua"<cooper.joshua@linux.alibaba.com>; jinma<jinma@linux.alibaba.com>; "cooper.qu"<cooper.qu@linux.alibaba.com>
主 题：Re: [PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
+DEF_RVV_FUNCTION (th_vlb, th_loadstore_width, full_preds, i8_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (th_vlh, th_loadstore_width, full_preds, i16_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (th_vlw, th_loadstore_width, full_preds, i32_v_scalar_const_ptr_ops)
I think we should remove those many data structure you added like: i8_v_scalar_const_ptr_ops
Instead, you should use all_v_scalar_const_ptr_ops
And revise th_loadstore_width, append the name according TYPE_UNSIGNED and
GET_MODE_BITSIZE (GET_MODE_INNER (TYPE_MODE (instance.op_info->args[i].get_tree_type (instance.type.index))))
juzhe.zhong@rivai.ai
From: Jun Sha (Joshua)
Date: 2024-01-10 17:27
To: gcc-patches
CC: jim.wilson.gcc; palmer; andrew; philipp.tomsich; jeffreyalaw; christoph.muellner; juzhe.zhong; Jun Sha (Joshua); Jin Ma; Xianmiao Qu
Subject: [PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics.
This patch only involves the generation of xtheadvector
special load/store instructions and vext instructions.
gcc/ChangeLog:
* config/riscv/riscv-vector-builtins-bases.cc
(class th_loadstore_width): Define new builtin bases.
(BASE): Define new builtin bases.
* config/riscv/riscv-vector-builtins-bases.h:
Define new builtin class.
* config/riscv/riscv-vector-builtins-functions.def (vlsegff):
Include thead-vector-builtins-functions.def.
* config/riscv/riscv-vector-builtins-shapes.cc
(struct th_loadstore_width_def): Define new builtin shapes.
(struct th_indexed_loadstore_width_def):
Define new builtin shapes.
(SHAPE): Define new builtin shapes.
* config/riscv/riscv-vector-builtins-shapes.h:
Define new builtin shapes.
* config/riscv/riscv-vector-builtins-types.def
(DEF_RVV_I8_OPS): Add datatypes for XTheadVector.
(DEF_RVV_I16_OPS): Add datatypes for XTheadVector.
(DEF_RVV_I32_OPS): Add datatypes for XTheadVector.
(DEF_RVV_U8_OPS): Add datatypes for XTheadVector.
(DEF_RVV_U16_OPS): Add datatypes for XTheadVector.
(DEF_RVV_U32_OPS): Add datatypes for XTheadVector.
(vint8m1_t): Add datatypes for XTheadVector.
(vint8m2_t): Likewise.
(vint8m4_t): Likewise.
(vint8m8_t): Likewise.
(vint16m1_t): Likewise.
(vint16m2_t): Likewise.
(vint16m4_t): Likewise.
(vint16m8_t): Likewise.
(vint32m1_t): Likewise.
(vint32m2_t): Likewise.
(vint32m4_t): Likewise.
(vint32m8_t): Likewise.
(vint64m1_t): Likewise.
(vint64m2_t): Likewise.
(vint64m4_t): Likewise.
(vint64m8_t): Likewise.
(vuint8m1_t): Likewise.
(vuint8m2_t): Likewise.
(vuint8m4_t): Likewise.
(vuint8m8_t): Likewise.
(vuint16m1_t): Likewise.
(vuint16m2_t): Likewise.
(vuint16m4_t): Likewise.
(vuint16m8_t): Likewise.
(vuint32m1_t): Likewise.
(vuint32m2_t): Likewise.
(vuint32m4_t): Likewise.
(vuint32m8_t): Likewise.
(vuint64m1_t): Likewise.
(vuint64m2_t): Likewise.
(vuint64m4_t): Likewise.
(vuint64m8_t): Likewise.
* config/riscv/riscv-vector-builtins.cc
(DEF_RVV_I8_OPS): Add datatypes for XTheadVector.
(DEF_RVV_I16_OPS): Add datatypes for XTheadVector.
(DEF_RVV_I32_OPS): Add datatypes for XTheadVector.
(DEF_RVV_U8_OPS): Add datatypes for XTheadVector.
(DEF_RVV_U16_OPS): Add datatypes for XTheadVector.
(DEF_RVV_U32_OPS): Add datatypes for XTheadVector.
* config/riscv/thead-vector-builtins-functions.def: New file.
* config/riscv/thead-vector.md: Add new patterns.
gcc/testsuite/ChangeLog:
* gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c: New test.
* gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c: New test.
Co-authored-by: Jin Ma <jinma@linux.alibaba.com>
Co-authored-by: Xianmiao Qu <cooper.qu@linux.alibaba.com>
Co-authored-by: Christoph Müllner <christoph.muellner@vrull.eu>
---
.../riscv/riscv-vector-builtins-bases.cc | 139 ++++++++
.../riscv/riscv-vector-builtins-bases.h | 31 ++
.../riscv/riscv-vector-builtins-shapes.cc | 98 ++++++
.../riscv/riscv-vector-builtins-shapes.h | 3 +
.../riscv/riscv-vector-builtins-types.def | 120 +++++++
gcc/config/riscv/riscv-vector-builtins.cc | 311 ++++++++++++++++++
gcc/config/riscv/riscv-vector-builtins.h | 3 +
gcc/config/riscv/t-riscv | 1 +
.../riscv/thead-vector-builtins-functions.def | 39 +++
gcc/config/riscv/thead-vector.md | 253 ++++++++++++++
.../riscv/rvv/xtheadvector/vlb-vsb.c | 68 ++++
.../riscv/rvv/xtheadvector/vlbu-vsb.c | 68 ++++
.../riscv/rvv/xtheadvector/vlh-vsh.c | 68 ++++
.../riscv/rvv/xtheadvector/vlhu-vsh.c | 68 ++++
.../riscv/rvv/xtheadvector/vlw-vsw.c | 68 ++++
.../riscv/rvv/xtheadvector/vlwu-vsw.c | 68 ++++
16 files changed, 1406 insertions(+)
create mode 100644 gcc/config/riscv/thead-vector-builtins-functions.def
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c
diff --git a/gcc/config/riscv/riscv-vector-builtins-bases.cc b/gcc/config/riscv/riscv-vector-builtins-bases.cc
index 46f1a1da33e..5f44f31a12b 100644
--- a/gcc/config/riscv/riscv-vector-builtins-bases.cc
+++ b/gcc/config/riscv/riscv-vector-builtins-bases.cc
@@ -2125,6 +2125,83 @@ public:
}
};
+/* Implements
+ * th.vl(b/h/w)[u].v/th.vs(b/h/w)[u].v/th.vls(b/h/w)[u].v/th.vss(b/h/w)[u].v/
+ * th.vlx(b/h/w)[u].v/th.vs[u]x(b/h/w).v
+ * codegen. */
+template<bool STORE_P, lst_type LST_TYPE, int UNSPEC>
+class th_loadstore_width : public function_base
+{
+public:
+ bool apply_tail_policy_p () const override { return !STORE_P; }
+ bool apply_mask_policy_p () const override { return !STORE_P; }
+
+ unsigned int call_properties (const function_instance &) const override
+ {
+ if (STORE_P)
+ return CP_WRITE_MEMORY;
+ else
+ return CP_READ_MEMORY;
+ }
+
+ bool can_be_overloaded_p (enum predication_type_index pred) const override
+ {
+ if (STORE_P || LST_TYPE == LST_INDEXED)
+ return true;
+ return pred != PRED_TYPE_none;
+ }
+
+ rtx expand (function_expander &e) const override
+ {
+ gcc_assert (TARGET_XTHEADVECTOR);
+ if (LST_TYPE == LST_INDEXED)
+ {
+ if (STORE_P)
+ return e.use_exact_insn (
+ code_for_pred_indexed_store_width (UNSPEC, UNSPEC,
+ e.vector_mode ()));
+ else
+ return e.use_exact_insn (
+ code_for_pred_indexed_load_width (UNSPEC, e.vector_mode ()));
+ }
+ else if (LST_TYPE == LST_STRIDED)
+ {
+ if (STORE_P)
+ return e.use_contiguous_store_insn (
+ code_for_pred_strided_store_width (UNSPEC, e.vector_mode ()));
+ else
+ return e.use_contiguous_load_insn (
+ code_for_pred_strided_load_width (UNSPEC, e.vector_mode ()));
+ }
+ else
+ {
+ if (STORE_P)
+ return e.use_contiguous_store_insn (
+ code_for_pred_store_width (UNSPEC, e.vector_mode ()));
+ else
+ return e.use_contiguous_load_insn (
+ code_for_pred_mov_width (UNSPEC, e.vector_mode ()));
+ }
+ }
+};
+
+/* Implements vext.x.v. */
+class th_extract : public function_base
+{
+public:
+ bool apply_vl_p () const override { return false; }
+ bool apply_tail_policy_p () const override { return false; }
+ bool apply_mask_policy_p () const override { return false; }
+ bool use_mask_predication_p () const override { return false; }
+ bool has_merge_operand_p () const override { return false; }
+
+ rtx expand (function_expander &e) const override
+ {
+ gcc_assert (TARGET_XTHEADVECTOR);
+ return e.use_exact_insn (code_for_pred_th_extract (e.vector_mode ()));
+ }
+};
+
/* Below implements are vector crypto */
/* Implements vandn.[vv,vx] */
class vandn : public function_base
@@ -2587,6 +2664,37 @@ static CONSTEXPR const seg_indexed_load<UNSPEC_ORDERED> vloxseg_obj;
static CONSTEXPR const seg_indexed_store<UNSPEC_UNORDERED> vsuxseg_obj;
static CONSTEXPR const seg_indexed_store<UNSPEC_ORDERED> vsoxseg_obj;
static CONSTEXPR const vlsegff vlsegff_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLB> th_vlb_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLBU> th_vlbu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLH> th_vlh_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLHU> th_vlhu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLW> th_vlw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_UNIT_STRIDE, UNSPEC_TH_VLWU> th_vlwu_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_UNIT_STRIDE, UNSPEC_TH_VLB> th_vsb_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_UNIT_STRIDE, UNSPEC_TH_VLH> th_vsh_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_UNIT_STRIDE, UNSPEC_TH_VLW> th_vsw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSB> th_vlsb_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSBU> th_vlsbu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSH> th_vlsh_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSHU> th_vlshu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSW> th_vlsw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_STRIDED, UNSPEC_TH_VLSWU> th_vlswu_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_STRIDED, UNSPEC_TH_VLSB> th_vssb_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_STRIDED, UNSPEC_TH_VLSH> th_vssh_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_STRIDED, UNSPEC_TH_VLSW> th_vssw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXB> th_vlxb_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXBU> th_vlxbu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXH> th_vlxh_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXHU> th_vlxhu_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXW> th_vlxw_obj;
+static CONSTEXPR const th_loadstore_width<false, LST_INDEXED, UNSPEC_TH_VLXWU> th_vlxwu_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VLXB> th_vsxb_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VLXH> th_vsxh_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VLXW> th_vsxw_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VSUXB> th_vsuxb_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VSUXH> th_vsuxh_obj;
+static CONSTEXPR const th_loadstore_width<true, LST_INDEXED, UNSPEC_TH_VSUXW> th_vsuxw_obj;
+static CONSTEXPR const th_extract th_vext_x_v_obj;
/* Crypto Vector */
static CONSTEXPR const vandn vandn_obj;
@@ -2878,6 +2986,37 @@ BASE (vloxseg)
BASE (vsuxseg)
BASE (vsoxseg)
BASE (vlsegff)
+BASE (th_vlb)
+BASE (th_vlh)
+BASE (th_vlw)
+BASE (th_vlbu)
+BASE (th_vlhu)
+BASE (th_vlwu)
+BASE (th_vsb)
+BASE (th_vsh)
+BASE (th_vsw)
+BASE (th_vlsb)
+BASE (th_vlsh)
+BASE (th_vlsw)
+BASE (th_vlsbu)
+BASE (th_vlshu)
+BASE (th_vlswu)
+BASE (th_vssb)
+BASE (th_vssh)
+BASE (th_vssw)
+BASE (th_vlxb)
+BASE (th_vlxh)
+BASE (th_vlxw)
+BASE (th_vlxbu)
+BASE (th_vlxhu)
+BASE (th_vlxwu)
+BASE (th_vsxb)
+BASE (th_vsxh)
+BASE (th_vsxw)
+BASE (th_vsuxb)
+BASE (th_vsuxh)
+BASE (th_vsuxw)
+BASE (th_vext_x_v)
/* Crypto vector */
BASE (vandn)
BASE (vbrev)
diff --git a/gcc/config/riscv/riscv-vector-builtins-bases.h b/gcc/config/riscv/riscv-vector-builtins-bases.h
index 1122e3801a7..df43adf9a17 100644
--- a/gcc/config/riscv/riscv-vector-builtins-bases.h
+++ b/gcc/config/riscv/riscv-vector-builtins-bases.h
@@ -299,6 +299,37 @@ extern const function_base *const vloxseg;
extern const function_base *const vsuxseg;
extern const function_base *const vsoxseg;
extern const function_base *const vlsegff;
+extern const function_base *const th_vlb;
+extern const function_base *const th_vlh;
+extern const function_base *const th_vlw;
+extern const function_base *const th_vlbu;
+extern const function_base *const th_vlhu;
+extern const function_base *const th_vlwu;
+extern const function_base *const th_vsb;
+extern const function_base *const th_vsh;
+extern const function_base *const th_vsw;
+extern const function_base *const th_vlsb;
+extern const function_base *const th_vlsh;
+extern const function_base *const th_vlsw;
+extern const function_base *const th_vlsbu;
+extern const function_base *const th_vlshu;
+extern const function_base *const th_vlswu;
+extern const function_base *const th_vssb;
+extern const function_base *const th_vssh;
+extern const function_base *const th_vssw;
+extern const function_base *const th_vlxb;
+extern const function_base *const th_vlxh;
+extern const function_base *const th_vlxw;
+extern const function_base *const th_vlxbu;
+extern const function_base *const th_vlxhu;
+extern const function_base *const th_vlxwu;
+extern const function_base *const th_vsxb;
+extern const function_base *const th_vsxh;
+extern const function_base *const th_vsxw;
+extern const function_base *const th_vsuxb;
+extern const function_base *const th_vsuxh;
+extern const function_base *const th_vsuxw;
+extern const function_base *const th_vext_x_v;
/* Below function_base are Vectro Crypto*/
extern const function_base *const vandn;
extern const function_base *const vbrev;
diff --git a/gcc/config/riscv/riscv-vector-builtins-shapes.cc b/gcc/config/riscv/riscv-vector-builtins-shapes.cc
index 1e4f4d53de6..489a95cf684 100644
--- a/gcc/config/riscv/riscv-vector-builtins-shapes.cc
+++ b/gcc/config/riscv/riscv-vector-builtins-shapes.cc
@@ -211,6 +211,86 @@ struct indexed_loadstore_def : public function_shape
}
};
+/* th_loadstore_width_def class. */
+struct th_loadstore_width_def : public build_base
+{
+ char *get_name (function_builder &b, const function_instance &instance,
+ bool overloaded_p) const override
+ {
+ /* Return nullptr if it can not be overloaded. */
+ if (overloaded_p && !instance.base->can_be_overloaded_p (instance.pred))
+ return nullptr;
+
+ b.append_base_name (instance.base_name);
+
+ /* vop_v --> vop_v_<type>. */
+ if (!overloaded_p)
+ {
+ /* vop --> vop_v. */
+ b.append_name (operand_suffixes[instance.op_info->op]);
+ /* vop_v --> vop_v_<type>. */
+ b.append_name (type_suffixes[instance.type.index].vector);
+ }
+
+ /* According to rvv-intrinsic-doc, it does not add "_m" suffix
+ for vop_m C++ overloaded API. */
+ if (overloaded_p && instance.pred == PRED_TYPE_m)
+ return b.finish_name ();
+ b.append_name (predication_suffixes[instance.pred]);
+ return b.finish_name ();
+ }
+};
+
+
+/* th_indexed_loadstore_width_def class. */
+struct th_indexed_loadstore_width_def : public function_shape
+{
+ void build (function_builder &b,
+ const function_group_info &group) const override
+ {
+ for (unsigned int pred_idx = 0; group.preds[pred_idx] != NUM_PRED_TYPES;
+ ++pred_idx)
+ {
+ for (unsigned int vec_type_idx = 0;
+ group.ops_infos.types[vec_type_idx].index != NUM_VECTOR_TYPES;
+ ++vec_type_idx)
+ {
+ tree index_type = group.ops_infos.args[1].get_tree_type (
+ group.ops_infos.types[vec_type_idx].index);
+ if (!index_type)
+ continue;
+ build_one (b, group, pred_idx, vec_type_idx);
+ }
+ }
+ }
+
+ char *get_name (function_builder &b, const function_instance &instance,
+ bool overloaded_p) const override
+ {
+
+ /* Return nullptr if it can not be overloaded. */
+ if (overloaded_p && !instance.base->can_be_overloaded_p (instance.pred))
+ return nullptr;
+
+ b.append_base_name (instance.base_name);
+ /* vop_v --> vop_v_<type>. */
+ if (!overloaded_p)
+ {
+ /* vop --> vop_v. */
+ b.append_name (operand_suffixes[instance.op_info->op]);
+ /* vop_v --> vop_v_<type>. */
+ b.append_name (type_suffixes[instance.type.index].vector);
+ }
+
+ /* According to rvv-intrinsic-doc, it does not add "_m" suffix
+ for vop_m C++ overloaded API. */
+ if (overloaded_p && instance.pred == PRED_TYPE_m)
+ return b.finish_name ();
+ b.append_name (predication_suffixes[instance.pred]);
+ return b.finish_name ();
+ }
+};
+
/* alu_def class. */
struct alu_def : public build_base
{
@@ -632,6 +712,21 @@ struct reduc_alu_def : public build_base
}
};
+/* th_extract_def class. */
+struct th_extract_def : public build_base
+{
+ char *get_name (function_builder &b, const function_instance &instance,
+ bool overloaded_p) const override
+ {
+ b.append_base_name (instance.base_name);
+ if (overloaded_p)
+ return b.finish_name ();
+ b.append_name (type_suffixes[instance.type.index].vector);
+ b.append_name (type_suffixes[instance.type.index].scalar);
+ return b.finish_name ();
+ }
+};
+
/* scalar_move_def class. */
struct scalar_move_def : public build_base
{
@@ -1094,6 +1189,8 @@ SHAPE(vsetvl, vsetvl)
SHAPE(vsetvl, vsetvlmax)
SHAPE(loadstore, loadstore)
SHAPE(indexed_loadstore, indexed_loadstore)
+SHAPE(th_loadstore_width, th_loadstore_width)
+SHAPE(th_indexed_loadstore_width, th_indexed_loadstore_width)
SHAPE(alu, alu)
SHAPE(alu_frm, alu_frm)
SHAPE(widen_alu, widen_alu)
@@ -1106,6 +1203,7 @@ SHAPE(move, move)
SHAPE(mask_alu, mask_alu)
SHAPE(reduc_alu, reduc_alu)
SHAPE(reduc_alu_frm, reduc_alu_frm)
+SHAPE(th_extract, th_extract)
SHAPE(scalar_move, scalar_move)
SHAPE(vundefined, vundefined)
SHAPE(misc, misc)
diff --git a/gcc/config/riscv/riscv-vector-builtins-shapes.h b/gcc/config/riscv/riscv-vector-builtins-shapes.h
index ac2a28ce017..a7624d0fabd 100644
--- a/gcc/config/riscv/riscv-vector-builtins-shapes.h
+++ b/gcc/config/riscv/riscv-vector-builtins-shapes.h
@@ -28,6 +28,8 @@ extern const function_shape *const vsetvl;
extern const function_shape *const vsetvlmax;
extern const function_shape *const loadstore;
extern const function_shape *const indexed_loadstore;
+extern const function_shape *const th_loadstore_width;
+extern const function_shape *const th_indexed_loadstore_width;
extern const function_shape *const alu;
extern const function_shape *const alu_frm;
extern const function_shape *const widen_alu;
@@ -41,6 +43,7 @@ extern const function_shape *const mask_alu;
extern const function_shape *const reduc_alu;
extern const function_shape *const reduc_alu_frm;
extern const function_shape *const scalar_move;
+extern const function_shape *const th_extract;
extern const function_shape *const vundefined;
extern const function_shape *const misc;
extern const function_shape *const vset;
diff --git a/gcc/config/riscv/riscv-vector-builtins-types.def b/gcc/config/riscv/riscv-vector-builtins-types.def
index 61019a56844..abfeb4fcd9b 100644
--- a/gcc/config/riscv/riscv-vector-builtins-types.def
+++ b/gcc/config/riscv/riscv-vector-builtins-types.def
@@ -24,12 +24,48 @@ along with GCC; see the file COPYING3. If not see
#define DEF_RVV_I_OPS(TYPE, REQUIRE)
#endif
+/* Use "DEF_RVV_I8_OPS" macro include some signed integer (i8/i16/i32/i64)
+ which will be iterated and registered as intrinsic functions. */
+#ifndef DEF_RVV_I8_OPS
+#define DEF_RVV_I8_OPS(TYPE, REQUIRE)
+#endif
+
+/* Use "DEF_RVV_I16_OPS" macro include some signed integer (i16/i32/i64)
+ which will be iterated and registered as intrinsic functions. */
+#ifndef DEF_RVV_I16_OPS
+#define DEF_RVV_I16_OPS(TYPE, REQUIRE)
+#endif
+
+/* Use "DEF_RVV_I32_OPS" macro include some signed integer (i32/i64)
+ which will be iterated and registered as intrinsic functions. */
+#ifndef DEF_RVV_I32_OPS
+#define DEF_RVV_I32_OPS(TYPE, REQUIRE)
+#endif
+
/* Use "DEF_RVV_U_OPS" macro include all unsigned integer which will be
iterated and registered as intrinsic functions. */
#ifndef DEF_RVV_U_OPS
#define DEF_RVV_U_OPS(TYPE, REQUIRE)
#endif
+/* Use "DEF_RVV_U8_OPS" macro include some unsigned integer (i8/i16/i32/i64)
+ which will be iterated and registered as intrinsic functions. */
+#ifndef DEF_RVV_U8_OPS
+#define DEF_RVV_U8_OPS(TYPE, REQUIRE)
+#endif
+
+/* Use "DEF_RVV_U16_OPS" macro include some unsigned integer (i16/i32/i64)
+ which will be iterated and registered as intrinsic functions. */
+#ifndef DEF_RVV_U16_OPS
+#define DEF_RVV_U16_OPS(TYPE, REQUIRE)
+#endif
+
+/* Use "DEF_RVV_U32_OPS" macro include some unsigned integer (i32/i64)
+ which will be iterated and registered as intrinsic functions. */
+#ifndef DEF_RVV_U32_OPS
+#define DEF_RVV_U32_OPS(TYPE, REQUIRE)
+#endif
+
/* Use "DEF_RVV_F_OPS" macro include all floating-point which will be
iterated and registered as intrinsic functions. */
#ifndef DEF_RVV_F_OPS
@@ -374,6 +410,45 @@ DEF_RVV_I_OPS (vint64m2_t, RVV_REQUIRE_ELEN_64)
DEF_RVV_I_OPS (vint64m4_t, RVV_REQUIRE_ELEN_64)
DEF_RVV_I_OPS (vint64m8_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_I8_OPS (vint8m1_t, 0)
+DEF_RVV_I8_OPS (vint8m2_t, 0)
+DEF_RVV_I8_OPS (vint8m4_t, 0)
+DEF_RVV_I8_OPS (vint8m8_t, 0)
+DEF_RVV_I8_OPS (vint16m1_t, 0)
+DEF_RVV_I8_OPS (vint16m2_t, 0)
+DEF_RVV_I8_OPS (vint16m4_t, 0)
+DEF_RVV_I8_OPS (vint16m8_t, 0)
+DEF_RVV_I8_OPS (vint32m1_t, 0)
+DEF_RVV_I8_OPS (vint32m2_t, 0)
+DEF_RVV_I8_OPS (vint32m4_t, 0)
+DEF_RVV_I8_OPS (vint32m8_t, 0)
+DEF_RVV_I8_OPS (vint64m1_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_I8_OPS (vint64m2_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_I8_OPS (vint64m4_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_I8_OPS (vint64m8_t, RVV_REQUIRE_ELEN_64)
+
+DEF_RVV_I16_OPS (vint16m1_t, 0)
+DEF_RVV_I16_OPS (vint16m2_t, 0)
+DEF_RVV_I16_OPS (vint16m4_t, 0)
+DEF_RVV_I16_OPS (vint16m8_t, 0)
+DEF_RVV_I16_OPS (vint32m1_t, 0)
+DEF_RVV_I16_OPS (vint32m2_t, 0)
+DEF_RVV_I16_OPS (vint32m4_t, 0)
+DEF_RVV_I16_OPS (vint32m8_t, 0)
+DEF_RVV_I16_OPS (vint64m1_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_I16_OPS (vint64m2_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_I16_OPS (vint64m4_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_I16_OPS (vint64m8_t, RVV_REQUIRE_ELEN_64)
+
+DEF_RVV_I32_OPS (vint32m1_t, 0)
+DEF_RVV_I32_OPS (vint32m2_t, 0)
+DEF_RVV_I32_OPS (vint32m4_t, 0)
+DEF_RVV_I32_OPS (vint32m8_t, 0)
+DEF_RVV_I32_OPS (vint64m1_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_I32_OPS (vint64m2_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_I32_OPS (vint64m4_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_I32_OPS (vint64m8_t, RVV_REQUIRE_ELEN_64)
+
DEF_RVV_U_OPS (vuint8mf8_t, RVV_REQUIRE_MIN_VLEN_64)
DEF_RVV_U_OPS (vuint8mf4_t, 0)
DEF_RVV_U_OPS (vuint8mf2_t, 0)
@@ -397,6 +472,45 @@ DEF_RVV_U_OPS (vuint64m2_t, RVV_REQUIRE_ELEN_64)
DEF_RVV_U_OPS (vuint64m4_t, RVV_REQUIRE_ELEN_64)
DEF_RVV_U_OPS (vuint64m8_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_U8_OPS (vuint8m1_t, 0)
+DEF_RVV_U8_OPS (vuint8m2_t, 0)
+DEF_RVV_U8_OPS (vuint8m4_t, 0)
+DEF_RVV_U8_OPS (vuint8m8_t, 0)
+DEF_RVV_U8_OPS (vuint16m1_t, 0)
+DEF_RVV_U8_OPS (vuint16m2_t, 0)
+DEF_RVV_U8_OPS (vuint16m4_t, 0)
+DEF_RVV_U8_OPS (vuint16m8_t, 0)
+DEF_RVV_U8_OPS (vuint32m1_t, 0)
+DEF_RVV_U8_OPS (vuint32m2_t, 0)
+DEF_RVV_U8_OPS (vuint32m4_t, 0)
+DEF_RVV_U8_OPS (vuint32m8_t, 0)
+DEF_RVV_U8_OPS (vuint64m1_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_U8_OPS (vuint64m2_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_U8_OPS (vuint64m4_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_U8_OPS (vuint64m8_t, RVV_REQUIRE_ELEN_64)
+
+DEF_RVV_U16_OPS (vuint16m1_t, 0)
+DEF_RVV_U16_OPS (vuint16m2_t, 0)
+DEF_RVV_U16_OPS (vuint16m4_t, 0)
+DEF_RVV_U16_OPS (vuint16m8_t, 0)
+DEF_RVV_U16_OPS (vuint32m1_t, 0)
+DEF_RVV_U16_OPS (vuint32m2_t, 0)
+DEF_RVV_U16_OPS (vuint32m4_t, 0)
+DEF_RVV_U16_OPS (vuint32m8_t, 0)
+DEF_RVV_U16_OPS (vuint64m1_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_U16_OPS (vuint64m2_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_U16_OPS (vuint64m4_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_U16_OPS (vuint64m8_t, RVV_REQUIRE_ELEN_64)
+
+DEF_RVV_U32_OPS (vuint32m1_t, 0)
+DEF_RVV_U32_OPS (vuint32m2_t, 0)
+DEF_RVV_U32_OPS (vuint32m4_t, 0)
+DEF_RVV_U32_OPS (vuint32m8_t, 0)
+DEF_RVV_U32_OPS (vuint64m1_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_U32_OPS (vuint64m2_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_U32_OPS (vuint64m4_t, RVV_REQUIRE_ELEN_64)
+DEF_RVV_U32_OPS (vuint64m8_t, RVV_REQUIRE_ELEN_64)
+
DEF_RVV_F_OPS (vfloat16mf4_t, RVV_REQUIRE_ELEN_FP_16 | RVV_REQUIRE_MIN_VLEN_64)
DEF_RVV_F_OPS (vfloat16mf2_t, RVV_REQUIRE_ELEN_FP_16)
DEF_RVV_F_OPS (vfloat16m1_t, RVV_REQUIRE_ELEN_FP_16)
@@ -1379,7 +1493,13 @@ DEF_RVV_CRYPTO_SEW64_OPS (vuint64m4_t, RVV_REQUIRE_ELEN_64)
DEF_RVV_CRYPTO_SEW64_OPS (vuint64m8_t, RVV_REQUIRE_ELEN_64)
#undef DEF_RVV_I_OPS
+#undef DEF_RVV_I8_OPS
+#undef DEF_RVV_I16_OPS
+#undef DEF_RVV_I32_OPS
#undef DEF_RVV_U_OPS
+#undef DEF_RVV_U8_OPS
+#undef DEF_RVV_U16_OPS
+#undef DEF_RVV_U32_OPS
#undef DEF_RVV_F_OPS
#undef DEF_RVV_B_OPS
#undef DEF_RVV_WEXTI_OPS
diff --git a/gcc/config/riscv/riscv-vector-builtins.cc b/gcc/config/riscv/riscv-vector-builtins.cc
index 25e0b6e56de..f429f12dc18 100644
--- a/gcc/config/riscv/riscv-vector-builtins.cc
+++ b/gcc/config/riscv/riscv-vector-builtins.cc
@@ -246,6 +246,63 @@ static const rvv_type_info iu_ops[] = {
#include "riscv-vector-builtins-types.def"
{NUM_VECTOR_TYPES, 0}};
+/* A list of all integer will be registered for intrinsic functions. */
+static const rvv_type_info i8_ops[] = {
+#define DEF_RVV_I8_OPS(TYPE, REQUIRE) {VECTOR_TYPE_##TYPE, REQUIRE},
+#include "riscv-vector-builtins-types.def"
+ {NUM_VECTOR_TYPES, 0}};
+
+/* A list of all integer will be registered for intrinsic functions. */
+static const rvv_type_info i16_ops[] = {
+#define DEF_RVV_I16_OPS(TYPE, REQUIRE) {VECTOR_TYPE_##TYPE, REQUIRE},
+#include "riscv-vector-builtins-types.def"
+ {NUM_VECTOR_TYPES, 0}};
+
+/* A list of all integer will be registered for intrinsic functions. */
+static const rvv_type_info i32_ops[] = {
+#define DEF_RVV_I32_OPS(TYPE, REQUIRE) {VECTOR_TYPE_##TYPE, REQUIRE},
+#include "riscv-vector-builtins-types.def"
+ {NUM_VECTOR_TYPES, 0}};
+
+/* A list of all integer will be registered for intrinsic functions. */
+static const rvv_type_info u8_ops[] = {
+#define DEF_RVV_U8_OPS(TYPE, REQUIRE) {VECTOR_TYPE_##TYPE, REQUIRE},
+#include "riscv-vector-builtins-types.def"
+ {NUM_VECTOR_TYPES, 0}};
+
+/* A list of all integer will be registered for intrinsic functions. */
+static const rvv_type_info u16_ops[] = {
+#define DEF_RVV_U16_OPS(TYPE, REQUIRE) {VECTOR_TYPE_##TYPE, REQUIRE},
+#include "riscv-vector-builtins-types.def"
+ {NUM_VECTOR_TYPES, 0}};
+
+/* A list of all integer will be registered for intrinsic functions. */
+static const rvv_type_info u32_ops[] = {
+#define DEF_RVV_U32_OPS(TYPE, REQUIRE) {VECTOR_TYPE_##TYPE, REQUIRE},
+#include "riscv-vector-builtins-types.def"
+ {NUM_VECTOR_TYPES, 0}};
+
+/* A list of all integer will be registered for intrinsic functions. */
+static const rvv_type_info iu8_ops[] = {
+#define DEF_RVV_I8_OPS(TYPE, REQUIRE) {VECTOR_TYPE_##TYPE, REQUIRE},
+#define DEF_RVV_U8_OPS(TYPE, REQUIRE) {VECTOR_TYPE_##TYPE, REQUIRE},
+#include "riscv-vector-builtins-types.def"
+ {NUM_VECTOR_TYPES, 0}};
+
+/* A list of all integer will be registered for intrinsic functions. */
+static const rvv_type_info iu16_ops[] = {
+#define DEF_RVV_I16_OPS(TYPE, REQUIRE) {VECTOR_TYPE_##TYPE, REQUIRE},
+#define DEF_RVV_U16_OPS(TYPE, REQUIRE) {VECTOR_TYPE_##TYPE, REQUIRE},
+#include "riscv-vector-builtins-types.def"
+ {NUM_VECTOR_TYPES, 0}};
+
+/* A list of all integer will be registered for intrinsic functions. */
+static const rvv_type_info iu32_ops[] = {
+#define DEF_RVV_I32_OPS(TYPE, REQUIRE) {VECTOR_TYPE_##TYPE, REQUIRE},
+#define DEF_RVV_U32_OPS(TYPE, REQUIRE) {VECTOR_TYPE_##TYPE, REQUIRE},
+#include "riscv-vector-builtins-types.def"
+ {NUM_VECTOR_TYPES, 0}};
+
/* A list of all types will be registered for intrinsic functions. */
static const rvv_type_info all_ops[] = {
#define DEF_RVV_I_OPS(TYPE, REQUIRE) {VECTOR_TYPE_##TYPE, REQUIRE},
@@ -934,6 +991,32 @@ static CONSTEXPR const rvv_arg_type_info ext_vcreate_args[]
= {rvv_arg_type_info (RVV_BASE_vector),
rvv_arg_type_info_end};
+/* A list of args for vector_type func (const scalar_type *, size_t)
+ * function. */
+static CONSTEXPR const rvv_arg_type_info scalar_const_ptr_size_args[]
+ = {rvv_arg_type_info (RVV_BASE_scalar_const_ptr),
+ rvv_arg_type_info (RVV_BASE_size), rvv_arg_type_info_end};
+
+/* A list of args for vector_type func (const scalar_type *, eew8_index_type)
+ * function. */
+static CONSTEXPR const rvv_arg_type_info scalar_const_ptr_index_args[]
+ = {rvv_arg_type_info (RVV_BASE_scalar_const_ptr),
+ rvv_arg_type_info (RVV_BASE_unsigned_vector), rvv_arg_type_info_end};
+
+/* A list of args for void func (scalar_type *, eew8_index_type, vector_type)
+ * function. */
+static CONSTEXPR const rvv_arg_type_info scalar_ptr_index_args[]
+ = {rvv_arg_type_info (RVV_BASE_scalar_ptr),
+ rvv_arg_type_info (RVV_BASE_unsigned_vector),
+ rvv_arg_type_info (RVV_BASE_vector), rvv_arg_type_info_end};
+
+/* A list of args for void func (scalar_type *, size_t, vector_type)
+ * function. */
+static CONSTEXPR const rvv_arg_type_info scalar_ptr_size_args[]
+ = {rvv_arg_type_info (RVV_BASE_scalar_ptr),
+ rvv_arg_type_info (RVV_BASE_size), rvv_arg_type_info (RVV_BASE_vector),
+ rvv_arg_type_info_end};
+
/* A list of none preds that will be registered for intrinsic functions. */
static CONSTEXPR const predication_type_index none_preds[]
= {PRED_TYPE_none, NUM_PRED_TYPES};
@@ -1455,6 +1538,14 @@ static CONSTEXPR const rvv_op_info iu_shift_vvv_ops
rvv_arg_type_info (RVV_BASE_vector), /* Return type */
shift_vv_args /* Args */};
+/* A static operand information for scalar_type func (vector_type, size_t)
+ * function registration. */
+static CONSTEXPR const rvv_op_info iu_x_s_u_ops
+ = {iu_ops, /* Types */
+ OP_TYPE_vx, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_scalar), /* Return type */
+ v_size_args /* Args */};
+
/* A static operand information for vector_type func (vector_type, size_t)
* function registration. */
static CONSTEXPR const rvv_op_info iu_shift_vvx_ops
@@ -2638,6 +2729,222 @@ static CONSTEXPR const rvv_op_info all_v_vcreate_lmul4_x2_ops
rvv_arg_type_info (RVV_BASE_vlmul_ext_x2), /* Return type */
ext_vcreate_args /* Args */};
+/* A static operand information for vector_type func (const scalar_type *)
+ * function registration. */
+static CONSTEXPR const rvv_op_info i8_v_scalar_const_ptr_ops
+ = {i8_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *)
+ * function registration. */
+static CONSTEXPR const rvv_op_info i16_v_scalar_const_ptr_ops
+ = {i16_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *)
+ * function registration. */
+static CONSTEXPR const rvv_op_info i32_v_scalar_const_ptr_ops
+ = {i32_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *)
+ * function registration. */
+static CONSTEXPR const rvv_op_info u8_v_scalar_const_ptr_ops
+ = {u8_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *)
+ * function registration. */
+static CONSTEXPR const rvv_op_info u16_v_scalar_const_ptr_ops
+ = {u16_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *)
+ * function registration. */
+static CONSTEXPR const rvv_op_info u32_v_scalar_const_ptr_ops
+ = {u32_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * size_t) function registration. */
+static CONSTEXPR const rvv_op_info i8_v_scalar_const_ptr_size_ops
+ = {i8_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_size_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * size_t) function registration. */
+static CONSTEXPR const rvv_op_info i16_v_scalar_const_ptr_size_ops
+ = {i16_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_size_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * size_t) function registration. */
+static CONSTEXPR const rvv_op_info i32_v_scalar_const_ptr_size_ops
+ = {i32_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_size_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * size_t) function registration. */
+static CONSTEXPR const rvv_op_info u8_v_scalar_const_ptr_size_ops
+ = {u8_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_size_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * size_t) function registration. */
+static CONSTEXPR const rvv_op_info u16_v_scalar_const_ptr_size_ops
+ = {u16_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_size_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * size_t) function registration. */
+static CONSTEXPR const rvv_op_info u32_v_scalar_const_ptr_size_ops
+ = {u32_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_size_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * eew8_index_type) function registration. */
+static CONSTEXPR const rvv_op_info i8_v_scalar_const_ptr_index_ops
+ = {i8_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_index_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * eew8_index_type) function registration. */
+static CONSTEXPR const rvv_op_info u8_v_scalar_const_ptr_index_ops
+ = {u8_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_index_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * eew8_index_type) function registration. */
+static CONSTEXPR const rvv_op_info i16_v_scalar_const_ptr_index_ops
+ = {i16_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_index_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * eew8_index_type) function registration. */
+static CONSTEXPR const rvv_op_info u16_v_scalar_const_ptr_index_ops
+ = {u16_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_index_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * eew8_index_type) function registration. */
+static CONSTEXPR const rvv_op_info i32_v_scalar_const_ptr_index_ops
+ = {i32_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_index_args /* Args */};
+
+/* A static operand information for vector_type func (const scalar_type *,
+ * eew8_index_type) function registration. */
+static CONSTEXPR const rvv_op_info u32_v_scalar_const_ptr_index_ops
+ = {u32_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_vector), /* Return type */
+ scalar_const_ptr_index_args /* Args */};
+
+/* A static operand information for void func (scalar_type *, eew8_index_type,
+ * vector_type) function registration. */
+static CONSTEXPR const rvv_op_info iu8_v_scalar_ptr_index_ops
+ = {iu8_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_void), /* Return type */
+ scalar_ptr_index_args /* Args */};
+
+/* A static operand information for void func (scalar_type *, eew16_index_type,
+ * vector_type) function registration. */
+static CONSTEXPR const rvv_op_info iu16_v_scalar_ptr_index_ops
+ = {iu16_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_void), /* Return type */
+ scalar_ptr_index_args /* Args */};
+
+/* A static operand information for void func (scalar_type *, eew32_index_type,
+ * vector_type) function registration. */
+static CONSTEXPR const rvv_op_info iu32_v_scalar_ptr_index_ops
+ = {iu32_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_void), /* Return type */
+ scalar_ptr_index_args /* Args */};
+
+/* A static operand information for void func (scalar_type *, vector_type,
+ * function registration. */
+static CONSTEXPR const rvv_op_info iu8_v_scalar_ptr_ops
+ = {iu8_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_void), /* Return type */
+ scalar_ptr_args /* Args */};
+
+/* A static operand information for void func (scalar_type *, vector_type)
+ * function registration. */
+static CONSTEXPR const rvv_op_info iu16_v_scalar_ptr_ops
+ = {iu16_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_void), /* Return type */
+ scalar_ptr_args /* Args */};
+
+/* A static operand information for void func (scalar_type *, vector_type)
+ * function registration. */
+static CONSTEXPR const rvv_op_info iu32_v_scalar_ptr_ops
+ = {iu32_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_void), /* Return type */
+ scalar_ptr_args /* Args */};
+
+/* A static operand information for void func (scalar_type *, size_t,
+ * vector_type) function registration. */
+static CONSTEXPR const rvv_op_info iu8_v_scalar_ptr_size_ops
+ = {iu8_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_void), /* Return type */
+ scalar_ptr_size_args /* Args */};
+
+/* A static operand information for void func (scalar_type *, size_t,
+ * vector_type) function registration. */
+static CONSTEXPR const rvv_op_info iu16_v_scalar_ptr_size_ops
+ = {iu16_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_void), /* Return type */
+ scalar_ptr_size_args /* Args */};
+
+/* A static operand information for void func (scalar_type *, size_t,
+ * vector_type) function registration. */
+static CONSTEXPR const rvv_op_info iu32_v_scalar_ptr_size_ops
+ = {iu32_ops, /* Types */
+ OP_TYPE_v, /* Suffix */
+ rvv_arg_type_info (RVV_BASE_void), /* Return type */
+ scalar_ptr_size_args /* Args */};
+
/* A static operand information for vector_type func (vector_type).
Some ins just supports SEW=32, such as crypto vectol Zvkg extension.
* function registration. */
@@ -2816,6 +3123,10 @@ static function_group_info function_groups[] = {
#define DEF_RVV_FUNCTION(NAME, SHAPE, PREDS, OPS_INFO) \
{#NAME, &bases::NAME, &shapes::SHAPE, PREDS, OPS_INFO, REQUIRED_EXTENSIONS},
#include "riscv-vector-builtins-functions.def"
+#undef DEF_RVV_FUNCTION
+#define DEF_RVV_FUNCTION(NAME, SHAPE, PREDS, OPS_INFO) \
+ {#NAME, &bases::NAME, &shapes::SHAPE, PREDS, OPS_INFO, REQUIRED_EXTENSIONS},
+#include "thead-vector-builtins-functions.def"
};
/* The RVV types, with their built-in
diff --git a/gcc/config/riscv/riscv-vector-builtins.h b/gcc/config/riscv/riscv-vector-builtins.h
index 54c8824ff92..a8ee39a3cb2 100644
--- a/gcc/config/riscv/riscv-vector-builtins.h
+++ b/gcc/config/riscv/riscv-vector-builtins.h
@@ -114,6 +114,7 @@ static const unsigned int CP_WRITE_CSR = 1U << 5;
enum required_ext
{
VECTOR_EXT, /* Vector extension */
+ XTHEADVECTOR_EXT, /* XTheadVector extension */
ZVBB_EXT, /* Cryto vector Zvbb sub-ext */
ZVBB_OR_ZVKB_EXT, /* Cryto vector Zvbb or zvkb sub-ext */
ZVBC_EXT, /* Crypto vector Zvbc sub-ext */
@@ -234,6 +235,8 @@ struct function_group_info
{
case VECTOR_EXT:
return TARGET_VECTOR;
+ case XTHEADVECTOR_EXT:
+ return TARGET_XTHEADVECTOR;
case ZVBB_EXT:
return TARGET_ZVBB;
case ZVBB_OR_ZVKB_EXT:
diff --git a/gcc/config/riscv/t-riscv b/gcc/config/riscv/t-riscv
index 32de6b851c1..38494320d8b 100644
--- a/gcc/config/riscv/t-riscv
+++ b/gcc/config/riscv/t-riscv
@@ -1,6 +1,7 @@
RISCV_BUILTINS_H = $(srcdir)/config/riscv/riscv-vector-builtins.h \
$(srcdir)/config/riscv/riscv-vector-builtins.def \
$(srcdir)/config/riscv/riscv-vector-builtins-functions.def \
+ $(srcdir)/config/riscv/thead-vector-builtins-functions.def \
riscv-vector-type-indexer.gen.def
riscv-builtins.o: $(srcdir)/config/riscv/riscv-builtins.cc $(CONFIG_H) \
diff --git a/gcc/config/riscv/thead-vector-builtins-functions.def b/gcc/config/riscv/thead-vector-builtins-functions.def
new file mode 100644
index 00000000000..667820d4c3e
--- /dev/null
+++ b/gcc/config/riscv/thead-vector-builtins-functions.def
@@ -0,0 +1,39 @@
+#ifndef DEF_RVV_FUNCTION
+#define DEF_RVV_FUNCTION(NAME, SHAPE, PREDS, OPS_INFO)
+#endif
+
+#define REQUIRED_EXTENSIONS XTHEADVECTOR_EXT
+DEF_RVV_FUNCTION (th_vlb, th_loadstore_width, full_preds, i8_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (th_vlh, th_loadstore_width, full_preds, i16_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (th_vlw, th_loadstore_width, full_preds, i32_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (th_vlbu, th_loadstore_width, full_preds, u8_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (th_vlhu, th_loadstore_width, full_preds, u16_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (th_vlwu, th_loadstore_width, full_preds, u32_v_scalar_const_ptr_ops)
+DEF_RVV_FUNCTION (th_vsb, th_loadstore_width, none_m_preds, iu8_v_scalar_ptr_ops)
+DEF_RVV_FUNCTION (th_vsh, th_loadstore_width, none_m_preds, iu16_v_scalar_ptr_ops)
+DEF_RVV_FUNCTION (th_vsw, th_loadstore_width, none_m_preds, iu32_v_scalar_ptr_ops)
+DEF_RVV_FUNCTION (th_vlsb, th_loadstore_width, full_preds, i8_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (th_vlsh, th_loadstore_width, full_preds, i16_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (th_vlsw, th_loadstore_width, full_preds, i32_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (th_vlsbu, th_loadstore_width, full_preds, u8_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (th_vlshu, th_loadstore_width, full_preds, u16_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (th_vlswu, th_loadstore_width, full_preds, u32_v_scalar_const_ptr_size_ops)
+DEF_RVV_FUNCTION (th_vssb, th_loadstore_width, none_m_preds, iu8_v_scalar_ptr_size_ops)
+DEF_RVV_FUNCTION (th_vssh, th_loadstore_width, none_m_preds, iu16_v_scalar_ptr_size_ops)
+DEF_RVV_FUNCTION (th_vssw, th_loadstore_width, none_m_preds, iu32_v_scalar_ptr_size_ops)
+DEF_RVV_FUNCTION (th_vlxb, th_indexed_loadstore_width, full_preds, i8_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (th_vlxh, th_indexed_loadstore_width, full_preds, i16_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (th_vlxw, th_indexed_loadstore_width, full_preds, i32_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (th_vlxbu, th_indexed_loadstore_width, full_preds, u8_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (th_vlxhu, th_indexed_loadstore_width, full_preds, u16_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (th_vlxwu, th_indexed_loadstore_width, full_preds, u32_v_scalar_const_ptr_index_ops)
+DEF_RVV_FUNCTION (th_vsxb, th_indexed_loadstore_width, none_m_preds, iu8_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (th_vsxh, th_indexed_loadstore_width, none_m_preds, iu16_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (th_vsxw, th_indexed_loadstore_width, none_m_preds, iu32_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (th_vsuxb, th_indexed_loadstore_width, none_m_preds, iu8_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (th_vsuxh, th_indexed_loadstore_width, none_m_preds, iu16_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (th_vsuxw, th_indexed_loadstore_width, none_m_preds, iu32_v_scalar_ptr_index_ops)
+DEF_RVV_FUNCTION (th_vext_x_v, th_extract, none_preds, iu_x_s_u_ops)
+#undef REQUIRED_EXTENSIONS
+
+#undef DEF_RVV_FUNCTION
diff --git a/gcc/config/riscv/thead-vector.md b/gcc/config/riscv/thead-vector.md
index 696b815252d..0f3700d9269 100644
--- a/gcc/config/riscv/thead-vector.md
+++ b/gcc/config/riscv/thead-vector.md
@@ -1,7 +1,95 @@
(define_c_enum "unspec" [
+ UNSPEC_TH_VLB
+ UNSPEC_TH_VLBU
+ UNSPEC_TH_VLH
+ UNSPEC_TH_VLHU
+ UNSPEC_TH_VLW
+ UNSPEC_TH_VLWU
+
+ UNSPEC_TH_VLSB
+ UNSPEC_TH_VLSBU
+ UNSPEC_TH_VLSH
+ UNSPEC_TH_VLSHU
+ UNSPEC_TH_VLSW
+ UNSPEC_TH_VLSWU
+
+ UNSPEC_TH_VLXB
+ UNSPEC_TH_VLXBU
+ UNSPEC_TH_VLXH
+ UNSPEC_TH_VLXHU
+ UNSPEC_TH_VLXW
+ UNSPEC_TH_VLXWU
+
+ UNSPEC_TH_VSUXB
+ UNSPEC_TH_VSUXH
+ UNSPEC_TH_VSUXW
+
UNSPEC_TH_VWLDST
])
+(define_int_iterator UNSPEC_TH_VLMEM_OP [
+ UNSPEC_TH_VLB UNSPEC_TH_VLBU
+ UNSPEC_TH_VLH UNSPEC_TH_VLHU
+ UNSPEC_TH_VLW UNSPEC_TH_VLWU
+])
+
+(define_int_iterator UNSPEC_TH_VLSMEM_OP [
+ UNSPEC_TH_VLSB UNSPEC_TH_VLSBU
+ UNSPEC_TH_VLSH UNSPEC_TH_VLSHU
+ UNSPEC_TH_VLSW UNSPEC_TH_VLSWU
+])
+
+(define_int_iterator UNSPEC_TH_VLXMEM_OP [
+ UNSPEC_TH_VLXB UNSPEC_TH_VLXBU
+ UNSPEC_TH_VLXH UNSPEC_TH_VLXHU
+ UNSPEC_TH_VLXW UNSPEC_TH_VLXWU
+])
+
+(define_int_attr vlmem_op_attr [
+ (UNSPEC_TH_VLB "b") (UNSPEC_TH_VLBU "bu")
+ (UNSPEC_TH_VLH "h") (UNSPEC_TH_VLHU "hu")
+ (UNSPEC_TH_VLW "w") (UNSPEC_TH_VLWU "wu")
+ (UNSPEC_TH_VLSB "b") (UNSPEC_TH_VLSBU "bu")
+ (UNSPEC_TH_VLSH "h") (UNSPEC_TH_VLSHU "hu")
+ (UNSPEC_TH_VLSW "w") (UNSPEC_TH_VLSWU "wu")
+ (UNSPEC_TH_VLXB "b") (UNSPEC_TH_VLXBU "bu")
+ (UNSPEC_TH_VLXH "h") (UNSPEC_TH_VLXHU "hu")
+ (UNSPEC_TH_VLXW "w") (UNSPEC_TH_VLXWU "wu")
+ (UNSPEC_TH_VSUXB "b")
+ (UNSPEC_TH_VSUXH "h")
+ (UNSPEC_TH_VSUXW "w")
+])
+
+(define_int_attr vlmem_order_attr [
+ (UNSPEC_TH_VLXB "")
+ (UNSPEC_TH_VLXH "")
+ (UNSPEC_TH_VLXW "")
+ (UNSPEC_TH_VSUXB "u")
+ (UNSPEC_TH_VSUXH "u")
+ (UNSPEC_TH_VSUXW "u")
+])
+
+(define_int_iterator UNSPEC_TH_VSMEM_OP [
+ UNSPEC_TH_VLB
+ UNSPEC_TH_VLH
+ UNSPEC_TH_VLW
+])
+
+(define_int_iterator UNSPEC_TH_VSSMEM_OP [
+ UNSPEC_TH_VLSB
+ UNSPEC_TH_VLSH
+ UNSPEC_TH_VLSW
+])
+
+(define_int_iterator UNSPEC_TH_VSXMEM_OP [
+ UNSPEC_TH_VLXB
+ UNSPEC_TH_VLXH
+ UNSPEC_TH_VLXW
+ UNSPEC_TH_VSUXB
+ UNSPEC_TH_VSUXH
+ UNSPEC_TH_VSUXW
+])
+
(define_mode_iterator V_VLS_VT [V VLS VT])
(define_mode_iterator V_VB_VLS_VT [V VB VLS VT])
@@ -100,3 +188,168 @@
}
[(set_attr "type" "vldm,vstm,vmalu,vmalu,vmalu")
(set_attr "mode" "<MODE>")])
+
+(define_expand "@pred_mov_width<vlmem_op_attr><mode>"
+ [(set (match_operand:V_VLS 0 "nonimmediate_operand")
+ (if_then_else:V_VLS
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand")
+ (match_operand 4 "vector_length_operand")
+ (match_operand 5 "const_int_operand")
+ (match_operand 6 "const_int_operand")
+ (match_operand 7 "const_int_operand")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLMEM_OP)
+ (match_operand:V_VLS 3 "vector_move_operand")
+ (match_operand:V_VLS 2 "vector_merge_operand")))]
+ "TARGET_XTHEADVECTOR"
+ {})
+
+(define_insn_and_split "*pred_mov_width<vlmem_op_attr><mode>"
+ [(set (match_operand:V_VLS 0 "nonimmediate_operand" "=vr, vr, vd, m, vr, vr")
+ (if_then_else:V_VLS
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1, Wc1, vm, vmWc1, Wc1, Wc1")
+ (match_operand 4 "vector_length_operand" " rK, rK, rK, rK, rK, rK")
+ (match_operand 5 "const_int_operand" " i, i, i, i, i, i")
+ (match_operand 6 "const_int_operand" " i, i, i, i, i, i")
+ (match_operand 7 "const_int_operand" " i, i, i, i, i, i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLMEM_OP)
+ (match_operand:V_VLS 3 "reg_or_mem_operand" " m, m, m, vr, vr, vr")
+ (match_operand:V_VLS 2 "vector_merge_operand" " 0, vu, vu, vu, vu, 0")))]
+ "(TARGET_XTHEADVECTOR
+ && (register_operand (operands[0], <MODE>mode)
+ || register_operand (operands[3], <MODE>mode)))"
+ "@
+ vl<vlmem_op_attr>.v\t%0,%3%p1
+ vl<vlmem_op_attr>.v\t%0,%3
+ vl<vlmem_op_attr>.v\t%0,%3,%1.t
+ vs<vlmem_op_attr>.v\t%3,%0%p1
+ vmv.v.v\t%0,%3
+ vmv.v.v\t%0,%3"
+ "&& register_operand (operands[0], <MODE>mode)
+ && register_operand (operands[3], <MODE>mode)
+ && satisfies_constraint_vu (operands[2])
+ && INTVAL (operands[7]) == riscv_vector::VLMAX"
+ [(set (match_dup 0) (match_dup 3))]
+ ""
+ [(set_attr "type" "vlde,vlde,vlde,vste,vimov,vimov")
+ (set_attr "mode" "<MODE>")])
+
+(define_insn "@pred_store_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "memory_operand" "+m")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1")
+ (match_operand 3 "vector_length_operand" " rK")
+ (match_operand 4 "const_int_operand" " i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VSMEM_OP)
+ (match_operand:VI 2 "register_operand" " vr")
+ (match_dup 0)))]
+ "TARGET_XTHEADVECTOR"
+ "vs<vlmem_op_attr>.v\t%2,%0%p1"
+ [(set_attr "type" "vste")
+ (set_attr "mode" "<MODE>")
+ (set (attr "avl_type_idx") (const_int 4))
+ (set_attr "vl_op_idx" "3")])
+
+(define_insn "@pred_strided_load_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "register_operand" "=vr, vr, vd")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1, Wc1, vm")
+ (match_operand 5 "vector_length_operand" " rK, rK, rK")
+ (match_operand 6 "const_int_operand" " i, i, i")
+ (match_operand 7 "const_int_operand" " i, i, i")
+ (match_operand 8 "const_int_operand" " i, i, i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLSMEM_OP)
+ (unspec:VI
+ [(match_operand:VI 3 "memory_operand" " m, m, m")
+ (match_operand 4 "pmode_reg_or_0_operand" " rJ, rJ, rJ")] UNSPEC_TH_VLSMEM_OP)
+ (match_operand:VI 2 "vector_merge_operand" " 0, vu, vu")))]
+ "TARGET_XTHEADVECTOR"
+ "vls<vlmem_op_attr>.v\t%0,%3,%z4%p1"
+ [(set_attr "type" "vlds")
+ (set_attr "mode" "<MODE>")])
+
+(define_insn "@pred_strided_store_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "memory_operand" "+m")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" "vmWc1")
+ (match_operand 4 "vector_length_operand" " rK")
+ (match_operand 5 "const_int_operand" " i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VSSMEM_OP)
+ (unspec:VI
+ [(match_operand 2 "pmode_reg_or_0_operand" " rJ")
+ (match_operand:VI 3 "register_operand" " vr")] UNSPEC_TH_VSSMEM_OP)
+ (match_dup 0)))]
+ "TARGET_XTHEADVECTOR"
+ "vss<vlmem_op_attr>.v\t%3,%0,%z2%p1"
+ [(set_attr "type" "vsts")
+ (set_attr "mode" "<MODE>")
+ (set (attr "avl_type_idx") (const_int 5))])
+
+(define_insn "@pred_indexed_load_width<vlmem_op_attr><mode>"
+ [(set (match_operand:VI 0 "register_operand" "=vd, vr,vd, vr")
+ (if_then_else:VI
+ (unspec:<VM>
+ [(match_operand:<VM> 1 "vector_mask_operand" " vm,Wc1,vm,Wc1")
+ (match_operand 5 "vector_length_operand" " rK, rK,rK, rK")
+ (match_operand 6 "const_int_operand" " i, i, i, i")
+ (match_operand 7 "const_int_operand" " i, i, i, i")
+ (match_operand 8 "const_int_operand" " i, i, i, i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VLXMEM_OP)
+ (unspec:VI
+ [(match_operand 3 "pmode_reg_or_0_operand" " rJ, rJ,rJ, rJ")
+ (mem:BLK (scratch))
+ (match_operand:VI 4 "register_operand" " vr, vr,vr, vr")] UNSPEC_TH_VLXMEM_OP)
+ (match_operand:VI 2 "vector_merge_operand" " vu, vu, 0, 0")))]
+ "TARGET_XTHEADVECTOR"
+ "vlx<vlmem_op_attr>.v\t%0,(%z3),%4%p1"
+ [(set_attr "type" "vldux")
+ (set_attr "mode" "<MODE>")])
+
+(define_insn "@pred_indexed_<vlmem_order_attr>store_width<vlmem_op_attr><mode>"
+ [(set (mem:BLK (scratch))
+ (unspec:BLK
+ [(unspec:<VM>
+ [(match_operand:<VM> 0 "vector_mask_operand" "vmWc1")
+ (match_operand 4 "vector_length_operand" " rK")
+ (match_operand 5 "const_int_operand" " i")
+ (reg:SI VL_REGNUM)
+ (reg:SI VTYPE_REGNUM)] UNSPEC_TH_VSXMEM_OP)
+ (match_operand 1 "pmode_reg_or_0_operand" " rJ")
+ (match_operand:VI 2 "register_operand" " vr")
+ (match_operand:VI 3 "register_operand" " vr")] UNSPEC_TH_VSXMEM_OP))]
+ "TARGET_XTHEADVECTOR"
+ "vs<vlmem_order_attr>x<vlmem_op_attr>.v\t%3,(%z1),%2%p0"
+ [(set_attr "type" "vstux")
+ (set_attr "mode" "<MODE>")])
+
+(define_expand "@pred_th_extract<mode>"
+ [(set (match_operand:<VEL> 0 "register_operand")
+ (unspec:<VEL>
+ [(vec_select:<VEL>
+ (match_operand:V_VLSI 1 "register_operand")
+ (parallel [(match_operand:DI 2 "register_operand" "r")]))
+ (reg:SI VTYPE_REGNUM)] UNSPEC_VPREDICATE))]
+ "TARGET_XTHEADVECTOR"
+{})
+
+(define_insn "*pred_th_extract<mode>"
+ [(set (match_operand:<VEL> 0 "register_operand" "=r")
+ (unspec:<VEL>
+ [(vec_select:<VEL>
+ (match_operand:V_VLSI 1 "register_operand" "vr")
+ (parallel [(match_operand:DI 2 "register_operand" "r")]))
+ (reg:SI VTYPE_REGNUM)] UNSPEC_VPREDICATE))]
+ "TARGET_XTHEADVECTOR"
+ "vext.x.v\t%0,%1,%2"
+ [(set_attr "type" "vimovvx")
+ (set_attr "mode" "<MODE>")])
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c
new file mode 100644
index 00000000000..4e192bbf025
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlb-vsb.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+
+** th.vadd\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+
+** th.vsb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out)
+{
+ vint32m1_t v = __riscv_th_vlb_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlb_v_i32m1_tu (v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vv_i32m1 (v2, v2, 4);
+ vint32m1_t v4 = __riscv_vadd_vv_i32m1_tu (v3, v2, v2, 4);
+ __riscv_th_vsb_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlb.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+
+** th.vadd\.vv\tv[1-9][0-9]?,\s*v[0-9]+,\s*v[0-9]+,\s*v0.t
+** th.vsb.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlb_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlb_v_i32m1_m (mask, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vv_i32m1 (v2, v2, 4);
+ vint32m1_t v4 = __riscv_vadd_vv_i32m1_m (mask, v3, v3, 4);
+ __riscv_th_vsb_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlb.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+
+** th.vadd\.vv\tv[1-9][0-9]?,\s*v[0-9]+,\s*v[0-9]+,\s*v0.t
+** th.vsb.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlb_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlb_v_i32m1_tumu (mask, v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vv_i32m1 (v2, v2, 4);
+ vint32m1_t v4 = __riscv_vadd_vv_i32m1_tumu (mask, v3, v2, v2, 4);
+ __riscv_th_vsb_v_i32m1 (out, v4, 4);
+}
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c
new file mode 100644
index 00000000000..1538afec68e
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlbu-vsb.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlbu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlbu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vsb\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, uint32_t x)
+{
+ vuint32m1_t v = __riscv_th_vlbu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlbu_v_u32m1_tu (v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tu (v3, v2, -16, 4);
+ __riscv_th_vsb_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlbu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsb.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlbu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlbu_v_u32m1_m (mask, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_m (mask, v3, -16, 4);
+ __riscv_th_vsb_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlbu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlbu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsb.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlbu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlbu_v_u32m1_tumu (mask, v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tumu (mask, v3, v2, -16, 4);
+ __riscv_th_vsb_v_u32m1 (out, v4, 4);
+}
\ No newline at end of file
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c
new file mode 100644
index 00000000000..bf4924a1d76
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlh-vsh.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vsh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, int32_t x)
+{
+ vint32m1_t v = __riscv_th_vlh_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlh_v_i32m1_tu (v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, -16, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_tu (v3, v2, -16, 4);
+ __riscv_th_vsh_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlh.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsh.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, int32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlh_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlh_v_i32m1_m (mask, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, -16, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_m (mask, v3, -16, 4);
+ __riscv_th_vsh_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlh.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsh.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, int32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlh_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlh_v_i32m1_tumu (mask, v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, -16, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_tumu (mask, v3, v2, -16, 4);
+ __riscv_th_vsh_v_i32m1 (out, v4, 4);
+}
\ No newline at end of file
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c
new file mode 100644
index 00000000000..8c451845175
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlhu-vsh.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlhu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlhu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vsh\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, uint32_t x)
+{
+ vuint32m1_t v = __riscv_th_vlhu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlhu_v_u32m1_tu (v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tu (v3, v2, -16, 4);
+ __riscv_th_vsh_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlhu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsh.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlhu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlhu_v_u32m1_m (mask, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_m (mask, v3, -16, 4);
+ __riscv_th_vsh_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlhu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlhu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsh.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlhu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlhu_v_u32m1_tumu (mask, v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tumu (mask, v3, v2, -16, 4);
+ __riscv_th_vsh_v_u32m1 (out, v4, 4);
+}
\ No newline at end of file
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c
new file mode 100644
index 00000000000..0f5b09684a5
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlw-vsw.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vx\tv[0-9]+,\s*v[0-9]+,\s*[a-x0-9]+
+** th.vadd\.vx\tv[0-9]+,\s*v[0-9]+,\s*[a-x0-9]+
+** th.vsw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, int32_t x)
+{
+ vint32m1_t v = __riscv_th_vlw_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlw_v_i32m1_tu (v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, x, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_tu (v3, v2, x, 4);
+ __riscv_th_vsw_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlw.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vx\tv[0-9]+,\s*v[0-9]+,\s*[a-x0-9]+
+** th.vadd\.vx\tv[1-9][0-9]?,\s*v[0-9]+,\s*[a-x0-9]+,\s*v0.t
+** th.vsw.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, int32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlw_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlw_v_i32m1_m (mask, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, x, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_m (mask, v3, x, 4);
+ __riscv_th_vsw_v_i32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlw.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vx\tv[0-9]+,\s*v[0-9]+,\s*[a-x0-9]+
+** th.vadd\.vx\tv[1-9][0-9]?,\s*v[0-9]+,\s*[a-x0-9]+,\s*v0.t
+** th.vsw.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, int32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vint32m1_t v = __riscv_th_vlw_v_i32m1 (in, 4);
+ vint32m1_t v2 = __riscv_th_vlw_v_i32m1_tumu (mask, v, in, 4);
+ vint32m1_t v3 = __riscv_vadd_vx_i32m1 (v2, x, 4);
+ vint32m1_t v4 = __riscv_vadd_vx_i32m1_tumu (mask, v3, v2, x, 4);
+ __riscv_th_vsw_v_i32m1 (out, v4, 4);
+}
\ No newline at end of file
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c
new file mode 100644
index 00000000000..aaa75be023d
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/xtheadvector/vlwu-vsw.c
@@ -0,0 +1,68 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv32gcxtheadvector -mabi=ilp32d -O3" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+#include "riscv_vector.h"
+
+/*
+** f1:
+** th.vsetivli\tzero,4,e32,m1,tu,ma
+** th.vlwu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlwu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vsw\.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f1 (void * in, void *out, uint32_t x)
+{
+ vuint32m1_t v = __riscv_th_vlwu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlwu_v_u32m1_tu (v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tu (v3, v2, -16, 4);
+ __riscv_th_vsw_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f2:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,ta,ma
+** th.vlwu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsw.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f2 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlwu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlwu_v_u32m1_m (mask, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_m (mask, v3, -16, 4);
+ __riscv_th_vsw_v_u32m1 (out, v4, 4);
+}
+
+/*
+** f3:
+** th.vsetvli\t[a-x0-9]+,zero,e8,mf4,ta,ma
+** th.vlm.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vsetivli\tzero,4,e32,m1,tu,mu
+** th.vlwu\.v\tv[0-9]+,0\([a-x0-9]+\)
+** th.vlwu.v\tv[0-9]+,0\([a-x0-9]+\),v0.t
+** th.vadd\.vi\tv[0-9]+,\s*v[0-9]+,\s*-16
+** th.vadd\.vi\tv[1-9][0-9]?,\s*v[0-9]+,\s*-16,\s*v0.t
+** th.vsw.v\tv[0-9]+,0\([a-x0-9]+\)
+** ret
+*/
+void f3 (void * in, void *out, uint32_t x)
+{
+ vbool32_t mask = *(vbool32_t*)in;
+ asm volatile ("":::"memory");
+ vuint32m1_t v = __riscv_th_vlwu_v_u32m1 (in, 4);
+ vuint32m1_t v2 = __riscv_th_vlwu_v_u32m1_tumu (mask, v, in, 4);
+ vuint32m1_t v3 = __riscv_vadd_vx_u32m1 (v2, -16, 4);
+ vuint32m1_t v4 = __riscv_vadd_vx_u32m1_tumu (mask, v3, v2, -16, 4);
+ __riscv_th_vsw_v_u32m1 (out, v4, 4);
+}
\ No newline at end of file
--
2.17.1
[-- Attachment #2: temp4cj.png --]
[-- Type: application/octet-stream, Size: 500533 bytes --]
[-- Attachment #3: temp4cj.png --]
[-- Type: application/octet-stream, Size: 676546 bytes --]
^ permalink raw reply [flat|nested] 17+ messages in thread
end of thread, other threads:[~2024-01-12 3:26 UTC | newest]
Thread overview: 17+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <84F7577214BA46AD+202401101900269053910@rivai.ai>
2024-01-10 11:06 ` Re：Re：[PATCH v5] RISC-V: Add support for xtheadvector-specific intrinsics joshua
2024-01-10 11:08 ` joshua
2024-01-10 11:09 ` Re：[PATCH " juzhe.zhong
2024-01-10 11:14 ` Re：Re：[PATCH " joshua
2024-01-10 11:16 ` Re：[PATCH " juzhe.zhong
2024-01-12 1:14 回复：[PATCH " juzhe.zhong
2024-01-12 3:26 ` Re：Re：[PATCH " joshua
-- strict thread matches above, loose matches on Subject: below --
2024-01-11 12:33 Re：[PATCH " juzhe.zhong
2024-01-11 12:36 ` Re：Re：[PATCH " joshua
2024-01-11 12:28 Re：[PATCH " juzhe.zhong
2024-01-11 12:31 ` Re：Re：[PATCH " joshua
2024-01-11 12:13 Re：[PATCH " juzhe.zhong
2024-01-11 12:18 ` Re：Re：[PATCH " joshua
2024-01-11 10:55 Re：[PATCH " juzhe.zhong
2024-01-11 14:11 ` Re：Re：[PATCH " joshua
2024-01-11 9:32 Re：[PATCH " juzhe.zhong
2024-01-11 9:38 ` Re：Re：[PATCH " joshua
2024-01-11 12:05 ` joshua
2024-01-11 9:28 Re：[PATCH " juzhe.zhong
2024-01-11 9:35 ` Re：Re：[PATCH " joshua
2024-01-11 9:24 Re：[PATCH " juzhe.zhong
2024-01-11 9:29 ` Re：Re：[PATCH " joshua
2024-01-11 9:17 Re：[PATCH " juzhe.zhong
2024-01-11 9:21 ` Re：Re：[PATCH " joshua
2024-01-11 9:26 ` joshua
2024-01-10 10:03 Re：[PATCH " juzhe.zhong
2024-01-10 10:57 ` Re：Re：[PATCH " joshua
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).