From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-oi1-x229.google.com (mail-oi1-x229.google.com [IPv6:2607:f8b0:4864:20::229]) by sourceware.org (Postfix) with ESMTPS id 2F3033858D28 for ; Mon, 30 Jan 2023 16:49:47 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 2F3033858D28 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=gmail.com Received: by mail-oi1-x229.google.com with SMTP id r205so10555783oib.9 for ; Mon, 30 Jan 2023 08:49:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=BN5CsC3O0eldlK5mLBo7ADBhJ1uHwoHeuyrdNqkMl+Y=; b=atqyo+jhnHojGBWAznikt1GZ0GGn2cJ+asknkgQ9OEXzwX8Mrj0lE2v2BLktELmhoR 7dp27QG3ASMUJV9QQsVHjnxSliiU1UOIxe3HJS+VBEfCa0VJEAqi7hnF2UVGkxkDbxJe 8YnU9aAvL9DnqWh9lPKza1ddC37FV/1khCYb37hCSNVLl3SXG6lVzABnD4syYfsiUKcV 7rJSjitxjrwTrN9SEjNKrCj7C/xUH/wGRfWKq2mDkOcwCYYefLbYpClXryr3lLdfgUK1 yOhHhPumgAPJ5rteg5B0E6mSnyN36nix9IVe/RkpU/5LCExoKvtHyzIOWbGagKtbZ8Fp GHLQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=BN5CsC3O0eldlK5mLBo7ADBhJ1uHwoHeuyrdNqkMl+Y=; b=Mlh8xOHx4IxAYGnRmUScQrutQdffDeIhg2cmybNREycpQ8rJmxEktBQGPNEv0hFH3S JjpaChtRTOV3oEzswQDsqml2VVTLaWFPkHyEAhqokM8ZKR8znlIvEA4rL/v2buEoLFd0 U+QLgh3+/xAmH0UMMakdflxdbLEgcvnzo9aKaTMu5uNyJB4L+LNIreyf5ral6QoHemL2 PqNkxj99lHlj6qlcYaMpdpIE8RdBBa3C0SiuFt46ssPSiklx8KPKUQQQYol/Nen8vSxU RXn80S7hEQiDpqbb/wWwhCq6XL44IrTSy4dlmjW8rkLEmxH/5A2C1oQvMDAxZzRurSs7 uIRA== X-Gm-Message-State: AO0yUKVApwZFh0CMH+uaw25FGyJcYFriTbKNnZkT4m17N+bTv7OYObjg TgMJp8vrb17CgOrDTcN5+6THvBw5c7HhLAXskrk= X-Google-Smtp-Source: AK7set9ZLlaJ4pba5xjgMNwQMUS7wXLFhATpZc4hPsp1MDZ718maPFDMqxBijO0ULCfLsctJXrONXpQ/9sqqp7Jl0o0= X-Received: by 2002:a54:4702:0:b0:378:30b3:b3ea with SMTP id k2-20020a544702000000b0037830b3b3eamr239959oik.11.1675097385781; Mon, 30 Jan 2023 08:49:45 -0800 (PST) MIME-Version: 1.0 References: <20230129153233.219454-1-juzhe.zhong@rivai.ai> In-Reply-To: <20230129153233.219454-1-juzhe.zhong@rivai.ai> From: Kito Cheng Date: Tue, 31 Jan 2023 00:49:18 +0800 Message-ID: Subject: Re: [PATCH] RISC-V: Add indexed loads/stores C/C++ intrinsic support To: juzhe.zhong@rivai.ai Cc: gcc-patches@gcc.gnu.org, palmer@dabbelt.com Content-Type: text/plain; charset="UTF-8" X-Spam-Status: No, score=-7.8 required=5.0 tests=BAYES_00,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM,GIT_PATCH_0,KAM_ASCII_DIVIDERS,KAM_SHORT,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS,TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: committed, thanks! On Sun, Jan 29, 2023 at 11:33 PM wrote: > > From: Ju-Zhe Zhong > > gcc/ChangeLog: > > * config/riscv/riscv-protos.h (get_vector_mode): New function. > * config/riscv/riscv-v.cc (get_vector_mode): Ditto. > * config/riscv/riscv-vector-builtins-bases.cc (enum lst_type): New enum. > (class loadstore): Adjust for indexed loads/stores support. > (BASE): Ditto. > * config/riscv/riscv-vector-builtins-bases.h: New function declare. > * config/riscv/riscv-vector-builtins-functions.def (vluxei8): Ditto. > (vluxei16): Ditto. > (vluxei32): Ditto. > (vluxei64): Ditto. > (vloxei8): Ditto. > (vloxei16): Ditto. > (vloxei32): Ditto. > (vloxei64): Ditto. > (vsuxei8): Ditto. > (vsuxei16): Ditto. > (vsuxei32): Ditto. > (vsuxei64): Ditto. > (vsoxei8): Ditto. > (vsoxei16): Ditto. > (vsoxei32): Ditto. > (vsoxei64): Ditto. > * config/riscv/riscv-vector-builtins-shapes.cc (struct indexed_loadstore_def): New class. > (SHAPE): Ditto. > * config/riscv/riscv-vector-builtins-shapes.h: Ditto. > * config/riscv/riscv-vector-builtins.cc (required_extensions_p): Adjust for indexed loads/stores support. > (check_required_extensions): Ditto. > (rvv_arg_type_info::get_base_vector_type): New function. > (rvv_arg_type_info::get_tree_type): Ditto. > (function_builder::add_unique_function): Adjust for indexed loads/stores support. > (function_expander::use_exact_insn): New function. > * config/riscv/riscv-vector-builtins.h (enum rvv_base_type): Adjust for indexed loads/stores support. > (struct rvv_arg_type_info): Ditto. > (function_expander::index_mode): New function. > (function_base::apply_tail_policy_p): Ditto. > (function_base::apply_mask_policy_p): Ditto. > * config/riscv/vector-iterators.md (unspec): New unspec. > * config/riscv/vector.md (unspec): Ditto. > (@pred_indexed_load): New pattern. > (@pred_indexed_store): Ditto. > (@pred_indexed_load): Ditto. > (@pred_indexed_store): Ditto. > (@pred_indexed_load): Ditto. > (@pred_indexed_store): Ditto. > (@pred_indexed_load): Ditto. > (@pred_indexed_store): Ditto. > (@pred_indexed_load): Ditto. > (@pred_indexed_store): Ditto. > (@pred_indexed_load): Ditto. > (@pred_indexed_store): Ditto. > (@pred_indexed_load): Ditto. > (@pred_indexed_store): Ditto. > > --- > gcc/config/riscv/riscv-protos.h | 1 + > gcc/config/riscv/riscv-v.cc | 22 ++ > .../riscv/riscv-vector-builtins-bases.cc | 92 +++++- > .../riscv/riscv-vector-builtins-bases.h | 16 + > .../riscv/riscv-vector-builtins-functions.def | 16 + > .../riscv/riscv-vector-builtins-shapes.cc | 49 +++ > .../riscv/riscv-vector-builtins-shapes.h | 1 + > gcc/config/riscv/riscv-vector-builtins.cc | 249 +++++++++++++- > gcc/config/riscv/riscv-vector-builtins.h | 36 ++ > gcc/config/riscv/vector-iterators.md | 87 +++++ > gcc/config/riscv/vector.md | 309 +++++++++++++++++- > 11 files changed, 845 insertions(+), 33 deletions(-) > > diff --git a/gcc/config/riscv/riscv-protos.h b/gcc/config/riscv/riscv-protos.h > index 1facad74b3c..ceae4007fd1 100644 > --- a/gcc/config/riscv/riscv-protos.h > +++ b/gcc/config/riscv/riscv-protos.h > @@ -172,6 +172,7 @@ enum mask_policy > enum tail_policy get_prefer_tail_policy (); > enum mask_policy get_prefer_mask_policy (); > rtx get_avl_type_rtx (enum avl_type); > +opt_machine_mode get_vector_mode (scalar_mode, poly_uint64); > } > > /* We classify builtin types into two classes: > diff --git a/gcc/config/riscv/riscv-v.cc b/gcc/config/riscv/riscv-v.cc > index 47f2cef459b..992b3437926 100644 > --- a/gcc/config/riscv/riscv-v.cc > +++ b/gcc/config/riscv/riscv-v.cc > @@ -349,4 +349,26 @@ get_avl_type_rtx (enum avl_type type) > return gen_int_mode (type, Pmode); > } > > +/* Return the RVV vector mode that has NUNITS elements of mode INNER_MODE. > + This function is not only used by builtins, but also will be used by > + auto-vectorization in the future. */ > +opt_machine_mode > +get_vector_mode (scalar_mode inner_mode, poly_uint64 nunits) > +{ > + enum mode_class mclass; > + if (inner_mode == E_BImode) > + mclass = MODE_VECTOR_BOOL; > + else if (FLOAT_MODE_P (inner_mode)) > + mclass = MODE_VECTOR_FLOAT; > + else > + mclass = MODE_VECTOR_INT; > + machine_mode mode; > + FOR_EACH_MODE_IN_CLASS (mode, mclass) > + if (inner_mode == GET_MODE_INNER (mode) > + && known_eq (nunits, GET_MODE_NUNITS (mode)) > + && riscv_v_ext_vector_mode_p (mode)) > + return mode; > + return opt_machine_mode (); > +} > + > } // namespace riscv_vector > diff --git a/gcc/config/riscv/riscv-vector-builtins-bases.cc b/gcc/config/riscv/riscv-vector-builtins-bases.cc > index f9a16c68e07..129e89f443e 100644 > --- a/gcc/config/riscv/riscv-vector-builtins-bases.cc > +++ b/gcc/config/riscv/riscv-vector-builtins-bases.cc > @@ -48,6 +48,16 @@ using namespace riscv_vector; > > namespace riscv_vector { > > +/* Enumerates types of loads/stores operations. > + It's only used in here so we don't define it > + in riscv-vector-builtins-bases.h. */ > +enum lst_type > +{ > + LST_UNIT_STRIDE, > + LST_STRIDED, > + LST_INDEXED, > +}; > + > /* Implements vsetvl && vsetvlmax. */ > template > class vsetvl : public function_base > @@ -84,10 +94,16 @@ public: > } > }; > > -/* Implements vle.v/vse.v/vlm.v/vsm.v/vlse.v/vsse.v codegen. */ > -template > +/* Implements > + * vle.v/vse.v/vlm.v/vsm.v/vlse.v/vsse.v/vluxei.v/vloxei.v/vsuxei.v/vsoxei.v > + * codegen. */ > +template > class loadstore : public function_base > { > +public: > + bool apply_tail_policy_p () const override { return !STORE_P; } > + bool apply_mask_policy_p () const override { return !STORE_P; } > + > unsigned int call_properties (const function_instance &) const override > { > if (STORE_P) > @@ -98,27 +114,39 @@ class loadstore : public function_base > > bool can_be_overloaded_p (enum predication_type_index pred) const override > { > - if (STORE_P) > + if (STORE_P || LST_TYPE == LST_INDEXED) > return true; > return pred != PRED_TYPE_none && pred != PRED_TYPE_mu; > } > > rtx expand (function_expander &e) const override > { > - if (STORE_P) > + if (LST_TYPE == LST_INDEXED) > + { > + int unspec = ORDERED_P ? UNSPEC_ORDERED : UNSPEC_UNORDERED; > + if (STORE_P) > + return e.use_exact_insn ( > + code_for_pred_indexed_store (unspec, e.vector_mode (), > + e.index_mode ())); > + else > + return e.use_exact_insn ( > + code_for_pred_indexed_load (unspec, e.vector_mode (), > + e.index_mode ())); > + } > + else if (LST_TYPE == LST_STRIDED) > { > - if (STRIDED_P) > + if (STORE_P) > return e.use_contiguous_store_insn ( > code_for_pred_strided_store (e.vector_mode ())); > else > - return e.use_contiguous_store_insn ( > - code_for_pred_store (e.vector_mode ())); > + return e.use_contiguous_load_insn ( > + code_for_pred_strided_load (e.vector_mode ())); > } > else > { > - if (STRIDED_P) > - return e.use_contiguous_load_insn ( > - code_for_pred_strided_load (e.vector_mode ())); > + if (STORE_P) > + return e.use_contiguous_store_insn ( > + code_for_pred_store (e.vector_mode ())); > else > return e.use_contiguous_load_insn ( > code_for_pred_mov (e.vector_mode ())); > @@ -128,12 +156,28 @@ class loadstore : public function_base > > static CONSTEXPR const vsetvl vsetvl_obj; > static CONSTEXPR const vsetvl vsetvlmax_obj; > -static CONSTEXPR const loadstore vle_obj; > -static CONSTEXPR const loadstore vse_obj; > -static CONSTEXPR const loadstore vlm_obj; > -static CONSTEXPR const loadstore vsm_obj; > -static CONSTEXPR const loadstore vlse_obj; > -static CONSTEXPR const loadstore vsse_obj; > +static CONSTEXPR const loadstore vle_obj; > +static CONSTEXPR const loadstore vse_obj; > +static CONSTEXPR const loadstore vlm_obj; > +static CONSTEXPR const loadstore vsm_obj; > +static CONSTEXPR const loadstore vlse_obj; > +static CONSTEXPR const loadstore vsse_obj; > +static CONSTEXPR const loadstore vluxei8_obj; > +static CONSTEXPR const loadstore vluxei16_obj; > +static CONSTEXPR const loadstore vluxei32_obj; > +static CONSTEXPR const loadstore vluxei64_obj; > +static CONSTEXPR const loadstore vloxei8_obj; > +static CONSTEXPR const loadstore vloxei16_obj; > +static CONSTEXPR const loadstore vloxei32_obj; > +static CONSTEXPR const loadstore vloxei64_obj; > +static CONSTEXPR const loadstore vsuxei8_obj; > +static CONSTEXPR const loadstore vsuxei16_obj; > +static CONSTEXPR const loadstore vsuxei32_obj; > +static CONSTEXPR const loadstore vsuxei64_obj; > +static CONSTEXPR const loadstore vsoxei8_obj; > +static CONSTEXPR const loadstore vsoxei16_obj; > +static CONSTEXPR const loadstore vsoxei32_obj; > +static CONSTEXPR const loadstore vsoxei64_obj; > > /* Declare the function base NAME, pointing it to an instance > of class _obj. */ > @@ -148,5 +192,21 @@ BASE (vlm) > BASE (vsm) > BASE (vlse) > BASE (vsse) > +BASE (vluxei8) > +BASE (vluxei16) > +BASE (vluxei32) > +BASE (vluxei64) > +BASE (vloxei8) > +BASE (vloxei16) > +BASE (vloxei32) > +BASE (vloxei64) > +BASE (vsuxei8) > +BASE (vsuxei16) > +BASE (vsuxei32) > +BASE (vsuxei64) > +BASE (vsoxei8) > +BASE (vsoxei16) > +BASE (vsoxei32) > +BASE (vsoxei64) > > } // end namespace riscv_vector > diff --git a/gcc/config/riscv/riscv-vector-builtins-bases.h b/gcc/config/riscv/riscv-vector-builtins-bases.h > index 93999e2cbee..238c01dbf1f 100644 > --- a/gcc/config/riscv/riscv-vector-builtins-bases.h > +++ b/gcc/config/riscv/riscv-vector-builtins-bases.h > @@ -32,6 +32,22 @@ extern const function_base *const vlm; > extern const function_base *const vsm; > extern const function_base *const vlse; > extern const function_base *const vsse; > +extern const function_base *const vluxei8; > +extern const function_base *const vluxei16; > +extern const function_base *const vluxei32; > +extern const function_base *const vluxei64; > +extern const function_base *const vloxei8; > +extern const function_base *const vloxei16; > +extern const function_base *const vloxei32; > +extern const function_base *const vloxei64; > +extern const function_base *const vsuxei8; > +extern const function_base *const vsuxei16; > +extern const function_base *const vsuxei32; > +extern const function_base *const vsuxei64; > +extern const function_base *const vsoxei8; > +extern const function_base *const vsoxei16; > +extern const function_base *const vsoxei32; > +extern const function_base *const vsoxei64; > } > > } // end namespace riscv_vector > diff --git a/gcc/config/riscv/riscv-vector-builtins-functions.def b/gcc/config/riscv/riscv-vector-builtins-functions.def > index 1ddde7b9d76..9719b9b4bf1 100644 > --- a/gcc/config/riscv/riscv-vector-builtins-functions.def > +++ b/gcc/config/riscv/riscv-vector-builtins-functions.def > @@ -46,5 +46,21 @@ DEF_RVV_FUNCTION (vlm, loadstore, none_preds, b_v_scalar_const_ptr_ops) > DEF_RVV_FUNCTION (vsm, loadstore, none_preds, b_v_scalar_ptr_ops) > DEF_RVV_FUNCTION (vlse, loadstore, full_preds, all_v_scalar_const_ptr_ptrdiff_ops) > DEF_RVV_FUNCTION (vsse, loadstore, none_m_preds, all_v_scalar_ptr_ptrdiff_ops) > +DEF_RVV_FUNCTION (vluxei8, indexed_loadstore, full_preds, all_v_scalar_const_ptr_uint8_index_ops) > +DEF_RVV_FUNCTION (vluxei16, indexed_loadstore, full_preds, all_v_scalar_const_ptr_uint16_index_ops) > +DEF_RVV_FUNCTION (vluxei32, indexed_loadstore, full_preds, all_v_scalar_const_ptr_uint32_index_ops) > +DEF_RVV_FUNCTION (vluxei64, indexed_loadstore, full_preds, all_v_scalar_const_ptr_uint64_index_ops) > +DEF_RVV_FUNCTION (vloxei8, indexed_loadstore, full_preds, all_v_scalar_const_ptr_uint8_index_ops) > +DEF_RVV_FUNCTION (vloxei16, indexed_loadstore, full_preds, all_v_scalar_const_ptr_uint16_index_ops) > +DEF_RVV_FUNCTION (vloxei32, indexed_loadstore, full_preds, all_v_scalar_const_ptr_uint32_index_ops) > +DEF_RVV_FUNCTION (vloxei64, indexed_loadstore, full_preds, all_v_scalar_const_ptr_uint64_index_ops) > +DEF_RVV_FUNCTION (vsuxei8, indexed_loadstore, none_m_preds, all_v_scalar_ptr_uint8_index_ops) > +DEF_RVV_FUNCTION (vsuxei16, indexed_loadstore, none_m_preds, all_v_scalar_ptr_uint16_index_ops) > +DEF_RVV_FUNCTION (vsuxei32, indexed_loadstore, none_m_preds, all_v_scalar_ptr_uint32_index_ops) > +DEF_RVV_FUNCTION (vsuxei64, indexed_loadstore, none_m_preds, all_v_scalar_ptr_uint64_index_ops) > +DEF_RVV_FUNCTION (vsoxei8, indexed_loadstore, none_m_preds, all_v_scalar_ptr_uint8_index_ops) > +DEF_RVV_FUNCTION (vsoxei16, indexed_loadstore, none_m_preds, all_v_scalar_ptr_uint16_index_ops) > +DEF_RVV_FUNCTION (vsoxei32, indexed_loadstore, none_m_preds, all_v_scalar_ptr_uint32_index_ops) > +DEF_RVV_FUNCTION (vsoxei64, indexed_loadstore, none_m_preds, all_v_scalar_ptr_uint64_index_ops) > > #undef DEF_RVV_FUNCTION > diff --git a/gcc/config/riscv/riscv-vector-builtins-shapes.cc b/gcc/config/riscv/riscv-vector-builtins-shapes.cc > index ef3a31555db..d261dfbceb7 100644 > --- a/gcc/config/riscv/riscv-vector-builtins-shapes.cc > +++ b/gcc/config/riscv/riscv-vector-builtins-shapes.cc > @@ -137,8 +137,57 @@ struct loadstore_def : public build_base > } > }; > > +/* indexed_loadstore_def class. */ > +struct indexed_loadstore_def : public function_shape > +{ > + void build (function_builder &b, > + const function_group_info &group) const override > + { > + for (unsigned int pred_idx = 0; group.preds[pred_idx] != NUM_PRED_TYPES; > + ++pred_idx) > + { > + for (unsigned int vec_type_idx = 0; > + group.ops_infos.types[vec_type_idx].index != NUM_VECTOR_TYPES; > + ++vec_type_idx) > + { > + tree index_type = group.ops_infos.args[1].get_tree_type ( > + group.ops_infos.types[vec_type_idx].index); > + if (!index_type) > + continue; > + build_one (b, group, pred_idx, vec_type_idx); > + } > + } > + } > + > + char *get_name (function_builder &b, const function_instance &instance, > + bool overloaded_p) const override > + { > + /* Return nullptr if it can not be overloaded. */ > + if (overloaded_p && !instance.base->can_be_overloaded_p (instance.pred)) > + return nullptr; > + > + b.append_base_name (instance.base_name); > + /* vop_v --> vop_v_. */ > + if (!overloaded_p) > + { > + /* vop --> vop_v. */ > + b.append_name (operand_suffixes[instance.op_info->op]); > + /* vop_v --> vop_v_. */ > + b.append_name (type_suffixes[instance.type.index].vector); > + } > + > + /* According to rvv-intrinsic-doc, it does not add "_m" suffix > + for vop_m C++ overloaded API. */ > + if (overloaded_p && instance.pred == PRED_TYPE_m) > + return b.finish_name (); > + b.append_name (predication_suffixes[instance.pred]); > + return b.finish_name (); > + } > +}; > + > SHAPE(vsetvl, vsetvl) > SHAPE(vsetvl, vsetvlmax) > SHAPE(loadstore, loadstore) > +SHAPE(indexed_loadstore, indexed_loadstore) > > } // end namespace riscv_vector > diff --git a/gcc/config/riscv/riscv-vector-builtins-shapes.h b/gcc/config/riscv/riscv-vector-builtins-shapes.h > index 642769208ee..05bc68b7f12 100644 > --- a/gcc/config/riscv/riscv-vector-builtins-shapes.h > +++ b/gcc/config/riscv/riscv-vector-builtins-shapes.h > @@ -27,6 +27,7 @@ namespace shapes { > extern const function_shape *const vsetvl; > extern const function_shape *const vsetvlmax; > extern const function_shape *const loadstore; > +extern const function_shape *const indexed_loadstore; > } > > } // end namespace riscv_vector > diff --git a/gcc/config/riscv/riscv-vector-builtins.cc b/gcc/config/riscv/riscv-vector-builtins.cc > index 593a5f08e69..df4e2ee1841 100644 > --- a/gcc/config/riscv/riscv-vector-builtins.cc > +++ b/gcc/config/riscv/riscv-vector-builtins.cc > @@ -180,6 +180,58 @@ static CONSTEXPR const rvv_arg_type_info scalar_ptr_ptrdiff_args[] > rvv_arg_type_info (RVV_BASE_ptrdiff), rvv_arg_type_info (RVV_BASE_vector), > rvv_arg_type_info_end}; > > +/* A list of args for vector_type func (const scalar_type *, uint8_index_type) > + * function. */ > +static CONSTEXPR const rvv_arg_type_info scalar_const_ptr_uint8_index_args[] > + = {rvv_arg_type_info (RVV_BASE_scalar_const_ptr), > + rvv_arg_type_info (RVV_BASE_uint8_index), rvv_arg_type_info_end}; > + > +/* A list of args for vector_type func (const scalar_type *, uint16_index_type) > + * function. */ > +static CONSTEXPR const rvv_arg_type_info scalar_const_ptr_uint16_index_args[] > + = {rvv_arg_type_info (RVV_BASE_scalar_const_ptr), > + rvv_arg_type_info (RVV_BASE_uint16_index), rvv_arg_type_info_end}; > + > +/* A list of args for vector_type func (const scalar_type *, uint32_index_type) > + * function. */ > +static CONSTEXPR const rvv_arg_type_info scalar_const_ptr_uint32_index_args[] > + = {rvv_arg_type_info (RVV_BASE_scalar_const_ptr), > + rvv_arg_type_info (RVV_BASE_uint32_index), rvv_arg_type_info_end}; > + > +/* A list of args for vector_type func (const scalar_type *, uint64_index_type) > + * function. */ > +static CONSTEXPR const rvv_arg_type_info scalar_const_ptr_uint64_index_args[] > + = {rvv_arg_type_info (RVV_BASE_scalar_const_ptr), > + rvv_arg_type_info (RVV_BASE_uint64_index), rvv_arg_type_info_end}; > + > +/* A list of args for void func (scalar_type *, uint8_index_type, vector_type) > + * function. */ > +static CONSTEXPR const rvv_arg_type_info scalar_ptr_uint8_index_args[] > + = {rvv_arg_type_info (RVV_BASE_scalar_ptr), > + rvv_arg_type_info (RVV_BASE_uint8_index), > + rvv_arg_type_info (RVV_BASE_vector), rvv_arg_type_info_end}; > + > +/* A list of args for void func (scalar_type *, uint16_index_type, vector_type) > + * function. */ > +static CONSTEXPR const rvv_arg_type_info scalar_ptr_uint16_index_args[] > + = {rvv_arg_type_info (RVV_BASE_scalar_ptr), > + rvv_arg_type_info (RVV_BASE_uint16_index), > + rvv_arg_type_info (RVV_BASE_vector), rvv_arg_type_info_end}; > + > +/* A list of args for void func (scalar_type *, uint32_index_type, vector_type) > + * function. */ > +static CONSTEXPR const rvv_arg_type_info scalar_ptr_uint32_index_args[] > + = {rvv_arg_type_info (RVV_BASE_scalar_ptr), > + rvv_arg_type_info (RVV_BASE_uint32_index), > + rvv_arg_type_info (RVV_BASE_vector), rvv_arg_type_info_end}; > + > +/* A list of args for void func (scalar_type *, uint64_index_type, vector_type) > + * function. */ > +static CONSTEXPR const rvv_arg_type_info scalar_ptr_uint64_index_args[] > + = {rvv_arg_type_info (RVV_BASE_scalar_ptr), > + rvv_arg_type_info (RVV_BASE_uint64_index), > + rvv_arg_type_info (RVV_BASE_vector), rvv_arg_type_info_end}; > + > /* A list of none preds that will be registered for intrinsic functions. */ > static CONSTEXPR const predication_type_index none_preds[] > = {PRED_TYPE_none, NUM_PRED_TYPES}; > @@ -248,6 +300,38 @@ static CONSTEXPR const rvv_op_info all_v_scalar_const_ptr_ptrdiff_ops > rvv_arg_type_info (RVV_BASE_vector), /* Return type */ > scalar_const_ptr_ptrdiff_args /* Args */}; > > +/* A static operand information for vector_type func (const scalar_type *, > + * uint8_index_type) function registration. */ > +static CONSTEXPR const rvv_op_info all_v_scalar_const_ptr_uint8_index_ops > + = {all_ops, /* Types */ > + OP_TYPE_v, /* Suffix */ > + rvv_arg_type_info (RVV_BASE_vector), /* Return type */ > + scalar_const_ptr_uint8_index_args /* Args */}; > + > +/* A static operand information for vector_type func (const scalar_type *, > + * uint16_index_type) function registration. */ > +static CONSTEXPR const rvv_op_info all_v_scalar_const_ptr_uint16_index_ops > + = {all_ops, /* Types */ > + OP_TYPE_v, /* Suffix */ > + rvv_arg_type_info (RVV_BASE_vector), /* Return type */ > + scalar_const_ptr_uint16_index_args /* Args */}; > + > +/* A static operand information for vector_type func (const scalar_type *, > + * uint32_index_type) function registration. */ > +static CONSTEXPR const rvv_op_info all_v_scalar_const_ptr_uint32_index_ops > + = {all_ops, /* Types */ > + OP_TYPE_v, /* Suffix */ > + rvv_arg_type_info (RVV_BASE_vector), /* Return type */ > + scalar_const_ptr_uint32_index_args /* Args */}; > + > +/* A static operand information for vector_type func (const scalar_type *, > + * uint64_index_type) function registration. */ > +static CONSTEXPR const rvv_op_info all_v_scalar_const_ptr_uint64_index_ops > + = {all_ops, /* Types */ > + OP_TYPE_v, /* Suffix */ > + rvv_arg_type_info (RVV_BASE_vector), /* Return type */ > + scalar_const_ptr_uint64_index_args /* Args */}; > + > /* A static operand information for void func (scalar_type *, ptrdiff_t, > * vector_type) function registration. */ > static CONSTEXPR const rvv_op_info all_v_scalar_ptr_ptrdiff_ops > @@ -256,6 +340,38 @@ static CONSTEXPR const rvv_op_info all_v_scalar_ptr_ptrdiff_ops > rvv_arg_type_info (RVV_BASE_void), /* Return type */ > scalar_ptr_ptrdiff_args /* Args */}; > > +/* A static operand information for void func (scalar_type *, uint8_index_type, > + * vector_type) function registration. */ > +static CONSTEXPR const rvv_op_info all_v_scalar_ptr_uint8_index_ops > + = {all_ops, /* Types */ > + OP_TYPE_v, /* Suffix */ > + rvv_arg_type_info (RVV_BASE_void), /* Return type */ > + scalar_ptr_uint8_index_args /* Args */}; > + > +/* A static operand information for void func (scalar_type *, uint16_index_type, > + * vector_type) function registration. */ > +static CONSTEXPR const rvv_op_info all_v_scalar_ptr_uint16_index_ops > + = {all_ops, /* Types */ > + OP_TYPE_v, /* Suffix */ > + rvv_arg_type_info (RVV_BASE_void), /* Return type */ > + scalar_ptr_uint16_index_args /* Args */}; > + > +/* A static operand information for void func (scalar_type *, uint32_index_type, > + * vector_type) function registration. */ > +static CONSTEXPR const rvv_op_info all_v_scalar_ptr_uint32_index_ops > + = {all_ops, /* Types */ > + OP_TYPE_v, /* Suffix */ > + rvv_arg_type_info (RVV_BASE_void), /* Return type */ > + scalar_ptr_uint32_index_args /* Args */}; > + > +/* A static operand information for void func (scalar_type *, uint64_index_type, > + * vector_type) function registration. */ > +static CONSTEXPR const rvv_op_info all_v_scalar_ptr_uint64_index_ops > + = {all_ops, /* Types */ > + OP_TYPE_v, /* Suffix */ > + rvv_arg_type_info (RVV_BASE_void), /* Return type */ > + scalar_ptr_uint64_index_args /* Args */}; > + > /* A list of all RVV intrinsic functions. */ > static function_group_info function_groups[] = { > #define DEF_RVV_FUNCTION(NAME, SHAPE, PREDS, OPS_INFO) \ > @@ -444,11 +560,41 @@ register_vector_type (vector_type_index type) > builtin_types[type].vector_ptr = build_pointer_type (vectype); > } > > +/* Return true if the type has required_extensions. */ > +static bool > +required_extensions_p (enum rvv_base_type type) > +{ > + return type == RVV_BASE_vector || type == RVV_BASE_uint8_index > + || type == RVV_BASE_uint16_index || type == RVV_BASE_uint32_index > + || type == RVV_BASE_uint64_index; > +} > + > /* Check whether all the RVV_REQUIRE_* values in REQUIRED_EXTENSIONS are > enabled. */ > static bool > -check_required_extensions (uint64_t required_extensions) > +check_required_extensions (const function_instance &instance) > { > + rvv_type_info type_info = instance.type; > + uint64_t required_extensions = type_info.required_extensions; > + const rvv_op_info *op_info = instance.op_info; > + tree type = builtin_types[type_info.index].vector; > + for (unsigned i = 0; op_info->args[i].base_type != NUM_BASE_TYPES; ++i) > + { > + if (!required_extensions_p (op_info->args[i].base_type)) > + continue; > + > + enum vector_type_index vector_type > + = op_info->args[i].get_base_vector_type (type); > + if (vector_type == NUM_VECTOR_TYPES) > + continue; > + required_extensions |= op_info->types[vector_type].required_extensions; > + > + /* According to RVV ISA, EEW=64 index of indexed loads/stores require > + XLEN = 64. */ > + if (op_info->args[i].base_type == RVV_BASE_uint64_index) > + required_extensions |= RVV_REQUIRE_RV64BIT; > + } > + > uint64_t riscv_isa_flags = 0; > > if (TARGET_VECTOR_ELEN_FP_32) > @@ -502,6 +648,56 @@ get_mask_policy_for_pred (enum predication_type_index pred) > return gen_int_mode (get_prefer_mask_policy (), Pmode); > } > > +vector_type_index > +rvv_arg_type_info::get_base_vector_type (tree type) const > +{ > + if (!type) > + return NUM_VECTOR_TYPES; > + poly_int64 nunits = GET_MODE_NUNITS (TYPE_MODE (type)); > + machine_mode inner_mode; > + bool unsigned_p = TYPE_UNSIGNED (type); > + switch (base_type) > + { > + case RVV_BASE_uint8_index: > + inner_mode = E_QImode; > + unsigned_p = true; > + break; > + case RVV_BASE_uint16_index: > + inner_mode = E_HImode; > + unsigned_p = true; > + break; > + case RVV_BASE_uint32_index: > + inner_mode = E_SImode; > + unsigned_p = true; > + break; > + case RVV_BASE_uint64_index: > + inner_mode = E_DImode; > + unsigned_p = true; > + break; > + default: > + return NUM_VECTOR_TYPES; > + } > + > + opt_machine_mode mode > + = get_vector_mode (as_a (inner_mode), nunits); > + > + if (!mode.exists ()) > + return NUM_VECTOR_TYPES; > + for (unsigned int i = 0; i < NUM_VECTOR_TYPES + 1; i++) > + { > + tree vector_type = builtin_types[i].vector; > + if (!vector_type) > + continue; > + > + if (TYPE_UNSIGNED (vector_type) != unsigned_p) > + continue; > + > + if (TYPE_MODE (vector_type) == mode.require ()) > + return (enum vector_type_index) i; > + } > + return NUM_VECTOR_TYPES; > +} > + > tree > rvv_arg_type_info::get_tree_type (vector_type_index type_idx) const > { > @@ -537,9 +733,20 @@ rvv_arg_type_info::get_tree_type (vector_type_index type_idx) const > return long_unsigned_type_node; > case RVV_BASE_long: > return long_integer_type_node; > + case RVV_BASE_uint8_index: > + case RVV_BASE_uint16_index: > + case RVV_BASE_uint32_index: > + case RVV_BASE_uint64_index: > + if (get_base_vector_type (builtin_types[type_idx].vector) > + != NUM_VECTOR_TYPES) > + return builtin_types[get_base_vector_type ( > + builtin_types[type_idx].vector)].vector; > + break; > default: > gcc_unreachable (); > } > + /* Return NULL_TREE if the type we don't want to register. */ > + return NULL_TREE; > } > > function_instance::function_instance (const char *base_name_in, > @@ -839,7 +1046,7 @@ function_builder::add_unique_function (const function_instance &instance, > vec &argument_types) > { > /* Do not add this function if it is invalid. */ > - if (!check_required_extensions (instance.type.required_extensions)) > + if (!check_required_extensions (instance)) > return; > > /* Add the function under its full (unique) name. */ > @@ -918,6 +1125,44 @@ function_expander::add_mem_operand (machine_mode mode, unsigned argno) > add_fixed_operand (mem); > } > > +/* Implement the call using instruction ICODE, with a 1:1 mapping between > + arguments and input operands. */ > +rtx > +function_expander::use_exact_insn (insn_code icode) > +{ > + machine_mode mode = TYPE_MODE (TREE_TYPE (exp)); > + tree mask_type = builtin_types[mask_types[type.index]].vector; > + machine_mode mask_mode = TYPE_MODE (mask_type); > + > + /* Record the offset to get the argument. */ > + int arg_offset = 0; > + > + if (use_real_mask_p (pred)) > + add_input_operand (arg_offset++); > + else > + add_all_one_mask_operand (mask_mode); > + > + /* Store operation doesn't have merge operand. */ > + if (!function_returns_void_p ()) > + { > + if (use_real_merge_p (pred)) > + add_input_operand (arg_offset++); > + else > + add_vundef_operand (mode); > + } > + > + for (int argno = arg_offset; argno < call_expr_nargs (exp); argno++) > + add_input_operand (argno); > + > + if (base->apply_tail_policy_p ()) > + add_input_operand (Pmode, get_tail_policy_for_pred (pred)); > + if (base->apply_mask_policy_p ()) > + add_input_operand (Pmode, get_mask_policy_for_pred (pred)); > + > + add_input_operand (Pmode, get_avl_type_rtx (avl_type::NONVLMAX)); > + return generate_insn (icode); > +} > + > /* Use contiguous load INSN. */ > rtx > function_expander::use_contiguous_load_insn (insn_code icode) > diff --git a/gcc/config/riscv/riscv-vector-builtins.h b/gcc/config/riscv/riscv-vector-builtins.h > index fb3f818e15a..7a4a4d4f74d 100644 > --- a/gcc/config/riscv/riscv-vector-builtins.h > +++ b/gcc/config/riscv/riscv-vector-builtins.h > @@ -147,6 +147,10 @@ enum rvv_base_type > RVV_BASE_ptrdiff, > RVV_BASE_unsigned_long, > RVV_BASE_long, > + RVV_BASE_uint8_index, > + RVV_BASE_uint16_index, > + RVV_BASE_uint32_index, > + RVV_BASE_uint64_index, > NUM_BASE_TYPES > }; > > @@ -176,6 +180,7 @@ struct rvv_arg_type_info > {} > enum rvv_base_type base_type; > > + vector_type_index get_base_vector_type (tree type) const; > tree get_tree_type (vector_type_index) const; > }; > > @@ -325,7 +330,9 @@ public: > void add_mem_operand (machine_mode, unsigned); > > machine_mode vector_mode (void) const; > + machine_mode index_mode (void) const; > > + rtx use_exact_insn (insn_code); > rtx use_contiguous_load_insn (insn_code); > rtx use_contiguous_store_insn (insn_code); > rtx generate_insn (insn_code); > @@ -358,6 +365,12 @@ public: > /* Return true if intrinsics should apply vl operand. */ > virtual bool apply_vl_p () const; > > + /* Return true if intrinsics should apply ta operand. */ > + virtual bool apply_tail_policy_p () const; > + > + /* Return true if intrinsics should apply ma operand. */ > + virtual bool apply_mask_policy_p () const; > + > /* Return true if intrinsic can be overloaded. */ > virtual bool can_be_overloaded_p (enum predication_type_index) const; > > @@ -444,6 +457,13 @@ function_expander::vector_mode (void) const > return TYPE_MODE (builtin_types[type.index].vector); > } > > +/* Return the machine_mode of the corresponding index type. */ > +inline machine_mode > +function_expander::index_mode (void) const > +{ > + return TYPE_MODE (op_info->args[1].get_tree_type (type.index)); > +} > + > /* Default implementation of function_base::call_properties, with conservatively > correct behavior for floating-point instructions. */ > inline unsigned int > @@ -463,6 +483,22 @@ function_base::apply_vl_p () const > return true; > } > > +/* We choose to apply ta operand by default since most of the intrinsics > + has vl operand. */ > +inline bool > +function_base::apply_tail_policy_p () const > +{ > + return true; > +} > + > +/* We choose to apply ma operand by default since most of the intrinsics > + has ma operand. */ > +inline bool > +function_base::apply_mask_policy_p () const > +{ > + return true; > +} > + > /* Since most of intrinsics can be overloaded, we set it true by default. */ > inline bool > function_base::can_be_overloaded_p (enum predication_type_index) const > diff --git a/gcc/config/riscv/vector-iterators.md b/gcc/config/riscv/vector-iterators.md > index c1ec23452bc..2ac75b3f94c 100644 > --- a/gcc/config/riscv/vector-iterators.md > +++ b/gcc/config/riscv/vector-iterators.md > @@ -18,6 +18,18 @@ > ;; along with GCC; see the file COPYING3. If not see > ;; . > > +(define_c_enum "unspec" [ > + UNSPEC_VSETVL > + UNSPEC_VUNDEF > + UNSPEC_VPREDICATE > + UNSPEC_VLMAX > + UNSPEC_STRIDED > + > + ;; It's used to specify ordered/unorderd operation. > + UNSPEC_ORDERED > + UNSPEC_UNORDERED > +]) > + > (define_mode_iterator V [ > VNx1QI VNx2QI VNx4QI VNx8QI VNx16QI VNx32QI (VNx64QI "TARGET_MIN_VLEN > 32") > VNx1HI VNx2HI VNx4HI VNx8HI VNx16HI (VNx32HI "TARGET_MIN_VLEN > 32") > @@ -35,6 +47,75 @@ > (VNx8DF "TARGET_VECTOR_ELEN_FP_64") > ]) > > +(define_mode_iterator VNX1_QHSD [ > + VNx1QI VNx1HI VNx1SI > + (VNx1DI "TARGET_MIN_VLEN > 32") > + (VNx1SF "TARGET_VECTOR_ELEN_FP_32") > + (VNx1DF "TARGET_VECTOR_ELEN_FP_64") > +]) > + > +(define_mode_iterator VNX2_QHSD [ > + VNx2QI VNx2HI VNx2SI > + (VNx2DI "TARGET_MIN_VLEN > 32") > + (VNx2SF "TARGET_VECTOR_ELEN_FP_32") > + (VNx2DF "TARGET_VECTOR_ELEN_FP_64") > +]) > + > +(define_mode_iterator VNX4_QHSD [ > + VNx4QI VNx4HI VNx4SI > + (VNx4DI "TARGET_MIN_VLEN > 32") > + (VNx4SF "TARGET_VECTOR_ELEN_FP_32") > + (VNx4DF "TARGET_VECTOR_ELEN_FP_64") > +]) > + > +(define_mode_iterator VNX8_QHSD [ > + VNx8QI VNx8HI VNx8SI > + (VNx8DI "TARGET_MIN_VLEN > 32") > + (VNx8SF "TARGET_VECTOR_ELEN_FP_32") > + (VNx8DF "TARGET_VECTOR_ELEN_FP_64") > +]) > + > +(define_mode_iterator VNX16_QHS [ > + VNx16QI VNx16HI (VNx16SI "TARGET_MIN_VLEN > 32") > + (VNx16SF "TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN > 32") > +]) > + > +(define_mode_iterator VNX32_QH [ > + VNx32QI (VNx32HI "TARGET_MIN_VLEN > 32") > +]) > + > +(define_mode_iterator VNX64_Q [ > + (VNx64QI "TARGET_MIN_VLEN > 32") > +]) > + > +(define_mode_iterator VNX1_QHSDI [ > + VNx1QI VNx1HI VNx1SI > + (VNx1DI "TARGET_64BIT && TARGET_MIN_VLEN > 32") > +]) > + > +(define_mode_iterator VNX2_QHSDI [ > + VNx2QI VNx2HI VNx2SI > + (VNx2DI "TARGET_64BIT && TARGET_MIN_VLEN > 32") > +]) > + > +(define_mode_iterator VNX4_QHSDI [ > + VNx4QI VNx4HI VNx4SI > + (VNx4DI "TARGET_64BIT && TARGET_MIN_VLEN > 32") > +]) > + > +(define_mode_iterator VNX8_QHSDI [ > + VNx8QI VNx8HI VNx8SI > + (VNx8DI "TARGET_64BIT && TARGET_MIN_VLEN > 32") > +]) > + > +(define_mode_iterator VNX16_QHSI [ > + VNx16QI VNx16HI (VNx16SI "TARGET_MIN_VLEN > 32") > +]) > + > +(define_mode_iterator VNX32_QHI [ > + VNx32QI (VNx32HI "TARGET_MIN_VLEN > 32") > +]) > + > (define_mode_iterator V_WHOLE [ > (VNx4QI "TARGET_MIN_VLEN == 32") VNx8QI VNx16QI VNx32QI (VNx64QI "TARGET_MIN_VLEN > 32") > (VNx2HI "TARGET_MIN_VLEN == 32") VNx4HI VNx8HI VNx16HI (VNx32HI "TARGET_MIN_VLEN > 32") > @@ -90,3 +171,9 @@ > (VNx1SF "32") (VNx2SF "32") (VNx4SF "32") (VNx8SF "32") (VNx16SF "32") > (VNx1DF "64") (VNx2DF "64") (VNx4DF "64") (VNx8DF "64") > ]) > + > +(define_int_iterator ORDER [UNSPEC_ORDERED UNSPEC_UNORDERED]) > + > +(define_int_attr order [ > + (UNSPEC_ORDERED "o") (UNSPEC_UNORDERED "u") > +]) > diff --git a/gcc/config/riscv/vector.md b/gcc/config/riscv/vector.md > index 1453be116a9..403d2f6cda3 100644 > --- a/gcc/config/riscv/vector.md > +++ b/gcc/config/riscv/vector.md > @@ -28,14 +28,6 @@ > > (include "vector-iterators.md") > > -(define_c_enum "unspec" [ > - UNSPEC_VSETVL > - UNSPEC_VUNDEF > - UNSPEC_VPREDICATE > - UNSPEC_VLMAX > - UNSPEC_STRIDED > -]) > - > (define_constants [ > (INVALID_ATTRIBUTE 255) > ]) > @@ -149,7 +141,7 @@ > > ;; It is valid for instruction that require sew/lmul ratio. > (define_attr "ratio" "" > - (cond [(eq_attr "type" "vimov,vfmov") > + (cond [(eq_attr "type" "vimov,vfmov,vldux,vldox,vstux,vstox") > (const_int INVALID_ATTRIBUTE) > (eq_attr "mode" "VNx1QI,VNx1BI") > (symbol_ref "riscv_vector::get_ratio(E_VNx1QImode)") > @@ -199,13 +191,13 @@ > > ;; The index of operand[] to get the merge op. > (define_attr "merge_op_idx" "" > - (cond [(eq_attr "type" "vlde,vimov,vfmov,vldm,vlds,vmalu") > + (cond [(eq_attr "type" "vlde,vimov,vfmov,vldm,vlds,vmalu,vldux,vldox") > (const_int 2)] > (const_int INVALID_ATTRIBUTE))) > > ;; The index of operand[] to get the avl op. > (define_attr "vl_op_idx" "" > - (cond [(eq_attr "type" "vlde,vste,vimov,vfmov,vldm,vstm,vmalu,vsts") > + (cond [(eq_attr "type" "vlde,vste,vimov,vfmov,vldm,vstm,vmalu,vsts,vstux,vstox") > (const_int 4) > > ;; If operands[3] of "vlds" is not vector mode, it is pred_broadcast. > @@ -213,7 +205,10 @@ > (eq_attr "type" "vlds") > (if_then_else (match_test "VECTOR_MODE_P (GET_MODE (operands[3]))") > (const_int 5) > - (const_int 4))] > + (const_int 4)) > + > + (eq_attr "type" "vldux,vldox") > + (const_int 5)] > (const_int INVALID_ATTRIBUTE))) > > ;; The tail policy op value. > @@ -226,7 +221,10 @@ > (eq_attr "type" "vlds") > (if_then_else (match_test "VECTOR_MODE_P (GET_MODE (operands[3]))") > (symbol_ref "riscv_vector::get_ta(operands[6])") > - (symbol_ref "riscv_vector::get_ta(operands[5])"))] > + (symbol_ref "riscv_vector::get_ta(operands[5])")) > + > + (eq_attr "type" "vldux,vldox") > + (symbol_ref "riscv_vector::get_ta(operands[6])")] > (const_int INVALID_ATTRIBUTE))) > > ;; The mask policy op value. > @@ -239,7 +237,10 @@ > (eq_attr "type" "vlds") > (if_then_else (match_test "VECTOR_MODE_P (GET_MODE (operands[3]))") > (symbol_ref "riscv_vector::get_ma(operands[7])") > - (symbol_ref "riscv_vector::get_ma(operands[6])"))] > + (symbol_ref "riscv_vector::get_ma(operands[6])")) > + > + (eq_attr "type" "vldux,vldox") > + (symbol_ref "riscv_vector::get_ma(operands[7])")] > (const_int INVALID_ATTRIBUTE))) > > ;; The avl type value. > @@ -254,7 +255,12 @@ > (eq_attr "type" "vlds") > (if_then_else (match_test "VECTOR_MODE_P (GET_MODE (operands[3]))") > (const_int INVALID_ATTRIBUTE) > - (symbol_ref "INTVAL (operands[7])"))] > + (symbol_ref "INTVAL (operands[7])")) > + > + (eq_attr "type" "vldux,vldox") > + (symbol_ref "INTVAL (operands[8])") > + (eq_attr "type" "vstux,vstox") > + (symbol_ref "INTVAL (operands[5])")] > (const_int INVALID_ATTRIBUTE))) > > ;; ----------------------------------------------------------------- > @@ -832,3 +838,276 @@ > "vsse.v\t%3,%0,%z2%p1" > [(set_attr "type" "vsts") > (set_attr "mode" "")]) > + > +;; ------------------------------------------------------------------------------- > +;; ---- Predicated indexed loads/stores > +;; ------------------------------------------------------------------------------- > +;; Includes: > +;; - 7.6. Vector Indexed Instructions > +;; ------------------------------------------------------------------------------- > + > +(define_insn "@pred_indexed_load" > + [(set (match_operand:VNX1_QHSD 0 "register_operand" "=&vr") > + (if_then_else:VNX1_QHSD > + (unspec: > + [(match_operand: 1 "vector_mask_operand" "vmWc1") > + (match_operand 5 "vector_length_operand" " rK") > + (match_operand 6 "const_int_operand" " i") > + (match_operand 7 "const_int_operand" " i") > + (match_operand 8 "const_int_operand" " i") > + (reg:SI VL_REGNUM) > + (reg:SI VTYPE_REGNUM)] UNSPEC_VPREDICATE) > + (unspec:VNX1_QHSD > + [(match_operand 3 "pmode_register_operand" " r") > + (mem:BLK (scratch)) > + (match_operand:VNX1_QHSDI 4 "register_operand" " vr")] ORDER) > + (match_operand:VNX1_QHSD 2 "vector_merge_operand" "0vu")))] > + "TARGET_VECTOR" > + "vlxei.v\t%0,(%3),%4%p1" > + [(set_attr "type" "vldx") > + (set_attr "mode" "")]) > + > +(define_insn "@pred_indexed_store" > + [(set (mem:BLK (scratch)) > + (unspec:BLK > + [(unspec: > + [(match_operand: 0 "vector_mask_operand" "vmWc1") > + (match_operand 4 "vector_length_operand" " rK") > + (match_operand 5 "const_int_operand" " i") > + (reg:SI VL_REGNUM) > + (reg:SI VTYPE_REGNUM)] UNSPEC_VPREDICATE) > + (match_operand 1 "pmode_register_operand" " r") > + (match_operand:VNX1_QHSDI 2 "register_operand" " vr") > + (match_operand:VNX1_QHSD 3 "register_operand" " vr")] ORDER))] > + "TARGET_VECTOR" > + "vsxei.v\t%3,(%1),%2%p0" > + [(set_attr "type" "vstx") > + (set_attr "mode" "")]) > + > +(define_insn "@pred_indexed_load" > + [(set (match_operand:VNX2_QHSD 0 "register_operand" "=&vr") > + (if_then_else:VNX2_QHSD > + (unspec: > + [(match_operand: 1 "vector_mask_operand" "vmWc1") > + (match_operand 5 "vector_length_operand" " rK") > + (match_operand 6 "const_int_operand" " i") > + (match_operand 7 "const_int_operand" " i") > + (match_operand 8 "const_int_operand" " i") > + (reg:SI VL_REGNUM) > + (reg:SI VTYPE_REGNUM)] UNSPEC_VPREDICATE) > + (unspec:VNX2_QHSD > + [(match_operand 3 "pmode_register_operand" " r") > + (mem:BLK (scratch)) > + (match_operand:VNX2_QHSDI 4 "register_operand" " vr")] ORDER) > + (match_operand:VNX2_QHSD 2 "vector_merge_operand" "0vu")))] > + "TARGET_VECTOR" > + "vlxei.v\t%0,(%3),%4%p1" > + [(set_attr "type" "vldx") > + (set_attr "mode" "")]) > + > +(define_insn "@pred_indexed_store" > + [(set (mem:BLK (scratch)) > + (unspec:BLK > + [(unspec: > + [(match_operand: 0 "vector_mask_operand" "vmWc1") > + (match_operand 4 "vector_length_operand" " rK") > + (match_operand 5 "const_int_operand" " i") > + (reg:SI VL_REGNUM) > + (reg:SI VTYPE_REGNUM)] UNSPEC_VPREDICATE) > + (match_operand 1 "pmode_register_operand" " r") > + (match_operand:VNX2_QHSDI 2 "register_operand" " vr") > + (match_operand:VNX2_QHSD 3 "register_operand" " vr")] ORDER))] > + "TARGET_VECTOR" > + "vsxei.v\t%3,(%1),%2%p0" > + [(set_attr "type" "vstx") > + (set_attr "mode" "")]) > + > +(define_insn "@pred_indexed_load" > + [(set (match_operand:VNX4_QHSD 0 "register_operand" "=&vr") > + (if_then_else:VNX4_QHSD > + (unspec: > + [(match_operand: 1 "vector_mask_operand" "vmWc1") > + (match_operand 5 "vector_length_operand" " rK") > + (match_operand 6 "const_int_operand" " i") > + (match_operand 7 "const_int_operand" " i") > + (match_operand 8 "const_int_operand" " i") > + (reg:SI VL_REGNUM) > + (reg:SI VTYPE_REGNUM)] UNSPEC_VPREDICATE) > + (unspec:VNX4_QHSD > + [(match_operand 3 "pmode_register_operand" " r") > + (mem:BLK (scratch)) > + (match_operand:VNX4_QHSDI 4 "register_operand" " vr")] ORDER) > + (match_operand:VNX4_QHSD 2 "vector_merge_operand" "0vu")))] > + "TARGET_VECTOR" > + "vlxei.v\t%0,(%3),%4%p1" > + [(set_attr "type" "vldx") > + (set_attr "mode" "")]) > + > +(define_insn "@pred_indexed_store" > + [(set (mem:BLK (scratch)) > + (unspec:BLK > + [(unspec: > + [(match_operand: 0 "vector_mask_operand" "vmWc1") > + (match_operand 4 "vector_length_operand" " rK") > + (match_operand 5 "const_int_operand" " i") > + (reg:SI VL_REGNUM) > + (reg:SI VTYPE_REGNUM)] UNSPEC_VPREDICATE) > + (match_operand 1 "pmode_register_operand" " r") > + (match_operand:VNX4_QHSDI 2 "register_operand" " vr") > + (match_operand:VNX4_QHSD 3 "register_operand" " vr")] ORDER))] > + "TARGET_VECTOR" > + "vsxei.v\t%3,(%1),%2%p0" > + [(set_attr "type" "vstx") > + (set_attr "mode" "")]) > + > +(define_insn "@pred_indexed_load" > + [(set (match_operand:VNX8_QHSD 0 "register_operand" "=&vr") > + (if_then_else:VNX8_QHSD > + (unspec: > + [(match_operand: 1 "vector_mask_operand" "vmWc1") > + (match_operand 5 "vector_length_operand" " rK") > + (match_operand 6 "const_int_operand" " i") > + (match_operand 7 "const_int_operand" " i") > + (match_operand 8 "const_int_operand" " i") > + (reg:SI VL_REGNUM) > + (reg:SI VTYPE_REGNUM)] UNSPEC_VPREDICATE) > + (unspec:VNX8_QHSD > + [(match_operand 3 "pmode_register_operand" " r") > + (mem:BLK (scratch)) > + (match_operand:VNX8_QHSDI 4 "register_operand" " vr")] ORDER) > + (match_operand:VNX8_QHSD 2 "vector_merge_operand" "0vu")))] > + "TARGET_VECTOR" > + "vlxei.v\t%0,(%3),%4%p1" > + [(set_attr "type" "vldx") > + (set_attr "mode" "")]) > + > +(define_insn "@pred_indexed_store" > + [(set (mem:BLK (scratch)) > + (unspec:BLK > + [(unspec: > + [(match_operand: 0 "vector_mask_operand" "vmWc1") > + (match_operand 4 "vector_length_operand" " rK") > + (match_operand 5 "const_int_operand" " i") > + (reg:SI VL_REGNUM) > + (reg:SI VTYPE_REGNUM)] UNSPEC_VPREDICATE) > + (match_operand 1 "pmode_register_operand" " r") > + (match_operand:VNX8_QHSDI 2 "register_operand" " vr") > + (match_operand:VNX8_QHSD 3 "register_operand" " vr")] ORDER))] > + "TARGET_VECTOR" > + "vsxei.v\t%3,(%1),%2%p0" > + [(set_attr "type" "vstx") > + (set_attr "mode" "")]) > + > +(define_insn "@pred_indexed_load" > + [(set (match_operand:VNX16_QHS 0 "register_operand" "=&vr") > + (if_then_else:VNX16_QHS > + (unspec: > + [(match_operand: 1 "vector_mask_operand" "vmWc1") > + (match_operand 5 "vector_length_operand" " rK") > + (match_operand 6 "const_int_operand" " i") > + (match_operand 7 "const_int_operand" " i") > + (match_operand 8 "const_int_operand" " i") > + (reg:SI VL_REGNUM) > + (reg:SI VTYPE_REGNUM)] UNSPEC_VPREDICATE) > + (unspec:VNX16_QHS > + [(match_operand 3 "pmode_register_operand" " r") > + (mem:BLK (scratch)) > + (match_operand:VNX16_QHSI 4 "register_operand" " vr")] ORDER) > + (match_operand:VNX16_QHS 2 "vector_merge_operand" "0vu")))] > + "TARGET_VECTOR" > + "vlxei.v\t%0,(%3),%4%p1" > + [(set_attr "type" "vldx") > + (set_attr "mode" "")]) > + > +(define_insn "@pred_indexed_store" > + [(set (mem:BLK (scratch)) > + (unspec:BLK > + [(unspec: > + [(match_operand: 0 "vector_mask_operand" "vmWc1") > + (match_operand 4 "vector_length_operand" " rK") > + (match_operand 5 "const_int_operand" " i") > + (reg:SI VL_REGNUM) > + (reg:SI VTYPE_REGNUM)] UNSPEC_VPREDICATE) > + (match_operand 1 "pmode_register_operand" " r") > + (match_operand:VNX16_QHSI 2 "register_operand" " vr") > + (match_operand:VNX16_QHS 3 "register_operand" " vr")] ORDER))] > + "TARGET_VECTOR" > + "vsxei.v\t%3,(%1),%2%p0" > + [(set_attr "type" "vstx") > + (set_attr "mode" "")]) > + > +(define_insn "@pred_indexed_load" > + [(set (match_operand:VNX32_QH 0 "register_operand" "=&vr") > + (if_then_else:VNX32_QH > + (unspec: > + [(match_operand: 1 "vector_mask_operand" "vmWc1") > + (match_operand 5 "vector_length_operand" " rK") > + (match_operand 6 "const_int_operand" " i") > + (match_operand 7 "const_int_operand" " i") > + (match_operand 8 "const_int_operand" " i") > + (reg:SI VL_REGNUM) > + (reg:SI VTYPE_REGNUM)] UNSPEC_VPREDICATE) > + (unspec:VNX32_QH > + [(match_operand 3 "pmode_register_operand" " r") > + (mem:BLK (scratch)) > + (match_operand:VNX32_QHI 4 "register_operand" " vr")] ORDER) > + (match_operand:VNX32_QH 2 "vector_merge_operand" "0vu")))] > + "TARGET_VECTOR" > + "vlxei.v\t%0,(%3),%4%p1" > + [(set_attr "type" "vldx") > + (set_attr "mode" "")]) > + > +(define_insn "@pred_indexed_store" > + [(set (mem:BLK (scratch)) > + (unspec:BLK > + [(unspec: > + [(match_operand: 0 "vector_mask_operand" "vmWc1") > + (match_operand 4 "vector_length_operand" " rK") > + (match_operand 5 "const_int_operand" " i") > + (reg:SI VL_REGNUM) > + (reg:SI VTYPE_REGNUM)] UNSPEC_VPREDICATE) > + (match_operand 1 "pmode_register_operand" " r") > + (match_operand:VNX32_QHI 2 "register_operand" " vr") > + (match_operand:VNX32_QH 3 "register_operand" " vr")] ORDER))] > + "TARGET_VECTOR" > + "vsxei.v\t%3,(%1),%2%p0" > + [(set_attr "type" "vstx") > + (set_attr "mode" "")]) > + > +(define_insn "@pred_indexed_load" > + [(set (match_operand:VNX64_Q 0 "register_operand" "=&vr") > + (if_then_else:VNX64_Q > + (unspec: > + [(match_operand: 1 "vector_mask_operand" "vmWc1") > + (match_operand 5 "vector_length_operand" " rK") > + (match_operand 6 "const_int_operand" " i") > + (match_operand 7 "const_int_operand" " i") > + (match_operand 8 "const_int_operand" " i") > + (reg:SI VL_REGNUM) > + (reg:SI VTYPE_REGNUM)] UNSPEC_VPREDICATE) > + (unspec:VNX64_Q > + [(match_operand 3 "pmode_register_operand" " r") > + (mem:BLK (scratch)) > + (match_operand:VNX64_Q 4 "register_operand" " vr")] ORDER) > + (match_operand:VNX64_Q 2 "vector_merge_operand" "0vu")))] > + "TARGET_VECTOR" > + "vlxei.v\t%0,(%3),%4%p1" > + [(set_attr "type" "vldx") > + (set_attr "mode" "")]) > + > +(define_insn "@pred_indexed_store" > + [(set (mem:BLK (scratch)) > + (unspec:BLK > + [(unspec: > + [(match_operand: 0 "vector_mask_operand" "vmWc1") > + (match_operand 4 "vector_length_operand" " rK") > + (match_operand 5 "const_int_operand" " i") > + (reg:SI VL_REGNUM) > + (reg:SI VTYPE_REGNUM)] UNSPEC_VPREDICATE) > + (match_operand 1 "pmode_register_operand" " r") > + (match_operand:VNX64_Q 2 "register_operand" " vr") > + (match_operand:VNX64_Q 3 "register_operand" " vr")] ORDER))] > + "TARGET_VECTOR" > + "vsxei.v\t%3,(%1),%2%p0" > + [(set_attr "type" "vstx") > + (set_attr "mode" "")]) > -- > 2.36.3 >