Commited, thanks Richard.


Bootstrap and regression passed.





------------------ Original ------------------
From:                                                                                                                        "Richard Biener"                                                                                    <gcc-patches@gcc.gnu.org&gt;;
Date:&nbsp;Fri, Jul 21, 2023 02:20 PM
To:&nbsp;"Juzhe-Zhong"<juzhe.zhong@rivai.ai&gt;;
Cc:&nbsp;"gcc-patches"<gcc-patches@gcc.gnu.org&gt;;"richard.sandiford"<richard.sandiford@arm.com&gt;;
Subject:&nbsp;Re: [PATCH] cleanup: make all cond_len_* and mask_len_* consistent on the order of mask and len



On Fri, 21 Jul 2023, Juzhe-Zhong wrote:

&gt; This patch is depending on:
&gt; https://gcc.gnu.org/pipermail/gcc-patches/2023-July/625121.html
&gt; 
&gt; Hi, Richard and Richi.
&gt; 
&gt; This patch is to align the order of mask and len.
&gt; 
&gt; Currently, According to this piece code:
&gt; 			if (final_len &amp;&amp; final_mask)
&gt; 			&nbsp; call = gimple_build_call_internal (
&gt; 			&nbsp;&nbsp;&nbsp; IFN_LEN_MASK_GATHER_LOAD, 7, dataref_ptr,
&gt; 			&nbsp;&nbsp;&nbsp; vec_offset, scale, zero, final_mask, final_len,
&gt; 			&nbsp;&nbsp;&nbsp; bias);
&gt; 
&gt; You can see the order of mask and len, is {mask,len,bias}.
&gt; "mask" comes before "len". The reason of this order is that we want to
&gt; reuse the current codes of MASK_GATHER_LOAD/MASK_SCATTER_STORE.
&gt; 
&gt; Same situation for COND_LEN_*, we want to reuse the codes of COND_*.
&gt; 
&gt; Reusing codes from the existing MASK_* or COND_* can allow us not to
&gt; change the codes too much and make the codes elegant and easy to maintain &amp;&amp; read.
&gt; 
&gt; To avoid any confusions of auto-vectorization patterns that includes both mask and len,
&gt; 
&gt; this patch align the order of mask and len for both Gimple IR and RTL pattern into
&gt; 
&gt; {mask, len, bias} to make everything cleaner and more elegant.
&gt; 
&gt; Bootstrap and Regression is on the way.

OK.

&gt; gcc/ChangeLog:
&gt; 
&gt; 	* config/riscv/autovec.md: Align order of mask and len.
&gt; 	* config/riscv/riscv-v.cc (expand_load_store): Ditto.
&gt; 	(expand_gather_scatter): Ditto.
&gt; 	* doc/md.texi: Ditto.
&gt; 	* internal-fn.cc (add_len_and_mask_args): Ditto.
&gt; 	(add_mask_and_len_args): Ditto.
&gt; 	(expand_partial_load_optab_fn): Ditto.
&gt; 	(expand_partial_store_optab_fn): Ditto.
&gt; 	(expand_scatter_store_optab_fn): Ditto.
&gt; 	(expand_gather_load_optab_fn): Ditto.
&gt; 	(internal_fn_len_index): Ditto.
&gt; 	(internal_fn_mask_index): Ditto.
&gt; 	(internal_len_load_store_bias): Ditto.
&gt; 	* tree-vect-stmts.cc (vectorizable_store): Ditto.
&gt; 	(vectorizable_load): Ditto.
&gt; 
&gt; ---
&gt;&nbsp; gcc/config/riscv/autovec.md | 96 ++++++++++++++++++-------------------
&gt;&nbsp; gcc/config/riscv/riscv-v.cc | 12 ++---
&gt;&nbsp; gcc/doc/md.texi&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; | 36 +++++++-------
&gt;&nbsp; gcc/internal-fn.cc&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; | 50 +++++++++----------
&gt;&nbsp; gcc/tree-vect-stmts.cc&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; |&nbsp; 8 ++--
&gt;&nbsp; 5 files changed, 101 insertions(+), 101 deletions(-)
&gt; 
&gt; diff --git a/gcc/config/riscv/autovec.md b/gcc/config/riscv/autovec.md
&gt; index 7eb96d42c18..d899922586a 100644
&gt; --- a/gcc/config/riscv/autovec.md
&gt; +++ b/gcc/config/riscv/autovec.md
&gt; @@ -25,9 +25,9 @@
&gt;&nbsp; (define_expand "mask_len_load<mode&gt;<vm&gt;"
&gt;&nbsp;&nbsp;&nbsp; [(match_operand:V 0 "register_operand")
&gt;&nbsp;&nbsp;&nbsp;&nbsp; (match_operand:V 1 "memory_operand")
&gt; -&nbsp;&nbsp; (match_operand 2 "autovec_length_operand")
&gt; -&nbsp;&nbsp; (match_operand 3 "const_0_operand")
&gt; -&nbsp;&nbsp; (match_operand:<VM&gt; 4 "vector_mask_operand")]
&gt; +&nbsp;&nbsp; (match_operand:<VM&gt; 2 "vector_mask_operand")
&gt; +&nbsp;&nbsp; (match_operand 3 "autovec_length_operand")
&gt; +&nbsp;&nbsp; (match_operand 4 "const_0_operand")]
&gt;&nbsp;&nbsp;&nbsp; "TARGET_VECTOR"
&gt;&nbsp; {
&gt;&nbsp;&nbsp;&nbsp; riscv_vector::expand_load_store (operands, true);
&gt; @@ -37,9 +37,9 @@
&gt;&nbsp; (define_expand "mask_len_store<mode&gt;<vm&gt;"
&gt;&nbsp;&nbsp;&nbsp; [(match_operand:V 0 "memory_operand")
&gt;&nbsp;&nbsp;&nbsp;&nbsp; (match_operand:V 1 "register_operand")
&gt; -&nbsp;&nbsp; (match_operand 2 "autovec_length_operand")
&gt; -&nbsp;&nbsp; (match_operand 3 "const_0_operand")
&gt; -&nbsp;&nbsp; (match_operand:<VM&gt; 4 "vector_mask_operand")]
&gt; +&nbsp;&nbsp; (match_operand:<VM&gt; 2 "vector_mask_operand")
&gt; +&nbsp;&nbsp; (match_operand 3 "autovec_length_operand")
&gt; +&nbsp;&nbsp; (match_operand 4 "const_0_operand")]
&gt;&nbsp;&nbsp;&nbsp; "TARGET_VECTOR"
&gt;&nbsp; {
&gt;&nbsp;&nbsp;&nbsp; riscv_vector::expand_load_store (operands, false);
&gt; @@ -67,9 +67,9 @@
&gt;&nbsp;&nbsp;&nbsp;&nbsp; (match_operand:RATIO64I 2 "register_operand")
&gt;&nbsp;&nbsp;&nbsp;&nbsp; (match_operand 3 "<RATIO64:gs_extension&gt;")
&gt;&nbsp;&nbsp;&nbsp;&nbsp; (match_operand 4 "<RATIO64:gs_scale&gt;")
&gt; -&nbsp;&nbsp; (match_operand 5 "autovec_length_operand")
&gt; -&nbsp;&nbsp; (match_operand 6 "const_0_operand")
&gt; -&nbsp;&nbsp; (match_operand:<RATIO64:VM&gt; 7 "vector_mask_operand")]
&gt; +&nbsp;&nbsp; (match_operand:<RATIO64:VM&gt; 5 "vector_mask_operand")
&gt; +&nbsp;&nbsp; (match_operand 6 "autovec_length_operand")
&gt; +&nbsp;&nbsp; (match_operand 7 "const_0_operand")]
&gt;&nbsp;&nbsp;&nbsp; "TARGET_VECTOR"
&gt;&nbsp; {
&gt;&nbsp;&nbsp;&nbsp; riscv_vector::expand_gather_scatter (operands, true);
&gt; @@ -82,9 +82,9 @@
&gt;&nbsp;&nbsp;&nbsp;&nbsp; (match_operand:RATIO32I 2 "register_operand")
&gt;&nbsp;&nbsp;&nbsp;&nbsp; (match_operand 3 "<RATIO32:gs_extension&gt;")
&gt;&nbsp;&nbsp;&nbsp;&nbsp; (match_operand 4 "<RATIO32:gs_scale&gt;")
&gt; -&nbsp;&nbsp; (match_operand 5 "autovec_length_operand")
&gt; -&nbsp;&nbsp; (match_operand 6 "const_0_operand")
&gt; -&nbsp;&nbsp; (match_operand:<RATIO32:VM&gt; 7 "vector_mask_operand")]
&gt; +&nbsp;&nbsp; (match_operand:<RATIO32:VM&gt; 5 "vector_mask_operand")
&gt; +&nbsp;&nbsp; (match_operand 6 "autovec_length_operand")
&gt; +&nbsp;&nbsp; (match_operand 7 "const_0_operand")]
&gt;&nbsp;&nbsp;&nbsp; "TARGET_VECTOR"
&gt;&nbsp; {
&gt;&nbsp;&nbsp;&nbsp; riscv_vector::expand_gather_scatter (operands, true);
&gt; @@ -97,9 +97,9 @@
&gt;&nbsp;&nbsp;&nbsp;&nbsp; (match_operand:RATIO16I 2 "register_operand")
&gt;&nbsp;&nbsp;&nbsp;&nbsp; (match_operand 3 "<RATIO16:gs_extension&gt;")
&gt;&nbsp;&nbsp;&nbsp;&nbsp; (match_operand 4 "<RATIO16:gs_scale&gt;")
&gt; -&nbsp;&nbsp; (match_operand 5 "autovec_length_operand")
&gt; -&nbsp;&nbsp; (match_operand 6 "const_0_operand")
&gt; -&nbsp;&nbsp; (match_operand:<RATIO16:VM&gt; 7 "vector_mask_operand")]
&gt; +&nbsp;&nbsp; (match_operand:<RATIO16:VM&gt; 5 "vector_mask_operand")
&gt; +&nbsp;&nbsp; (match_operand 6 "autovec_length_operand")
&gt; +&nbsp;&nbsp; (match_operand 7 "const_0_operand")]
&gt;&nbsp;&nbsp;&nbsp; "TARGET_VECTOR"
&gt;&nbsp; {
&gt;&nbsp;&nbsp;&nbsp; riscv_vector::expand_gather_scatter (operands, true);
&gt; @@ -112,9 +112,9 @@
&gt;&nbsp;&nbsp;&nbsp;&nbsp; (match_operand:RATIO8I 2 "register_operand")
&gt;&nbsp;&nbsp;&nbsp;&nbsp; (match_operand 3 "<RATIO8:gs_extension&gt;")
&gt;&nbsp;&nbsp;&nbsp;&nbsp; (match_operand 4 "<RATIO8:gs_scale&gt;")
&gt; -&nbsp;&nbsp; (match_operand 5 "autovec_length_operand")
&gt; -&nbsp;&nbsp; (match_operand 6 "const_0_operand")
&gt; -&nbsp;&nbsp; (match_operand:<RATIO8:VM&gt; 7 "vector_mask_operand")]
&gt; +&nbsp;&nbsp; (match_operand:<RATIO8:VM&gt; 5 "vector_mask_operand")
&gt; +&nbsp;&nbsp; (match_operand 6 "autovec_length_operand")
&gt; +&nbsp;&nbsp; (match_operand 7 "const_0_operand")]
&gt;&nbsp;&nbsp;&nbsp; "TARGET_VECTOR"
&gt;&nbsp; {
&gt;&nbsp;&nbsp;&nbsp; riscv_vector::expand_gather_scatter (operands, true);
&gt; @@ -127,9 +127,9 @@
&gt;&nbsp;&nbsp;&nbsp;&nbsp; (match_operand:RATIO4I 2 "register_operand")
&gt;&nbsp;&nbsp;&nbsp;&nbsp; (match_operand 3 "<RATIO4:gs_extension&gt;")
&gt;&nbsp;&nbsp;&nbsp;&nbsp; (match_operand 4 "<RATIO4:gs_scale&gt;")
&gt; -&nbsp;&nbsp; (match_operand 5 "autovec_length_operand")
&gt; -&nbsp;&nbsp; (match_operand 6 "const_0_operand")
&gt; -&nbsp;&nbsp; (match_operand:<RATIO4:VM&gt; 7 "vector_mask_operand")]
&gt; +&nbsp;&nbsp; (match_operand:<RATIO4:VM&gt; 5 "vector_mask_operand")
&gt; +&nbsp;&nbsp; (match_operand 6 "autovec_length_operand")
&gt; +&nbsp;&nbsp; (match_operand 7 "const_0_operand")]
&gt;&nbsp;&nbsp;&nbsp; "TARGET_VECTOR"
&gt;&nbsp; {
&gt;&nbsp;&nbsp;&nbsp; riscv_vector::expand_gather_scatter (operands, true);
&gt; @@ -142,9 +142,9 @@
&gt;&nbsp;&nbsp;&nbsp;&nbsp; (match_operand:RATIO2I 2 "register_operand")
&gt;&nbsp;&nbsp;&nbsp;&nbsp; (match_operand 3 "<RATIO2:gs_extension&gt;")
&gt;&nbsp;&nbsp;&nbsp;&nbsp; (match_operand 4 "<RATIO2:gs_scale&gt;")
&gt; -&nbsp;&nbsp; (match_operand 5 "autovec_length_operand")
&gt; -&nbsp;&nbsp; (match_operand 6 "const_0_operand")
&gt; -&nbsp;&nbsp; (match_operand:<RATIO2:VM&gt; 7 "vector_mask_operand")]
&gt; +&nbsp;&nbsp; (match_operand:<RATIO2:VM&gt; 5 "vector_mask_operand")
&gt; +&nbsp;&nbsp; (match_operand 6 "autovec_length_operand")
&gt; +&nbsp;&nbsp; (match_operand 7 "const_0_operand")]
&gt;&nbsp;&nbsp;&nbsp; "TARGET_VECTOR"
&gt;&nbsp; {
&gt;&nbsp;&nbsp;&nbsp; riscv_vector::expand_gather_scatter (operands, true);
&gt; @@ -161,9 +161,9 @@
&gt;&nbsp;&nbsp;&nbsp;&nbsp; (match_operand:RATIO1 2 "register_operand")
&gt;&nbsp;&nbsp;&nbsp;&nbsp; (match_operand 3 "<RATIO1:gs_extension&gt;")
&gt;&nbsp;&nbsp;&nbsp;&nbsp; (match_operand 4 "<RATIO1:gs_scale&gt;")
&gt; -&nbsp;&nbsp; (match_operand 5 "autovec_length_operand")
&gt; -&nbsp;&nbsp; (match_operand 6 "const_0_operand")
&gt; -&nbsp;&nbsp; (match_operand:<RATIO1:VM&gt; 7 "vector_mask_operand")]
&gt; +&nbsp;&nbsp; (match_operand:<RATIO1:VM&gt; 5 "vector_mask_operand")
&gt; +&nbsp;&nbsp; (match_operand 6 "autovec_length_operand")
&gt; +&nbsp;&nbsp; (match_operand 7 "const_0_operand")]
&gt;&nbsp;&nbsp;&nbsp; "TARGET_VECTOR"
&gt;&nbsp; {
&gt;&nbsp;&nbsp;&nbsp; riscv_vector::expand_gather_scatter (operands, true);
&gt; @@ -180,9 +180,9 @@
&gt;&nbsp;&nbsp;&nbsp;&nbsp; (match_operand 2 "<RATIO64:gs_extension&gt;")
&gt;&nbsp;&nbsp;&nbsp;&nbsp; (match_operand 3 "<RATIO64:gs_scale&gt;")
&gt;&nbsp;&nbsp;&nbsp;&nbsp; (match_operand:RATIO64 4 "register_operand")
&gt; -&nbsp;&nbsp; (match_operand 5 "autovec_length_operand")
&gt; -&nbsp;&nbsp; (match_operand 6 "const_0_operand")
&gt; -&nbsp;&nbsp; (match_operand:<RATIO64:VM&gt; 7 "vector_mask_operand")]
&gt; +&nbsp;&nbsp; (match_operand:<RATIO64:VM&gt; 5 "vector_mask_operand")
&gt; +&nbsp;&nbsp; (match_operand 6 "autovec_length_operand")
&gt; +&nbsp;&nbsp; (match_operand 7 "const_0_operand")]
&gt;&nbsp;&nbsp;&nbsp; "TARGET_VECTOR"
&gt;&nbsp; {
&gt;&nbsp;&nbsp;&nbsp; riscv_vector::expand_gather_scatter (operands, false);
&gt; @@ -195,9 +195,9 @@
&gt;&nbsp;&nbsp;&nbsp;&nbsp; (match_operand 2 "<RATIO32:gs_extension&gt;")
&gt;&nbsp;&nbsp;&nbsp;&nbsp; (match_operand 3 "<RATIO32:gs_scale&gt;")
&gt;&nbsp;&nbsp;&nbsp;&nbsp; (match_operand:RATIO32 4 "register_operand")
&gt; -&nbsp;&nbsp; (match_operand 5 "autovec_length_operand")
&gt; -&nbsp;&nbsp; (match_operand 6 "const_0_operand")
&gt; -&nbsp;&nbsp; (match_operand:<RATIO32:VM&gt; 7 "vector_mask_operand")]
&gt; +&nbsp;&nbsp; (match_operand:<RATIO32:VM&gt; 5 "vector_mask_operand")
&gt; +&nbsp;&nbsp; (match_operand 6 "autovec_length_operand")
&gt; +&nbsp;&nbsp; (match_operand 7 "const_0_operand")]
&gt;&nbsp;&nbsp;&nbsp; "TARGET_VECTOR"
&gt;&nbsp; {
&gt;&nbsp;&nbsp;&nbsp; riscv_vector::expand_gather_scatter (operands, false);
&gt; @@ -210,9 +210,9 @@
&gt;&nbsp;&nbsp;&nbsp;&nbsp; (match_operand 2 "<RATIO16:gs_extension&gt;")
&gt;&nbsp;&nbsp;&nbsp;&nbsp; (match_operand 3 "<RATIO16:gs_scale&gt;")
&gt;&nbsp;&nbsp;&nbsp;&nbsp; (match_operand:RATIO16 4 "register_operand")
&gt; -&nbsp;&nbsp; (match_operand 5 "autovec_length_operand")
&gt; -&nbsp;&nbsp; (match_operand 6 "const_0_operand")
&gt; -&nbsp;&nbsp; (match_operand:<RATIO16:VM&gt; 7 "vector_mask_operand")]
&gt; +&nbsp;&nbsp; (match_operand:<RATIO16:VM&gt; 5 "vector_mask_operand")
&gt; +&nbsp;&nbsp; (match_operand 6 "autovec_length_operand")
&gt; +&nbsp;&nbsp; (match_operand 7 "const_0_operand")]
&gt;&nbsp;&nbsp;&nbsp; "TARGET_VECTOR"
&gt;&nbsp; {
&gt;&nbsp;&nbsp;&nbsp; riscv_vector::expand_gather_scatter (operands, false);
&gt; @@ -225,9 +225,9 @@
&gt;&nbsp;&nbsp;&nbsp;&nbsp; (match_operand 2 "<RATIO8:gs_extension&gt;")
&gt;&nbsp;&nbsp;&nbsp;&nbsp; (match_operand 3 "<RATIO8:gs_scale&gt;")
&gt;&nbsp;&nbsp;&nbsp;&nbsp; (match_operand:RATIO8 4 "register_operand")
&gt; -&nbsp;&nbsp; (match_operand 5 "autovec_length_operand")
&gt; -&nbsp;&nbsp; (match_operand 6 "const_0_operand")
&gt; -&nbsp;&nbsp; (match_operand:<RATIO8:VM&gt; 7 "vector_mask_operand")]
&gt; +&nbsp;&nbsp; (match_operand:<RATIO8:VM&gt; 5 "vector_mask_operand")
&gt; +&nbsp;&nbsp; (match_operand 6 "autovec_length_operand")
&gt; +&nbsp;&nbsp; (match_operand 7 "const_0_operand")]
&gt;&nbsp;&nbsp;&nbsp; "TARGET_VECTOR"
&gt;&nbsp; {
&gt;&nbsp;&nbsp;&nbsp; riscv_vector::expand_gather_scatter (operands, false);
&gt; @@ -240,9 +240,9 @@
&gt;&nbsp;&nbsp;&nbsp;&nbsp; (match_operand 2 "<RATIO4:gs_extension&gt;")
&gt;&nbsp;&nbsp;&nbsp;&nbsp; (match_operand 3 "<RATIO4:gs_scale&gt;")
&gt;&nbsp;&nbsp;&nbsp;&nbsp; (match_operand:RATIO4 4 "register_operand")
&gt; -&nbsp;&nbsp; (match_operand 5 "autovec_length_operand")
&gt; -&nbsp;&nbsp; (match_operand 6 "const_0_operand")
&gt; -&nbsp;&nbsp; (match_operand:<RATIO4:VM&gt; 7 "vector_mask_operand")]
&gt; +&nbsp;&nbsp; (match_operand:<RATIO4:VM&gt; 5 "vector_mask_operand")
&gt; +&nbsp;&nbsp; (match_operand 6 "autovec_length_operand")
&gt; +&nbsp;&nbsp; (match_operand 7 "const_0_operand")]
&gt;&nbsp;&nbsp;&nbsp; "TARGET_VECTOR"
&gt;&nbsp; {
&gt;&nbsp;&nbsp;&nbsp; riscv_vector::expand_gather_scatter (operands, false);
&gt; @@ -255,9 +255,9 @@
&gt;&nbsp;&nbsp;&nbsp;&nbsp; (match_operand 2 "<RATIO2:gs_extension&gt;")
&gt;&nbsp;&nbsp;&nbsp;&nbsp; (match_operand 3 "<RATIO2:gs_scale&gt;")
&gt;&nbsp;&nbsp;&nbsp;&nbsp; (match_operand:RATIO2 4 "register_operand")
&gt; -&nbsp;&nbsp; (match_operand 5 "autovec_length_operand")
&gt; -&nbsp;&nbsp; (match_operand 6 "const_0_operand")
&gt; -&nbsp;&nbsp; (match_operand:<RATIO2:VM&gt; 7 "vector_mask_operand")]
&gt; +&nbsp;&nbsp; (match_operand:<RATIO2:VM&gt; 5 "vector_mask_operand")
&gt; +&nbsp;&nbsp; (match_operand 6 "autovec_length_operand")
&gt; +&nbsp;&nbsp; (match_operand 7 "const_0_operand")]
&gt;&nbsp;&nbsp;&nbsp; "TARGET_VECTOR"
&gt;&nbsp; {
&gt;&nbsp;&nbsp;&nbsp; riscv_vector::expand_gather_scatter (operands, false);
&gt; @@ -274,9 +274,9 @@
&gt;&nbsp;&nbsp;&nbsp;&nbsp; (match_operand 2 "<RATIO1:gs_extension&gt;")
&gt;&nbsp;&nbsp;&nbsp;&nbsp; (match_operand 3 "<RATIO1:gs_scale&gt;")
&gt;&nbsp;&nbsp;&nbsp;&nbsp; (match_operand:RATIO1 4 "register_operand")
&gt; -&nbsp;&nbsp; (match_operand 5 "autovec_length_operand")
&gt; -&nbsp;&nbsp; (match_operand 6 "const_0_operand")
&gt; -&nbsp;&nbsp; (match_operand:<RATIO1:VM&gt; 7 "vector_mask_operand")]
&gt; +&nbsp;&nbsp; (match_operand:<RATIO1:VM&gt; 5 "vector_mask_operand")
&gt; +&nbsp;&nbsp; (match_operand 6 "autovec_length_operand")
&gt; +&nbsp;&nbsp; (match_operand 7 "const_0_operand")]
&gt;&nbsp;&nbsp;&nbsp; "TARGET_VECTOR"
&gt;&nbsp; {
&gt;&nbsp;&nbsp;&nbsp; riscv_vector::expand_gather_scatter (operands, false);
&gt; diff --git a/gcc/config/riscv/riscv-v.cc b/gcc/config/riscv/riscv-v.cc
&gt; index e338be151d3..0ae87936c38 100644
&gt; --- a/gcc/config/riscv/riscv-v.cc
&gt; +++ b/gcc/config/riscv/riscv-v.cc
&gt; @@ -3093,13 +3093,13 @@ expand_select_vl (rtx *ops)
&gt;&nbsp;&nbsp;&nbsp; emit_insn (gen_no_side_effects_vsetvl_rtx (rvv_mode, ops[0], ops[1]));
&gt;&nbsp; }
&gt;&nbsp; 
&gt; -/* Expand LEN_MASK_{LOAD,STORE}.&nbsp; */
&gt; +/* Expand MASK_LEN_{LOAD,STORE}.&nbsp; */
&gt;&nbsp; void
&gt;&nbsp; expand_load_store (rtx *ops, bool is_load)
&gt;&nbsp; {
&gt;&nbsp;&nbsp;&nbsp; poly_int64 value;
&gt; -&nbsp; rtx len = ops[2];
&gt; -&nbsp; rtx mask = ops[4];
&gt; +&nbsp; rtx mask = ops[2];
&gt; +&nbsp; rtx len = ops[3];
&gt;&nbsp;&nbsp;&nbsp; machine_mode mode = GET_MODE (ops[0]);
&gt;&nbsp; 
&gt;&nbsp;&nbsp;&nbsp; if (poly_int_rtx_p (len, &amp;value) &amp;&amp; known_eq (value, GET_MODE_NUNITS (mode)))
&gt; @@ -3235,6 +3235,8 @@ expand_gather_scatter (rtx *ops, bool is_load)
&gt;&nbsp;&nbsp;&nbsp; rtx ptr, vec_offset, vec_reg, len, mask;
&gt;&nbsp;&nbsp;&nbsp; bool zero_extend_p;
&gt;&nbsp;&nbsp;&nbsp; int scale_log2;
&gt; +&nbsp; rtx mask = ops[5];
&gt; +&nbsp; rtx len = ops[6];
&gt;&nbsp;&nbsp;&nbsp; if (is_load)
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; {
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; vec_reg = ops[0];
&gt; @@ -3242,8 +3244,6 @@ expand_gather_scatter (rtx *ops, bool is_load)
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; vec_offset = ops[2];
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; zero_extend_p = INTVAL (ops[3]);
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; scale_log2 = exact_log2 (INTVAL (ops[4]));
&gt; -&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; len = ops[5];
&gt; -&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; mask = ops[7];
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; }
&gt;&nbsp;&nbsp;&nbsp; else
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; {
&gt; @@ -3252,8 +3252,6 @@ expand_gather_scatter (rtx *ops, bool is_load)
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; vec_offset = ops[1];
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; zero_extend_p = INTVAL (ops[2]);
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; scale_log2 = exact_log2 (INTVAL (ops[3]));
&gt; -&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; len = ops[5];
&gt; -&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; mask = ops[7];
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; }
&gt;&nbsp; 
&gt;&nbsp;&nbsp;&nbsp; machine_mode vec_mode = GET_MODE (vec_reg);
&gt; diff --git a/gcc/doc/md.texi b/gcc/doc/md.texi
&gt; index f864936d54a..9693b6bfe79 100644
&gt; --- a/gcc/doc/md.texi
&gt; +++ b/gcc/doc/md.texi
&gt; @@ -5042,12 +5042,12 @@ of the result should be set to zero.
&gt;&nbsp; 
&gt;&nbsp; @cindex @code{mask_len_gather_load@var{m}@var{n}} instruction pattern
&gt;&nbsp; @item @samp{mask_len_gather_load@var{m}@var{n}}
&gt; -Like @samp{gather_load@var{m}@var{n}}, but takes an extra length operand (operand 5),
&gt; -a bias operand (operand 6) as well as a mask operand (operand 7).&nbsp; Similar to mask_len_load,
&gt; -the instruction loads at most (operand 5 + operand 6) elements from memory.
&gt; +Like @samp{gather_load@var{m}@var{n}}, but takes an extra mask operand (operand 5),
&gt; +a len operand (operand 6) as well as a bias operand (operand 7).&nbsp; Similar to mask_len_load,
&gt; +the instruction loads at most (operand 6 + operand 7) elements from memory.
&gt;&nbsp; Bit @var{i} of the mask is set if element @var{i} of the result should
&gt;&nbsp; be loaded from memory and clear if element @var{i} of the result should be undefined.
&gt; -Mask elements @var{i} with @var{i} &gt; (operand 5 + operand 6) are ignored.
&gt; +Mask elements @var{i} with @var{i} &gt; (operand 6 + operand 7) are ignored.
&gt;&nbsp; 
&gt;&nbsp; @cindex @code{scatter_store@var{m}@var{n}} instruction pattern
&gt;&nbsp; @item @samp{scatter_store@var{m}@var{n}}
&gt; @@ -5080,11 +5080,11 @@ of the result should be stored to memory.
&gt;&nbsp; 
&gt;&nbsp; @cindex @code{mask_len_scatter_store@var{m}@var{n}} instruction pattern
&gt;&nbsp; @item @samp{mask_len_scatter_store@var{m}@var{n}}
&gt; -Like @samp{scatter_store@var{m}@var{n}}, but takes an extra length operand (operand 5),
&gt; -a bias operand (operand 6) as well as a mask operand (operand 7).&nbsp; The instruction stores
&gt; -at most (operand 5 + operand 6) elements of (operand 4) to memory.
&gt; +Like @samp{scatter_store@var{m}@var{n}}, but takes an extra mask operand (operand 5),
&gt; +a len operand (operand 6) as well as a bias operand (operand 7).&nbsp; The instruction stores
&gt; +at most (operand 6 + operand 7) elements of (operand 4) to memory.
&gt;&nbsp; Bit @var{i} of the mask is set if element @var{i} of (operand 4) should be stored.
&gt; -Mask elements @var{i} with @var{i} &gt; (operand 5 + operand 6) are ignored.
&gt; +Mask elements @var{i} with @var{i} &gt; (operand 6 + operand 7) are ignored.
&gt;&nbsp; 
&gt;&nbsp; @cindex @code{vec_set@var{m}} instruction pattern
&gt;&nbsp; @item @samp{vec_set@var{m}}
&gt; @@ -5324,15 +5324,15 @@ This pattern is not allowed to @code{FAIL}.
&gt;&nbsp; @cindex @code{mask_len_load@var{m}@var{n}} instruction pattern
&gt;&nbsp; @item @samp{mask_len_load@var{m}@var{n}}
&gt;&nbsp; Perform a masked load from the memory location pointed to by operand 1
&gt; -into register operand 0.&nbsp; (operand 2 + operand 3) elements are loaded from
&gt; +into register operand 0.&nbsp; (operand 3 + operand 4) elements are loaded from
&gt;&nbsp; memory and other elements in operand 0 are set to undefined values.
&gt;&nbsp; This is a combination of len_load and maskload.
&gt; -Operands 0 and 1 have mode @var{m}, which must be a vector mode.&nbsp; Operand 2
&gt; +Operands 0 and 1 have mode @var{m}, which must be a vector mode.&nbsp; Operand 3
&gt;&nbsp; has whichever integer mode the target prefers.&nbsp; A mask is specified in
&gt; -operand 4 which must be of type @var{n}.&nbsp; The mask has lower precedence than
&gt; +operand 2 which must be of type @var{n}.&nbsp; The mask has lower precedence than
&gt;&nbsp; the length and is itself subject to length masking,
&gt; -i.e. only mask indices < (operand 2 + operand 3) are used.
&gt; -Operand 3 conceptually has mode @code{QI}.
&gt; +i.e. only mask indices < (operand 3 + operand 4) are used.
&gt; +Operand 4 conceptually has mode @code{QI}.
&gt;&nbsp; 
&gt;&nbsp; Operand 2 can be a variable or a constant amount.&nbsp; Operand 4 specifies a
&gt;&nbsp; constant bias: it is either a constant 0 or a constant -1.&nbsp; The predicate on
&gt; @@ -5351,14 +5351,14 @@ This pattern is not allowed to @code{FAIL}.
&gt;&nbsp; @cindex @code{mask_len_store@var{m}@var{n}} instruction pattern
&gt;&nbsp; @item @samp{mask_len_store@var{m}@var{n}}
&gt;&nbsp; Perform a masked store from vector register operand 1 into memory operand 0.
&gt; -(operand 2 + operand 3) elements are stored to memory
&gt; +(operand 3 + operand 4) elements are stored to memory
&gt;&nbsp; and leave the other elements of operand 0 unchanged.
&gt;&nbsp; This is a combination of len_store and maskstore.
&gt; -Operands 0 and 1 have mode @var{m}, which must be a vector mode.&nbsp; Operand 2 has whichever
&gt; -integer mode the target prefers.&nbsp; A mask is specified in operand 4 which must be
&gt; +Operands 0 and 1 have mode @var{m}, which must be a vector mode.&nbsp; Operand 3 has whichever
&gt; +integer mode the target prefers.&nbsp; A mask is specified in operand 2 which must be
&gt;&nbsp; of type @var{n}.&nbsp; The mask has lower precedence than the length and is itself subject to
&gt; -length masking, i.e. only mask indices < (operand 2 + operand 3) are used.
&gt; -Operand 3 conceptually has mode @code{QI}.
&gt; +length masking, i.e. only mask indices < (operand 3 + operand 4) are used.
&gt; +Operand 4 conceptually has mode @code{QI}.
&gt;&nbsp; 
&gt;&nbsp; Operand 2 can be a variable or a constant amount.&nbsp; Operand 3 specifies a
&gt;&nbsp; constant bias: it is either a constant 0 or a constant -1.&nbsp; The predicate on
&gt; diff --git a/gcc/internal-fn.cc b/gcc/internal-fn.cc
&gt; index 969bfe9ab67..0c153a4ee0d 100644
&gt; --- a/gcc/internal-fn.cc
&gt; +++ b/gcc/internal-fn.cc
&gt; @@ -298,10 +298,10 @@ get_multi_vector_move (tree array_type, convert_optab optab)
&gt;&nbsp;&nbsp;&nbsp; return convert_optab_handler (optab, imode, vmode);
&gt;&nbsp; }
&gt;&nbsp; 
&gt; -/* Add len and mask arguments according to the STMT.&nbsp; */
&gt; +/* Add mask and len arguments according to the STMT.&nbsp; */
&gt;&nbsp; 
&gt;&nbsp; static unsigned int
&gt; -add_len_and_mask_args (expand_operand *ops, unsigned int opno, gcall *stmt)
&gt; +add_mask_and_len_args (expand_operand *ops, unsigned int opno, gcall *stmt)
&gt;&nbsp; {
&gt;&nbsp;&nbsp;&nbsp; internal_fn ifn = gimple_call_internal_fn (stmt);
&gt;&nbsp;&nbsp;&nbsp; int len_index = internal_fn_len_index (ifn);
&gt; @@ -309,6 +309,13 @@ add_len_and_mask_args (expand_operand *ops, unsigned int opno, gcall *stmt)
&gt;&nbsp;&nbsp;&nbsp; int bias_index = len_index + 1;
&gt;&nbsp;&nbsp;&nbsp; int mask_index = internal_fn_mask_index (ifn);
&gt;&nbsp;&nbsp;&nbsp; /* The order of arguments are always {len,bias,mask}.&nbsp; */
&gt; +&nbsp; if (mask_index &gt;= 0)
&gt; +&nbsp;&nbsp;&nbsp; {
&gt; +&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; tree mask = gimple_call_arg (stmt, mask_index);
&gt; +&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; rtx mask_rtx = expand_normal (mask);
&gt; +&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; create_input_operand (&amp;ops[opno++], mask_rtx,
&gt; +			&nbsp;&nbsp;&nbsp; TYPE_MODE (TREE_TYPE (mask)));
&gt; +&nbsp;&nbsp;&nbsp; }
&gt;&nbsp;&nbsp;&nbsp; if (len_index &gt;= 0)
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; {
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; tree len = gimple_call_arg (stmt, len_index);
&gt; @@ -320,13 +327,6 @@ add_len_and_mask_args (expand_operand *ops, unsigned int opno, gcall *stmt)
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; rtx bias = expand_normal (biast);
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; create_input_operand (&amp;ops[opno++], bias, QImode);
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; }
&gt; -&nbsp; if (mask_index &gt;= 0)
&gt; -&nbsp;&nbsp;&nbsp; {
&gt; -&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; tree mask = gimple_call_arg (stmt, mask_index);
&gt; -&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; rtx mask_rtx = expand_normal (mask);
&gt; -&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; create_input_operand (&amp;ops[opno++], mask_rtx,
&gt; -			&nbsp;&nbsp;&nbsp; TYPE_MODE (TREE_TYPE (mask)));
&gt; -&nbsp;&nbsp;&nbsp; }
&gt;&nbsp;&nbsp;&nbsp; return opno;
&gt;&nbsp; }
&gt;&nbsp; 
&gt; @@ -2944,7 +2944,7 @@ expand_partial_load_optab_fn (internal_fn ifn, gcall *stmt, convert_optab optab)
&gt;&nbsp;&nbsp;&nbsp; target = expand_expr (lhs, NULL_RTX, VOIDmode, EXPAND_WRITE);
&gt;&nbsp;&nbsp;&nbsp; create_output_operand (&amp;ops[i++], target, TYPE_MODE (type));
&gt;&nbsp;&nbsp;&nbsp; create_fixed_operand (&amp;ops[i++], mem);
&gt; -&nbsp; i = add_len_and_mask_args (ops, i, stmt);
&gt; +&nbsp; i = add_mask_and_len_args (ops, i, stmt);
&gt;&nbsp;&nbsp;&nbsp; expand_insn (icode, i, ops);
&gt;&nbsp; 
&gt;&nbsp;&nbsp;&nbsp; if (!rtx_equal_p (target, ops[0].value))
&gt; @@ -2986,7 +2986,7 @@ expand_partial_store_optab_fn (internal_fn ifn, gcall *stmt, convert_optab optab
&gt;&nbsp;&nbsp;&nbsp; reg = expand_normal (rhs);
&gt;&nbsp;&nbsp;&nbsp; create_fixed_operand (&amp;ops[i++], mem);
&gt;&nbsp;&nbsp;&nbsp; create_input_operand (&amp;ops[i++], reg, TYPE_MODE (type));
&gt; -&nbsp; i = add_len_and_mask_args (ops, i, stmt);
&gt; +&nbsp; i = add_mask_and_len_args (ops, i, stmt);
&gt;&nbsp;&nbsp;&nbsp; expand_insn (icode, i, ops);
&gt;&nbsp; }
&gt;&nbsp; 
&gt; @@ -3566,7 +3566,7 @@ expand_scatter_store_optab_fn (internal_fn, gcall *stmt, direct_optab optab)
&gt;&nbsp;&nbsp;&nbsp; create_integer_operand (&amp;ops[i++], TYPE_UNSIGNED (TREE_TYPE (offset)));
&gt;&nbsp;&nbsp;&nbsp; create_integer_operand (&amp;ops[i++], scale_int);
&gt;&nbsp;&nbsp;&nbsp; create_input_operand (&amp;ops[i++], rhs_rtx, TYPE_MODE (TREE_TYPE (rhs)));
&gt; -&nbsp; i = add_len_and_mask_args (ops, i, stmt);
&gt; +&nbsp; i = add_mask_and_len_args (ops, i, stmt);
&gt;&nbsp; 
&gt;&nbsp;&nbsp;&nbsp; insn_code icode = convert_optab_handler (optab, TYPE_MODE (TREE_TYPE (rhs)),
&gt;&nbsp; 					&nbsp;&nbsp; TYPE_MODE (TREE_TYPE (offset)));
&gt; @@ -3595,7 +3595,7 @@ expand_gather_load_optab_fn (internal_fn, gcall *stmt, direct_optab optab)
&gt;&nbsp;&nbsp;&nbsp; create_input_operand (&amp;ops[i++], offset_rtx, TYPE_MODE (TREE_TYPE (offset)));
&gt;&nbsp;&nbsp;&nbsp; create_integer_operand (&amp;ops[i++], TYPE_UNSIGNED (TREE_TYPE (offset)));
&gt;&nbsp;&nbsp;&nbsp; create_integer_operand (&amp;ops[i++], scale_int);
&gt; -&nbsp; i = add_len_and_mask_args (ops, i, stmt);
&gt; +&nbsp; i = add_mask_and_len_args (ops, i, stmt);
&gt;&nbsp;&nbsp;&nbsp; insn_code icode = convert_optab_handler (optab, TYPE_MODE (TREE_TYPE (lhs)),
&gt;&nbsp; 					&nbsp;&nbsp; TYPE_MODE (TREE_TYPE (offset)));
&gt;&nbsp;&nbsp;&nbsp; expand_insn (icode, i, ops);
&gt; @@ -4603,8 +4603,6 @@ internal_fn_len_index (internal_fn fn)
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; {
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; case IFN_LEN_LOAD:
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; case IFN_LEN_STORE:
&gt; -&nbsp;&nbsp;&nbsp; case IFN_MASK_LEN_LOAD:
&gt; -&nbsp;&nbsp;&nbsp; case IFN_MASK_LEN_STORE:
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; return 2;
&gt;&nbsp; 
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; case IFN_MASK_LEN_GATHER_LOAD:
&gt; @@ -4633,6 +4631,8 @@ internal_fn_len_index (internal_fn fn)
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; return 4;
&gt;&nbsp; 
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; case IFN_COND_LEN_NEG:
&gt; +&nbsp;&nbsp;&nbsp; case IFN_MASK_LEN_LOAD:
&gt; +&nbsp;&nbsp;&nbsp; case IFN_MASK_LEN_STORE:
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; return 3;
&gt;&nbsp; 
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; default:
&gt; @@ -4652,12 +4652,12 @@ internal_fn_mask_index (internal_fn fn)
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; case IFN_MASK_LOAD_LANES:
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; case IFN_MASK_STORE:
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; case IFN_MASK_STORE_LANES:
&gt; +&nbsp;&nbsp;&nbsp; case IFN_MASK_LEN_LOAD:
&gt; +&nbsp;&nbsp;&nbsp; case IFN_MASK_LEN_STORE:
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; return 2;
&gt;&nbsp; 
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; case IFN_MASK_GATHER_LOAD:
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; case IFN_MASK_SCATTER_STORE:
&gt; -&nbsp;&nbsp;&nbsp; case IFN_MASK_LEN_LOAD:
&gt; -&nbsp;&nbsp;&nbsp; case IFN_MASK_LEN_STORE:
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; case IFN_MASK_LEN_GATHER_LOAD:
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; case IFN_MASK_LEN_SCATTER_STORE:
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; return 4;
&gt; @@ -4742,17 +4742,18 @@ internal_check_ptrs_fn_supported_p (internal_fn ifn, tree type,
&gt;&nbsp; 	&nbsp; &amp;&amp; insn_operand_matches (icode, 4, GEN_INT (align)));
&gt;&nbsp; }
&gt;&nbsp; 
&gt; -/* Return the supported bias for IFN which is either IFN_LEN_LOAD
&gt; -&nbsp;&nbsp; or IFN_LEN_STORE.&nbsp; For now we only support the biases of 0 and -1
&gt; -&nbsp;&nbsp; (in case 0 is not an allowable length for len_load or len_store).
&gt; -&nbsp;&nbsp; If none of the biases match what the backend provides, return
&gt; -&nbsp;&nbsp; VECT_PARTIAL_BIAS_UNSUPPORTED.&nbsp; */
&gt; +/* Return the supported bias for IFN which is either IFN_{LEN_,MASK_LEN_,}LOAD
&gt; +&nbsp;&nbsp; or IFN_{LEN_,MASK_LEN_,}STORE.&nbsp; For now we only support the biases of 0 and
&gt; +&nbsp;&nbsp; -1 (in case 0 is not an allowable length for {len_,mask_len_}load or
&gt; +&nbsp;&nbsp; {len_,mask_len_}store). If none of the biases match what the backend
&gt; +&nbsp;&nbsp; provides, return VECT_PARTIAL_BIAS_UNSUPPORTED.&nbsp; */
&gt;&nbsp; 
&gt;&nbsp; signed char
&gt;&nbsp; internal_len_load_store_bias (internal_fn ifn, machine_mode mode)
&gt;&nbsp; {
&gt;&nbsp;&nbsp;&nbsp; optab optab = direct_internal_fn_optab (ifn);
&gt;&nbsp;&nbsp;&nbsp; insn_code icode = direct_optab_handler (optab, mode);
&gt; +&nbsp; int bias_no = 3;
&gt;&nbsp; 
&gt;&nbsp;&nbsp;&nbsp; if (icode == CODE_FOR_nothing)
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; {
&gt; @@ -4770,14 +4771,15 @@ internal_len_load_store_bias (internal_fn ifn, machine_mode mode)
&gt;&nbsp; 	&nbsp; optab = direct_internal_fn_optab (IFN_MASK_LEN_STORE);
&gt;&nbsp; 	}
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; icode = convert_optab_handler (optab, mode, mask_mode);
&gt; +&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; bias_no = 4;
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; }
&gt;&nbsp; 
&gt;&nbsp;&nbsp;&nbsp; if (icode != CODE_FOR_nothing)
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; {
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; /* For now we only support biases of 0 or -1.&nbsp; Try both of them.&nbsp; */
&gt; -&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; if (insn_operand_matches (icode, 3, GEN_INT (0)))
&gt; +&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; if (insn_operand_matches (icode, bias_no, GEN_INT (0)))
&gt;&nbsp; 	return 0;
&gt; -&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; if (insn_operand_matches (icode, 3, GEN_INT (-1)))
&gt; +&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; if (insn_operand_matches (icode, bias_no, GEN_INT (-1)))
&gt;&nbsp; 	return -1;
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; }
&gt;&nbsp; 
&gt; diff --git a/gcc/tree-vect-stmts.cc b/gcc/tree-vect-stmts.cc
&gt; index caa4c974cb9..d5b4f020332 100644
&gt; --- a/gcc/tree-vect-stmts.cc
&gt; +++ b/gcc/tree-vect-stmts.cc
&gt; @@ -9087,8 +9087,8 @@ vectorizable_store (vec_info *vinfo,
&gt;&nbsp; 		&nbsp; if (partial_ifn == IFN_MASK_LEN_STORE)
&gt;&nbsp; 		&nbsp;&nbsp;&nbsp; call = gimple_build_call_internal (IFN_MASK_LEN_STORE, 6,
&gt;&nbsp; 						&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; dataref_ptr, ptr,
&gt; -						&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; final_len, bias,
&gt; -						&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; final_mask, vec_oprnd);
&gt; +						&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; final_mask, final_len,
&gt; +						&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; bias, vec_oprnd);
&gt;&nbsp; 		&nbsp; else
&gt;&nbsp; 		&nbsp;&nbsp;&nbsp; call
&gt;&nbsp; 		&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; = gimple_build_call_internal (IFN_LEN_STORE, 5,
&gt; @@ -10654,8 +10654,8 @@ vectorizable_load (vec_info *vinfo,
&gt;&nbsp; 			if (partial_ifn == IFN_MASK_LEN_LOAD)
&gt;&nbsp; 			&nbsp; call = gimple_build_call_internal (IFN_MASK_LEN_LOAD,
&gt;&nbsp; 							&nbsp;&nbsp;&nbsp;&nbsp; 5, dataref_ptr,
&gt; -							&nbsp;&nbsp;&nbsp;&nbsp; ptr, final_len,
&gt; -							&nbsp;&nbsp;&nbsp;&nbsp; bias, final_mask);
&gt; +							&nbsp;&nbsp;&nbsp;&nbsp; ptr, final_mask,
&gt; +							&nbsp;&nbsp;&nbsp;&nbsp; final_len, bias);
&gt;&nbsp; 			else
&gt;&nbsp; 			&nbsp; call = gimple_build_call_internal (IFN_LEN_LOAD, 4,
&gt;&nbsp; 							&nbsp;&nbsp;&nbsp;&nbsp; dataref_ptr, ptr,
&gt; 

-- 
Richard Biener <rguenther@suse.de&gt;
SUSE Software Solutions Germany GmbH, Frankenstrasse 146, 90461 Nuernberg,
Germany; GF: Ivo Totev, Andrew Myers, Andrew McDonald, Boudien Moerman;
HRB 36809 (AG Nuernberg)