From: Joel Hutton <Joel.Hutton@arm.com>
To: Richard Biener <rguenther@suse.de>
Cc: Richard Sandiford <Richard.Sandiford@arm.com>,
"gcc-patches@gcc.gnu.org" <gcc-patches@gcc.gnu.org>,
Andre Simoes Dias Vieira <Andre.SimoesDiasVieira@arm.com>
Subject: RE: [ping][vect-patterns] Refactor widen_plus/widen_minus as internal_fns
Date: Thu, 30 Jun 2022 13:20:13 +0000 [thread overview]
Message-ID: <DB9PR08MB6603D656FD05B9D562033809F5BA9@DB9PR08MB6603.eurprd08.prod.outlook.com> (raw)
In-Reply-To: <sp30q1n6-orn4-p4ss-q36s-734854o1ss4@fhfr.qr>
[-- Attachment #1: Type: text/plain, Size: 647 bytes --]
> We can go with a private vect_gimple_build function until we sort out the API
> issue to unblock Tamar (I'll reply to Richards reply with further thoughts on
> this)
>
Done.
> > Similarly are you ok with the use of gimple_extract_op? I would lean
> towards using it as it is cleaner, but I don't have strong feelings.
>
> I don't like using gimple_extract_op here, I think I outlined a variant that is
> even shorter.
>
Done.
Updated patches attached, bootstrapped and regression tested on aarch64.
Tomorrow is my last working day at Arm, so it will likely be Andre that commits this/addresses any further comments.
[-- Attachment #2: 0001-Refactor-to-allow-internal_fn-s.patch --]
[-- Type: application/octet-stream, Size: 23390 bytes --]
From f1321c617838e94044cbae357a63db002fbd3edb Mon Sep 17 00:00:00 2001
From: Joel Hutton <joel.hutton@arm.com>
Date: Wed, 25 Aug 2021 14:31:15 +0100
Subject: [PATCH 1/3] Refactor to allow internal_fn's
Hi all,
This refactor allows widening patterns (such as widen_plus/widen_minus) to be represented as
either internal_fns or tree_codes.
[vect-patterns] Refactor as internal_fn's
Refactor vect-patterns to allow patterns to be internal_fns starting
with widening_plus/minus patterns
gcc/ChangeLog:
* tree-core.h (ECF_WIDEN): New flag.
* gimple-match.h (class code_helper): * tree-core.h (ECF_WIDEN): Flag to mark internal_fn as widening.
* tree-vect-patterns.cc (vect_recog_widen_op_pattern): Refactor to
use code_helper.
(vect_gimple_build): New function.
* tree-vect-stmts.cc (vect_gen_widened_results_half): Refactor to
use code_helper.
(vect_create_vectorized_promotion_stmts): Refactor to use
code_helper.
(vectorizable_conversion): Refactor to use code_helper.
gimple_call or gimple_assign.
(supportable_widening_operation): Refactor to use code_helper.
(supportable_narrowing_operation): Refactor to use code_helper.
* tree-vectorizer.h (supportable_widening_operation): Change
prototype to use code_helper.
(supportable_narrowing_operation): change prototype to use
code_helper.
(vect_gimple_build): New function prototype.
* tree.h (code_helper::safe_as_tree_code): New function.
helper functions.
(code_helper::safe_as_fn_code): New function.
---
gcc/tree-core.h | 3 +
gcc/tree-vect-patterns.cc | 34 ++++++-
gcc/tree-vect-stmts.cc | 208 +++++++++++++++++++++++++-------------
gcc/tree-vectorizer.h | 14 +--
gcc/tree.h | 13 +++
5 files changed, 189 insertions(+), 83 deletions(-)
diff --git a/gcc/tree-core.h b/gcc/tree-core.h
index ab5fa01e5cb5fb56c1964b93b014ed55a4aa704a..cff6211080bced0bffb39e98039a6550897acf77 100644
--- a/gcc/tree-core.h
+++ b/gcc/tree-core.h
@@ -96,6 +96,9 @@ struct die_struct;
/* Nonzero if this is a cold function. */
#define ECF_COLD (1 << 15)
+/* Nonzero if this is a widening function. */
+#define ECF_WIDEN (1 << 16)
+
/* Call argument flags. */
/* Nonzero if the argument is not used by the function. */
diff --git a/gcc/tree-vect-patterns.cc b/gcc/tree-vect-patterns.cc
index 8f624863971392c891fde7278949c8818f646576..d892158f024fc045b897aebe76f2e2b66211cf83 100644
--- a/gcc/tree-vect-patterns.cc
+++ b/gcc/tree-vect-patterns.cc
@@ -25,6 +25,8 @@ along with GCC; see the file COPYING3. If not see
#include "rtl.h"
#include "tree.h"
#include "gimple.h"
+#include "gimple-iterator.h"
+#include "gimple-fold.h"
#include "ssa.h"
#include "expmed.h"
#include "optabs-tree.h"
@@ -1348,7 +1350,7 @@ vect_recog_sad_pattern (vec_info *vinfo,
static gimple *
vect_recog_widen_op_pattern (vec_info *vinfo,
stmt_vec_info last_stmt_info, tree *type_out,
- tree_code orig_code, tree_code wide_code,
+ tree_code orig_code, code_helper wide_code,
bool shift_p, const char *name)
{
gimple *last_stmt = last_stmt_info->stmt;
@@ -1391,7 +1393,7 @@ vect_recog_widen_op_pattern (vec_info *vinfo,
vecctype = get_vectype_for_scalar_type (vinfo, ctype);
}
- enum tree_code dummy_code;
+ code_helper dummy_code;
int dummy_int;
auto_vec<tree> dummy_vec;
if (!vectype
@@ -1412,8 +1414,7 @@ vect_recog_widen_op_pattern (vec_info *vinfo,
2, oprnd, half_type, unprom, vectype);
tree var = vect_recog_temp_ssa_var (itype, NULL);
- gimple *pattern_stmt = gimple_build_assign (var, wide_code,
- oprnd[0], oprnd[1]);
+ gimple *pattern_stmt = vect_gimple_build (var, wide_code, oprnd[0], oprnd[1]);
if (vecctype != vecitype)
pattern_stmt = vect_convert_output (vinfo, last_stmt_info, ctype,
@@ -5968,3 +5969,28 @@ vect_pattern_recog (vec_info *vinfo)
/* After this no more add_stmt calls are allowed. */
vinfo->stmt_vec_info_ro = true;
}
+
+/* Build a GIMPLE_ASSIGN or GIMPLE_CALL with the tree_code,
+ or internal_fn contained in ch, respectively. */
+gimple *
+vect_gimple_build (tree lhs, code_helper ch, tree op0, tree op1)
+{
+ if (op0 == NULL_TREE)
+ return NULL;
+ if (ch.is_tree_code ())
+ return op1 == NULL_TREE ? gimple_build_assign (lhs, ch.safe_as_tree_code (),
+ op0) :
+ gimple_build_assign (lhs, ch.safe_as_tree_code (),
+ op0, op1);
+ else
+ {
+ internal_fn fn = as_internal_fn (ch.safe_as_fn_code ());
+ gimple* stmt;
+ if (op1 == NULL_TREE)
+ stmt = gimple_build_call_internal (fn, 1, op0);
+ else
+ stmt = gimple_build_call_internal (fn, 2, op0, op1);
+ gimple_call_set_lhs (stmt, lhs);
+ return stmt;
+ }
+}
diff --git a/gcc/tree-vect-stmts.cc b/gcc/tree-vect-stmts.cc
index 346d8ce280437e00bfeb19a4b4adc59eb96207f9..d6aabb873c86ab8ff0bae41c7f6c3bad34d583c5 100644
--- a/gcc/tree-vect-stmts.cc
+++ b/gcc/tree-vect-stmts.cc
@@ -4636,7 +4636,7 @@ vectorizable_simd_clone_call (vec_info *vinfo, stmt_vec_info stmt_info,
STMT_INFO is the original scalar stmt that we are vectorizing. */
static gimple *
-vect_gen_widened_results_half (vec_info *vinfo, enum tree_code code,
+vect_gen_widened_results_half (vec_info *vinfo, code_helper ch,
tree vec_oprnd0, tree vec_oprnd1, int op_type,
tree vec_dest, gimple_stmt_iterator *gsi,
stmt_vec_info stmt_info)
@@ -4645,12 +4645,11 @@ vect_gen_widened_results_half (vec_info *vinfo, enum tree_code code,
tree new_temp;
/* Generate half of the widened result: */
- gcc_assert (op_type == TREE_CODE_LENGTH (code));
if (op_type != binary_op)
vec_oprnd1 = NULL;
- new_stmt = gimple_build_assign (vec_dest, code, vec_oprnd0, vec_oprnd1);
+ new_stmt = vect_gimple_build (vec_dest, ch, vec_oprnd0, vec_oprnd1);
new_temp = make_ssa_name (vec_dest, new_stmt);
- gimple_assign_set_lhs (new_stmt, new_temp);
+ gimple_set_lhs (new_stmt, new_temp);
vect_finish_stmt_generation (vinfo, stmt_info, new_stmt, gsi);
return new_stmt;
@@ -4729,8 +4728,8 @@ vect_create_vectorized_promotion_stmts (vec_info *vinfo,
vec<tree> *vec_oprnds1,
stmt_vec_info stmt_info, tree vec_dest,
gimple_stmt_iterator *gsi,
- enum tree_code code1,
- enum tree_code code2, int op_type)
+ code_helper ch1,
+ code_helper ch2, int op_type)
{
int i;
tree vop0, vop1, new_tmp1, new_tmp2;
@@ -4746,10 +4745,10 @@ vect_create_vectorized_promotion_stmts (vec_info *vinfo,
vop1 = NULL_TREE;
/* Generate the two halves of promotion operation. */
- new_stmt1 = vect_gen_widened_results_half (vinfo, code1, vop0, vop1,
+ new_stmt1 = vect_gen_widened_results_half (vinfo, ch1, vop0, vop1,
op_type, vec_dest, gsi,
stmt_info);
- new_stmt2 = vect_gen_widened_results_half (vinfo, code2, vop0, vop1,
+ new_stmt2 = vect_gen_widened_results_half (vinfo, ch2, vop0, vop1,
op_type, vec_dest, gsi,
stmt_info);
if (is_gimple_call (new_stmt1))
@@ -4846,8 +4845,9 @@ vectorizable_conversion (vec_info *vinfo,
tree scalar_dest;
tree op0, op1 = NULL_TREE;
loop_vec_info loop_vinfo = dyn_cast <loop_vec_info> (vinfo);
- enum tree_code code, code1 = ERROR_MARK, code2 = ERROR_MARK;
- enum tree_code codecvt1 = ERROR_MARK, codecvt2 = ERROR_MARK;
+ tree_code tc1;
+ code_helper code, code1, code2;
+ code_helper codecvt1 = ERROR_MARK, codecvt2 = ERROR_MARK;
tree new_temp;
enum vect_def_type dt[2] = {vect_unknown_def_type, vect_unknown_def_type};
int ndts = 2;
@@ -4876,31 +4876,43 @@ vectorizable_conversion (vec_info *vinfo,
&& ! vec_stmt)
return false;
- gassign *stmt = dyn_cast <gassign *> (stmt_info->stmt);
- if (!stmt)
+ gimple* stmt = stmt_info->stmt;
+ if (!(is_gimple_assign (stmt) || is_gimple_call (stmt)))
return false;
- if (TREE_CODE (gimple_assign_lhs (stmt)) != SSA_NAME)
+ if (gimple_get_lhs (stmt) == NULL_TREE
+ || TREE_CODE (gimple_get_lhs (stmt)) != SSA_NAME)
return false;
- code = gimple_assign_rhs_code (stmt);
- if (!CONVERT_EXPR_CODE_P (code)
- && code != FIX_TRUNC_EXPR
- && code != FLOAT_EXPR
- && code != WIDEN_PLUS_EXPR
- && code != WIDEN_MINUS_EXPR
- && code != WIDEN_MULT_EXPR
- && code != WIDEN_LSHIFT_EXPR)
+ if (TREE_CODE (gimple_get_lhs (stmt)) != SSA_NAME)
+ return false;
+
+ if (is_gimple_assign (stmt))
+ {
+ code = gimple_assign_rhs_code (stmt);
+ op_type = TREE_CODE_LENGTH (code.safe_as_tree_code ());
+ }
+ else if (gimple_call_internal_p (stmt))
+ {
+ code = gimple_call_internal_fn (stmt);
+ op_type = gimple_call_num_args (stmt);
+ }
+ else
return false;
bool widen_arith = (code == WIDEN_PLUS_EXPR
- || code == WIDEN_MINUS_EXPR
- || code == WIDEN_MULT_EXPR
- || code == WIDEN_LSHIFT_EXPR);
- op_type = TREE_CODE_LENGTH (code);
+ || code == WIDEN_MINUS_EXPR
+ || code == WIDEN_MULT_EXPR
+ || code == WIDEN_LSHIFT_EXPR);
+
+ if (!widen_arith
+ && !CONVERT_EXPR_CODE_P (code)
+ && code != FIX_TRUNC_EXPR
+ && code != FLOAT_EXPR)
+ return false;
/* Check types of lhs and rhs. */
- scalar_dest = gimple_assign_lhs (stmt);
+ scalar_dest = gimple_get_lhs (stmt);
lhs_type = TREE_TYPE (scalar_dest);
vectype_out = STMT_VINFO_VECTYPE (stmt_info);
@@ -4938,10 +4950,14 @@ vectorizable_conversion (vec_info *vinfo,
if (op_type == binary_op)
{
- gcc_assert (code == WIDEN_MULT_EXPR || code == WIDEN_LSHIFT_EXPR
- || code == WIDEN_PLUS_EXPR || code == WIDEN_MINUS_EXPR);
+ gcc_assert (code == WIDEN_MULT_EXPR
+ || code == WIDEN_LSHIFT_EXPR
+ || code == WIDEN_PLUS_EXPR
+ || code == WIDEN_MINUS_EXPR);
- op1 = gimple_assign_rhs2 (stmt);
+
+ op1 = is_gimple_assign (stmt) ? gimple_assign_rhs2 (stmt) :
+ gimple_call_arg (stmt, 0);
tree vectype1_in;
if (!vect_is_simple_use (vinfo, stmt_info, slp_node, 1,
&op1, &slp_op1, &dt[1], &vectype1_in))
@@ -5025,8 +5041,12 @@ vectorizable_conversion (vec_info *vinfo,
&& code != FLOAT_EXPR
&& !CONVERT_EXPR_CODE_P (code))
return false;
- if (supportable_convert_operation (code, vectype_out, vectype_in, &code1))
+ if (supportable_convert_operation (code.safe_as_tree_code (), vectype_out,
+ vectype_in, &tc1))
+ {
+ code1 = tc1;
break;
+ }
/* FALLTHRU */
unsupported:
if (dump_enabled_p ())
@@ -5037,9 +5057,11 @@ vectorizable_conversion (vec_info *vinfo,
case WIDEN:
if (known_eq (nunits_in, nunits_out))
{
- if (!supportable_half_widening_operation (code, vectype_out,
- vectype_in, &code1))
+ if (!supportable_half_widening_operation (code.safe_as_tree_code (),
+ vectype_out, vectype_in,
+ &tc1))
goto unsupported;
+ code1 = tc1;
gcc_assert (!(multi_step_cvt && op_type == binary_op));
break;
}
@@ -5073,14 +5095,17 @@ vectorizable_conversion (vec_info *vinfo,
if (GET_MODE_SIZE (rhs_mode) == fltsz)
{
- if (!supportable_convert_operation (code, vectype_out,
- cvt_type, &codecvt1))
+ tc1 = ERROR_MARK;
+ if (!supportable_convert_operation (code.safe_as_tree_code (),
+ vectype_out,
+ cvt_type, &tc1))
goto unsupported;
+ codecvt1 = tc1;
}
- else if (!supportable_widening_operation (vinfo, code, stmt_info,
- vectype_out, cvt_type,
- &codecvt1, &codecvt2,
- &multi_step_cvt,
+ else if (!supportable_widening_operation (vinfo, code,
+ stmt_info, vectype_out,
+ cvt_type, &codecvt1,
+ &codecvt2, &multi_step_cvt,
&interm_types))
continue;
else
@@ -5088,8 +5113,9 @@ vectorizable_conversion (vec_info *vinfo,
if (supportable_widening_operation (vinfo, NOP_EXPR, stmt_info,
cvt_type,
- vectype_in, &code1, &code2,
- &multi_step_cvt, &interm_types))
+ vectype_in, &code1,
+ &code2, &multi_step_cvt,
+ &interm_types))
{
found_mode = true;
break;
@@ -5111,10 +5137,15 @@ vectorizable_conversion (vec_info *vinfo,
case NARROW:
gcc_assert (op_type == unary_op);
- if (supportable_narrowing_operation (code, vectype_out, vectype_in,
- &code1, &multi_step_cvt,
+ if (supportable_narrowing_operation (code.safe_as_tree_code (),
+ vectype_out,
+ vectype_in,
+ &tc1, &multi_step_cvt,
&interm_types))
- break;
+ {
+ code1 = tc1;
+ break;
+ }
if (code != FIX_TRUNC_EXPR
|| GET_MODE_SIZE (lhs_mode) >= GET_MODE_SIZE (rhs_mode))
@@ -5125,13 +5156,18 @@ vectorizable_conversion (vec_info *vinfo,
cvt_type = get_same_sized_vectype (cvt_type, vectype_in);
if (cvt_type == NULL_TREE)
goto unsupported;
- if (!supportable_convert_operation (code, cvt_type, vectype_in,
- &codecvt1))
+ if (!supportable_convert_operation (code.safe_as_tree_code (), cvt_type,
+ vectype_in,
+ &tc1))
goto unsupported;
+ codecvt1 = tc1;
if (supportable_narrowing_operation (NOP_EXPR, vectype_out, cvt_type,
- &code1, &multi_step_cvt,
+ &tc1, &multi_step_cvt,
&interm_types))
- break;
+ {
+ code1 = tc1;
+ break;
+ }
goto unsupported;
default:
@@ -5245,8 +5281,10 @@ vectorizable_conversion (vec_info *vinfo,
FOR_EACH_VEC_ELT (vec_oprnds0, i, vop0)
{
/* Arguments are ready, create the new vector stmt. */
- gcc_assert (TREE_CODE_LENGTH (code1) == unary_op);
- gassign *new_stmt = gimple_build_assign (vec_dest, code1, vop0);
+ gcc_assert (TREE_CODE_LENGTH ((tree_code) code1) == unary_op);
+ gassign *new_stmt = gimple_build_assign (vec_dest,
+ code1.safe_as_tree_code (),
+ vop0);
new_temp = make_ssa_name (vec_dest, new_stmt);
gimple_assign_set_lhs (new_stmt, new_temp);
vect_finish_stmt_generation (vinfo, stmt_info, new_stmt, gsi);
@@ -5278,7 +5316,7 @@ vectorizable_conversion (vec_info *vinfo,
for (i = multi_step_cvt; i >= 0; i--)
{
tree this_dest = vec_dsts[i];
- enum tree_code c1 = code1, c2 = code2;
+ code_helper c1 = code1, c2 = code2;
if (i == 0 && codecvt2 != ERROR_MARK)
{
c1 = codecvt1;
@@ -5288,7 +5326,8 @@ vectorizable_conversion (vec_info *vinfo,
vect_create_half_widening_stmts (vinfo, &vec_oprnds0,
&vec_oprnds1, stmt_info,
this_dest, gsi,
- c1, op_type);
+ c1.safe_as_tree_code (),
+ op_type);
else
vect_create_vectorized_promotion_stmts (vinfo, &vec_oprnds0,
&vec_oprnds1, stmt_info,
@@ -5301,9 +5340,11 @@ vectorizable_conversion (vec_info *vinfo,
gimple *new_stmt;
if (cvt_type)
{
- gcc_assert (TREE_CODE_LENGTH (codecvt1) == unary_op);
+ gcc_assert (TREE_CODE_LENGTH ((tree_code) codecvt1) == unary_op);
new_temp = make_ssa_name (vec_dest);
- new_stmt = gimple_build_assign (new_temp, codecvt1, vop0);
+ new_stmt = gimple_build_assign (new_temp,
+ codecvt1.safe_as_tree_code (),
+ vop0);
vect_finish_stmt_generation (vinfo, stmt_info, new_stmt, gsi);
}
else
@@ -5327,10 +5368,12 @@ vectorizable_conversion (vec_info *vinfo,
if (cvt_type)
FOR_EACH_VEC_ELT (vec_oprnds0, i, vop0)
{
- gcc_assert (TREE_CODE_LENGTH (codecvt1) == unary_op);
+ gcc_assert (TREE_CODE_LENGTH (((tree_code) codecvt1)) == unary_op);
new_temp = make_ssa_name (vec_dest);
gassign *new_stmt
- = gimple_build_assign (new_temp, codecvt1, vop0);
+ = gimple_build_assign (new_temp,
+ codecvt1.safe_as_tree_code (),
+ vop0);
vect_finish_stmt_generation (vinfo, stmt_info, new_stmt, gsi);
vec_oprnds0[i] = new_temp;
}
@@ -5338,7 +5381,8 @@ vectorizable_conversion (vec_info *vinfo,
vect_create_vectorized_demotion_stmts (vinfo, &vec_oprnds0,
multi_step_cvt,
stmt_info, vec_dsts, gsi,
- slp_node, code1);
+ slp_node,
+ code1.safe_as_tree_code ());
break;
}
if (!slp_node)
@@ -11926,9 +11970,11 @@ vect_maybe_update_slp_op_vectype (slp_tree op, tree vectype)
bool
supportable_widening_operation (vec_info *vinfo,
- enum tree_code code, stmt_vec_info stmt_info,
+ code_helper code,
+ stmt_vec_info stmt_info,
tree vectype_out, tree vectype_in,
- enum tree_code *code1, enum tree_code *code2,
+ code_helper *code1,
+ code_helper *code2,
int *multi_step_cvt,
vec<tree> *interm_types)
{
@@ -11939,7 +11985,7 @@ supportable_widening_operation (vec_info *vinfo,
optab optab1, optab2;
tree vectype = vectype_in;
tree wide_vectype = vectype_out;
- enum tree_code c1, c2;
+ code_helper c1 = MAX_TREE_CODES, c2 = MAX_TREE_CODES;
int i;
tree prev_type, intermediate_type;
machine_mode intermediate_mode, prev_mode;
@@ -11949,7 +11995,7 @@ supportable_widening_operation (vec_info *vinfo,
if (loop_info)
vect_loop = LOOP_VINFO_LOOP (loop_info);
- switch (code)
+ switch (code.safe_as_tree_code ())
{
case WIDEN_MULT_EXPR:
/* The result of a vectorized widening operation usually requires
@@ -11990,8 +12036,9 @@ supportable_widening_operation (vec_info *vinfo,
&& !nested_in_vect_loop_p (vect_loop, stmt_info)
&& supportable_widening_operation (vinfo, VEC_WIDEN_MULT_EVEN_EXPR,
stmt_info, vectype_out,
- vectype_in, code1, code2,
- multi_step_cvt, interm_types))
+ vectype_in, code1,
+ code2, multi_step_cvt,
+ interm_types))
{
/* Elements in a vector with vect_used_by_reduction property cannot
be reordered if the use chain with this property does not have the
@@ -12054,6 +12101,9 @@ supportable_widening_operation (vec_info *vinfo,
c2 = VEC_UNPACK_FIX_TRUNC_HI_EXPR;
break;
+ case MAX_TREE_CODES:
+ break;
+
default:
gcc_unreachable ();
}
@@ -12064,10 +12114,12 @@ supportable_widening_operation (vec_info *vinfo,
if (code == FIX_TRUNC_EXPR)
{
/* The signedness is determined from output operand. */
- optab1 = optab_for_tree_code (c1, vectype_out, optab_default);
- optab2 = optab_for_tree_code (c2, vectype_out, optab_default);
+ optab1 = optab_for_tree_code (c1.safe_as_tree_code (), vectype_out,
+ optab_default);
+ optab2 = optab_for_tree_code (c2.safe_as_tree_code (), vectype_out,
+ optab_default);
}
- else if (CONVERT_EXPR_CODE_P (code)
+ else if (CONVERT_EXPR_CODE_P (code.safe_as_tree_code ())
&& VECTOR_BOOLEAN_TYPE_P (wide_vectype)
&& VECTOR_BOOLEAN_TYPE_P (vectype)
&& TYPE_MODE (wide_vectype) == TYPE_MODE (vectype)
@@ -12080,8 +12132,10 @@ supportable_widening_operation (vec_info *vinfo,
}
else
{
- optab1 = optab_for_tree_code (c1, vectype, optab_default);
- optab2 = optab_for_tree_code (c2, vectype, optab_default);
+ optab1 = optab_for_tree_code (c1.safe_as_tree_code (), vectype,
+ optab_default);
+ optab2 = optab_for_tree_code (c2.safe_as_tree_code (), vectype,
+ optab_default);
}
if (!optab1 || !optab2)
@@ -12092,8 +12146,12 @@ supportable_widening_operation (vec_info *vinfo,
|| (icode2 = optab_handler (optab2, vec_mode)) == CODE_FOR_nothing)
return false;
- *code1 = c1;
- *code2 = c2;
+ if (code.is_tree_code ())
+ {
+ *code1 = c1;
+ *code2 = c2;
+ }
+
if (insn_data[icode1].operand[0].mode == TYPE_MODE (wide_vectype)
&& insn_data[icode2].operand[0].mode == TYPE_MODE (wide_vectype))
@@ -12114,7 +12172,7 @@ supportable_widening_operation (vec_info *vinfo,
prev_type = vectype;
prev_mode = vec_mode;
- if (!CONVERT_EXPR_CODE_P (code))
+ if (!CONVERT_EXPR_CODE_P ((tree_code) code))
return false;
/* We assume here that there will not be more than MAX_INTERM_CVT_STEPS
@@ -12145,8 +12203,12 @@ supportable_widening_operation (vec_info *vinfo,
}
else
{
- optab3 = optab_for_tree_code (c1, intermediate_type, optab_default);
- optab4 = optab_for_tree_code (c2, intermediate_type, optab_default);
+ optab3 = optab_for_tree_code (c1.safe_as_tree_code (),
+ intermediate_type,
+ optab_default);
+ optab4 = optab_for_tree_code (c2.safe_as_tree_code (),
+ intermediate_type,
+ optab_default);
}
if (!optab3 || !optab4
@@ -12205,7 +12267,7 @@ supportable_widening_operation (vec_info *vinfo,
bool
supportable_narrowing_operation (enum tree_code code,
tree vectype_out, tree vectype_in,
- enum tree_code *code1, int *multi_step_cvt,
+ tree_code *code1, int *multi_step_cvt,
vec<tree> *interm_types)
{
machine_mode vec_mode;
diff --git a/gcc/tree-vectorizer.h b/gcc/tree-vectorizer.h
index 642eb0aeb21264cd736a479b1ec25357abef29cd..6f70bd622c4a4dea8c432cd26c96d24af399ef3e 100644
--- a/gcc/tree-vectorizer.h
+++ b/gcc/tree-vectorizer.h
@@ -2120,13 +2120,12 @@ extern bool vect_is_simple_use (vec_info *, stmt_vec_info, slp_tree,
enum vect_def_type *,
tree *, stmt_vec_info * = NULL);
extern bool vect_maybe_update_slp_op_vectype (slp_tree, tree);
-extern bool supportable_widening_operation (vec_info *,
- enum tree_code, stmt_vec_info,
- tree, tree, enum tree_code *,
- enum tree_code *, int *,
- vec<tree> *);
+extern bool supportable_widening_operation (vec_info*, code_helper,
+ stmt_vec_info, tree, tree,
+ code_helper*, code_helper*,
+ int*, vec<tree> *);
extern bool supportable_narrowing_operation (enum tree_code, tree, tree,
- enum tree_code *, int *,
+ tree_code *, int *,
vec<tree> *);
extern unsigned record_stmt_cost (stmt_vector_for_cost *, int,
@@ -2558,4 +2557,7 @@ vect_is_integer_truncation (stmt_vec_info stmt_info)
&& TYPE_PRECISION (lhs_type) < TYPE_PRECISION (rhs_type));
}
+/* Build a GIMPLE_ASSIGN or GIMPLE_CALL with the tree_code,
+ or internal_fn contained in ch, respectively. */
+gimple * vect_gimple_build (tree, code_helper, tree, tree);
#endif /* GCC_TREE_VECTORIZER_H */
diff --git a/gcc/tree.h b/gcc/tree.h
index 6f6ad5a3a5f4dd4173482dfe259acf539ba24000..24b5184122550fe21ab0a5387867b6c65c20bb03 100644
--- a/gcc/tree.h
+++ b/gcc/tree.h
@@ -93,6 +93,8 @@ public:
bool is_internal_fn () const;
bool is_builtin_fn () const;
int get_rep () const { return rep; }
+ enum tree_code safe_as_tree_code () const;
+ combined_fn safe_as_fn_code () const;
bool operator== (const code_helper &other) { return rep == other.rep; }
bool operator!= (const code_helper &other) { return rep != other.rep; }
bool operator== (tree_code c) { return rep == code_helper (c).rep; }
@@ -102,6 +104,17 @@ private:
int rep;
};
+inline enum tree_code
+code_helper::safe_as_tree_code () const
+{
+ return is_tree_code () ? (tree_code)* this : MAX_TREE_CODES;
+}
+
+inline combined_fn
+code_helper::safe_as_fn_code () const {
+ return is_fn_code () ? (combined_fn) *this : CFN_LAST;
+}
+
inline code_helper::operator internal_fn () const
{
return as_internal_fn (combined_fn (*this));
--
2.17.1
[-- Attachment #3: 0002-Refactor-widen_plus-as-internal_fn.patch --]
[-- Type: application/octet-stream, Size: 22832 bytes --]
From 1e8afa697157c3cb520a36304326e14891444226 Mon Sep 17 00:00:00 2001
From: Joel Hutton <joel.hutton@arm.com>
Date: Wed, 26 Jan 2022 14:00:17 +0000
Subject: [PATCH 2/3] Refactor widen_plus as internal_fn
This patch replaces the existing tree_code widen_plus and widen_minus
patterns with internal_fn versions.
DEF_INTERNAL_OPTAB_MULTI_FN is like DEF_INTERNAL_OPTAB_FN except it provides convenience wrappers for defining conversions that require a hi/lo split, like widening and narrowing operations. Each definition for <NAME> will require an optab named <OPTAB> and two other optabs that you specify for signed and unsigned. The hi/lo pair is necessary because the widening operations take n narrow elements as inputs and return n/2 wide elements as outputs. The 'lo' operation operates on the first n/2 elements of input. The 'hi' operation operates on the second n/2 elements of input. Defining an internal_fn along with hi/lo variations allows a single internal function to be returned from a vect_recog function that will later be expanded to hi/lo.
DEF_INTERNAL_OPTAB_MULTI_FN is used in internal-fn.def to register a widening internal_fn. It is defined differently in different places and internal-fn.def is sourced from those places so the parameters given can be reused.
internal-fn.c: defined to expand to hi/lo signed/unsigned optabs, later defined to generate the 'expand_' functions for the hi/lo versions of the fn.
internal-fn.def: defined to invoke DEF_INTERNAL_OPTAB_FN for the original and hi/lo variants of the internal_fn
For example:
IFN_VEC_WIDEN_PLUS -> IFN_VEC_WIDEN_PLUS_HI, IFN_VEC_WIDEN_PLUS_LO
for aarch64: IFN_VEC_WIDEN_PLUS_HI -> vec_widen_<su>addl_hi_<mode> -> (u/s)addl2
IFN_VEC_WIDEN_PLUS_LO -> vec_widen_<su>addl_lo_<mode> -> (u/s)addl
This gives the same functionality as the previous WIDEN_PLUS/WIDEN_MINUS tree codes which are expanded into VEC_WIDEN_PLUS_LO, VEC_WIDEN_PLUS_HI.
gcc/ChangeLog:
2022-04-13 Joel Hutton <joel.hutton@arm.com>
2022-04-13 Tamar Christina <tamar.christina@arm.com>
* internal-fn.cc (INCLUDE_MAP): Include maps for use in optab
lookup.
(DEF_INTERNAL_OPTAB_MULTI_FN): Macro to define an internal_fn that
expands into multiple internal_fns (for widening).
(ifn_cmp): Function to compare ifn's for sorting/searching.
(lookup_multi_ifn_optab): Add lookup function.
(lookup_multi_internal_fn): Add lookup function.
(commutative_binary_fn_p): Add widen_plus fn's.
* internal-fn.def (DEF_INTERNAL_OPTAB_MULTI_FN): Define widening
plus,minus functions.
(VEC_WIDEN_PLUS): Replacement for VEC_WIDEN_PLUS tree code.
(VEC_WIDEN_MINUS): Replacement for VEC_WIDEN_MINUS tree code.
* internal-fn.h (GCC_INTERNAL_FN_H): Add headers.
(lookup_multi_ifn_optab): Add prototype.
(lookup_multi_internal_fn): Add prototype.
* optabs.cc (commutative_optab_p): Add widening plus, minus optabs.
* optabs.def (OPTAB_CD): widen add, sub optabs
* tree-core.h (ECF_MULTI): Flag to indicate if a function decays
into hi/lo parts.
* tree-vect-patterns.cc (vect_recog_widen_op_pattern): Support
patterns with a hi/lo split.
(vect_recog_widen_plus_pattern): Refactor to return
IFN_VECT_WIDEN_PLUS.
(vect_recog_widen_minus_pattern): Refactor to return new
IFN_VEC_WIDEN_MINUS.
* tree-vect-stmts.cc (vectorizable_conversion): Add widen plus/minus
ifn
support.
(supportable_widening_operation): Add widen plus/minus ifn support.
gcc/testsuite/ChangeLog:
* gcc.target/aarch64/vect-widen-add.c: Test that new
IFN_VEC_WIDEN_PLUS is being used.
* gcc.target/aarch64/vect-widen-sub.c: Test that new
IFN_VEC_WIDEN_MINUS is being used.
---
gcc/internal-fn.cc | 107 ++++++++++++++++++
gcc/internal-fn.def | 23 ++++
gcc/internal-fn.h | 7 ++
gcc/optabs.cc | 12 +-
gcc/optabs.def | 2 +
.../gcc.target/aarch64/vect-widen-add.c | 4 +-
.../gcc.target/aarch64/vect-widen-sub.c | 4 +-
gcc/tree-core.h | 4 +
gcc/tree-vect-patterns.cc | 37 ++++--
gcc/tree-vect-stmts.cc | 68 +++++++++--
10 files changed, 249 insertions(+), 19 deletions(-)
diff --git a/gcc/internal-fn.cc b/gcc/internal-fn.cc
index 91588f8bc9f7c3fe2bac17f3c4e6078cddb7b4d2..b2cb3e508027a84e4456d676d78b27b6c04b7b61 100644
--- a/gcc/internal-fn.cc
+++ b/gcc/internal-fn.cc
@@ -17,6 +17,7 @@ You should have received a copy of the GNU General Public License
along with GCC; see the file COPYING3. If not see
<http://www.gnu.org/licenses/>. */
+#define INCLUDE_MAP
#include "config.h"
#include "system.h"
#include "coretypes.h"
@@ -70,6 +71,26 @@ const int internal_fn_flags_array[] = {
0
};
+const enum internal_fn internal_fn_hilo_keys_array[] = {
+#undef DEF_INTERNAL_OPTAB_MULTI_FN
+#define DEF_INTERNAL_OPTAB_MULTI_FN(NAME, FLAGS, OPTAB, SOPTAB, UOPTAB, TYPE) \
+ IFN_##NAME##_LO, \
+ IFN_##NAME##_HI,
+#include "internal-fn.def"
+ IFN_LAST
+#undef DEF_INTERNAL_OPTAB_MULTI_FN
+};
+
+const optab internal_fn_hilo_values_array[] = {
+#undef DEF_INTERNAL_OPTAB_MULTI_FN
+#define DEF_INTERNAL_OPTAB_MULTI_FN(NAME, FLAGS, OPTAB, SOPTAB, UOPTAB, TYPE) \
+ SOPTAB##_lo_optab, UOPTAB##_lo_optab, \
+ SOPTAB##_hi_optab, UOPTAB##_hi_optab,
+#include "internal-fn.def"
+ unknown_optab, unknown_optab
+#undef DEF_INTERNAL_OPTAB_MULTI_FN
+};
+
/* Return the internal function called NAME, or IFN_LAST if there's
no such function. */
@@ -90,6 +111,62 @@ lookup_internal_fn (const char *name)
return entry ? *entry : IFN_LAST;
}
+static int
+ifn_cmp (const void *a_, const void *b_)
+{
+ typedef std::pair<enum internal_fn, unsigned> ifn_pair;
+ auto *a = (const std::pair<ifn_pair, optab> *)a_;
+ auto *b = (const std::pair<ifn_pair, optab> *)b_;
+ return (int) (a->first.first) - (b->first.first);
+}
+
+/* Return the optab belonging to the given internal function NAME for the given
+ SIGN or unknown_optab. */
+
+optab
+lookup_multi_ifn_optab (enum internal_fn fn, unsigned sign)
+{
+ typedef std::pair<enum internal_fn, unsigned> ifn_pair;
+ typedef auto_vec <std::pair<ifn_pair, optab>>fn_to_optab_map_type;
+ static fn_to_optab_map_type *fn_to_optab_map;
+
+ if (!fn_to_optab_map)
+ {
+ unsigned num
+ = sizeof (internal_fn_hilo_keys_array) / sizeof (enum internal_fn);
+ fn_to_optab_map = new fn_to_optab_map_type ();
+ for (unsigned int i = 0; i < num - 1; ++i)
+ {
+ enum internal_fn fn = internal_fn_hilo_keys_array[i];
+ optab v1 = internal_fn_hilo_values_array[2*i];
+ optab v2 = internal_fn_hilo_values_array[2*i + 1];
+ ifn_pair key1 (fn, 0);
+ fn_to_optab_map->safe_push ({key1, v1});
+ ifn_pair key2 (fn, 1);
+ fn_to_optab_map->safe_push ({key2, v2});
+ }
+ fn_to_optab_map->qsort (ifn_cmp);
+ }
+
+ ifn_pair new_pair (fn, sign ? 1 : 0);
+ optab tmp;
+ std::pair<ifn_pair,optab> pair_wrap (new_pair, tmp);
+ auto entry = fn_to_optab_map->bsearch (&pair_wrap, ifn_cmp);
+ return entry != fn_to_optab_map->end () ? entry->second : unknown_optab;
+}
+
+extern void
+lookup_multi_internal_fn (enum internal_fn ifn, enum internal_fn *lo,
+ enum internal_fn *hi)
+{
+ int ecf_flags = internal_fn_flags (ifn);
+ gcc_assert (ecf_flags & ECF_MULTI);
+
+ *lo = internal_fn (ifn + 1);
+ *hi = internal_fn (ifn + 2);
+}
+
+
/* Fnspec of each internal function, indexed by function number. */
const_tree internal_fn_fnspec_array[IFN_LAST + 1];
@@ -3928,6 +4005,9 @@ commutative_binary_fn_p (internal_fn fn)
case IFN_UBSAN_CHECK_MUL:
case IFN_ADD_OVERFLOW:
case IFN_MUL_OVERFLOW:
+ case IFN_VEC_WIDEN_PLUS:
+ case IFN_VEC_WIDEN_PLUS_LO:
+ case IFN_VEC_WIDEN_PLUS_HI:
return true;
default:
@@ -4013,6 +4093,32 @@ set_edom_supported_p (void)
#endif
}
+#undef DEF_INTERNAL_OPTAB_MULTI_FN
+#define DEF_INTERNAL_OPTAB_MULTI_FN(CODE, FLAGS, OPTAB, SOPTAB, UOPTAB, TYPE) \
+ static void \
+ expand_##CODE (internal_fn, gcall *) \
+ { \
+ gcc_unreachable (); \
+ } \
+ static void \
+ expand_##CODE##_LO (internal_fn fn, gcall *stmt) \
+ { \
+ tree ty = TREE_TYPE (gimple_get_lhs (stmt)); \
+ if (!TYPE_UNSIGNED (ty)) \
+ expand_##TYPE##_optab_fn (fn, stmt, SOPTAB##_lo##_optab); \
+ else \
+ expand_##TYPE##_optab_fn (fn, stmt, UOPTAB##_lo##_optab); \
+ } \
+ static void \
+ expand_##CODE##_HI (internal_fn fn, gcall *stmt) \
+ { \
+ tree ty = TREE_TYPE (gimple_get_lhs (stmt)); \
+ if (!TYPE_UNSIGNED (ty)) \
+ expand_##TYPE##_optab_fn (fn, stmt, SOPTAB##_hi##_optab); \
+ else \
+ expand_##TYPE##_optab_fn (fn, stmt, UOPTAB##_hi##_optab); \
+ }
+
#define DEF_INTERNAL_OPTAB_FN(CODE, FLAGS, OPTAB, TYPE) \
static void \
expand_##CODE (internal_fn fn, gcall *stmt) \
@@ -4029,6 +4135,7 @@ set_edom_supported_p (void)
expand_##TYPE##_optab_fn (fn, stmt, which_optab); \
}
#include "internal-fn.def"
+#undef DEF_INTERNAL_OPTAB_MULTI_FN
/* Routines to expand each internal function, indexed by function number.
Each routine has the prototype:
diff --git a/gcc/internal-fn.def b/gcc/internal-fn.def
index d2d550d358606022b1cb44fa842f06e0be507bc3..4635a9c8af9ad27bb05d7510388d0fe2270428e5 100644
--- a/gcc/internal-fn.def
+++ b/gcc/internal-fn.def
@@ -82,6 +82,13 @@ along with GCC; see the file COPYING3. If not see
says that the function extends the C-level BUILT_IN_<NAME>{,L,LL,IMAX}
group of functions to any integral mode (including vector modes).
+ DEF_INTERNAL_OPTAB_MULTI_FN is like DEF_INTERNAL_OPTAB_FN except it
+ provides convenience wrappers for defining conversions that require a
+ hi/lo split, like widening and narrowing operations. Each definition
+ for <NAME> will require an optab named <OPTAB> and two other optabs that
+ you specify for signed and unsigned.
+
+
Each entry must have a corresponding expander of the form:
void expand_NAME (gimple_call stmt)
@@ -120,6 +127,14 @@ along with GCC; see the file COPYING3. If not see
DEF_INTERNAL_OPTAB_FN (NAME, FLAGS, OPTAB, TYPE)
#endif
+#ifndef DEF_INTERNAL_OPTAB_MULTI_FN
+#define DEF_INTERNAL_OPTAB_MULTI_FN(NAME, FLAGS, OPTAB, SOPTAB, UOPTAB, TYPE) \
+ DEF_INTERNAL_OPTAB_FN (NAME, FLAGS | ECF_MULTI, OPTAB, TYPE) \
+ DEF_INTERNAL_OPTAB_FN (NAME ## _LO, FLAGS, unknown, TYPE) \
+ DEF_INTERNAL_OPTAB_FN (NAME ## _HI, FLAGS, unknown, TYPE)
+#endif
+
+
DEF_INTERNAL_OPTAB_FN (MASK_LOAD, ECF_PURE, maskload, mask_load)
DEF_INTERNAL_OPTAB_FN (LOAD_LANES, ECF_CONST, vec_load_lanes, load_lanes)
DEF_INTERNAL_OPTAB_FN (MASK_LOAD_LANES, ECF_PURE,
@@ -292,6 +307,14 @@ DEF_INTERNAL_OPTAB_FN (COMPLEX_ADD_ROT270, ECF_CONST, cadd270, binary)
DEF_INTERNAL_OPTAB_FN (COMPLEX_MUL, ECF_CONST, cmul, binary)
DEF_INTERNAL_OPTAB_FN (COMPLEX_MUL_CONJ, ECF_CONST, cmul_conj, binary)
DEF_INTERNAL_OPTAB_FN (VEC_ADDSUB, ECF_CONST, vec_addsub, binary)
+DEF_INTERNAL_OPTAB_MULTI_FN (VEC_WIDEN_PLUS,
+ ECF_CONST | ECF_WIDEN | ECF_NOTHROW,
+ vec_widen_add, vec_widen_saddl, vec_widen_uaddl,
+ binary)
+DEF_INTERNAL_OPTAB_MULTI_FN (VEC_WIDEN_MINUS,
+ ECF_CONST | ECF_WIDEN | ECF_NOTHROW,
+ vec_widen_sub, vec_widen_ssubl, vec_widen_usubl,
+ binary)
DEF_INTERNAL_OPTAB_FN (VEC_FMADDSUB, ECF_CONST, vec_fmaddsub, ternary)
DEF_INTERNAL_OPTAB_FN (VEC_FMSUBADD, ECF_CONST, vec_fmsubadd, ternary)
diff --git a/gcc/internal-fn.h b/gcc/internal-fn.h
index 23c014a963c4d72da92c763db87ee486a2adb485..b35de19747d251d19dc13de1e0323368bd2ebdf2 100644
--- a/gcc/internal-fn.h
+++ b/gcc/internal-fn.h
@@ -20,6 +20,10 @@ along with GCC; see the file COPYING3. If not see
#ifndef GCC_INTERNAL_FN_H
#define GCC_INTERNAL_FN_H
+#include "insn-codes.h"
+#include "insn-opinit.h"
+
+
/* INTEGER_CST values for IFN_UNIQUE function arg-0.
UNSPEC: Undifferentiated UNIQUE.
@@ -112,6 +116,9 @@ internal_fn_name (enum internal_fn fn)
}
extern internal_fn lookup_internal_fn (const char *);
+extern optab lookup_multi_ifn_optab (enum internal_fn, unsigned);
+extern void lookup_multi_internal_fn (enum internal_fn, enum internal_fn *,
+ enum internal_fn *);
/* Return the ECF_* flags for function FN. */
diff --git a/gcc/optabs.cc b/gcc/optabs.cc
index a50dd798f2a454ac54e247f3e6cbab17577ea304..f9be369a6c5b99de5bbad664a11364d1c2cc4b95 100644
--- a/gcc/optabs.cc
+++ b/gcc/optabs.cc
@@ -1312,7 +1312,17 @@ commutative_optab_p (optab binoptab)
|| binoptab == smul_widen_optab
|| binoptab == umul_widen_optab
|| binoptab == smul_highpart_optab
- || binoptab == umul_highpart_optab);
+ || binoptab == umul_highpart_optab
+ || binoptab == vec_widen_add_optab
+ || binoptab == vec_widen_sub_optab
+ || binoptab == vec_widen_saddl_hi_optab
+ || binoptab == vec_widen_saddl_lo_optab
+ || binoptab == vec_widen_ssubl_hi_optab
+ || binoptab == vec_widen_ssubl_lo_optab
+ || binoptab == vec_widen_uaddl_hi_optab
+ || binoptab == vec_widen_uaddl_lo_optab
+ || binoptab == vec_widen_usubl_hi_optab
+ || binoptab == vec_widen_usubl_lo_optab);
}
/* X is to be used in mode MODE as operand OPN to BINOPTAB. If we're
diff --git a/gcc/optabs.def b/gcc/optabs.def
index 801310ebaa7d469520809bb7efed6820f8eb866b..a7881dcb49e4ef07d8f07aa31214eb3a7a944e2e 100644
--- a/gcc/optabs.def
+++ b/gcc/optabs.def
@@ -78,6 +78,8 @@ OPTAB_CD(smsub_widen_optab, "msub$b$a4")
OPTAB_CD(umsub_widen_optab, "umsub$b$a4")
OPTAB_CD(ssmsub_widen_optab, "ssmsub$b$a4")
OPTAB_CD(usmsub_widen_optab, "usmsub$a$b4")
+OPTAB_CD(vec_widen_add_optab, "add$a$b3")
+OPTAB_CD(vec_widen_sub_optab, "sub$a$b3")
OPTAB_CD(vec_load_lanes_optab, "vec_load_lanes$a$b")
OPTAB_CD(vec_store_lanes_optab, "vec_store_lanes$a$b")
OPTAB_CD(vec_mask_load_lanes_optab, "vec_mask_load_lanes$a$b")
diff --git a/gcc/testsuite/gcc.target/aarch64/vect-widen-add.c b/gcc/testsuite/gcc.target/aarch64/vect-widen-add.c
index 220bd9352a4c7acd2e3713e441d74898d3e92b30..7037673d32bd780e1c9b58a51e58e2bac3b30b7e 100644
--- a/gcc/testsuite/gcc.target/aarch64/vect-widen-add.c
+++ b/gcc/testsuite/gcc.target/aarch64/vect-widen-add.c
@@ -1,5 +1,5 @@
/* { dg-do run } */
-/* { dg-options "-O3 -save-temps" } */
+/* { dg-options "-O3 -save-temps -fdump-tree-vect-all" } */
#include <stdint.h>
#include <string.h>
@@ -86,6 +86,8 @@ main()
return 0;
}
+/* { dg-final { scan-tree-dump "add new stmt.*VEC_WIDEN_PLUS_LO" "vect" } } */
+/* { dg-final { scan-tree-dump "add new stmt.*VEC_WIDEN_PLUS_HI" "vect" } } */
/* { dg-final { scan-assembler-times {\tuaddl\t} 1} } */
/* { dg-final { scan-assembler-times {\tuaddl2\t} 1} } */
/* { dg-final { scan-assembler-times {\tsaddl\t} 1} } */
diff --git a/gcc/testsuite/gcc.target/aarch64/vect-widen-sub.c b/gcc/testsuite/gcc.target/aarch64/vect-widen-sub.c
index a2bed63affbd091977df95a126da1f5b8c1d41d2..83bc1edb6105f47114b665e24a13e6194b2179a2 100644
--- a/gcc/testsuite/gcc.target/aarch64/vect-widen-sub.c
+++ b/gcc/testsuite/gcc.target/aarch64/vect-widen-sub.c
@@ -1,5 +1,5 @@
/* { dg-do run } */
-/* { dg-options "-O3 -save-temps" } */
+/* { dg-options "-O3 -save-temps -fdump-tree-vect-all" } */
#include <stdint.h>
#include <string.h>
@@ -86,6 +86,8 @@ main()
return 0;
}
+/* { dg-final { scan-tree-dump "add new stmt.*VEC_WIDEN_MINUS_LO" "vect" } } */
+/* { dg-final { scan-tree-dump "add new stmt.*VEC_WIDEN_MINUS_HI" "vect" } } */
/* { dg-final { scan-assembler-times {\tusubl\t} 1} } */
/* { dg-final { scan-assembler-times {\tusubl2\t} 1} } */
/* { dg-final { scan-assembler-times {\tssubl\t} 1} } */
diff --git a/gcc/tree-core.h b/gcc/tree-core.h
index cff6211080bced0bffb39e98039a6550897acf77..d0c8b812cfb9c3ac83bf25fff0431b08cb7d823d 100644
--- a/gcc/tree-core.h
+++ b/gcc/tree-core.h
@@ -99,6 +99,10 @@ struct die_struct;
/* Nonzero if this is a widening function. */
#define ECF_WIDEN (1 << 16)
+/* Nonzero if this is a function that decomposes into a lo/hi operation. */
+#define ECF_MULTI (1 << 17)
+
+
/* Call argument flags. */
/* Nonzero if the argument is not used by the function. */
diff --git a/gcc/tree-vect-patterns.cc b/gcc/tree-vect-patterns.cc
index d892158f024fc045b897aebe76f2e2b66211cf83..62ca28d725ed4ac8d7e4d493119e40772a0fbac6 100644
--- a/gcc/tree-vect-patterns.cc
+++ b/gcc/tree-vect-patterns.cc
@@ -1351,14 +1351,16 @@ static gimple *
vect_recog_widen_op_pattern (vec_info *vinfo,
stmt_vec_info last_stmt_info, tree *type_out,
tree_code orig_code, code_helper wide_code,
- bool shift_p, const char *name)
+ bool shift_p, const char *name,
+ enum optab_subtype *subtype = NULL)
{
gimple *last_stmt = last_stmt_info->stmt;
vect_unpromoted_value unprom[2];
tree half_type;
if (!vect_widened_op_tree (vinfo, last_stmt_info, orig_code, orig_code,
- shift_p, 2, unprom, &half_type))
+ shift_p, 2, unprom, &half_type, subtype))
+
return NULL;
/* Pattern detected. */
@@ -1424,6 +1426,20 @@ vect_recog_widen_op_pattern (vec_info *vinfo,
type, pattern_stmt, vecctype);
}
+static gimple *
+vect_recog_widen_op_pattern (vec_info *vinfo,
+ stmt_vec_info last_stmt_info, tree *type_out,
+ tree_code orig_code, internal_fn wide_ifn,
+ bool shift_p, const char *name,
+ enum optab_subtype *subtype = NULL)
+{
+ combined_fn ifn = as_combined_fn (wide_ifn);
+ return vect_recog_widen_op_pattern (vinfo, last_stmt_info, type_out,
+ orig_code, ifn, shift_p, name,
+ subtype);
+}
+
+
/* Try to detect multiplication on widened inputs, converting MULT_EXPR
to WIDEN_MULT_EXPR. See vect_recog_widen_op_pattern for details. */
@@ -1437,26 +1453,30 @@ vect_recog_widen_mult_pattern (vec_info *vinfo, stmt_vec_info last_stmt_info,
}
/* Try to detect addition on widened inputs, converting PLUS_EXPR
- to WIDEN_PLUS_EXPR. See vect_recog_widen_op_pattern for details. */
+ to IFN_VEC_WIDEN_PLUS. See vect_recog_widen_op_pattern for details. */
static gimple *
vect_recog_widen_plus_pattern (vec_info *vinfo, stmt_vec_info last_stmt_info,
tree *type_out)
{
+ enum optab_subtype subtype;
return vect_recog_widen_op_pattern (vinfo, last_stmt_info, type_out,
- PLUS_EXPR, WIDEN_PLUS_EXPR, false,
- "vect_recog_widen_plus_pattern");
+ PLUS_EXPR, IFN_VEC_WIDEN_PLUS,
+ false, "vect_recog_widen_plus_pattern",
+ &subtype);
}
/* Try to detect subtraction on widened inputs, converting MINUS_EXPR
- to WIDEN_MINUS_EXPR. See vect_recog_widen_op_pattern for details. */
+ to IFN_VEC_WIDEN_MINUS. See vect_recog_widen_op_pattern for details. */
static gimple *
vect_recog_widen_minus_pattern (vec_info *vinfo, stmt_vec_info last_stmt_info,
tree *type_out)
{
+ enum optab_subtype subtype;
return vect_recog_widen_op_pattern (vinfo, last_stmt_info, type_out,
- MINUS_EXPR, WIDEN_MINUS_EXPR, false,
- "vect_recog_widen_minus_pattern");
+ MINUS_EXPR, IFN_VEC_WIDEN_MINUS,
+ false, "vect_recog_widen_minus_pattern",
+ &subtype);
}
/* Function vect_recog_popcount_pattern
@@ -5629,6 +5649,7 @@ static vect_recog_func vect_vect_recog_func_ptrs[] = {
{ vect_recog_mask_conversion_pattern, "mask_conversion" },
{ vect_recog_widen_plus_pattern, "widen_plus" },
{ vect_recog_widen_minus_pattern, "widen_minus" },
+ /* These must come after the double widening ones. */
};
const unsigned int NUM_PATTERNS = ARRAY_SIZE (vect_vect_recog_func_ptrs);
diff --git a/gcc/tree-vect-stmts.cc b/gcc/tree-vect-stmts.cc
index d6aabb873c86ab8ff0bae41c7f6c3bad34d583c5..6fa0669fdfc8630842b3f9f32f4b4a253e79bb92 100644
--- a/gcc/tree-vect-stmts.cc
+++ b/gcc/tree-vect-stmts.cc
@@ -4903,7 +4903,9 @@ vectorizable_conversion (vec_info *vinfo,
bool widen_arith = (code == WIDEN_PLUS_EXPR
|| code == WIDEN_MINUS_EXPR
|| code == WIDEN_MULT_EXPR
- || code == WIDEN_LSHIFT_EXPR);
+ || code == WIDEN_LSHIFT_EXPR
+ || code == IFN_VEC_WIDEN_PLUS
+ || code == IFN_VEC_WIDEN_MINUS);
if (!widen_arith
&& !CONVERT_EXPR_CODE_P (code)
@@ -4953,7 +4955,9 @@ vectorizable_conversion (vec_info *vinfo,
gcc_assert (code == WIDEN_MULT_EXPR
|| code == WIDEN_LSHIFT_EXPR
|| code == WIDEN_PLUS_EXPR
- || code == WIDEN_MINUS_EXPR);
+ || code == WIDEN_MINUS_EXPR
+ || code == IFN_VEC_WIDEN_PLUS
+ || code == IFN_VEC_WIDEN_MINUS);
op1 = is_gimple_assign (stmt) ? gimple_assign_rhs2 (stmt) :
@@ -12130,14 +12134,62 @@ supportable_widening_operation (vec_info *vinfo,
optab1 = vec_unpacks_sbool_lo_optab;
optab2 = vec_unpacks_sbool_hi_optab;
}
- else
- {
- optab1 = optab_for_tree_code (c1.safe_as_tree_code (), vectype,
- optab_default);
- optab2 = optab_for_tree_code (c2.safe_as_tree_code (), vectype,
- optab_default);
+
+ if (code.is_fn_code ())
+ {
+ internal_fn ifn = as_internal_fn (code.safe_as_fn_code ());
+ int ecf_flags = internal_fn_flags (ifn);
+ gcc_assert (ecf_flags & ECF_MULTI);
+
+ switch (code.safe_as_fn_code ())
+ {
+ case CFN_VEC_WIDEN_PLUS:
+ break;
+ case CFN_VEC_WIDEN_MINUS:
+ break;
+ case CFN_LAST:
+ default:
+ return false;
+ }
+
+ internal_fn lo, hi;
+ lookup_multi_internal_fn (ifn, &lo, &hi);
+ *code1 = as_combined_fn (lo);
+ *code2 = as_combined_fn (hi);
+ optab1 = lookup_multi_ifn_optab (lo, !TYPE_UNSIGNED (vectype));
+ optab2 = lookup_multi_ifn_optab (hi, !TYPE_UNSIGNED (vectype));
}
+ if (code.is_tree_code ())
+ {
+ if (code == FIX_TRUNC_EXPR)
+ {
+ /* The signedness is determined from output operand. */
+ optab1 = optab_for_tree_code (c1.safe_as_tree_code (), vectype_out,
+ optab_default);
+ optab2 = optab_for_tree_code (c2.safe_as_tree_code (), vectype_out,
+ optab_default);
+ }
+ else if (CONVERT_EXPR_CODE_P (code.safe_as_tree_code ())
+ && VECTOR_BOOLEAN_TYPE_P (wide_vectype)
+ && VECTOR_BOOLEAN_TYPE_P (vectype)
+ && TYPE_MODE (wide_vectype) == TYPE_MODE (vectype)
+ && SCALAR_INT_MODE_P (TYPE_MODE (vectype)))
+ {
+ /* If the input and result modes are the same, a different optab
+ is needed where we pass in the number of units in vectype. */
+ optab1 = vec_unpacks_sbool_lo_optab;
+ optab2 = vec_unpacks_sbool_hi_optab;
+ }
+ else
+ {
+ optab1 = optab_for_tree_code (c1.safe_as_tree_code (), vectype,
+ optab_default);
+ optab2 = optab_for_tree_code (c2.safe_as_tree_code (), vectype,
+ optab_default);
+ }
+ }
+
if (!optab1 || !optab2)
return false;
--
2.17.1
[-- Attachment #4: 0003-Remove-widen_plus-minus_expr-tree-codes.patch --]
[-- Type: application/octet-stream, Size: 19552 bytes --]
From 60664218e6e59510f02fb64b49a236e9e5b26c9f Mon Sep 17 00:00:00 2001
From: Joel Hutton <joel.hutton@arm.com>
Date: Fri, 28 Jan 2022 12:04:44 +0000
Subject: [PATCH 3/3] Remove widen_plus/minus_expr tree codes
This patch removes the old widen plus/minus tree codes which have been
replaced by internal functions.
gcc/ChangeLog:
* doc/generic.texi: Remove old tree codes.
* expr.cc (expand_expr_real_2): Remove old tree code cases.
* gimple-pretty-print.cc (dump_binary_rhs): Remove old tree code
cases.
* optabs-tree.cc (optab_for_tree_code): Remove old tree code cases.
(supportable_half_widening_operation): Remove old tree code cases.
* tree-cfg.cc (verify_gimple_assign_binary): Remove old tree code
cases.
* tree-inline.cc (estimate_operator_cost): Remove old tree code
cases.
* tree-pretty-print.cc (dump_generic_node): Remove tree code definition.
(op_symbol_code): Remove old tree code
cases.
* tree-vect-data-refs.cc (vect_get_smallest_scalar_type): Remove old tree code
cases.
(vect_analyze_data_ref_accesses): Remove old tree code
cases.
* tree-vect-generic.cc (expand_vector_operations_1): Remove old tree code
cases.
* tree-vect-patterns.cc (vect_widened_op_tree): Refactor ot replace
usage in vect_recog_sad_pattern.
(vect_recog_sad_pattern): Replace tree code widening pattern with
internal function.
(vect_recog_average_pattern): Replace tree code widening pattern
with internal function.
* tree-vect-stmts.cc (vectorizable_conversion): Remove old tree code
cases.
(supportable_widening_operation): Remove old tree code
cases.
* tree.def (WIDEN_PLUS_EXPR): Remove tree code definition.
(WIDEN_MINUS_EXPR): Remove tree code definition.
(VEC_WIDEN_PLUS_HI_EXPR): Remove tree code definition.
(VEC_WIDEN_PLUS_LO_EXPR): Remove tree code definition.
(VEC_WIDEN_MINUS_HI_EXPR): Remove tree code definition.
(VEC_WIDEN_MINUS_LO_EXPR): Remove tree code definition.
---
gcc/doc/generic.texi | 31 -------------------------------
gcc/expr.cc | 6 ------
gcc/gimple-pretty-print.cc | 4 ----
gcc/optabs-tree.cc | 24 ------------------------
gcc/tree-cfg.cc | 6 ------
gcc/tree-inline.cc | 6 ------
gcc/tree-pretty-print.cc | 12 ------------
gcc/tree-vect-data-refs.cc | 8 +++-----
gcc/tree-vect-generic.cc | 4 ----
gcc/tree-vect-patterns.cc | 36 +++++++++++++++++++++++++-----------
gcc/tree-vect-stmts.cc | 18 ++----------------
gcc/tree.def | 6 ------
12 files changed, 30 insertions(+), 131 deletions(-)
diff --git a/gcc/doc/generic.texi b/gcc/doc/generic.texi
index e5f9d1be8ea81f3da002ec3bb925590d331a2551..344045efd419b0cc3a11771acf70d2fd279c48ac 100644
--- a/gcc/doc/generic.texi
+++ b/gcc/doc/generic.texi
@@ -1811,10 +1811,6 @@ a value from @code{enum annot_expr_kind}, the third is an @code{INTEGER_CST}.
@tindex VEC_RSHIFT_EXPR
@tindex VEC_WIDEN_MULT_HI_EXPR
@tindex VEC_WIDEN_MULT_LO_EXPR
-@tindex VEC_WIDEN_PLUS_HI_EXPR
-@tindex VEC_WIDEN_PLUS_LO_EXPR
-@tindex VEC_WIDEN_MINUS_HI_EXPR
-@tindex VEC_WIDEN_MINUS_LO_EXPR
@tindex VEC_UNPACK_HI_EXPR
@tindex VEC_UNPACK_LO_EXPR
@tindex VEC_UNPACK_FLOAT_HI_EXPR
@@ -1861,33 +1857,6 @@ vector of @code{N/2} products. In the case of @code{VEC_WIDEN_MULT_LO_EXPR} the
low @code{N/2} elements of the two vector are multiplied to produce the
vector of @code{N/2} products.
-@item VEC_WIDEN_PLUS_HI_EXPR
-@itemx VEC_WIDEN_PLUS_LO_EXPR
-These nodes represent widening vector addition of the high and low parts of
-the two input vectors, respectively. Their operands are vectors that contain
-the same number of elements (@code{N}) of the same integral type. The result
-is a vector that contains half as many elements, of an integral type whose size
-is twice as wide. In the case of @code{VEC_WIDEN_PLUS_HI_EXPR} the high
-@code{N/2} elements of the two vectors are added to produce the vector of
-@code{N/2} products. In the case of @code{VEC_WIDEN_PLUS_LO_EXPR} the low
-@code{N/2} elements of the two vectors are added to produce the vector of
-@code{N/2} products.
-
-@item VEC_WIDEN_MINUS_HI_EXPR
-@itemx VEC_WIDEN_MINUS_LO_EXPR
-These nodes represent widening vector subtraction of the high and low parts of
-the two input vectors, respectively. Their operands are vectors that contain
-the same number of elements (@code{N}) of the same integral type. The high/low
-elements of the second vector are subtracted from the high/low elements of the
-first. The result is a vector that contains half as many elements, of an
-integral type whose size is twice as wide. In the case of
-@code{VEC_WIDEN_MINUS_HI_EXPR} the high @code{N/2} elements of the second
-vector are subtracted from the high @code{N/2} of the first to produce the
-vector of @code{N/2} products. In the case of
-@code{VEC_WIDEN_MINUS_LO_EXPR} the low @code{N/2} elements of the second
-vector are subtracted from the low @code{N/2} of the first to produce the
-vector of @code{N/2} products.
-
@item VEC_UNPACK_HI_EXPR
@itemx VEC_UNPACK_LO_EXPR
These nodes represent unpacking of the high and low parts of the input vector,
diff --git a/gcc/expr.cc b/gcc/expr.cc
index 5d66c9f21f0ccd2eafb322eb9001f0dc873e35b4..b80385d51ba22172750d94535e04c82f75661255 100644
--- a/gcc/expr.cc
+++ b/gcc/expr.cc
@@ -9386,8 +9386,6 @@ expand_expr_real_2 (sepops ops, rtx target, machine_mode tmode,
target, unsignedp);
return target;
- case WIDEN_PLUS_EXPR:
- case WIDEN_MINUS_EXPR:
case WIDEN_MULT_EXPR:
/* If first operand is constant, swap them.
Thus the following special case checks need only
@@ -10165,10 +10163,6 @@ expand_expr_real_2 (sepops ops, rtx target, machine_mode tmode,
return temp;
}
- case VEC_WIDEN_PLUS_HI_EXPR:
- case VEC_WIDEN_PLUS_LO_EXPR:
- case VEC_WIDEN_MINUS_HI_EXPR:
- case VEC_WIDEN_MINUS_LO_EXPR:
case VEC_WIDEN_MULT_HI_EXPR:
case VEC_WIDEN_MULT_LO_EXPR:
case VEC_WIDEN_MULT_EVEN_EXPR:
diff --git a/gcc/gimple-pretty-print.cc b/gcc/gimple-pretty-print.cc
index ebd87b20a0adc080c4a8f9429e75f49b96e72f9a..2a1a5b7f811ca341e8ee7e85a9701d3a37ff80bf 100644
--- a/gcc/gimple-pretty-print.cc
+++ b/gcc/gimple-pretty-print.cc
@@ -459,10 +459,6 @@ dump_binary_rhs (pretty_printer *buffer, const gassign *gs, int spc,
case VEC_PACK_FLOAT_EXPR:
case VEC_WIDEN_LSHIFT_HI_EXPR:
case VEC_WIDEN_LSHIFT_LO_EXPR:
- case VEC_WIDEN_PLUS_HI_EXPR:
- case VEC_WIDEN_PLUS_LO_EXPR:
- case VEC_WIDEN_MINUS_HI_EXPR:
- case VEC_WIDEN_MINUS_LO_EXPR:
case VEC_SERIES_EXPR:
for (p = get_tree_code_name (code); *p; p++)
pp_character (buffer, TOUPPER (*p));
diff --git a/gcc/optabs-tree.cc b/gcc/optabs-tree.cc
index 8383fe820b080f6d66f83dcf3b77d3c9f869f4bc..2f5f93dc6624f86f6b5618cf6e7aa2b508053a64 100644
--- a/gcc/optabs-tree.cc
+++ b/gcc/optabs-tree.cc
@@ -190,22 +190,6 @@ optab_for_tree_code (enum tree_code code, const_tree type,
return (TYPE_UNSIGNED (type)
? vec_widen_ushiftl_lo_optab : vec_widen_sshiftl_lo_optab);
- case VEC_WIDEN_PLUS_LO_EXPR:
- return (TYPE_UNSIGNED (type)
- ? vec_widen_uaddl_lo_optab : vec_widen_saddl_lo_optab);
-
- case VEC_WIDEN_PLUS_HI_EXPR:
- return (TYPE_UNSIGNED (type)
- ? vec_widen_uaddl_hi_optab : vec_widen_saddl_hi_optab);
-
- case VEC_WIDEN_MINUS_LO_EXPR:
- return (TYPE_UNSIGNED (type)
- ? vec_widen_usubl_lo_optab : vec_widen_ssubl_lo_optab);
-
- case VEC_WIDEN_MINUS_HI_EXPR:
- return (TYPE_UNSIGNED (type)
- ? vec_widen_usubl_hi_optab : vec_widen_ssubl_hi_optab);
-
case VEC_UNPACK_HI_EXPR:
return (TYPE_UNSIGNED (type)
? vec_unpacku_hi_optab : vec_unpacks_hi_optab);
@@ -312,8 +296,6 @@ optab_for_tree_code (enum tree_code code, const_tree type,
'hi'/'lo' pair using codes such as VEC_WIDEN_MINUS_HI/LO.
Supported widening operations:
- WIDEN_MINUS_EXPR
- WIDEN_PLUS_EXPR
WIDEN_MULT_EXPR
WIDEN_LSHIFT_EXPR
@@ -345,12 +327,6 @@ supportable_half_widening_operation (enum tree_code code, tree vectype_out,
case WIDEN_LSHIFT_EXPR:
*code1 = LSHIFT_EXPR;
break;
- case WIDEN_MINUS_EXPR:
- *code1 = MINUS_EXPR;
- break;
- case WIDEN_PLUS_EXPR:
- *code1 = PLUS_EXPR;
- break;
case WIDEN_MULT_EXPR:
*code1 = MULT_EXPR;
break;
diff --git a/gcc/tree-cfg.cc b/gcc/tree-cfg.cc
index bfcb1425f7e2e46e3d525808adda11560041dd68..757c6c73e351c13bc6695699d9f449530546f70f 100644
--- a/gcc/tree-cfg.cc
+++ b/gcc/tree-cfg.cc
@@ -3951,8 +3951,6 @@ verify_gimple_assign_binary (gassign *stmt)
return false;
}
- case WIDEN_PLUS_EXPR:
- case WIDEN_MINUS_EXPR:
case PLUS_EXPR:
case MINUS_EXPR:
{
@@ -4073,10 +4071,6 @@ verify_gimple_assign_binary (gassign *stmt)
return false;
}
- case VEC_WIDEN_MINUS_HI_EXPR:
- case VEC_WIDEN_MINUS_LO_EXPR:
- case VEC_WIDEN_PLUS_HI_EXPR:
- case VEC_WIDEN_PLUS_LO_EXPR:
case VEC_WIDEN_MULT_HI_EXPR:
case VEC_WIDEN_MULT_LO_EXPR:
case VEC_WIDEN_MULT_EVEN_EXPR:
diff --git a/gcc/tree-inline.cc b/gcc/tree-inline.cc
index 043e1d5987a4c4b0159109dafb85a805ca828c1e..c0bebb7f4de36838341ed62389ad0e2b79f03034 100644
--- a/gcc/tree-inline.cc
+++ b/gcc/tree-inline.cc
@@ -4288,8 +4288,6 @@ estimate_operator_cost (enum tree_code code, eni_weights *weights,
case REALIGN_LOAD_EXPR:
- case WIDEN_PLUS_EXPR:
- case WIDEN_MINUS_EXPR:
case WIDEN_SUM_EXPR:
case WIDEN_MULT_EXPR:
case DOT_PROD_EXPR:
@@ -4298,10 +4296,6 @@ estimate_operator_cost (enum tree_code code, eni_weights *weights,
case WIDEN_MULT_MINUS_EXPR:
case WIDEN_LSHIFT_EXPR:
- case VEC_WIDEN_PLUS_HI_EXPR:
- case VEC_WIDEN_PLUS_LO_EXPR:
- case VEC_WIDEN_MINUS_HI_EXPR:
- case VEC_WIDEN_MINUS_LO_EXPR:
case VEC_WIDEN_MULT_HI_EXPR:
case VEC_WIDEN_MULT_LO_EXPR:
case VEC_WIDEN_MULT_EVEN_EXPR:
diff --git a/gcc/tree-pretty-print.cc b/gcc/tree-pretty-print.cc
index bfabe9e76279d7c3383b684ed61cc92228de4500..0ca8802576656f098e60cb77fa4312d1375ff3f0 100644
--- a/gcc/tree-pretty-print.cc
+++ b/gcc/tree-pretty-print.cc
@@ -2846,8 +2846,6 @@ dump_generic_node (pretty_printer *pp, tree node, int spc, dump_flags_t flags,
break;
/* Binary arithmetic and logic expressions. */
- case WIDEN_PLUS_EXPR:
- case WIDEN_MINUS_EXPR:
case WIDEN_SUM_EXPR:
case WIDEN_MULT_EXPR:
case MULT_EXPR:
@@ -3811,10 +3809,6 @@ dump_generic_node (pretty_printer *pp, tree node, int spc, dump_flags_t flags,
case VEC_SERIES_EXPR:
case VEC_WIDEN_MULT_HI_EXPR:
case VEC_WIDEN_MULT_LO_EXPR:
- case VEC_WIDEN_PLUS_HI_EXPR:
- case VEC_WIDEN_PLUS_LO_EXPR:
- case VEC_WIDEN_MINUS_HI_EXPR:
- case VEC_WIDEN_MINUS_LO_EXPR:
case VEC_WIDEN_MULT_EVEN_EXPR:
case VEC_WIDEN_MULT_ODD_EXPR:
case VEC_WIDEN_LSHIFT_HI_EXPR:
@@ -4332,12 +4326,6 @@ op_symbol_code (enum tree_code code)
case WIDEN_LSHIFT_EXPR:
return "w<<";
- case WIDEN_PLUS_EXPR:
- return "w+";
-
- case WIDEN_MINUS_EXPR:
- return "w-";
-
case POINTER_PLUS_EXPR:
return "+";
diff --git a/gcc/tree-vect-data-refs.cc b/gcc/tree-vect-data-refs.cc
index d20a10a1524164eef788ab4b88ba57c7a09c3387..98dd56ff022233ccead36a1f5a5e896e352f9f5b 100644
--- a/gcc/tree-vect-data-refs.cc
+++ b/gcc/tree-vect-data-refs.cc
@@ -136,8 +136,6 @@ vect_get_smallest_scalar_type (stmt_vec_info stmt_info, tree scalar_type)
|| gimple_assign_rhs_code (assign) == WIDEN_SUM_EXPR
|| gimple_assign_rhs_code (assign) == WIDEN_MULT_EXPR
|| gimple_assign_rhs_code (assign) == WIDEN_LSHIFT_EXPR
- || gimple_assign_rhs_code (assign) == WIDEN_PLUS_EXPR
- || gimple_assign_rhs_code (assign) == WIDEN_MINUS_EXPR
|| gimple_assign_rhs_code (assign) == FLOAT_EXPR)
{
tree rhs_type = TREE_TYPE (gimple_assign_rhs1 (assign));
@@ -3172,8 +3170,8 @@ vect_analyze_data_ref_accesses (vec_info *vinfo,
break;
/* Check that the DR_INITs are compile-time constants. */
- if (!tree_fits_shwi_p (DR_INIT (dra))
- || !tree_fits_shwi_p (DR_INIT (drb)))
+ if (TREE_CODE (DR_INIT (dra)) != INTEGER_CST
+ || TREE_CODE (DR_INIT (drb)) != INTEGER_CST)
break;
/* Different .GOMP_SIMD_LANE calls still give the same lane,
@@ -3225,7 +3223,7 @@ vect_analyze_data_ref_accesses (vec_info *vinfo,
unsigned HOST_WIDE_INT step
= absu_hwi (tree_to_shwi (DR_STEP (dra)));
if (step != 0
- && step <= ((unsigned HOST_WIDE_INT)init_b - init_a))
+ && step <= (unsigned HOST_WIDE_INT)(init_b - init_a))
break;
}
}
diff --git a/gcc/tree-vect-generic.cc b/gcc/tree-vect-generic.cc
index 350129555a0c71c0896c4f1003163f3b3557c11b..066f05873118c2288c90604e6287c91ef9aed72b 100644
--- a/gcc/tree-vect-generic.cc
+++ b/gcc/tree-vect-generic.cc
@@ -2209,10 +2209,6 @@ expand_vector_operations_1 (gimple_stmt_iterator *gsi,
arguments, not the widened result. VEC_UNPACK_FLOAT_*_EXPR is
calculated in the same way above. */
if (code == WIDEN_SUM_EXPR
- || code == VEC_WIDEN_PLUS_HI_EXPR
- || code == VEC_WIDEN_PLUS_LO_EXPR
- || code == VEC_WIDEN_MINUS_HI_EXPR
- || code == VEC_WIDEN_MINUS_LO_EXPR
|| code == VEC_WIDEN_MULT_HI_EXPR
|| code == VEC_WIDEN_MULT_LO_EXPR
|| code == VEC_WIDEN_MULT_EVEN_EXPR
diff --git a/gcc/tree-vect-patterns.cc b/gcc/tree-vect-patterns.cc
index 62ca28d725ed4ac8d7e4d493119e40772a0fbac6..9cd3989656c024b1d0394b2fcde6f6d774dff74e 100644
--- a/gcc/tree-vect-patterns.cc
+++ b/gcc/tree-vect-patterns.cc
@@ -559,21 +559,29 @@ vect_joust_widened_type (tree type, tree new_type, tree *common_type)
static unsigned int
vect_widened_op_tree (vec_info *vinfo, stmt_vec_info stmt_info, tree_code code,
- tree_code widened_code, bool shift_p,
+ code_helper widened_code, bool shift_p,
unsigned int max_nops,
vect_unpromoted_value *unprom, tree *common_type,
enum optab_subtype *subtype = NULL)
{
/* Check for an integer operation with the right code. */
- gassign *assign = dyn_cast <gassign *> (stmt_info->stmt);
- if (!assign)
+ gimple* stmt = stmt_info->stmt;
+ if (!(is_gimple_assign (stmt) || is_gimple_call (stmt)))
return 0;
- tree_code rhs_code = gimple_assign_rhs_code (assign);
- if (rhs_code != code && rhs_code != widened_code)
+ code_helper rhs_code;
+ if (is_gimple_assign (stmt))
+ rhs_code = gimple_assign_rhs_code (stmt);
+ else
+ rhs_code = gimple_call_combined_fn (stmt);
+
+ if (rhs_code.safe_as_tree_code () != code
+ && rhs_code.get_rep () != widened_code.get_rep ())
return 0;
- tree type = TREE_TYPE (gimple_assign_lhs (assign));
+ tree lhs = is_gimple_assign (stmt) ? gimple_assign_lhs (stmt):
+ gimple_call_lhs (stmt);
+ tree type = TREE_TYPE (lhs);
if (!INTEGRAL_TYPE_P (type))
return 0;
@@ -586,7 +594,11 @@ vect_widened_op_tree (vec_info *vinfo, stmt_vec_info stmt_info, tree_code code,
{
vect_unpromoted_value *this_unprom = &unprom[next_op];
unsigned int nops = 1;
- tree op = gimple_op (assign, i + 1);
+ tree op;
+ if (is_gimple_assign (stmt))
+ op = gimple_op (stmt, i + 1);
+ else
+ op = gimple_call_arg (stmt, i);
if (i == 1 && TREE_CODE (op) == INTEGER_CST)
{
/* We already have a common type from earlier operands.
@@ -1299,8 +1311,9 @@ vect_recog_sad_pattern (vec_info *vinfo,
/* FORNOW. Can continue analyzing the def-use chain when this stmt in a phi
inside the loop (in case we are analyzing an outer-loop). */
vect_unpromoted_value unprom[2];
- if (!vect_widened_op_tree (vinfo, diff_stmt_vinfo, MINUS_EXPR, WIDEN_MINUS_EXPR,
- false, 2, unprom, &half_type))
+ if (!vect_widened_op_tree (vinfo, diff_stmt_vinfo, MINUS_EXPR,
+ CFN_VEC_WIDEN_MINUS, false, 2, unprom,
+ &half_type))
return NULL;
vect_pattern_detected ("vect_recog_sad_pattern", last_stmt);
@@ -2337,9 +2350,10 @@ vect_recog_average_pattern (vec_info *vinfo,
internal_fn ifn = IFN_AVG_FLOOR;
vect_unpromoted_value unprom[3];
tree new_type;
+ enum optab_subtype subtype;
unsigned int nops = vect_widened_op_tree (vinfo, plus_stmt_info, PLUS_EXPR,
- WIDEN_PLUS_EXPR, false, 3,
- unprom, &new_type);
+ CFN_VEC_WIDEN_PLUS, false, 3,
+ unprom, &new_type, &subtype);
if (nops == 0)
return NULL;
if (nops == 3)
diff --git a/gcc/tree-vect-stmts.cc b/gcc/tree-vect-stmts.cc
index 6fa0669fdfc8630842b3f9f32f4b4a253e79bb92..92b17e6d0ec18ce3d90290dba9efec5d1968264c 100644
--- a/gcc/tree-vect-stmts.cc
+++ b/gcc/tree-vect-stmts.cc
@@ -4900,9 +4900,7 @@ vectorizable_conversion (vec_info *vinfo,
else
return false;
- bool widen_arith = (code == WIDEN_PLUS_EXPR
- || code == WIDEN_MINUS_EXPR
- || code == WIDEN_MULT_EXPR
+ bool widen_arith = (code == WIDEN_MULT_EXPR
|| code == WIDEN_LSHIFT_EXPR
|| code == IFN_VEC_WIDEN_PLUS
|| code == IFN_VEC_WIDEN_MINUS);
@@ -4954,8 +4952,6 @@ vectorizable_conversion (vec_info *vinfo,
{
gcc_assert (code == WIDEN_MULT_EXPR
|| code == WIDEN_LSHIFT_EXPR
- || code == WIDEN_PLUS_EXPR
- || code == WIDEN_MINUS_EXPR
|| code == IFN_VEC_WIDEN_PLUS
|| code == IFN_VEC_WIDEN_MINUS);
@@ -11986,7 +11982,7 @@ supportable_widening_operation (vec_info *vinfo,
class loop *vect_loop = NULL;
machine_mode vec_mode;
enum insn_code icode1, icode2;
- optab optab1, optab2;
+ optab optab1 = unknown_optab, optab2 = unknown_optab;
tree vectype = vectype_in;
tree wide_vectype = vectype_out;
code_helper c1 = MAX_TREE_CODES, c2 = MAX_TREE_CODES;
@@ -12080,16 +12076,6 @@ supportable_widening_operation (vec_info *vinfo,
c2 = VEC_WIDEN_LSHIFT_HI_EXPR;
break;
- case WIDEN_PLUS_EXPR:
- c1 = VEC_WIDEN_PLUS_LO_EXPR;
- c2 = VEC_WIDEN_PLUS_HI_EXPR;
- break;
-
- case WIDEN_MINUS_EXPR:
- c1 = VEC_WIDEN_MINUS_LO_EXPR;
- c2 = VEC_WIDEN_MINUS_HI_EXPR;
- break;
-
CASE_CONVERT:
c1 = VEC_UNPACK_LO_EXPR;
c2 = VEC_UNPACK_HI_EXPR;
diff --git a/gcc/tree.def b/gcc/tree.def
index 62650b6934b337c5d56e5393dc114173d72c9aa9..9b2dce3576440c445d3240b9ed937fe67c9a5992 100644
--- a/gcc/tree.def
+++ b/gcc/tree.def
@@ -1383,8 +1383,6 @@ DEFTREECODE (WIDEN_MULT_MINUS_EXPR, "widen_mult_minus_expr", tcc_expression, 3)
the first argument from type t1 to type t2, and then shifting it
by the second argument. */
DEFTREECODE (WIDEN_LSHIFT_EXPR, "widen_lshift_expr", tcc_binary, 2)
-DEFTREECODE (WIDEN_PLUS_EXPR, "widen_plus_expr", tcc_binary, 2)
-DEFTREECODE (WIDEN_MINUS_EXPR, "widen_minus_expr", tcc_binary, 2)
/* Widening vector multiplication.
The two operands are vectors with N elements of size S. Multiplying the
@@ -1449,10 +1447,6 @@ DEFTREECODE (VEC_PACK_FLOAT_EXPR, "vec_pack_float_expr", tcc_binary, 2)
*/
DEFTREECODE (VEC_WIDEN_LSHIFT_HI_EXPR, "widen_lshift_hi_expr", tcc_binary, 2)
DEFTREECODE (VEC_WIDEN_LSHIFT_LO_EXPR, "widen_lshift_lo_expr", tcc_binary, 2)
-DEFTREECODE (VEC_WIDEN_PLUS_HI_EXPR, "widen_plus_hi_expr", tcc_binary, 2)
-DEFTREECODE (VEC_WIDEN_PLUS_LO_EXPR, "widen_plus_lo_expr", tcc_binary, 2)
-DEFTREECODE (VEC_WIDEN_MINUS_HI_EXPR, "widen_minus_hi_expr", tcc_binary, 2)
-DEFTREECODE (VEC_WIDEN_MINUS_LO_EXPR, "widen_minus_lo_expr", tcc_binary, 2)
/* PREDICT_EXPR. Specify hint for branch prediction. The
PREDICT_EXPR_PREDICTOR specify predictor and PREDICT_EXPR_OUTCOME the
--
2.17.1
next prev parent reply other threads:[~2022-06-30 13:20 UTC|newest]
Thread overview: 54+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-05-25 9:11 Joel Hutton
2022-05-27 13:23 ` Richard Biener
2022-05-31 10:07 ` Joel Hutton
2022-05-31 16:46 ` Tamar Christina
2022-06-01 10:11 ` Richard Biener
2022-06-06 17:20 ` Joel Hutton
2022-06-07 8:18 ` Richard Sandiford
2022-06-07 9:01 ` Joel Hutton
2022-06-09 14:03 ` Joel Hutton
2022-06-13 9:02 ` Richard Biener
2022-06-30 13:20 ` Joel Hutton [this message]
2022-07-12 12:32 ` Richard Biener
2023-03-17 10:14 ` Andre Vieira (lists)
2023-03-17 11:52 ` Richard Biener
2023-04-20 13:23 ` Andre Vieira (lists)
2023-04-24 11:57 ` Richard Biener
2023-04-24 13:01 ` Richard Sandiford
2023-04-25 12:30 ` Richard Biener
2023-04-28 16:06 ` Andre Vieira (lists)
2023-04-25 9:55 ` Andre Vieira (lists)
2023-04-28 12:36 ` [PATCH 1/3] Refactor to allow internal_fn's Andre Vieira (lists)
2023-05-03 11:55 ` Richard Biener
2023-05-04 15:20 ` Andre Vieira (lists)
2023-05-05 6:09 ` Richard Biener
2023-05-12 12:14 ` Andre Vieira (lists)
2023-05-12 13:18 ` Richard Biener
2023-04-28 12:37 ` [PATCH 2/3] Refactor widen_plus as internal_fn Andre Vieira (lists)
2023-05-03 12:11 ` Richard Biener
2023-05-03 19:07 ` Richard Sandiford
2023-05-12 12:16 ` Andre Vieira (lists)
2023-05-12 13:28 ` Richard Biener
2023-05-12 13:55 ` Andre Vieira (lists)
2023-05-12 14:01 ` Richard Sandiford
2023-05-15 10:20 ` Richard Biener
2023-05-15 10:47 ` Richard Sandiford
2023-05-15 11:01 ` Richard Biener
2023-05-15 11:10 ` Richard Sandiford
2023-05-15 11:53 ` Andre Vieira (lists)
2023-05-15 12:21 ` Richard Biener
2023-05-18 17:15 ` Andre Vieira (lists)
2023-05-22 13:06 ` Richard Biener
2023-06-01 16:27 ` Andre Vieira (lists)
2023-06-02 12:00 ` Richard Sandiford
2023-06-06 19:00 ` Jakub Jelinek
2023-06-06 21:28 ` [PATCH] modula2: Fix bootstrap Jakub Jelinek
2023-06-06 22:18 ` Gaius Mulley
2023-06-07 8:42 ` Andre Vieira (lists)
2023-06-13 14:48 ` Jakub Jelinek
2023-04-28 12:37 ` [PATCH 3/3] Remove widen_plus/minus_expr tree codes Andre Vieira (lists)
2023-05-03 12:29 ` Richard Biener
2023-05-10 9:15 ` Andre Vieira (lists)
2023-05-12 12:18 ` Andre Vieira (lists)
2022-06-13 9:18 ` [ping][vect-patterns] Refactor widen_plus/widen_minus as internal_fns Richard Biener
-- strict thread matches above, loose matches on Subject: below --
2021-11-25 10:08 Joel Hutton
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=DB9PR08MB6603D656FD05B9D562033809F5BA9@DB9PR08MB6603.eurprd08.prod.outlook.com \
--to=joel.hutton@arm.com \
--cc=Andre.SimoesDiasVieira@arm.com \
--cc=Richard.Sandiford@arm.com \
--cc=gcc-patches@gcc.gnu.org \
--cc=rguenther@suse.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).