From: Richard Sandiford <richard.sandiford@arm.com>
To: Oluwatamilore Adebayo <oluwatamilore.adebayo@arm.com>
Cc: <gcc-patches@gcc.gnu.org>, <richard.guenther@gmail.com>
Subject: Re: [PATCH 1/2] Missed opportunity to use [SU]ABD
Date: Wed, 14 Jun 2023 12:15:08 +0100 [thread overview]
Message-ID: <mpt5y7qi9c3.fsf@arm.com> (raw)
In-Reply-To: <20230613082643.149991-1-oluwatamilore.adebayo@arm.com> (Oluwatamilore Adebayo's message of "Tue, 13 Jun 2023 09:26:43 +0100")
Oluwatamilore Adebayo <oluwatamilore.adebayo@arm.com> writes:
> From: oluade01 <oluwatamilore.adebayo@arm.com>
>
> This adds a recognition pattern for the non-widening
> absolute difference (ABD).
>
> gcc/ChangeLog:
>
> * doc/md.texi (sabd, uabd): Document them.
> * internal-fn.def (ABD): Use new optab.
> * optabs.def (sabd_optab, uabd_optab): New optabs,
> * tree-vect-patterns.cc (vect_recog_absolute_difference):
> Recognize the following idiom abs (a - b).
> (vect_recog_sad_pattern): Refactor to use
> vect_recog_absolute_difference.
> (vect_recog_abd_pattern): Use patterns found by
> vect_recog_absolute_difference to build a new ABD
> internal call.
> ---
> gcc/doc/md.texi | 10 ++
> gcc/internal-fn.def | 3 +
> gcc/optabs.def | 2 +
> gcc/tree-vect-patterns.cc | 245 +++++++++++++++++++++++++++++++++-----
> 4 files changed, 230 insertions(+), 30 deletions(-)
>
> diff --git a/gcc/doc/md.texi b/gcc/doc/md.texi
> index 6a435eb44610960513e9739ac9ac1e8a27182c10..e11b10d2fca11016232921bc85e47975f700e6c6 100644
> --- a/gcc/doc/md.texi
> +++ b/gcc/doc/md.texi
> @@ -5787,6 +5787,16 @@ Other shift and rotate instructions, analogous to the
> Vector shift and rotate instructions that take vectors as operand 2
> instead of a scalar type.
>
> +@cindex @code{uabd@var{m}} instruction pattern
> +@cindex @code{sabd@var{m}} instruction pattern
> +@item @samp{uabd@var{m}}, @samp{sabd@var{m}}
> +Signed and unsigned absolute difference instructions. These
> +instructions find the difference between operands 1 and 2
> +then return the absolute value. A C code equivalent would be:
> +@smallexample
> +op0 = op1 > op2 ? op1 - op2 : op2 - op1;
> +@end smallexample
> +
> @cindex @code{avg@var{m}3_floor} instruction pattern
> @cindex @code{uavg@var{m}3_floor} instruction pattern
> @item @samp{avg@var{m}3_floor}
> diff --git a/gcc/internal-fn.def b/gcc/internal-fn.def
> index 3ac9d82aace322bd8ef108596e5583daa18c76e3..116965f4830cec8f60642ff011a86b6562e2c509 100644
> --- a/gcc/internal-fn.def
> +++ b/gcc/internal-fn.def
> @@ -191,6 +191,9 @@ DEF_INTERNAL_OPTAB_FN (FMS, ECF_CONST, fms, ternary)
> DEF_INTERNAL_OPTAB_FN (FNMA, ECF_CONST, fnma, ternary)
> DEF_INTERNAL_OPTAB_FN (FNMS, ECF_CONST, fnms, ternary)
>
> +DEF_INTERNAL_SIGNED_OPTAB_FN (ABD, ECF_CONST | ECF_NOTHROW, first,
> + sabd, uabd, binary)
> +
> DEF_INTERNAL_SIGNED_OPTAB_FN (AVG_FLOOR, ECF_CONST | ECF_NOTHROW, first,
> savg_floor, uavg_floor, binary)
> DEF_INTERNAL_SIGNED_OPTAB_FN (AVG_CEIL, ECF_CONST | ECF_NOTHROW, first,
> diff --git a/gcc/optabs.def b/gcc/optabs.def
> index 6c064ff4993620067d38742a0bfe0a3efb511069..35b835a6ac56d72417dac8ddfd77a8a7e2475e65 100644
> --- a/gcc/optabs.def
> +++ b/gcc/optabs.def
> @@ -359,6 +359,8 @@ OPTAB_D (mask_fold_left_plus_optab, "mask_fold_left_plus_$a")
> OPTAB_D (extract_last_optab, "extract_last_$a")
> OPTAB_D (fold_extract_last_optab, "fold_extract_last_$a")
>
> +OPTAB_D (uabd_optab, "uabd$a3")
> +OPTAB_D (sabd_optab, "sabd$a3")
> OPTAB_D (savg_floor_optab, "avg$a3_floor")
> OPTAB_D (uavg_floor_optab, "uavg$a3_floor")
> OPTAB_D (savg_ceil_optab, "avg$a3_ceil")
> diff --git a/gcc/tree-vect-patterns.cc b/gcc/tree-vect-patterns.cc
> index dc102c919352a0328cf86eabceb3a38c41a7e4fd..4b63febc33e90b3caa854404a241afb2f09d755e 100644
> --- a/gcc/tree-vect-patterns.cc
> +++ b/gcc/tree-vect-patterns.cc
> @@ -782,6 +782,86 @@ vect_split_statement (vec_info *vinfo, stmt_vec_info stmt2_info, tree new_rhs,
> }
> }
>
> +/* Look for the following pattern
> + X = x[i]
> + Y = y[i]
> + DIFF = X - Y
> + DAD = ABS_EXPR<DIFF>
> +
> + ABS_STMT should point to a statement of code ABS_EXPR or ABSU_EXPR.
> + HALF_TYPE and UNPROM will be set should the statement be found to
> + be a widened operation.
> + DIFF_STMT will be set to the MINUS_EXPR
> + statement that precedes the ABS_STMT unless vect_widened_op_tree
> + succeeds.
> + */
> +static bool
> +vect_recog_absolute_difference (vec_info *vinfo, gassign *abs_stmt,
> + tree *half_type,
> + vect_unpromoted_value unprom[2],
> + gassign **diff_stmt)
> +{
> + if (!abs_stmt)
> + return false;
> +
> + /* FORNOW. Can continue analyzing the def-use chain when this stmt in a phi
> + inside the loop (in case we are analyzing an outer-loop). */
> + enum tree_code code = gimple_assign_rhs_code (abs_stmt);
> + if (code != ABS_EXPR && code != ABSU_EXPR)
> + return false;
> +
> + tree abs_oprnd = gimple_assign_rhs1 (abs_stmt);
> + tree abs_type = TREE_TYPE (abs_oprnd);
> + if (!abs_oprnd)
> + return false;
> + if (!ANY_INTEGRAL_TYPE_P (abs_type)
> + || TYPE_OVERFLOW_WRAPS (abs_type)
> + || TYPE_UNSIGNED (abs_type))
> + return false;
> +
> + /* Peel off conversions from the ABS input. This can involve sign
> + changes (e.g. from an unsigned subtraction to a signed ABS input)
> + or signed promotion, but it can't include unsigned promotion.
> + (Note that ABS of an unsigned promotion should have been folded
> + away before now anyway.) */
> + vect_unpromoted_value unprom_diff;
> + abs_oprnd = vect_look_through_possible_promotion (vinfo, abs_oprnd,
> + &unprom_diff);
> + if (!abs_oprnd)
> + return false;
> + if (TYPE_PRECISION (unprom_diff.type) != TYPE_PRECISION (abs_type)
> + && TYPE_UNSIGNED (unprom_diff.type))
> + return false;
> +
> + /* We then detect if the operand of abs_expr is defined by a minus_expr. */
> + stmt_vec_info diff_stmt_vinfo = vect_get_internal_def (vinfo, abs_oprnd);
> + if (!diff_stmt_vinfo)
> + return false;
> +
> + /* FORNOW. Can continue analyzing the def-use chain when this stmt in a phi
> + inside the loop (in case we are analyzing an outer-loop). */
> + if (vect_widened_op_tree (vinfo, diff_stmt_vinfo,
> + MINUS_EXPR, IFN_VEC_WIDEN_MINUS,
> + false, 2, unprom, half_type))
> + return true;
> +
> + /* Failed to find a widen operation so we check for a regular MINUS_EXPR. */
> + gassign *diff = dyn_cast <gassign *> (STMT_VINFO_STMT (diff_stmt_vinfo));
> + if (!diff || gimple_assign_rhs_code (diff) != MINUS_EXPR)
> + return false;
> +
> + if (diff_stmt)
> + *diff_stmt = diff;
> +
> + if (TYPE_OVERFLOW_UNDEFINED (TREE_TYPE (abs_oprnd)))
> + {
> + *half_type = NULL_TREE;
> + return true;
> + }
An earlier version had:
+ if (diff_oprnds && diff
+ && gimple_assign_rhs_code (diff) == MINUS_EXPR)
which IMO correctly made this path conditional on whether the caller
was prepared to accept a MINUS_EXPR on unwidened inputs. The code
above instead returns true for such a MINUS_EXPR even if diff_stmt
is null.
So I think this should be:
/* Failed to find a widen operation so we check for a regular MINUS_EXPR. */
gassign *diff = dyn_cast <gassign *> (STMT_VINFO_STMT (diff_stmt_vinfo));
if (diff_stmt
&& diff
&& gimple_assign_rhs_code (diff) == MINUS_EXPR
&& TYPE_OVERFLOW_UNDEFINED (TREE_TYPE (abs_oprnd)))
{
*diff_stmt = diff;
*half_type = NULL_TREE;
return true;
}
With that change, there would be no need for a separate half_type
check...
> +
> + return false;
> +}
> +
> /* Convert UNPROM to TYPE and return the result, adding new statements
> to STMT_INFO's pattern definition statements if no better way is
> available. VECTYPE is the vector form of TYPE.
> @@ -1320,41 +1400,31 @@ vect_recog_sad_pattern (vec_info *vinfo,
> /* FORNOW. Can continue analyzing the def-use chain when this stmt in a phi
> inside the loop (in case we are analyzing an outer-loop). */
> gassign *abs_stmt = dyn_cast <gassign *> (abs_stmt_vinfo->stmt);
> - if (!abs_stmt
> - || (gimple_assign_rhs_code (abs_stmt) != ABS_EXPR
> - && gimple_assign_rhs_code (abs_stmt) != ABSU_EXPR))
> - return NULL;
> + vect_unpromoted_value unprom[2];
>
> - tree abs_oprnd = gimple_assign_rhs1 (abs_stmt);
> - tree abs_type = TREE_TYPE (abs_oprnd);
> - if (TYPE_UNSIGNED (abs_type))
> - return NULL;
> + if (!abs_stmt)
> + {
> + gcall *abd_stmt = dyn_cast <gcall *> (abs_stmt_vinfo->stmt);
> + if (!abd_stmt
> + || !gimple_call_internal_p (abd_stmt)
> + || gimple_call_internal_fn (abd_stmt) != IFN_ABD)
> + return NULL;
>
> - /* Peel off conversions from the ABS input. This can involve sign
> - changes (e.g. from an unsigned subtraction to a signed ABS input)
> - or signed promotion, but it can't include unsigned promotion.
> - (Note that ABS of an unsigned promotion should have been folded
> - away before now anyway.) */
> - vect_unpromoted_value unprom_diff;
> - abs_oprnd = vect_look_through_possible_promotion (vinfo, abs_oprnd,
> - &unprom_diff);
> - if (!abs_oprnd)
> - return NULL;
> - if (TYPE_PRECISION (unprom_diff.type) != TYPE_PRECISION (abs_type)
> - && TYPE_UNSIGNED (unprom_diff.type))
> - return NULL;
> + tree abd_oprnd0 = gimple_call_arg (abd_stmt, 0);
> + tree abd_oprnd1 = gimple_call_arg (abd_stmt, 1);
>
> - /* We then detect if the operand of abs_expr is defined by a minus_expr. */
> - stmt_vec_info diff_stmt_vinfo = vect_get_internal_def (vinfo, abs_oprnd);
> - if (!diff_stmt_vinfo)
> + if (!vect_look_through_possible_promotion (vinfo, abd_oprnd0, &unprom[0])
> + || !vect_look_through_possible_promotion (vinfo, abd_oprnd1,
> + &unprom[1]))
> + return NULL;
> +
> + half_type = unprom[0].type;
> + }
> + else if (!vect_recog_absolute_difference (vinfo, abs_stmt, &half_type,
> + unprom, NULL))
> return NULL;
>
> - /* FORNOW. Can continue analyzing the def-use chain when this stmt in a phi
> - inside the loop (in case we are analyzing an outer-loop). */
> - vect_unpromoted_value unprom[2];
> - if (!vect_widened_op_tree (vinfo, diff_stmt_vinfo, MINUS_EXPR,
> - IFN_VEC_WIDEN_MINUS,
> - false, 2, unprom, &half_type))
> + if (!half_type)
> return NULL;
...here.
>
> vect_pattern_detected ("vect_recog_sad_pattern", last_stmt);
> @@ -1376,6 +1446,120 @@ vect_recog_sad_pattern (vec_info *vinfo,
> return pattern_stmt;
> }
>
> +/* Function vect_recog_abd_pattern
> +
> + Try to find the following ABsolute Difference (ABD) pattern:
> +
> + VTYPE x, y, out;
> + type diff;
> + loop i in range:
> + S1 diff = x[i] - y[i]
> + S2 out[i] = ABS_EXPR <diff>;
> +
> + where 'type' is a integer and 'VTYPE' is a vector of integers
> + the same size as 'type'
> +
> + Input:
> +
> + * STMT_VINFO: The stmt from which the pattern search begins
> +
> + Output:
> +
> + * TYPE_out: The type of the output of this pattern
> +
> + * Return value: A new stmt that will be used to replace the sequence of
> + stmts that constitute the pattern; either SABD or UABD:
> + SABD_EXPR<x, y, out>
> + UABD_EXPR<x, y, out>
> + */
> +
> +static gimple *
> +vect_recog_abd_pattern (vec_info *vinfo,
> + stmt_vec_info stmt_vinfo, tree *type_out)
> +{
> + /* Look for the following patterns
> + X = x[i]
> + Y = y[i]
> + DIFF = X - Y
> + DAD = ABS_EXPR<DIFF>
> + out[i] = DAD
> +
> + In which
> + - X, Y, DIFF, DAD all have the same type
> + - x, y, out are all vectors of the same type
> + */
> +
> + gassign *last_stmt = dyn_cast <gassign *> (STMT_VINFO_STMT (stmt_vinfo));
> + if (!last_stmt)
> + return NULL;
> +
> + tree out_type = TREE_TYPE (gimple_assign_lhs (last_stmt));
> +
> + vect_unpromoted_value unprom[2];
> + gassign *diff_stmt;
> + tree half_type;
> + if (!vect_recog_absolute_difference (vinfo, last_stmt, &half_type,
> + unprom, &diff_stmt))
> + return NULL;
> +
> + tree abd_type = out_type, vectype;
> + tree abd_oprnds[2];
> + bool extend = false;
> + if (half_type)
> + {
> + vectype = get_vectype_for_scalar_type (vinfo, half_type);
> + if (!vectype)
> + return NULL;
> +
> + abd_type = half_type;
> + vect_convert_inputs (vinfo, stmt_vinfo, 2, abd_oprnds,
> + half_type, unprom, vectype);
> +
> + extend = TYPE_PRECISION (abd_type) < TYPE_PRECISION (out_type);
> + }
> + else
> + {
> + unprom[0].op = gimple_assign_rhs1 (diff_stmt);
> + unprom[1].op = gimple_assign_rhs2 (diff_stmt);
> + tree signed_out = signed_type_for (out_type);
> + vectype = get_vectype_for_scalar_type (vinfo, signed_out);
> + if (!vectype)
> + return NULL;
> +
> + vect_convert_inputs (vinfo, stmt_vinfo, 2, abd_oprnds,
> + signed_out, unprom, vectype);
> + }
We should postpone doing the vect_convert_inputs until...
> +
> + vect_pattern_detected ("vect_recog_abd_pattern", last_stmt);
> +
> + if (!vectype
> + || !direct_internal_fn_supported_p (IFN_ABD, vectype,
> + OPTIMIZE_FOR_SPEED))
> + return NULL;
...here. The following should work for both cases:
vect_convert_inputs (vinfo, stmt_vinfo, 2, abd_oprnds,
TREE_TYPE (vectype), unprom, vectype);
LGTM otherwise, thanks.
Richard
> +
> + *type_out = get_vectype_for_scalar_type (vinfo, out_type);
> +
> + tree abd_result = vect_recog_temp_ssa_var (abd_type, NULL);
> + gcall *abd_stmt = gimple_build_call_internal (IFN_ABD, 2,
> + abd_oprnds[0], abd_oprnds[1]);
> + gimple_call_set_lhs (abd_stmt, abd_result);
> + gimple_set_location (abd_stmt, gimple_location (last_stmt));
> +
> + if (!extend)
> + return abd_stmt;
> +
> + gimple *stmt = abd_stmt;
> + if (!TYPE_UNSIGNED (abd_type))
> + {
> + tree unsign = unsigned_type_for (abd_type);
> + tree unsign_vectype = get_vectype_for_scalar_type (vinfo, unsign);
> + stmt = vect_convert_output (vinfo, stmt_vinfo, unsign, stmt,
> + unsign_vectype);
> + }
> +
> + return vect_convert_output (vinfo, stmt_vinfo, out_type, stmt, vectype);
> +}
> +
> /* Recognize an operation that performs ORIG_CODE on widened inputs,
> so that it can be treated as though it had the form:
>
> @@ -6471,6 +6655,7 @@ struct vect_recog_func
> static vect_recog_func vect_vect_recog_func_ptrs[] = {
> { vect_recog_bitfield_ref_pattern, "bitfield_ref" },
> { vect_recog_bit_insert_pattern, "bit_insert" },
> + { vect_recog_abd_pattern, "abd" },
> { vect_recog_over_widening_pattern, "over_widening" },
> /* Must come after over_widening, which narrows the shift as much as
> possible beforehand. */
next prev parent reply other threads:[~2023-06-14 11:15 UTC|newest]
Thread overview: 24+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-05-09 16:07 [PATCH] vect: " Oluwatamilore Adebayo
2023-05-10 9:01 ` Richard Sandiford
2023-05-10 9:49 ` Richard Biener
2023-05-10 9:51 ` Richard Biener
2023-05-10 15:27 ` Richard Sandiford
2023-05-17 12:21 ` oluwatamilore.adebayo
2023-05-18 8:39 ` [PATCH 1/4] " Oluwatamilore.Adebayo
2023-05-18 17:59 ` Richard Sandiford
2023-05-22 13:32 ` Richard Biener
2023-05-23 14:27 ` [PATCH] vect: " Oluwatamilore Adebayo
2023-05-23 14:34 ` [PATCH 1/2] " Oluwatamilore Adebayo
2023-05-24 9:48 ` Richard Sandiford
2023-06-06 9:50 ` [PATCH] vect: " Oluwatamilore Adebayo
2023-06-06 9:53 ` [PATCH 1/2] " Oluwatamilore Adebayo
2023-06-06 12:56 ` Richard Sandiford
2023-06-06 14:34 ` Oluwatamilore Adebayo
2023-06-08 10:28 ` [PATCH] vect: " Oluwatamilore Adebayo
2023-06-08 10:31 ` [PATCH 1/2] " Oluwatamilore Adebayo
2023-06-13 8:26 ` Oluwatamilore Adebayo
2023-06-14 11:15 ` Richard Sandiford [this message]
2023-06-14 15:26 ` Oluwatamilore Adebayo
2023-06-15 6:38 ` Richard Sandiford
2023-05-10 13:29 ` [PATCH] vect: " Oluwatamilore Adebayo
2023-05-15 12:35 ` Oluwatamilore Adebayo
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=mpt5y7qi9c3.fsf@arm.com \
--to=richard.sandiford@arm.com \
--cc=gcc-patches@gcc.gnu.org \
--cc=oluwatamilore.adebayo@arm.com \
--cc=richard.guenther@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).