* [ping][vect-patterns] Refactor widen_plus/widen_minus as internal_fns
@ 2021-11-25 10:08 Joel Hutton
0 siblings, 0 replies; 22+ messages in thread
From: Joel Hutton @ 2021-11-25 10:08 UTC (permalink / raw)
To: Richard Biener; +Cc: gcc-patches, Richard Sandiford
Just a quick ping to check this hasn't been forgotten.
> -----Original Message-----
> From: Joel Hutton
> Sent: 12 November 2021 11:42
> To: Richard Biener <rguenther@suse.de>
> Cc: gcc-patches@gcc.gnu.org; Richard Sandiford
> <Richard.Sandiford@arm.com>
> Subject: RE: [vect-patterns] Refactor widen_plus/widen_minus as
> internal_fns
>
> > please use #define INCLUDE_MAP before the system.h include instead.
> > Is it really necessary to build a new std::map for each optab lookup?!
> > That looks quite ugly and inefficient. We'd usually - if necessary at
> > all - build a auto_vec<std::pair<key,data> > and .sort () and .bsearch () it.
> Ok, I'll rework this part. In the meantime, to address your other comment.
>
> > I'm not sure I understand DEF_INTERNAL_OPTAB_MULTI_FN, neither this
> > cover letter nor the patch ChangeLog explains anything.
>
> I'll attempt to clarify, if this makes things clearer I can include this in the
> commit message of the respun patch:
>
> DEF_INTERNAL_OPTAB_MULTI_FN is like DEF_INTERNAL_OPTAB_FN except it
> provides convenience wrappers for defining conversions that require a hi/lo
> split, like widening and narrowing operations. Each definition for <NAME>
> will require an optab named <OPTAB> and two other optabs that you specify
> for signed and unsigned. The hi/lo pair is necessary because the widening
> operations take n narrow elements as inputs and return n/2 wide elements
> as outputs. The 'lo' operation operates on the first n/2 elements of input.
> The 'hi' operation operates on the second n/2 elements of input. Defining an
> internal_fn along with hi/lo variations allows a single internal function to be
> returned from a vect_recog function that will later be expanded to hi/lo.
>
> DEF_INTERNAL_OPTAB_MULTI_FN is used in internal-fn.def to register a
> widening internal_fn. It is defined differently in different places and internal-
> fn.def is sourced from those places so the parameters given can be reused.
> internal-fn.c: defined to expand to hi/lo signed/unsigned optabs, later
> defined to generate the 'expand_' functions for the hi/lo versions of the fn.
> internal-fn.def: defined to invoke DEF_INTERNAL_OPTAB_FN for the original
> and hi/lo variants of the internal_fn
>
> For example:
> IFN_VEC_WIDEN_PLUS -> IFN_VEC_WIDEN_PLUS_HI,
> IFN_VEC_WIDEN_PLUS_LO
> for aarch64: IFN_VEC_WIDEN_PLUS_HI -> vec_widen_<su>addl_hi_<mode>
> -> (u/s)addl2
> IFN_VEC_WIDEN_PLUS_LO -> vec_widen_<su>addl_lo_<mode>
> -> (u/s)addl
>
> This gives the same functionality as the previous
> WIDEN_PLUS/WIDEN_MINUS tree codes which are expanded into
> VEC_WIDEN_PLUS_LO, VEC_WIDEN_PLUS_HI.
>
> Let me know if I'm not expressing this clearly.
>
> Thanks,
> Joel
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [ping][vect-patterns] Refactor widen_plus/widen_minus as internal_fns
2023-04-25 12:30 ` Richard Biener
@ 2023-04-28 16:06 ` Andre Vieira (lists)
0 siblings, 0 replies; 22+ messages in thread
From: Andre Vieira (lists) @ 2023-04-28 16:06 UTC (permalink / raw)
To: Richard Biener, Richard Sandiford; +Cc: Richard Biener, gcc-patches
On 25/04/2023 13:30, Richard Biener wrote:
> On Mon, 24 Apr 2023, Richard Sandiford wrote:
>
>> Richard Biener <richard.guenther@gmail.com> writes:
>>> On Thu, Apr 20, 2023 at 3:24?PM Andre Vieira (lists) via Gcc-patches
>>> <gcc-patches@gcc.gnu.org> wrote:
>>>>
>>>> Rebased all three patches and made some small changes to the second one:
>>>> - removed sub and abd optabs from commutative_optab_p, I suspect this
>>>> was a copy paste mistake,
>>>> - removed what I believe to be a superfluous switch case in vectorizable
>>>> conversion, the one that was here:
>>>> + if (code.is_fn_code ())
>>>> + {
>>>> + internal_fn ifn = as_internal_fn (code.as_fn_code ());
>>>> + int ecf_flags = internal_fn_flags (ifn);
>>>> + gcc_assert (ecf_flags & ECF_MULTI);
>>>> +
>>>> + switch (code.as_fn_code ())
>>>> + {
>>>> + case CFN_VEC_WIDEN_PLUS:
>>>> + break;
>>>> + case CFN_VEC_WIDEN_MINUS:
>>>> + break;
>>>> + case CFN_LAST:
>>>> + default:
>>>> + return false;
>>>> + }
>>>> +
>>>> + internal_fn lo, hi;
>>>> + lookup_multi_internal_fn (ifn, &lo, &hi);
>>>> + *code1 = as_combined_fn (lo);
>>>> + *code2 = as_combined_fn (hi);
>>>> + optab1 = lookup_multi_ifn_optab (lo, !TYPE_UNSIGNED (vectype));
>>>> + optab2 = lookup_multi_ifn_optab (hi, !TYPE_UNSIGNED (vectype));
>>>> }
>>>>
>>>> I don't think we need to check they are a specfic fn code, as we look-up
>>>> optabs and if they succeed then surely we can vectorize?
>>>>
>>>> OK for trunk?
>>>
>>> In the first patch I see some uses of safe_as_tree_code like
>>>
>>> + if (ch.is_tree_code ())
>>> + return op1 == NULL_TREE ? gimple_build_assign (lhs,
>>> ch.safe_as_tree_code (),
>>> + op0) :
>>> + gimple_build_assign (lhs, ch.safe_as_tree_code (),
>>> + op0, op1);
>>> + else
>>> + {
>>> + internal_fn fn = as_internal_fn (ch.safe_as_fn_code ());
>>> + gimple* stmt;
>>>
>>> where the context actually requires a valid tree code. Please change those
>>> to force to tree code / ifn code. Just use explicit casts here and the other
>>> places that are similar. Before the as_internal_fn just put a
>>> gcc_assert (ch.is_internal_fn ()).
>>
>> Also, doesn't the above ?: simplify to the "else" arm? Null trailing
>> arguments would be ignored for unary operators.
>>
>> I wasn't sure what to make of the op0 handling:
>>
>>> +/* Build a GIMPLE_ASSIGN or GIMPLE_CALL with the tree_code,
>>> + or internal_fn contained in ch, respectively. */
>>> +gimple *
>>> +vect_gimple_build (tree lhs, code_helper ch, tree op0, tree op1)
>>> +{
>>> + if (op0 == NULL_TREE)
>>> + return NULL;
>>
>> Can that happen, and if so, does returning null make sense?
>> Maybe an assert would be safer.
>
> Yeah, I was hoping to have a look whether the new gimple_build
> overloads could be used to make this all better (but hoped we can
> finally get this series in in some way).
>
> Richard.
Yeah, in the newest version of the first patch of the series I found
that most of the time I can get away with only really needing to
distinguish between tree_code and internal_fn when building gimple, for
which it currently uses vect_gimple_build, but it does feel like that
could easily be a gimple function.
Having said that, as I partially mention in the patch, I didn't rewrite
the optabs-tree supportable_half_widening and supportable_conversion (or
whatever they are called) because those also at some point need to
access the stmt and there is a massive difference in how we handle
gassigns and gcall's from that perspective, but maybe we can generalize
that too somehow...
Anyway have a look at the new versions (posted just some minutes after
the email I'm replying too haha! timing :P)
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [ping][vect-patterns] Refactor widen_plus/widen_minus as internal_fns
2023-04-24 13:01 ` Richard Sandiford
@ 2023-04-25 12:30 ` Richard Biener
2023-04-28 16:06 ` Andre Vieira (lists)
0 siblings, 1 reply; 22+ messages in thread
From: Richard Biener @ 2023-04-25 12:30 UTC (permalink / raw)
To: Richard Sandiford; +Cc: Richard Biener, Andre Vieira (lists), gcc-patches
On Mon, 24 Apr 2023, Richard Sandiford wrote:
> Richard Biener <richard.guenther@gmail.com> writes:
> > On Thu, Apr 20, 2023 at 3:24?PM Andre Vieira (lists) via Gcc-patches
> > <gcc-patches@gcc.gnu.org> wrote:
> >>
> >> Rebased all three patches and made some small changes to the second one:
> >> - removed sub and abd optabs from commutative_optab_p, I suspect this
> >> was a copy paste mistake,
> >> - removed what I believe to be a superfluous switch case in vectorizable
> >> conversion, the one that was here:
> >> + if (code.is_fn_code ())
> >> + {
> >> + internal_fn ifn = as_internal_fn (code.as_fn_code ());
> >> + int ecf_flags = internal_fn_flags (ifn);
> >> + gcc_assert (ecf_flags & ECF_MULTI);
> >> +
> >> + switch (code.as_fn_code ())
> >> + {
> >> + case CFN_VEC_WIDEN_PLUS:
> >> + break;
> >> + case CFN_VEC_WIDEN_MINUS:
> >> + break;
> >> + case CFN_LAST:
> >> + default:
> >> + return false;
> >> + }
> >> +
> >> + internal_fn lo, hi;
> >> + lookup_multi_internal_fn (ifn, &lo, &hi);
> >> + *code1 = as_combined_fn (lo);
> >> + *code2 = as_combined_fn (hi);
> >> + optab1 = lookup_multi_ifn_optab (lo, !TYPE_UNSIGNED (vectype));
> >> + optab2 = lookup_multi_ifn_optab (hi, !TYPE_UNSIGNED (vectype));
> >> }
> >>
> >> I don't think we need to check they are a specfic fn code, as we look-up
> >> optabs and if they succeed then surely we can vectorize?
> >>
> >> OK for trunk?
> >
> > In the first patch I see some uses of safe_as_tree_code like
> >
> > + if (ch.is_tree_code ())
> > + return op1 == NULL_TREE ? gimple_build_assign (lhs,
> > ch.safe_as_tree_code (),
> > + op0) :
> > + gimple_build_assign (lhs, ch.safe_as_tree_code (),
> > + op0, op1);
> > + else
> > + {
> > + internal_fn fn = as_internal_fn (ch.safe_as_fn_code ());
> > + gimple* stmt;
> >
> > where the context actually requires a valid tree code. Please change those
> > to force to tree code / ifn code. Just use explicit casts here and the other
> > places that are similar. Before the as_internal_fn just put a
> > gcc_assert (ch.is_internal_fn ()).
>
> Also, doesn't the above ?: simplify to the "else" arm? Null trailing
> arguments would be ignored for unary operators.
>
> I wasn't sure what to make of the op0 handling:
>
> > +/* Build a GIMPLE_ASSIGN or GIMPLE_CALL with the tree_code,
> > + or internal_fn contained in ch, respectively. */
> > +gimple *
> > +vect_gimple_build (tree lhs, code_helper ch, tree op0, tree op1)
> > +{
> > + if (op0 == NULL_TREE)
> > + return NULL;
>
> Can that happen, and if so, does returning null make sense?
> Maybe an assert would be safer.
Yeah, I was hoping to have a look whether the new gimple_build
overloads could be used to make this all better (but hoped we can
finally get this series in in some way).
Richard.
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [ping][vect-patterns] Refactor widen_plus/widen_minus as internal_fns
2023-04-24 11:57 ` Richard Biener
2023-04-24 13:01 ` Richard Sandiford
@ 2023-04-25 9:55 ` Andre Vieira (lists)
1 sibling, 0 replies; 22+ messages in thread
From: Andre Vieira (lists) @ 2023-04-25 9:55 UTC (permalink / raw)
To: Richard Biener; +Cc: Richard Biener, Richard Sandiford, gcc-patches
On 24/04/2023 12:57, Richard Biener wrote:
> On Thu, Apr 20, 2023 at 3:24 PM Andre Vieira (lists) via Gcc-patches
> <gcc-patches@gcc.gnu.org> wrote:
>>
>> Rebased all three patches and made some small changes to the second one:
>> - removed sub and abd optabs from commutative_optab_p, I suspect this
>> was a copy paste mistake,
>> - removed what I believe to be a superfluous switch case in vectorizable
>> conversion, the one that was here:
>> + if (code.is_fn_code ())
>> + {
>> + internal_fn ifn = as_internal_fn (code.as_fn_code ());
>> + int ecf_flags = internal_fn_flags (ifn);
>> + gcc_assert (ecf_flags & ECF_MULTI);
>> +
>> + switch (code.as_fn_code ())
>> + {
>> + case CFN_VEC_WIDEN_PLUS:
>> + break;
>> + case CFN_VEC_WIDEN_MINUS:
>> + break;
>> + case CFN_LAST:
>> + default:
>> + return false;
>> + }
>> +
>> + internal_fn lo, hi;
>> + lookup_multi_internal_fn (ifn, &lo, &hi);
>> + *code1 = as_combined_fn (lo);
>> + *code2 = as_combined_fn (hi);
>> + optab1 = lookup_multi_ifn_optab (lo, !TYPE_UNSIGNED (vectype));
>> + optab2 = lookup_multi_ifn_optab (hi, !TYPE_UNSIGNED (vectype));
>> }
>>
>> I don't think we need to check they are a specfic fn code, as we look-up
>> optabs and if they succeed then surely we can vectorize?
>>
>> OK for trunk?
>
> In the first patch I see some uses of safe_as_tree_code like
>
> + if (ch.is_tree_code ())
> + return op1 == NULL_TREE ? gimple_build_assign (lhs,
> ch.safe_as_tree_code (),
> + op0) :
> + gimple_build_assign (lhs, ch.safe_as_tree_code (),
> + op0, op1);
> + else
> + {
> + internal_fn fn = as_internal_fn (ch.safe_as_fn_code ());
> + gimple* stmt;
>
> where the context actually requires a valid tree code. Please change those
> to force to tree code / ifn code. Just use explicit casts here and the other
> places that are similar. Before the as_internal_fn just put a
> gcc_assert (ch.is_internal_fn ()).
>
> Maybe the need for the (ugly) safe_as_tree_code/fn_code goes away then.
>
> Otherwise patch1 looks OK.
>
> Unfortunately there are no ChangeLog / patch descriptions on the changes.
> patch2 has
>
> - tree_code rhs_code = gimple_assign_rhs_code (assign);
> - if (rhs_code != code && rhs_code != widened_code)
> + code_helper rhs_code;
> + if (is_gimple_assign (stmt))
> + {
> + rhs_code = gimple_assign_rhs_code (stmt);
> + if (rhs_code.safe_as_tree_code () != code
> + && rhs_code.get_rep () != widened_code.get_rep ())
> + return 0;
> + }
> + else if (is_gimple_call (stmt))
> + {
> + rhs_code = gimple_call_combined_fn (stmt);
> + if (rhs_code.get_rep () != widened_code.get_rep ())
> + return 0;
> + }
>
> that looks mightly complicated - esp. the use of get_rep ()
> looks dangerous? What's the intent of this? Not that I
> understand the existing code much. A comment would
> clearly help (also indicating test coverage).
I don't think the use of get_rep here is dangerous, it's meant to avoid
having to check whether widened_code is the same 'kind' as rhs_code.
With get_rep we don't have to do this check first because tree_codes
will have positive reps and combined_fns negative reps. Having said
that, this can all be simplified and we don't need to use get_rep either
as the == operator has been overloaded to use get_rep and even use the
constructor on the rhs of the ==, so I suggest moving the check after
assigning rhs_code and just doing:
if (rhs_code != code
&& rhs_code != widened_code)
return 0;
>
>
>> Kind regards,
>> Andre
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [ping][vect-patterns] Refactor widen_plus/widen_minus as internal_fns
2023-04-24 11:57 ` Richard Biener
@ 2023-04-24 13:01 ` Richard Sandiford
2023-04-25 12:30 ` Richard Biener
2023-04-25 9:55 ` Andre Vieira (lists)
1 sibling, 1 reply; 22+ messages in thread
From: Richard Sandiford @ 2023-04-24 13:01 UTC (permalink / raw)
To: Richard Biener; +Cc: Andre Vieira (lists), Richard Biener, gcc-patches
Richard Biener <richard.guenther@gmail.com> writes:
> On Thu, Apr 20, 2023 at 3:24 PM Andre Vieira (lists) via Gcc-patches
> <gcc-patches@gcc.gnu.org> wrote:
>>
>> Rebased all three patches and made some small changes to the second one:
>> - removed sub and abd optabs from commutative_optab_p, I suspect this
>> was a copy paste mistake,
>> - removed what I believe to be a superfluous switch case in vectorizable
>> conversion, the one that was here:
>> + if (code.is_fn_code ())
>> + {
>> + internal_fn ifn = as_internal_fn (code.as_fn_code ());
>> + int ecf_flags = internal_fn_flags (ifn);
>> + gcc_assert (ecf_flags & ECF_MULTI);
>> +
>> + switch (code.as_fn_code ())
>> + {
>> + case CFN_VEC_WIDEN_PLUS:
>> + break;
>> + case CFN_VEC_WIDEN_MINUS:
>> + break;
>> + case CFN_LAST:
>> + default:
>> + return false;
>> + }
>> +
>> + internal_fn lo, hi;
>> + lookup_multi_internal_fn (ifn, &lo, &hi);
>> + *code1 = as_combined_fn (lo);
>> + *code2 = as_combined_fn (hi);
>> + optab1 = lookup_multi_ifn_optab (lo, !TYPE_UNSIGNED (vectype));
>> + optab2 = lookup_multi_ifn_optab (hi, !TYPE_UNSIGNED (vectype));
>> }
>>
>> I don't think we need to check they are a specfic fn code, as we look-up
>> optabs and if they succeed then surely we can vectorize?
>>
>> OK for trunk?
>
> In the first patch I see some uses of safe_as_tree_code like
>
> + if (ch.is_tree_code ())
> + return op1 == NULL_TREE ? gimple_build_assign (lhs,
> ch.safe_as_tree_code (),
> + op0) :
> + gimple_build_assign (lhs, ch.safe_as_tree_code (),
> + op0, op1);
> + else
> + {
> + internal_fn fn = as_internal_fn (ch.safe_as_fn_code ());
> + gimple* stmt;
>
> where the context actually requires a valid tree code. Please change those
> to force to tree code / ifn code. Just use explicit casts here and the other
> places that are similar. Before the as_internal_fn just put a
> gcc_assert (ch.is_internal_fn ()).
Also, doesn't the above ?: simplify to the "else" arm? Null trailing
arguments would be ignored for unary operators.
I wasn't sure what to make of the op0 handling:
> +/* Build a GIMPLE_ASSIGN or GIMPLE_CALL with the tree_code,
> + or internal_fn contained in ch, respectively. */
> +gimple *
> +vect_gimple_build (tree lhs, code_helper ch, tree op0, tree op1)
> +{
> + if (op0 == NULL_TREE)
> + return NULL;
Can that happen, and if so, does returning null make sense?
Maybe an assert would be safer.
Thanks,
Richard
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [ping][vect-patterns] Refactor widen_plus/widen_minus as internal_fns
2023-04-20 13:23 ` Andre Vieira (lists)
@ 2023-04-24 11:57 ` Richard Biener
2023-04-24 13:01 ` Richard Sandiford
2023-04-25 9:55 ` Andre Vieira (lists)
0 siblings, 2 replies; 22+ messages in thread
From: Richard Biener @ 2023-04-24 11:57 UTC (permalink / raw)
To: Andre Vieira (lists); +Cc: Richard Biener, Richard Sandiford, gcc-patches
On Thu, Apr 20, 2023 at 3:24 PM Andre Vieira (lists) via Gcc-patches
<gcc-patches@gcc.gnu.org> wrote:
>
> Rebased all three patches and made some small changes to the second one:
> - removed sub and abd optabs from commutative_optab_p, I suspect this
> was a copy paste mistake,
> - removed what I believe to be a superfluous switch case in vectorizable
> conversion, the one that was here:
> + if (code.is_fn_code ())
> + {
> + internal_fn ifn = as_internal_fn (code.as_fn_code ());
> + int ecf_flags = internal_fn_flags (ifn);
> + gcc_assert (ecf_flags & ECF_MULTI);
> +
> + switch (code.as_fn_code ())
> + {
> + case CFN_VEC_WIDEN_PLUS:
> + break;
> + case CFN_VEC_WIDEN_MINUS:
> + break;
> + case CFN_LAST:
> + default:
> + return false;
> + }
> +
> + internal_fn lo, hi;
> + lookup_multi_internal_fn (ifn, &lo, &hi);
> + *code1 = as_combined_fn (lo);
> + *code2 = as_combined_fn (hi);
> + optab1 = lookup_multi_ifn_optab (lo, !TYPE_UNSIGNED (vectype));
> + optab2 = lookup_multi_ifn_optab (hi, !TYPE_UNSIGNED (vectype));
> }
>
> I don't think we need to check they are a specfic fn code, as we look-up
> optabs and if they succeed then surely we can vectorize?
>
> OK for trunk?
In the first patch I see some uses of safe_as_tree_code like
+ if (ch.is_tree_code ())
+ return op1 == NULL_TREE ? gimple_build_assign (lhs,
ch.safe_as_tree_code (),
+ op0) :
+ gimple_build_assign (lhs, ch.safe_as_tree_code (),
+ op0, op1);
+ else
+ {
+ internal_fn fn = as_internal_fn (ch.safe_as_fn_code ());
+ gimple* stmt;
where the context actually requires a valid tree code. Please change those
to force to tree code / ifn code. Just use explicit casts here and the other
places that are similar. Before the as_internal_fn just put a
gcc_assert (ch.is_internal_fn ()).
Maybe the need for the (ugly) safe_as_tree_code/fn_code goes away then.
Otherwise patch1 looks OK.
Unfortunately there are no ChangeLog / patch descriptions on the changes.
patch2 has
- tree_code rhs_code = gimple_assign_rhs_code (assign);
- if (rhs_code != code && rhs_code != widened_code)
+ code_helper rhs_code;
+ if (is_gimple_assign (stmt))
+ {
+ rhs_code = gimple_assign_rhs_code (stmt);
+ if (rhs_code.safe_as_tree_code () != code
+ && rhs_code.get_rep () != widened_code.get_rep ())
+ return 0;
+ }
+ else if (is_gimple_call (stmt))
+ {
+ rhs_code = gimple_call_combined_fn (stmt);
+ if (rhs_code.get_rep () != widened_code.get_rep ())
+ return 0;
+ }
that looks mightly complicated - esp. the use of get_rep ()
looks dangerous? What's the intent of this? Not that I
understand the existing code much. A comment would
clearly help (also indicating test coverage).
> Kind regards,
> Andre
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [ping][vect-patterns] Refactor widen_plus/widen_minus as internal_fns
2023-03-17 11:52 ` Richard Biener
@ 2023-04-20 13:23 ` Andre Vieira (lists)
2023-04-24 11:57 ` Richard Biener
0 siblings, 1 reply; 22+ messages in thread
From: Andre Vieira (lists) @ 2023-04-20 13:23 UTC (permalink / raw)
To: Richard Biener; +Cc: Richard Sandiford, gcc-patches
[-- Attachment #1: Type: text/plain, Size: 1105 bytes --]
Rebased all three patches and made some small changes to the second one:
- removed sub and abd optabs from commutative_optab_p, I suspect this
was a copy paste mistake,
- removed what I believe to be a superfluous switch case in vectorizable
conversion, the one that was here:
+ if (code.is_fn_code ())
+ {
+ internal_fn ifn = as_internal_fn (code.as_fn_code ());
+ int ecf_flags = internal_fn_flags (ifn);
+ gcc_assert (ecf_flags & ECF_MULTI);
+
+ switch (code.as_fn_code ())
+ {
+ case CFN_VEC_WIDEN_PLUS:
+ break;
+ case CFN_VEC_WIDEN_MINUS:
+ break;
+ case CFN_LAST:
+ default:
+ return false;
+ }
+
+ internal_fn lo, hi;
+ lookup_multi_internal_fn (ifn, &lo, &hi);
+ *code1 = as_combined_fn (lo);
+ *code2 = as_combined_fn (hi);
+ optab1 = lookup_multi_ifn_optab (lo, !TYPE_UNSIGNED (vectype));
+ optab2 = lookup_multi_ifn_optab (hi, !TYPE_UNSIGNED (vectype));
}
I don't think we need to check they are a specfic fn code, as we look-up
optabs and if they succeed then surely we can vectorize?
OK for trunk?
Kind regards,
Andre
[-- Attachment #2: ifn0.patch --]
[-- Type: text/plain, Size: 21121 bytes --]
diff --git a/gcc/tree-vect-patterns.cc b/gcc/tree-vect-patterns.cc
index 8802141cd6edb298866025b8a55843eae1f0eb17..68dfba266d679c9738a3d5d70551a91cbdafcf66 100644
--- a/gcc/tree-vect-patterns.cc
+++ b/gcc/tree-vect-patterns.cc
@@ -25,6 +25,8 @@ along with GCC; see the file COPYING3. If not see
#include "rtl.h"
#include "tree.h"
#include "gimple.h"
+#include "gimple-iterator.h"
+#include "gimple-fold.h"
#include "ssa.h"
#include "expmed.h"
#include "optabs-tree.h"
@@ -1391,7 +1393,7 @@ vect_recog_sad_pattern (vec_info *vinfo,
static gimple *
vect_recog_widen_op_pattern (vec_info *vinfo,
stmt_vec_info last_stmt_info, tree *type_out,
- tree_code orig_code, tree_code wide_code,
+ tree_code orig_code, code_helper wide_code,
bool shift_p, const char *name)
{
gimple *last_stmt = last_stmt_info->stmt;
@@ -1434,7 +1436,7 @@ vect_recog_widen_op_pattern (vec_info *vinfo,
vecctype = get_vectype_for_scalar_type (vinfo, ctype);
}
- enum tree_code dummy_code;
+ code_helper dummy_code;
int dummy_int;
auto_vec<tree> dummy_vec;
if (!vectype
@@ -1455,8 +1457,7 @@ vect_recog_widen_op_pattern (vec_info *vinfo,
2, oprnd, half_type, unprom, vectype);
tree var = vect_recog_temp_ssa_var (itype, NULL);
- gimple *pattern_stmt = gimple_build_assign (var, wide_code,
- oprnd[0], oprnd[1]);
+ gimple *pattern_stmt = vect_gimple_build (var, wide_code, oprnd[0], oprnd[1]);
if (vecctype != vecitype)
pattern_stmt = vect_convert_output (vinfo, last_stmt_info, ctype,
@@ -6406,3 +6407,28 @@ vect_pattern_recog (vec_info *vinfo)
/* After this no more add_stmt calls are allowed. */
vinfo->stmt_vec_info_ro = true;
}
+
+/* Build a GIMPLE_ASSIGN or GIMPLE_CALL with the tree_code,
+ or internal_fn contained in ch, respectively. */
+gimple *
+vect_gimple_build (tree lhs, code_helper ch, tree op0, tree op1)
+{
+ if (op0 == NULL_TREE)
+ return NULL;
+ if (ch.is_tree_code ())
+ return op1 == NULL_TREE ? gimple_build_assign (lhs, ch.safe_as_tree_code (),
+ op0) :
+ gimple_build_assign (lhs, ch.safe_as_tree_code (),
+ op0, op1);
+ else
+ {
+ internal_fn fn = as_internal_fn (ch.safe_as_fn_code ());
+ gimple* stmt;
+ if (op1 == NULL_TREE)
+ stmt = gimple_build_call_internal (fn, 1, op0);
+ else
+ stmt = gimple_build_call_internal (fn, 2, op0, op1);
+ gimple_call_set_lhs (stmt, lhs);
+ return stmt;
+ }
+}
diff --git a/gcc/tree-vect-stmts.cc b/gcc/tree-vect-stmts.cc
index 6b7dbfd4a231baec24e740ffe0ce0b0bf7a1de6b..715ec2e30a4de620b8a5076c0e7f2f7fd1b0654e 100644
--- a/gcc/tree-vect-stmts.cc
+++ b/gcc/tree-vect-stmts.cc
@@ -4768,7 +4768,7 @@ vectorizable_simd_clone_call (vec_info *vinfo, stmt_vec_info stmt_info,
STMT_INFO is the original scalar stmt that we are vectorizing. */
static gimple *
-vect_gen_widened_results_half (vec_info *vinfo, enum tree_code code,
+vect_gen_widened_results_half (vec_info *vinfo, code_helper ch,
tree vec_oprnd0, tree vec_oprnd1, int op_type,
tree vec_dest, gimple_stmt_iterator *gsi,
stmt_vec_info stmt_info)
@@ -4777,12 +4777,11 @@ vect_gen_widened_results_half (vec_info *vinfo, enum tree_code code,
tree new_temp;
/* Generate half of the widened result: */
- gcc_assert (op_type == TREE_CODE_LENGTH (code));
if (op_type != binary_op)
vec_oprnd1 = NULL;
- new_stmt = gimple_build_assign (vec_dest, code, vec_oprnd0, vec_oprnd1);
+ new_stmt = vect_gimple_build (vec_dest, ch, vec_oprnd0, vec_oprnd1);
new_temp = make_ssa_name (vec_dest, new_stmt);
- gimple_assign_set_lhs (new_stmt, new_temp);
+ gimple_set_lhs (new_stmt, new_temp);
vect_finish_stmt_generation (vinfo, stmt_info, new_stmt, gsi);
return new_stmt;
@@ -4861,8 +4860,8 @@ vect_create_vectorized_promotion_stmts (vec_info *vinfo,
vec<tree> *vec_oprnds1,
stmt_vec_info stmt_info, tree vec_dest,
gimple_stmt_iterator *gsi,
- enum tree_code code1,
- enum tree_code code2, int op_type)
+ code_helper ch1,
+ code_helper ch2, int op_type)
{
int i;
tree vop0, vop1, new_tmp1, new_tmp2;
@@ -4878,10 +4877,10 @@ vect_create_vectorized_promotion_stmts (vec_info *vinfo,
vop1 = NULL_TREE;
/* Generate the two halves of promotion operation. */
- new_stmt1 = vect_gen_widened_results_half (vinfo, code1, vop0, vop1,
+ new_stmt1 = vect_gen_widened_results_half (vinfo, ch1, vop0, vop1,
op_type, vec_dest, gsi,
stmt_info);
- new_stmt2 = vect_gen_widened_results_half (vinfo, code2, vop0, vop1,
+ new_stmt2 = vect_gen_widened_results_half (vinfo, ch2, vop0, vop1,
op_type, vec_dest, gsi,
stmt_info);
if (is_gimple_call (new_stmt1))
@@ -4978,8 +4977,9 @@ vectorizable_conversion (vec_info *vinfo,
tree scalar_dest;
tree op0, op1 = NULL_TREE;
loop_vec_info loop_vinfo = dyn_cast <loop_vec_info> (vinfo);
- enum tree_code code, code1 = ERROR_MARK, code2 = ERROR_MARK;
- enum tree_code codecvt1 = ERROR_MARK, codecvt2 = ERROR_MARK;
+ tree_code tc1;
+ code_helper code, code1, code2;
+ code_helper codecvt1 = ERROR_MARK, codecvt2 = ERROR_MARK;
tree new_temp;
enum vect_def_type dt[2] = {vect_unknown_def_type, vect_unknown_def_type};
int ndts = 2;
@@ -5008,31 +5008,43 @@ vectorizable_conversion (vec_info *vinfo,
&& ! vec_stmt)
return false;
- gassign *stmt = dyn_cast <gassign *> (stmt_info->stmt);
- if (!stmt)
+ gimple* stmt = stmt_info->stmt;
+ if (!(is_gimple_assign (stmt) || is_gimple_call (stmt)))
return false;
- if (TREE_CODE (gimple_assign_lhs (stmt)) != SSA_NAME)
+ if (gimple_get_lhs (stmt) == NULL_TREE
+ || TREE_CODE (gimple_get_lhs (stmt)) != SSA_NAME)
return false;
- code = gimple_assign_rhs_code (stmt);
- if (!CONVERT_EXPR_CODE_P (code)
- && code != FIX_TRUNC_EXPR
- && code != FLOAT_EXPR
- && code != WIDEN_PLUS_EXPR
- && code != WIDEN_MINUS_EXPR
- && code != WIDEN_MULT_EXPR
- && code != WIDEN_LSHIFT_EXPR)
+ if (TREE_CODE (gimple_get_lhs (stmt)) != SSA_NAME)
+ return false;
+
+ if (is_gimple_assign (stmt))
+ {
+ code = gimple_assign_rhs_code (stmt);
+ op_type = TREE_CODE_LENGTH (code.safe_as_tree_code ());
+ }
+ else if (gimple_call_internal_p (stmt))
+ {
+ code = gimple_call_internal_fn (stmt);
+ op_type = gimple_call_num_args (stmt);
+ }
+ else
return false;
bool widen_arith = (code == WIDEN_PLUS_EXPR
- || code == WIDEN_MINUS_EXPR
- || code == WIDEN_MULT_EXPR
- || code == WIDEN_LSHIFT_EXPR);
- op_type = TREE_CODE_LENGTH (code);
+ || code == WIDEN_MINUS_EXPR
+ || code == WIDEN_MULT_EXPR
+ || code == WIDEN_LSHIFT_EXPR);
+
+ if (!widen_arith
+ && !CONVERT_EXPR_CODE_P (code)
+ && code != FIX_TRUNC_EXPR
+ && code != FLOAT_EXPR)
+ return false;
/* Check types of lhs and rhs. */
- scalar_dest = gimple_assign_lhs (stmt);
+ scalar_dest = gimple_get_lhs (stmt);
lhs_type = TREE_TYPE (scalar_dest);
vectype_out = STMT_VINFO_VECTYPE (stmt_info);
@@ -5070,10 +5082,14 @@ vectorizable_conversion (vec_info *vinfo,
if (op_type == binary_op)
{
- gcc_assert (code == WIDEN_MULT_EXPR || code == WIDEN_LSHIFT_EXPR
- || code == WIDEN_PLUS_EXPR || code == WIDEN_MINUS_EXPR);
+ gcc_assert (code == WIDEN_MULT_EXPR
+ || code == WIDEN_LSHIFT_EXPR
+ || code == WIDEN_PLUS_EXPR
+ || code == WIDEN_MINUS_EXPR);
+
- op1 = gimple_assign_rhs2 (stmt);
+ op1 = is_gimple_assign (stmt) ? gimple_assign_rhs2 (stmt) :
+ gimple_call_arg (stmt, 0);
tree vectype1_in;
if (!vect_is_simple_use (vinfo, stmt_info, slp_node, 1,
&op1, &slp_op1, &dt[1], &vectype1_in))
@@ -5157,8 +5173,12 @@ vectorizable_conversion (vec_info *vinfo,
&& code != FLOAT_EXPR
&& !CONVERT_EXPR_CODE_P (code))
return false;
- if (supportable_convert_operation (code, vectype_out, vectype_in, &code1))
+ if (supportable_convert_operation (code.safe_as_tree_code (), vectype_out,
+ vectype_in, &tc1))
+ {
+ code1 = tc1;
break;
+ }
/* FALLTHRU */
unsupported:
if (dump_enabled_p ())
@@ -5169,9 +5189,11 @@ vectorizable_conversion (vec_info *vinfo,
case WIDEN:
if (known_eq (nunits_in, nunits_out))
{
- if (!supportable_half_widening_operation (code, vectype_out,
- vectype_in, &code1))
+ if (!supportable_half_widening_operation (code.safe_as_tree_code (),
+ vectype_out, vectype_in,
+ &tc1))
goto unsupported;
+ code1 = tc1;
gcc_assert (!(multi_step_cvt && op_type == binary_op));
break;
}
@@ -5205,14 +5227,17 @@ vectorizable_conversion (vec_info *vinfo,
if (GET_MODE_SIZE (rhs_mode) == fltsz)
{
- if (!supportable_convert_operation (code, vectype_out,
- cvt_type, &codecvt1))
+ tc1 = ERROR_MARK;
+ if (!supportable_convert_operation (code.safe_as_tree_code (),
+ vectype_out,
+ cvt_type, &tc1))
goto unsupported;
+ codecvt1 = tc1;
}
- else if (!supportable_widening_operation (vinfo, code, stmt_info,
- vectype_out, cvt_type,
- &codecvt1, &codecvt2,
- &multi_step_cvt,
+ else if (!supportable_widening_operation (vinfo, code,
+ stmt_info, vectype_out,
+ cvt_type, &codecvt1,
+ &codecvt2, &multi_step_cvt,
&interm_types))
continue;
else
@@ -5220,8 +5245,9 @@ vectorizable_conversion (vec_info *vinfo,
if (supportable_widening_operation (vinfo, NOP_EXPR, stmt_info,
cvt_type,
- vectype_in, &code1, &code2,
- &multi_step_cvt, &interm_types))
+ vectype_in, &code1,
+ &code2, &multi_step_cvt,
+ &interm_types))
{
found_mode = true;
break;
@@ -5243,10 +5269,15 @@ vectorizable_conversion (vec_info *vinfo,
case NARROW:
gcc_assert (op_type == unary_op);
- if (supportable_narrowing_operation (code, vectype_out, vectype_in,
- &code1, &multi_step_cvt,
+ if (supportable_narrowing_operation (code.safe_as_tree_code (),
+ vectype_out,
+ vectype_in,
+ &tc1, &multi_step_cvt,
&interm_types))
- break;
+ {
+ code1 = tc1;
+ break;
+ }
if (code != FIX_TRUNC_EXPR
|| GET_MODE_SIZE (lhs_mode) >= GET_MODE_SIZE (rhs_mode))
@@ -5257,13 +5288,18 @@ vectorizable_conversion (vec_info *vinfo,
cvt_type = get_same_sized_vectype (cvt_type, vectype_in);
if (cvt_type == NULL_TREE)
goto unsupported;
- if (!supportable_convert_operation (code, cvt_type, vectype_in,
- &codecvt1))
+ if (!supportable_convert_operation (code.safe_as_tree_code (), cvt_type,
+ vectype_in,
+ &tc1))
goto unsupported;
+ codecvt1 = tc1;
if (supportable_narrowing_operation (NOP_EXPR, vectype_out, cvt_type,
- &code1, &multi_step_cvt,
+ &tc1, &multi_step_cvt,
&interm_types))
- break;
+ {
+ code1 = tc1;
+ break;
+ }
goto unsupported;
default:
@@ -5377,8 +5413,10 @@ vectorizable_conversion (vec_info *vinfo,
FOR_EACH_VEC_ELT (vec_oprnds0, i, vop0)
{
/* Arguments are ready, create the new vector stmt. */
- gcc_assert (TREE_CODE_LENGTH (code1) == unary_op);
- gassign *new_stmt = gimple_build_assign (vec_dest, code1, vop0);
+ gcc_assert (TREE_CODE_LENGTH ((tree_code) code1) == unary_op);
+ gassign *new_stmt = gimple_build_assign (vec_dest,
+ code1.safe_as_tree_code (),
+ vop0);
new_temp = make_ssa_name (vec_dest, new_stmt);
gimple_assign_set_lhs (new_stmt, new_temp);
vect_finish_stmt_generation (vinfo, stmt_info, new_stmt, gsi);
@@ -5410,7 +5448,7 @@ vectorizable_conversion (vec_info *vinfo,
for (i = multi_step_cvt; i >= 0; i--)
{
tree this_dest = vec_dsts[i];
- enum tree_code c1 = code1, c2 = code2;
+ code_helper c1 = code1, c2 = code2;
if (i == 0 && codecvt2 != ERROR_MARK)
{
c1 = codecvt1;
@@ -5420,7 +5458,8 @@ vectorizable_conversion (vec_info *vinfo,
vect_create_half_widening_stmts (vinfo, &vec_oprnds0,
&vec_oprnds1, stmt_info,
this_dest, gsi,
- c1, op_type);
+ c1.safe_as_tree_code (),
+ op_type);
else
vect_create_vectorized_promotion_stmts (vinfo, &vec_oprnds0,
&vec_oprnds1, stmt_info,
@@ -5433,9 +5472,11 @@ vectorizable_conversion (vec_info *vinfo,
gimple *new_stmt;
if (cvt_type)
{
- gcc_assert (TREE_CODE_LENGTH (codecvt1) == unary_op);
+ gcc_assert (TREE_CODE_LENGTH ((tree_code) codecvt1) == unary_op);
new_temp = make_ssa_name (vec_dest);
- new_stmt = gimple_build_assign (new_temp, codecvt1, vop0);
+ new_stmt = gimple_build_assign (new_temp,
+ codecvt1.safe_as_tree_code (),
+ vop0);
vect_finish_stmt_generation (vinfo, stmt_info, new_stmt, gsi);
}
else
@@ -5459,10 +5500,12 @@ vectorizable_conversion (vec_info *vinfo,
if (cvt_type)
FOR_EACH_VEC_ELT (vec_oprnds0, i, vop0)
{
- gcc_assert (TREE_CODE_LENGTH (codecvt1) == unary_op);
+ gcc_assert (TREE_CODE_LENGTH (((tree_code) codecvt1)) == unary_op);
new_temp = make_ssa_name (vec_dest);
gassign *new_stmt
- = gimple_build_assign (new_temp, codecvt1, vop0);
+ = gimple_build_assign (new_temp,
+ codecvt1.safe_as_tree_code (),
+ vop0);
vect_finish_stmt_generation (vinfo, stmt_info, new_stmt, gsi);
vec_oprnds0[i] = new_temp;
}
@@ -5470,7 +5513,8 @@ vectorizable_conversion (vec_info *vinfo,
vect_create_vectorized_demotion_stmts (vinfo, &vec_oprnds0,
multi_step_cvt,
stmt_info, vec_dsts, gsi,
- slp_node, code1);
+ slp_node,
+ code1.safe_as_tree_code ());
break;
}
if (!slp_node)
@@ -12151,9 +12195,11 @@ vect_maybe_update_slp_op_vectype (slp_tree op, tree vectype)
bool
supportable_widening_operation (vec_info *vinfo,
- enum tree_code code, stmt_vec_info stmt_info,
+ code_helper code,
+ stmt_vec_info stmt_info,
tree vectype_out, tree vectype_in,
- enum tree_code *code1, enum tree_code *code2,
+ code_helper *code1,
+ code_helper *code2,
int *multi_step_cvt,
vec<tree> *interm_types)
{
@@ -12164,7 +12210,7 @@ supportable_widening_operation (vec_info *vinfo,
optab optab1, optab2;
tree vectype = vectype_in;
tree wide_vectype = vectype_out;
- enum tree_code c1, c2;
+ code_helper c1 = MAX_TREE_CODES, c2 = MAX_TREE_CODES;
int i;
tree prev_type, intermediate_type;
machine_mode intermediate_mode, prev_mode;
@@ -12174,7 +12220,7 @@ supportable_widening_operation (vec_info *vinfo,
if (loop_info)
vect_loop = LOOP_VINFO_LOOP (loop_info);
- switch (code)
+ switch (code.safe_as_tree_code ())
{
case WIDEN_MULT_EXPR:
/* The result of a vectorized widening operation usually requires
@@ -12215,8 +12261,9 @@ supportable_widening_operation (vec_info *vinfo,
&& !nested_in_vect_loop_p (vect_loop, stmt_info)
&& supportable_widening_operation (vinfo, VEC_WIDEN_MULT_EVEN_EXPR,
stmt_info, vectype_out,
- vectype_in, code1, code2,
- multi_step_cvt, interm_types))
+ vectype_in, code1,
+ code2, multi_step_cvt,
+ interm_types))
{
/* Elements in a vector with vect_used_by_reduction property cannot
be reordered if the use chain with this property does not have the
@@ -12279,6 +12326,9 @@ supportable_widening_operation (vec_info *vinfo,
c2 = VEC_UNPACK_FIX_TRUNC_HI_EXPR;
break;
+ case MAX_TREE_CODES:
+ break;
+
default:
gcc_unreachable ();
}
@@ -12289,10 +12339,12 @@ supportable_widening_operation (vec_info *vinfo,
if (code == FIX_TRUNC_EXPR)
{
/* The signedness is determined from output operand. */
- optab1 = optab_for_tree_code (c1, vectype_out, optab_default);
- optab2 = optab_for_tree_code (c2, vectype_out, optab_default);
+ optab1 = optab_for_tree_code (c1.safe_as_tree_code (), vectype_out,
+ optab_default);
+ optab2 = optab_for_tree_code (c2.safe_as_tree_code (), vectype_out,
+ optab_default);
}
- else if (CONVERT_EXPR_CODE_P (code)
+ else if (CONVERT_EXPR_CODE_P (code.safe_as_tree_code ())
&& VECTOR_BOOLEAN_TYPE_P (wide_vectype)
&& VECTOR_BOOLEAN_TYPE_P (vectype)
&& TYPE_MODE (wide_vectype) == TYPE_MODE (vectype)
@@ -12305,8 +12357,10 @@ supportable_widening_operation (vec_info *vinfo,
}
else
{
- optab1 = optab_for_tree_code (c1, vectype, optab_default);
- optab2 = optab_for_tree_code (c2, vectype, optab_default);
+ optab1 = optab_for_tree_code (c1.safe_as_tree_code (), vectype,
+ optab_default);
+ optab2 = optab_for_tree_code (c2.safe_as_tree_code (), vectype,
+ optab_default);
}
if (!optab1 || !optab2)
@@ -12317,8 +12371,12 @@ supportable_widening_operation (vec_info *vinfo,
|| (icode2 = optab_handler (optab2, vec_mode)) == CODE_FOR_nothing)
return false;
- *code1 = c1;
- *code2 = c2;
+ if (code.is_tree_code ())
+ {
+ *code1 = c1;
+ *code2 = c2;
+ }
+
if (insn_data[icode1].operand[0].mode == TYPE_MODE (wide_vectype)
&& insn_data[icode2].operand[0].mode == TYPE_MODE (wide_vectype))
@@ -12339,7 +12397,7 @@ supportable_widening_operation (vec_info *vinfo,
prev_type = vectype;
prev_mode = vec_mode;
- if (!CONVERT_EXPR_CODE_P (code))
+ if (!CONVERT_EXPR_CODE_P ((tree_code) code))
return false;
/* We assume here that there will not be more than MAX_INTERM_CVT_STEPS
@@ -12379,8 +12437,12 @@ supportable_widening_operation (vec_info *vinfo,
}
else
{
- optab3 = optab_for_tree_code (c1, intermediate_type, optab_default);
- optab4 = optab_for_tree_code (c2, intermediate_type, optab_default);
+ optab3 = optab_for_tree_code (c1.safe_as_tree_code (),
+ intermediate_type,
+ optab_default);
+ optab4 = optab_for_tree_code (c2.safe_as_tree_code (),
+ intermediate_type,
+ optab_default);
}
if (!optab3 || !optab4
@@ -12439,7 +12501,7 @@ supportable_widening_operation (vec_info *vinfo,
bool
supportable_narrowing_operation (enum tree_code code,
tree vectype_out, tree vectype_in,
- enum tree_code *code1, int *multi_step_cvt,
+ tree_code *code1, int *multi_step_cvt,
vec<tree> *interm_types)
{
machine_mode vec_mode;
diff --git a/gcc/tree-vectorizer.h b/gcc/tree-vectorizer.h
index 9cf2fb23fe397b467d89aa7cc5ebeaa293ed4cce..d241eba6ef3302225bbe37b374baa11e6472c280 100644
--- a/gcc/tree-vectorizer.h
+++ b/gcc/tree-vectorizer.h
@@ -2139,13 +2139,12 @@ extern bool vect_is_simple_use (vec_info *, stmt_vec_info, slp_tree,
enum vect_def_type *,
tree *, stmt_vec_info * = NULL);
extern bool vect_maybe_update_slp_op_vectype (slp_tree, tree);
-extern bool supportable_widening_operation (vec_info *,
- enum tree_code, stmt_vec_info,
- tree, tree, enum tree_code *,
- enum tree_code *, int *,
- vec<tree> *);
+extern bool supportable_widening_operation (vec_info*, code_helper,
+ stmt_vec_info, tree, tree,
+ code_helper*, code_helper*,
+ int*, vec<tree> *);
extern bool supportable_narrowing_operation (enum tree_code, tree, tree,
- enum tree_code *, int *,
+ tree_code *, int *,
vec<tree> *);
extern unsigned record_stmt_cost (stmt_vector_for_cost *, int,
@@ -2583,4 +2582,7 @@ vect_is_integer_truncation (stmt_vec_info stmt_info)
&& TYPE_PRECISION (lhs_type) < TYPE_PRECISION (rhs_type));
}
+/* Build a GIMPLE_ASSIGN or GIMPLE_CALL with the tree_code,
+ or internal_fn contained in ch, respectively. */
+gimple * vect_gimple_build (tree, code_helper, tree, tree);
#endif /* GCC_TREE_VECTORIZER_H */
diff --git a/gcc/tree.h b/gcc/tree.h
index abcdb5638d49aea4ccc46efa8e540b1fa78aa27a..a250a80e0321241e1158086acb2dd837d5827e10 100644
--- a/gcc/tree.h
+++ b/gcc/tree.h
@@ -93,6 +93,8 @@ public:
bool is_internal_fn () const;
bool is_builtin_fn () const;
int get_rep () const { return rep; }
+ enum tree_code safe_as_tree_code () const;
+ combined_fn safe_as_fn_code () const;
bool operator== (const code_helper &other) { return rep == other.rep; }
bool operator!= (const code_helper &other) { return rep != other.rep; }
bool operator== (tree_code c) { return rep == code_helper (c).rep; }
@@ -102,6 +104,17 @@ private:
int rep;
};
+inline enum tree_code
+code_helper::safe_as_tree_code () const
+{
+ return is_tree_code () ? (tree_code)* this : MAX_TREE_CODES;
+}
+
+inline combined_fn
+code_helper::safe_as_fn_code () const {
+ return is_fn_code () ? (combined_fn) *this : CFN_LAST;
+}
+
inline code_helper::operator internal_fn () const
{
return as_internal_fn (combined_fn (*this));
[-- Attachment #3: ifn1.patch --]
[-- Type: text/plain, Size: 18605 bytes --]
diff --git a/gcc/internal-fn.cc b/gcc/internal-fn.cc
index 6e81dc05e0e0714256759b0594816df451415a2d..e4d815cd577d266d2bccf6fb68d62aac91a8b4cf 100644
--- a/gcc/internal-fn.cc
+++ b/gcc/internal-fn.cc
@@ -17,6 +17,7 @@ You should have received a copy of the GNU General Public License
along with GCC; see the file COPYING3. If not see
<http://www.gnu.org/licenses/>. */
+#define INCLUDE_MAP
#include "config.h"
#include "system.h"
#include "coretypes.h"
@@ -70,6 +71,26 @@ const int internal_fn_flags_array[] = {
0
};
+const enum internal_fn internal_fn_hilo_keys_array[] = {
+#undef DEF_INTERNAL_OPTAB_HILO_FN
+#define DEF_INTERNAL_OPTAB_HILO_FN(NAME, FLAGS, OPTAB, SOPTAB, UOPTAB, TYPE) \
+ IFN_##NAME##_LO, \
+ IFN_##NAME##_HI,
+#include "internal-fn.def"
+ IFN_LAST
+#undef DEF_INTERNAL_OPTAB_HILO_FN
+};
+
+const optab internal_fn_hilo_values_array[] = {
+#undef DEF_INTERNAL_OPTAB_HILO_FN
+#define DEF_INTERNAL_OPTAB_HILO_FN(NAME, FLAGS, OPTAB, SOPTAB, UOPTAB, TYPE) \
+ SOPTAB##_lo_optab, UOPTAB##_lo_optab, \
+ SOPTAB##_hi_optab, UOPTAB##_hi_optab,
+#include "internal-fn.def"
+ unknown_optab, unknown_optab
+#undef DEF_INTERNAL_OPTAB_HILO_FN
+};
+
/* Return the internal function called NAME, or IFN_LAST if there's
no such function. */
@@ -90,6 +111,61 @@ lookup_internal_fn (const char *name)
return entry ? *entry : IFN_LAST;
}
+static int
+ifn_cmp (const void *a_, const void *b_)
+{
+ typedef std::pair<enum internal_fn, unsigned> ifn_pair;
+ auto *a = (const std::pair<ifn_pair, optab> *)a_;
+ auto *b = (const std::pair<ifn_pair, optab> *)b_;
+ return (int) (a->first.first) - (b->first.first);
+}
+
+/* Return the optab belonging to the given internal function NAME for the given
+ SIGN or unknown_optab. */
+
+optab
+lookup_hilo_ifn_optab (enum internal_fn fn, unsigned sign)
+{
+ typedef std::pair<enum internal_fn, unsigned> ifn_pair;
+ typedef auto_vec <std::pair<ifn_pair, optab>>fn_to_optab_map_type;
+ static fn_to_optab_map_type *fn_to_optab_map;
+
+ if (!fn_to_optab_map)
+ {
+ unsigned num
+ = sizeof (internal_fn_hilo_keys_array) / sizeof (enum internal_fn);
+ fn_to_optab_map = new fn_to_optab_map_type ();
+ for (unsigned int i = 0; i < num - 1; ++i)
+ {
+ enum internal_fn fn = internal_fn_hilo_keys_array[i];
+ optab v1 = internal_fn_hilo_values_array[2*i];
+ optab v2 = internal_fn_hilo_values_array[2*i + 1];
+ ifn_pair key1 (fn, 0);
+ fn_to_optab_map->safe_push ({key1, v1});
+ ifn_pair key2 (fn, 1);
+ fn_to_optab_map->safe_push ({key2, v2});
+ }
+ fn_to_optab_map->qsort (ifn_cmp);
+ }
+
+ ifn_pair new_pair (fn, sign ? 1 : 0);
+ optab tmp;
+ std::pair<ifn_pair,optab> pair_wrap (new_pair, tmp);
+ auto entry = fn_to_optab_map->bsearch (&pair_wrap, ifn_cmp);
+ return entry != fn_to_optab_map->end () ? entry->second : unknown_optab;
+}
+
+extern void
+lookup_hilo_internal_fn (enum internal_fn ifn, enum internal_fn *lo,
+ enum internal_fn *hi)
+{
+ gcc_assert (decomposes_to_hilo_fn_p (ifn));
+
+ *lo = internal_fn (ifn + 1);
+ *hi = internal_fn (ifn + 2);
+}
+
+
/* Fnspec of each internal function, indexed by function number. */
const_tree internal_fn_fnspec_array[IFN_LAST + 1];
@@ -3970,6 +4046,9 @@ commutative_binary_fn_p (internal_fn fn)
case IFN_UBSAN_CHECK_MUL:
case IFN_ADD_OVERFLOW:
case IFN_MUL_OVERFLOW:
+ case IFN_VEC_WIDEN_PLUS:
+ case IFN_VEC_WIDEN_PLUS_LO:
+ case IFN_VEC_WIDEN_PLUS_HI:
return true;
default:
@@ -4043,6 +4122,42 @@ first_commutative_argument (internal_fn fn)
}
}
+/* Return true if FN has a wider output type than its argument types. */
+
+bool
+widening_fn_p (internal_fn fn)
+{
+ switch (fn)
+ {
+ case IFN_VEC_WIDEN_PLUS:
+ case IFN_VEC_WIDEN_MINUS:
+ return true;
+
+ default:
+ return false;
+ }
+}
+
+/* Return true if FN decomposes to _hi and _lo IFN. If true this should also
+ be a widening function. */
+
+bool
+decomposes_to_hilo_fn_p (internal_fn fn)
+{
+ if (!widening_fn_p (fn))
+ return false;
+
+ switch (fn)
+ {
+ case IFN_VEC_WIDEN_PLUS:
+ case IFN_VEC_WIDEN_MINUS:
+ return true;
+
+ default:
+ return false;
+ }
+}
+
/* Return true if IFN_SET_EDOM is supported. */
bool
@@ -4055,6 +4170,32 @@ set_edom_supported_p (void)
#endif
}
+#undef DEF_INTERNAL_OPTAB_HILO_FN
+#define DEF_INTERNAL_OPTAB_HILO_FN(CODE, FLAGS, OPTAB, SOPTAB, UOPTAB, TYPE) \
+ static void \
+ expand_##CODE (internal_fn, gcall *) \
+ { \
+ gcc_unreachable (); \
+ } \
+ static void \
+ expand_##CODE##_LO (internal_fn fn, gcall *stmt) \
+ { \
+ tree ty = TREE_TYPE (gimple_get_lhs (stmt)); \
+ if (!TYPE_UNSIGNED (ty)) \
+ expand_##TYPE##_optab_fn (fn, stmt, SOPTAB##_lo##_optab); \
+ else \
+ expand_##TYPE##_optab_fn (fn, stmt, UOPTAB##_lo##_optab); \
+ } \
+ static void \
+ expand_##CODE##_HI (internal_fn fn, gcall *stmt) \
+ { \
+ tree ty = TREE_TYPE (gimple_get_lhs (stmt)); \
+ if (!TYPE_UNSIGNED (ty)) \
+ expand_##TYPE##_optab_fn (fn, stmt, SOPTAB##_hi##_optab); \
+ else \
+ expand_##TYPE##_optab_fn (fn, stmt, UOPTAB##_hi##_optab); \
+ }
+
#define DEF_INTERNAL_OPTAB_FN(CODE, FLAGS, OPTAB, TYPE) \
static void \
expand_##CODE (internal_fn fn, gcall *stmt) \
@@ -4071,6 +4212,7 @@ set_edom_supported_p (void)
expand_##TYPE##_optab_fn (fn, stmt, which_optab); \
}
#include "internal-fn.def"
+#undef DEF_INTERNAL_OPTAB_HILO_FN
/* Routines to expand each internal function, indexed by function number.
Each routine has the prototype:
diff --git a/gcc/internal-fn.def b/gcc/internal-fn.def
index 7fe742c2ae713e7152ab05cfdfba86e4e0aa3456..347ed667d92620e0ee3ea15c58ecac6c242ebe73 100644
--- a/gcc/internal-fn.def
+++ b/gcc/internal-fn.def
@@ -85,6 +85,13 @@ along with GCC; see the file COPYING3. If not see
says that the function extends the C-level BUILT_IN_<NAME>{,L,LL,IMAX}
group of functions to any integral mode (including vector modes).
+ DEF_INTERNAL_OPTAB_HILO_FN is like DEF_INTERNAL_OPTAB_FN except it
+ provides convenience wrappers for defining conversions that require a
+ hi/lo split, like widening and narrowing operations. Each definition
+ for <NAME> will require an optab named <OPTAB> and two other optabs that
+ you specify for signed and unsigned.
+
+
Each entry must have a corresponding expander of the form:
void expand_NAME (gimple_call stmt)
@@ -123,6 +130,14 @@ along with GCC; see the file COPYING3. If not see
DEF_INTERNAL_OPTAB_FN (NAME, FLAGS, OPTAB, TYPE)
#endif
+#ifndef DEF_INTERNAL_OPTAB_HILO_FN
+#define DEF_INTERNAL_OPTAB_HILO_FN(NAME, FLAGS, OPTAB, SOPTAB, UOPTAB, TYPE) \
+ DEF_INTERNAL_OPTAB_FN (NAME, FLAGS, OPTAB, TYPE) \
+ DEF_INTERNAL_OPTAB_FN (NAME ## _LO, FLAGS, unknown, TYPE) \
+ DEF_INTERNAL_OPTAB_FN (NAME ## _HI, FLAGS, unknown, TYPE)
+#endif
+
+
DEF_INTERNAL_OPTAB_FN (MASK_LOAD, ECF_PURE, maskload, mask_load)
DEF_INTERNAL_OPTAB_FN (LOAD_LANES, ECF_CONST, vec_load_lanes, load_lanes)
DEF_INTERNAL_OPTAB_FN (MASK_LOAD_LANES, ECF_PURE,
@@ -315,6 +330,14 @@ DEF_INTERNAL_OPTAB_FN (COMPLEX_ADD_ROT270, ECF_CONST, cadd270, binary)
DEF_INTERNAL_OPTAB_FN (COMPLEX_MUL, ECF_CONST, cmul, binary)
DEF_INTERNAL_OPTAB_FN (COMPLEX_MUL_CONJ, ECF_CONST, cmul_conj, binary)
DEF_INTERNAL_OPTAB_FN (VEC_ADDSUB, ECF_CONST, vec_addsub, binary)
+DEF_INTERNAL_OPTAB_HILO_FN (VEC_WIDEN_PLUS,
+ ECF_CONST | ECF_NOTHROW,
+ vec_widen_add, vec_widen_saddl, vec_widen_uaddl,
+ binary)
+DEF_INTERNAL_OPTAB_HILO_FN (VEC_WIDEN_MINUS,
+ ECF_CONST | ECF_NOTHROW,
+ vec_widen_sub, vec_widen_ssubl, vec_widen_usubl,
+ binary)
DEF_INTERNAL_OPTAB_FN (VEC_FMADDSUB, ECF_CONST, vec_fmaddsub, ternary)
DEF_INTERNAL_OPTAB_FN (VEC_FMSUBADD, ECF_CONST, vec_fmsubadd, ternary)
diff --git a/gcc/internal-fn.h b/gcc/internal-fn.h
index 08922ed4254898f5fffca3f33973e96ed9ce772f..6a5f8762e872ad2ef64ce2986a678e3b40622d81 100644
--- a/gcc/internal-fn.h
+++ b/gcc/internal-fn.h
@@ -20,6 +20,10 @@ along with GCC; see the file COPYING3. If not see
#ifndef GCC_INTERNAL_FN_H
#define GCC_INTERNAL_FN_H
+#include "insn-codes.h"
+#include "insn-opinit.h"
+
+
/* INTEGER_CST values for IFN_UNIQUE function arg-0.
UNSPEC: Undifferentiated UNIQUE.
@@ -112,6 +116,9 @@ internal_fn_name (enum internal_fn fn)
}
extern internal_fn lookup_internal_fn (const char *);
+extern optab lookup_hilo_ifn_optab (enum internal_fn, unsigned);
+extern void lookup_hilo_internal_fn (enum internal_fn, enum internal_fn *,
+ enum internal_fn *);
/* Return the ECF_* flags for function FN. */
@@ -210,6 +217,8 @@ extern bool commutative_binary_fn_p (internal_fn);
extern bool commutative_ternary_fn_p (internal_fn);
extern int first_commutative_argument (internal_fn);
extern bool associative_binary_fn_p (internal_fn);
+extern bool widening_fn_p (internal_fn);
+extern bool decomposes_to_hilo_fn_p (internal_fn);
extern bool set_edom_supported_p (void);
diff --git a/gcc/optabs.cc b/gcc/optabs.cc
index c8e39c82d57a7d726e7da33d247b80f32ec9236c..d4dd7ee3d34d01c32ab432ae4e4ce9e4b522b2f7 100644
--- a/gcc/optabs.cc
+++ b/gcc/optabs.cc
@@ -1314,7 +1314,12 @@ commutative_optab_p (optab binoptab)
|| binoptab == smul_widen_optab
|| binoptab == umul_widen_optab
|| binoptab == smul_highpart_optab
- || binoptab == umul_highpart_optab);
+ || binoptab == umul_highpart_optab
+ || binoptab == vec_widen_add_optab
+ || binoptab == vec_widen_saddl_hi_optab
+ || binoptab == vec_widen_saddl_lo_optab
+ || binoptab == vec_widen_uaddl_hi_optab
+ || binoptab == vec_widen_uaddl_lo_optab);
}
/* X is to be used in mode MODE as operand OPN to BINOPTAB. If we're
diff --git a/gcc/optabs.def b/gcc/optabs.def
index 695f5911b300c9ca5737de9be809fa01aabe5e01..e064189103b3be70644468d11f3c91ac45ffe0d0 100644
--- a/gcc/optabs.def
+++ b/gcc/optabs.def
@@ -78,6 +78,8 @@ OPTAB_CD(smsub_widen_optab, "msub$b$a4")
OPTAB_CD(umsub_widen_optab, "umsub$b$a4")
OPTAB_CD(ssmsub_widen_optab, "ssmsub$b$a4")
OPTAB_CD(usmsub_widen_optab, "usmsub$a$b4")
+OPTAB_CD(vec_widen_add_optab, "add$a$b3")
+OPTAB_CD(vec_widen_sub_optab, "sub$a$b3")
OPTAB_CD(vec_load_lanes_optab, "vec_load_lanes$a$b")
OPTAB_CD(vec_store_lanes_optab, "vec_store_lanes$a$b")
OPTAB_CD(vec_mask_load_lanes_optab, "vec_mask_load_lanes$a$b")
diff --git a/gcc/testsuite/gcc.target/aarch64/vect-widen-add.c b/gcc/testsuite/gcc.target/aarch64/vect-widen-add.c
index 220bd9352a4c7acd2e3713e441d74898d3e92b30..7037673d32bd780e1c9b58a51e58e2bac3b30b7e 100644
--- a/gcc/testsuite/gcc.target/aarch64/vect-widen-add.c
+++ b/gcc/testsuite/gcc.target/aarch64/vect-widen-add.c
@@ -1,5 +1,5 @@
/* { dg-do run } */
-/* { dg-options "-O3 -save-temps" } */
+/* { dg-options "-O3 -save-temps -fdump-tree-vect-all" } */
#include <stdint.h>
#include <string.h>
@@ -86,6 +86,8 @@ main()
return 0;
}
+/* { dg-final { scan-tree-dump "add new stmt.*VEC_WIDEN_PLUS_LO" "vect" } } */
+/* { dg-final { scan-tree-dump "add new stmt.*VEC_WIDEN_PLUS_HI" "vect" } } */
/* { dg-final { scan-assembler-times {\tuaddl\t} 1} } */
/* { dg-final { scan-assembler-times {\tuaddl2\t} 1} } */
/* { dg-final { scan-assembler-times {\tsaddl\t} 1} } */
diff --git a/gcc/testsuite/gcc.target/aarch64/vect-widen-sub.c b/gcc/testsuite/gcc.target/aarch64/vect-widen-sub.c
index a2bed63affbd091977df95a126da1f5b8c1d41d2..83bc1edb6105f47114b665e24a13e6194b2179a2 100644
--- a/gcc/testsuite/gcc.target/aarch64/vect-widen-sub.c
+++ b/gcc/testsuite/gcc.target/aarch64/vect-widen-sub.c
@@ -1,5 +1,5 @@
/* { dg-do run } */
-/* { dg-options "-O3 -save-temps" } */
+/* { dg-options "-O3 -save-temps -fdump-tree-vect-all" } */
#include <stdint.h>
#include <string.h>
@@ -86,6 +86,8 @@ main()
return 0;
}
+/* { dg-final { scan-tree-dump "add new stmt.*VEC_WIDEN_MINUS_LO" "vect" } } */
+/* { dg-final { scan-tree-dump "add new stmt.*VEC_WIDEN_MINUS_HI" "vect" } } */
/* { dg-final { scan-assembler-times {\tusubl\t} 1} } */
/* { dg-final { scan-assembler-times {\tusubl2\t} 1} } */
/* { dg-final { scan-assembler-times {\tssubl\t} 1} } */
diff --git a/gcc/tree-vect-patterns.cc b/gcc/tree-vect-patterns.cc
index 68dfba266d679c9738a3d5d70551a91cbdafcf66..1a514461b2ca416f45a5fa9abe417980d33ef4df 100644
--- a/gcc/tree-vect-patterns.cc
+++ b/gcc/tree-vect-patterns.cc
@@ -1394,14 +1394,16 @@ static gimple *
vect_recog_widen_op_pattern (vec_info *vinfo,
stmt_vec_info last_stmt_info, tree *type_out,
tree_code orig_code, code_helper wide_code,
- bool shift_p, const char *name)
+ bool shift_p, const char *name,
+ enum optab_subtype *subtype = NULL)
{
gimple *last_stmt = last_stmt_info->stmt;
vect_unpromoted_value unprom[2];
tree half_type;
if (!vect_widened_op_tree (vinfo, last_stmt_info, orig_code, orig_code,
- shift_p, 2, unprom, &half_type))
+ shift_p, 2, unprom, &half_type, subtype))
+
return NULL;
/* Pattern detected. */
@@ -1467,6 +1469,20 @@ vect_recog_widen_op_pattern (vec_info *vinfo,
type, pattern_stmt, vecctype);
}
+static gimple *
+vect_recog_widen_op_pattern (vec_info *vinfo,
+ stmt_vec_info last_stmt_info, tree *type_out,
+ tree_code orig_code, internal_fn wide_ifn,
+ bool shift_p, const char *name,
+ enum optab_subtype *subtype = NULL)
+{
+ combined_fn ifn = as_combined_fn (wide_ifn);
+ return vect_recog_widen_op_pattern (vinfo, last_stmt_info, type_out,
+ orig_code, ifn, shift_p, name,
+ subtype);
+}
+
+
/* Try to detect multiplication on widened inputs, converting MULT_EXPR
to WIDEN_MULT_EXPR. See vect_recog_widen_op_pattern for details. */
@@ -1480,26 +1496,30 @@ vect_recog_widen_mult_pattern (vec_info *vinfo, stmt_vec_info last_stmt_info,
}
/* Try to detect addition on widened inputs, converting PLUS_EXPR
- to WIDEN_PLUS_EXPR. See vect_recog_widen_op_pattern for details. */
+ to IFN_VEC_WIDEN_PLUS. See vect_recog_widen_op_pattern for details. */
static gimple *
vect_recog_widen_plus_pattern (vec_info *vinfo, stmt_vec_info last_stmt_info,
tree *type_out)
{
+ enum optab_subtype subtype;
return vect_recog_widen_op_pattern (vinfo, last_stmt_info, type_out,
- PLUS_EXPR, WIDEN_PLUS_EXPR, false,
- "vect_recog_widen_plus_pattern");
+ PLUS_EXPR, IFN_VEC_WIDEN_PLUS,
+ false, "vect_recog_widen_plus_pattern",
+ &subtype);
}
/* Try to detect subtraction on widened inputs, converting MINUS_EXPR
- to WIDEN_MINUS_EXPR. See vect_recog_widen_op_pattern for details. */
+ to IFN_VEC_WIDEN_MINUS. See vect_recog_widen_op_pattern for details. */
static gimple *
vect_recog_widen_minus_pattern (vec_info *vinfo, stmt_vec_info last_stmt_info,
tree *type_out)
{
+ enum optab_subtype subtype;
return vect_recog_widen_op_pattern (vinfo, last_stmt_info, type_out,
- MINUS_EXPR, WIDEN_MINUS_EXPR, false,
- "vect_recog_widen_minus_pattern");
+ MINUS_EXPR, IFN_VEC_WIDEN_MINUS,
+ false, "vect_recog_widen_minus_pattern",
+ &subtype);
}
/* Function vect_recog_popcount_pattern
@@ -6067,6 +6087,7 @@ static vect_recog_func vect_vect_recog_func_ptrs[] = {
{ vect_recog_mask_conversion_pattern, "mask_conversion" },
{ vect_recog_widen_plus_pattern, "widen_plus" },
{ vect_recog_widen_minus_pattern, "widen_minus" },
+ /* These must come after the double widening ones. */
};
const unsigned int NUM_PATTERNS = ARRAY_SIZE (vect_vect_recog_func_ptrs);
diff --git a/gcc/tree-vect-stmts.cc b/gcc/tree-vect-stmts.cc
index 715ec2e30a4de620b8a5076c0e7f2f7fd1b0654e..f4806073f48d4dedea3ac9bd855792b152d78919 100644
--- a/gcc/tree-vect-stmts.cc
+++ b/gcc/tree-vect-stmts.cc
@@ -5035,7 +5035,9 @@ vectorizable_conversion (vec_info *vinfo,
bool widen_arith = (code == WIDEN_PLUS_EXPR
|| code == WIDEN_MINUS_EXPR
|| code == WIDEN_MULT_EXPR
- || code == WIDEN_LSHIFT_EXPR);
+ || code == WIDEN_LSHIFT_EXPR
+ || code == IFN_VEC_WIDEN_PLUS
+ || code == IFN_VEC_WIDEN_MINUS);
if (!widen_arith
&& !CONVERT_EXPR_CODE_P (code)
@@ -5085,7 +5087,9 @@ vectorizable_conversion (vec_info *vinfo,
gcc_assert (code == WIDEN_MULT_EXPR
|| code == WIDEN_LSHIFT_EXPR
|| code == WIDEN_PLUS_EXPR
- || code == WIDEN_MINUS_EXPR);
+ || code == WIDEN_MINUS_EXPR
+ || code == IFN_VEC_WIDEN_PLUS
+ || code == IFN_VEC_WIDEN_MINUS);
op1 = is_gimple_assign (stmt) ? gimple_assign_rhs2 (stmt) :
@@ -12355,14 +12359,50 @@ supportable_widening_operation (vec_info *vinfo,
optab1 = vec_unpacks_sbool_lo_optab;
optab2 = vec_unpacks_sbool_hi_optab;
}
- else
- {
- optab1 = optab_for_tree_code (c1.safe_as_tree_code (), vectype,
- optab_default);
- optab2 = optab_for_tree_code (c2.safe_as_tree_code (), vectype,
- optab_default);
+
+ if (code.is_fn_code ())
+ {
+ internal_fn ifn = as_internal_fn (code.safe_as_fn_code ());
+ gcc_assert (decomposes_to_hilo_fn_p (ifn));
+
+ internal_fn lo, hi;
+ lookup_hilo_internal_fn (ifn, &lo, &hi);
+ *code1 = as_combined_fn (lo);
+ *code2 = as_combined_fn (hi);
+ optab1 = lookup_hilo_ifn_optab (lo, !TYPE_UNSIGNED (vectype));
+ optab2 = lookup_hilo_ifn_optab (hi, !TYPE_UNSIGNED (vectype));
}
+ if (code.is_tree_code ())
+ {
+ if (code == FIX_TRUNC_EXPR)
+ {
+ /* The signedness is determined from output operand. */
+ optab1 = optab_for_tree_code (c1.safe_as_tree_code (), vectype_out,
+ optab_default);
+ optab2 = optab_for_tree_code (c2.safe_as_tree_code (), vectype_out,
+ optab_default);
+ }
+ else if (CONVERT_EXPR_CODE_P (code.safe_as_tree_code ())
+ && VECTOR_BOOLEAN_TYPE_P (wide_vectype)
+ && VECTOR_BOOLEAN_TYPE_P (vectype)
+ && TYPE_MODE (wide_vectype) == TYPE_MODE (vectype)
+ && SCALAR_INT_MODE_P (TYPE_MODE (vectype)))
+ {
+ /* If the input and result modes are the same, a different optab
+ is needed where we pass in the number of units in vectype. */
+ optab1 = vec_unpacks_sbool_lo_optab;
+ optab2 = vec_unpacks_sbool_hi_optab;
+ }
+ else
+ {
+ optab1 = optab_for_tree_code (c1.safe_as_tree_code (), vectype,
+ optab_default);
+ optab2 = optab_for_tree_code (c2.safe_as_tree_code (), vectype,
+ optab_default);
+ }
+ }
+
if (!optab1 || !optab2)
return false;
[-- Attachment #4: ifn2.patch --]
[-- Type: text/plain, Size: 19234 bytes --]
diff --git a/gcc/cfgexpand.cc b/gcc/cfgexpand.cc
index 1a1b26b1c6c23ce273bcd08dc9a973f777174007..25b1558dcb941ea491a19aeeb2cd8f4d2dbdf7c6 100644
--- a/gcc/cfgexpand.cc
+++ b/gcc/cfgexpand.cc
@@ -5365,10 +5365,6 @@ expand_debug_expr (tree exp)
case VEC_WIDEN_MULT_ODD_EXPR:
case VEC_WIDEN_LSHIFT_HI_EXPR:
case VEC_WIDEN_LSHIFT_LO_EXPR:
- case VEC_WIDEN_PLUS_HI_EXPR:
- case VEC_WIDEN_PLUS_LO_EXPR:
- case VEC_WIDEN_MINUS_HI_EXPR:
- case VEC_WIDEN_MINUS_LO_EXPR:
case VEC_PERM_EXPR:
case VEC_DUPLICATE_EXPR:
case VEC_SERIES_EXPR:
@@ -5405,8 +5401,6 @@ expand_debug_expr (tree exp)
case WIDEN_MULT_EXPR:
case WIDEN_MULT_PLUS_EXPR:
case WIDEN_MULT_MINUS_EXPR:
- case WIDEN_PLUS_EXPR:
- case WIDEN_MINUS_EXPR:
if (SCALAR_INT_MODE_P (GET_MODE (op0))
&& SCALAR_INT_MODE_P (mode))
{
@@ -5419,10 +5413,6 @@ expand_debug_expr (tree exp)
op1 = simplify_gen_unary (ZERO_EXTEND, mode, op1, inner_mode);
else
op1 = simplify_gen_unary (SIGN_EXTEND, mode, op1, inner_mode);
- if (TREE_CODE (exp) == WIDEN_PLUS_EXPR)
- return simplify_gen_binary (PLUS, mode, op0, op1);
- else if (TREE_CODE (exp) == WIDEN_MINUS_EXPR)
- return simplify_gen_binary (MINUS, mode, op0, op1);
op0 = simplify_gen_binary (MULT, mode, op0, op1);
if (TREE_CODE (exp) == WIDEN_MULT_EXPR)
return op0;
diff --git a/gcc/doc/generic.texi b/gcc/doc/generic.texi
index 2c14b7abce2db0a3da0a21e916907947cb56a265..3816abaaf4d364d604a44942317f96f3f303e5b6 100644
--- a/gcc/doc/generic.texi
+++ b/gcc/doc/generic.texi
@@ -1811,10 +1811,6 @@ a value from @code{enum annot_expr_kind}, the third is an @code{INTEGER_CST}.
@tindex VEC_RSHIFT_EXPR
@tindex VEC_WIDEN_MULT_HI_EXPR
@tindex VEC_WIDEN_MULT_LO_EXPR
-@tindex VEC_WIDEN_PLUS_HI_EXPR
-@tindex VEC_WIDEN_PLUS_LO_EXPR
-@tindex VEC_WIDEN_MINUS_HI_EXPR
-@tindex VEC_WIDEN_MINUS_LO_EXPR
@tindex VEC_UNPACK_HI_EXPR
@tindex VEC_UNPACK_LO_EXPR
@tindex VEC_UNPACK_FLOAT_HI_EXPR
@@ -1861,33 +1857,6 @@ vector of @code{N/2} products. In the case of @code{VEC_WIDEN_MULT_LO_EXPR} the
low @code{N/2} elements of the two vector are multiplied to produce the
vector of @code{N/2} products.
-@item VEC_WIDEN_PLUS_HI_EXPR
-@itemx VEC_WIDEN_PLUS_LO_EXPR
-These nodes represent widening vector addition of the high and low parts of
-the two input vectors, respectively. Their operands are vectors that contain
-the same number of elements (@code{N}) of the same integral type. The result
-is a vector that contains half as many elements, of an integral type whose size
-is twice as wide. In the case of @code{VEC_WIDEN_PLUS_HI_EXPR} the high
-@code{N/2} elements of the two vectors are added to produce the vector of
-@code{N/2} products. In the case of @code{VEC_WIDEN_PLUS_LO_EXPR} the low
-@code{N/2} elements of the two vectors are added to produce the vector of
-@code{N/2} products.
-
-@item VEC_WIDEN_MINUS_HI_EXPR
-@itemx VEC_WIDEN_MINUS_LO_EXPR
-These nodes represent widening vector subtraction of the high and low parts of
-the two input vectors, respectively. Their operands are vectors that contain
-the same number of elements (@code{N}) of the same integral type. The high/low
-elements of the second vector are subtracted from the high/low elements of the
-first. The result is a vector that contains half as many elements, of an
-integral type whose size is twice as wide. In the case of
-@code{VEC_WIDEN_MINUS_HI_EXPR} the high @code{N/2} elements of the second
-vector are subtracted from the high @code{N/2} of the first to produce the
-vector of @code{N/2} products. In the case of
-@code{VEC_WIDEN_MINUS_LO_EXPR} the low @code{N/2} elements of the second
-vector are subtracted from the low @code{N/2} of the first to produce the
-vector of @code{N/2} products.
-
@item VEC_UNPACK_HI_EXPR
@itemx VEC_UNPACK_LO_EXPR
These nodes represent unpacking of the high and low parts of the input vector,
diff --git a/gcc/expr.cc b/gcc/expr.cc
index f8f5cc5a6ca67f291b3c8b7246d593c0be80272f..454d1391b19a7d2aa53f0a88876d1eaf0494de51 100644
--- a/gcc/expr.cc
+++ b/gcc/expr.cc
@@ -9601,8 +9601,6 @@ expand_expr_real_2 (sepops ops, rtx target, machine_mode tmode,
target, unsignedp);
return target;
- case WIDEN_PLUS_EXPR:
- case WIDEN_MINUS_EXPR:
case WIDEN_MULT_EXPR:
/* If first operand is constant, swap them.
Thus the following special case checks need only
@@ -10380,10 +10378,6 @@ expand_expr_real_2 (sepops ops, rtx target, machine_mode tmode,
return temp;
}
- case VEC_WIDEN_PLUS_HI_EXPR:
- case VEC_WIDEN_PLUS_LO_EXPR:
- case VEC_WIDEN_MINUS_HI_EXPR:
- case VEC_WIDEN_MINUS_LO_EXPR:
case VEC_WIDEN_MULT_HI_EXPR:
case VEC_WIDEN_MULT_LO_EXPR:
case VEC_WIDEN_MULT_EVEN_EXPR:
diff --git a/gcc/gimple-pretty-print.cc b/gcc/gimple-pretty-print.cc
index 300e9d7ed1e7be73f30875e08c461a8880c3134e..d903826894e7f0dfd34dc0caad92eea3caa45e05 100644
--- a/gcc/gimple-pretty-print.cc
+++ b/gcc/gimple-pretty-print.cc
@@ -459,10 +459,6 @@ dump_binary_rhs (pretty_printer *buffer, const gassign *gs, int spc,
case VEC_PACK_FLOAT_EXPR:
case VEC_WIDEN_LSHIFT_HI_EXPR:
case VEC_WIDEN_LSHIFT_LO_EXPR:
- case VEC_WIDEN_PLUS_HI_EXPR:
- case VEC_WIDEN_PLUS_LO_EXPR:
- case VEC_WIDEN_MINUS_HI_EXPR:
- case VEC_WIDEN_MINUS_LO_EXPR:
case VEC_SERIES_EXPR:
for (p = get_tree_code_name (code); *p; p++)
pp_character (buffer, TOUPPER (*p));
diff --git a/gcc/gimple-range-op.cc b/gcc/gimple-range-op.cc
index 4ca32a7b5d52f8426b09d1446a336650e143b41f..5ae7f7596c6fc6f901e4e47ae44f00185f4602b2 100644
--- a/gcc/gimple-range-op.cc
+++ b/gcc/gimple-range-op.cc
@@ -797,12 +797,6 @@ gimple_range_op_handler::maybe_non_standard ()
if (gimple_code (m_stmt) == GIMPLE_ASSIGN)
switch (gimple_assign_rhs_code (m_stmt))
{
- case WIDEN_PLUS_EXPR:
- {
- signed_op = ptr_op_widen_plus_signed;
- unsigned_op = ptr_op_widen_plus_unsigned;
- }
- gcc_fallthrough ();
case WIDEN_MULT_EXPR:
{
m_valid = false;
diff --git a/gcc/optabs-tree.cc b/gcc/optabs-tree.cc
index 8010046c6a8b3e809c989ddef7a06ddaa68ae32a..ee1aa8c9676ee9c67edbf403e6295da391826a62 100644
--- a/gcc/optabs-tree.cc
+++ b/gcc/optabs-tree.cc
@@ -190,22 +190,6 @@ optab_for_tree_code (enum tree_code code, const_tree type,
return (TYPE_UNSIGNED (type)
? vec_widen_ushiftl_lo_optab : vec_widen_sshiftl_lo_optab);
- case VEC_WIDEN_PLUS_LO_EXPR:
- return (TYPE_UNSIGNED (type)
- ? vec_widen_uaddl_lo_optab : vec_widen_saddl_lo_optab);
-
- case VEC_WIDEN_PLUS_HI_EXPR:
- return (TYPE_UNSIGNED (type)
- ? vec_widen_uaddl_hi_optab : vec_widen_saddl_hi_optab);
-
- case VEC_WIDEN_MINUS_LO_EXPR:
- return (TYPE_UNSIGNED (type)
- ? vec_widen_usubl_lo_optab : vec_widen_ssubl_lo_optab);
-
- case VEC_WIDEN_MINUS_HI_EXPR:
- return (TYPE_UNSIGNED (type)
- ? vec_widen_usubl_hi_optab : vec_widen_ssubl_hi_optab);
-
case VEC_UNPACK_HI_EXPR:
return (TYPE_UNSIGNED (type)
? vec_unpacku_hi_optab : vec_unpacks_hi_optab);
@@ -312,8 +296,6 @@ optab_for_tree_code (enum tree_code code, const_tree type,
'hi'/'lo' pair using codes such as VEC_WIDEN_MINUS_HI/LO.
Supported widening operations:
- WIDEN_MINUS_EXPR
- WIDEN_PLUS_EXPR
WIDEN_MULT_EXPR
WIDEN_LSHIFT_EXPR
@@ -345,12 +327,6 @@ supportable_half_widening_operation (enum tree_code code, tree vectype_out,
case WIDEN_LSHIFT_EXPR:
*code1 = LSHIFT_EXPR;
break;
- case WIDEN_MINUS_EXPR:
- *code1 = MINUS_EXPR;
- break;
- case WIDEN_PLUS_EXPR:
- *code1 = PLUS_EXPR;
- break;
case WIDEN_MULT_EXPR:
*code1 = MULT_EXPR;
break;
diff --git a/gcc/tree-cfg.cc b/gcc/tree-cfg.cc
index a9fcc7fd050f871437ef336ecfb8d6cc81280ee0..f80cd1465df83b5540492e619e56b9af249e9f31 100644
--- a/gcc/tree-cfg.cc
+++ b/gcc/tree-cfg.cc
@@ -4017,8 +4017,6 @@ verify_gimple_assign_binary (gassign *stmt)
return false;
}
- case WIDEN_PLUS_EXPR:
- case WIDEN_MINUS_EXPR:
case PLUS_EXPR:
case MINUS_EXPR:
{
@@ -4139,10 +4137,6 @@ verify_gimple_assign_binary (gassign *stmt)
return false;
}
- case VEC_WIDEN_MINUS_HI_EXPR:
- case VEC_WIDEN_MINUS_LO_EXPR:
- case VEC_WIDEN_PLUS_HI_EXPR:
- case VEC_WIDEN_PLUS_LO_EXPR:
case VEC_WIDEN_MULT_HI_EXPR:
case VEC_WIDEN_MULT_LO_EXPR:
case VEC_WIDEN_MULT_EVEN_EXPR:
diff --git a/gcc/tree-inline.cc b/gcc/tree-inline.cc
index c702f0032a19203a7c536a01c1e7f47fc7b77add..6e5fd45a0c2435109dd3d50e8fc8e1d4969a1fd0 100644
--- a/gcc/tree-inline.cc
+++ b/gcc/tree-inline.cc
@@ -4273,8 +4273,6 @@ estimate_operator_cost (enum tree_code code, eni_weights *weights,
case REALIGN_LOAD_EXPR:
- case WIDEN_PLUS_EXPR:
- case WIDEN_MINUS_EXPR:
case WIDEN_SUM_EXPR:
case WIDEN_MULT_EXPR:
case DOT_PROD_EXPR:
@@ -4283,10 +4281,6 @@ estimate_operator_cost (enum tree_code code, eni_weights *weights,
case WIDEN_MULT_MINUS_EXPR:
case WIDEN_LSHIFT_EXPR:
- case VEC_WIDEN_PLUS_HI_EXPR:
- case VEC_WIDEN_PLUS_LO_EXPR:
- case VEC_WIDEN_MINUS_HI_EXPR:
- case VEC_WIDEN_MINUS_LO_EXPR:
case VEC_WIDEN_MULT_HI_EXPR:
case VEC_WIDEN_MULT_LO_EXPR:
case VEC_WIDEN_MULT_EVEN_EXPR:
diff --git a/gcc/tree-pretty-print.cc b/gcc/tree-pretty-print.cc
index 7947f9647a15110b52d195643ad7d28ee32d4236..9941d8bf80535a98e647b8928619a6bf08bc434c 100644
--- a/gcc/tree-pretty-print.cc
+++ b/gcc/tree-pretty-print.cc
@@ -2874,8 +2874,6 @@ dump_generic_node (pretty_printer *pp, tree node, int spc, dump_flags_t flags,
break;
/* Binary arithmetic and logic expressions. */
- case WIDEN_PLUS_EXPR:
- case WIDEN_MINUS_EXPR:
case WIDEN_SUM_EXPR:
case WIDEN_MULT_EXPR:
case MULT_EXPR:
@@ -3831,10 +3829,6 @@ dump_generic_node (pretty_printer *pp, tree node, int spc, dump_flags_t flags,
case VEC_SERIES_EXPR:
case VEC_WIDEN_MULT_HI_EXPR:
case VEC_WIDEN_MULT_LO_EXPR:
- case VEC_WIDEN_PLUS_HI_EXPR:
- case VEC_WIDEN_PLUS_LO_EXPR:
- case VEC_WIDEN_MINUS_HI_EXPR:
- case VEC_WIDEN_MINUS_LO_EXPR:
case VEC_WIDEN_MULT_EVEN_EXPR:
case VEC_WIDEN_MULT_ODD_EXPR:
case VEC_WIDEN_LSHIFT_HI_EXPR:
@@ -4352,12 +4346,6 @@ op_symbol_code (enum tree_code code)
case WIDEN_LSHIFT_EXPR:
return "w<<";
- case WIDEN_PLUS_EXPR:
- return "w+";
-
- case WIDEN_MINUS_EXPR:
- return "w-";
-
case POINTER_PLUS_EXPR:
return "+";
diff --git a/gcc/tree-vect-data-refs.cc b/gcc/tree-vect-data-refs.cc
index 8daf7bd7dd34d043b1d7b4cba1779f0ecf9f520a..213a3899a6c145bb057cd118bec1df7a05728aef 100644
--- a/gcc/tree-vect-data-refs.cc
+++ b/gcc/tree-vect-data-refs.cc
@@ -136,8 +136,6 @@ vect_get_smallest_scalar_type (stmt_vec_info stmt_info, tree scalar_type)
|| gimple_assign_rhs_code (assign) == WIDEN_SUM_EXPR
|| gimple_assign_rhs_code (assign) == WIDEN_MULT_EXPR
|| gimple_assign_rhs_code (assign) == WIDEN_LSHIFT_EXPR
- || gimple_assign_rhs_code (assign) == WIDEN_PLUS_EXPR
- || gimple_assign_rhs_code (assign) == WIDEN_MINUS_EXPR
|| gimple_assign_rhs_code (assign) == FLOAT_EXPR)
{
tree rhs_type = TREE_TYPE (gimple_assign_rhs1 (assign));
diff --git a/gcc/tree-vect-generic.cc b/gcc/tree-vect-generic.cc
index 445da53292e9d1d2db62ca962fc017bb0e6c9bbe..342ffc5fa7f3b8f37e6bd4658d2f1fccf1d2c7fa 100644
--- a/gcc/tree-vect-generic.cc
+++ b/gcc/tree-vect-generic.cc
@@ -2227,10 +2227,6 @@ expand_vector_operations_1 (gimple_stmt_iterator *gsi,
arguments, not the widened result. VEC_UNPACK_FLOAT_*_EXPR is
calculated in the same way above. */
if (code == WIDEN_SUM_EXPR
- || code == VEC_WIDEN_PLUS_HI_EXPR
- || code == VEC_WIDEN_PLUS_LO_EXPR
- || code == VEC_WIDEN_MINUS_HI_EXPR
- || code == VEC_WIDEN_MINUS_LO_EXPR
|| code == VEC_WIDEN_MULT_HI_EXPR
|| code == VEC_WIDEN_MULT_LO_EXPR
|| code == VEC_WIDEN_MULT_EVEN_EXPR
diff --git a/gcc/tree-vect-patterns.cc b/gcc/tree-vect-patterns.cc
index 1a514461b2ca416f45a5fa9abe417980d33ef4df..13c69133d7ae565cf0334390cb0c303c89f98ac8 100644
--- a/gcc/tree-vect-patterns.cc
+++ b/gcc/tree-vect-patterns.cc
@@ -561,21 +561,35 @@ vect_joust_widened_type (tree type, tree new_type, tree *common_type)
static unsigned int
vect_widened_op_tree (vec_info *vinfo, stmt_vec_info stmt_info, tree_code code,
- tree_code widened_code, bool shift_p,
+ code_helper widened_code, bool shift_p,
unsigned int max_nops,
vect_unpromoted_value *unprom, tree *common_type,
enum optab_subtype *subtype = NULL)
{
/* Check for an integer operation with the right code. */
- gassign *assign = dyn_cast <gassign *> (stmt_info->stmt);
- if (!assign)
+ gimple* stmt = stmt_info->stmt;
+ if (!(is_gimple_assign (stmt) || is_gimple_call (stmt)))
return 0;
- tree_code rhs_code = gimple_assign_rhs_code (assign);
- if (rhs_code != code && rhs_code != widened_code)
+ code_helper rhs_code;
+ if (is_gimple_assign (stmt))
+ {
+ rhs_code = gimple_assign_rhs_code (stmt);
+ if (rhs_code.safe_as_tree_code () != code
+ && rhs_code.get_rep () != widened_code.get_rep ())
+ return 0;
+ }
+ else if (is_gimple_call (stmt))
+ {
+ rhs_code = gimple_call_combined_fn (stmt);
+ if (rhs_code.get_rep () != widened_code.get_rep ())
+ return 0;
+ }
+ else
return 0;
- tree type = TREE_TYPE (gimple_assign_lhs (assign));
+ tree lhs = gimple_get_lhs (stmt);
+ tree type = TREE_TYPE (lhs);
if (!INTEGRAL_TYPE_P (type))
return 0;
@@ -588,7 +602,7 @@ vect_widened_op_tree (vec_info *vinfo, stmt_vec_info stmt_info, tree_code code,
{
vect_unpromoted_value *this_unprom = &unprom[next_op];
unsigned int nops = 1;
- tree op = gimple_op (assign, i + 1);
+ tree op = gimple_arg (stmt, i);
if (i == 1 && TREE_CODE (op) == INTEGER_CST)
{
/* We already have a common type from earlier operands.
@@ -1342,8 +1356,9 @@ vect_recog_sad_pattern (vec_info *vinfo,
/* FORNOW. Can continue analyzing the def-use chain when this stmt in a phi
inside the loop (in case we are analyzing an outer-loop). */
vect_unpromoted_value unprom[2];
- if (!vect_widened_op_tree (vinfo, diff_stmt_vinfo, MINUS_EXPR, WIDEN_MINUS_EXPR,
- false, 2, unprom, &half_type))
+ if (!vect_widened_op_tree (vinfo, diff_stmt_vinfo, MINUS_EXPR,
+ CFN_VEC_WIDEN_MINUS, false, 2, unprom,
+ &half_type))
return NULL;
vect_pattern_detected ("vect_recog_sad_pattern", last_stmt);
@@ -2696,9 +2711,10 @@ vect_recog_average_pattern (vec_info *vinfo,
internal_fn ifn = IFN_AVG_FLOOR;
vect_unpromoted_value unprom[3];
tree new_type;
+ enum optab_subtype subtype;
unsigned int nops = vect_widened_op_tree (vinfo, plus_stmt_info, PLUS_EXPR,
- WIDEN_PLUS_EXPR, false, 3,
- unprom, &new_type);
+ CFN_VEC_WIDEN_PLUS, false, 3,
+ unprom, &new_type, &subtype);
if (nops == 0)
return NULL;
if (nops == 3)
diff --git a/gcc/tree-vect-stmts.cc b/gcc/tree-vect-stmts.cc
index f4806073f48d4dedea3ac9bd855792b152d78919..38f4680d45ab80e8f86327327c13667d96bc5bea 100644
--- a/gcc/tree-vect-stmts.cc
+++ b/gcc/tree-vect-stmts.cc
@@ -5032,9 +5032,7 @@ vectorizable_conversion (vec_info *vinfo,
else
return false;
- bool widen_arith = (code == WIDEN_PLUS_EXPR
- || code == WIDEN_MINUS_EXPR
- || code == WIDEN_MULT_EXPR
+ bool widen_arith = (code == WIDEN_MULT_EXPR
|| code == WIDEN_LSHIFT_EXPR
|| code == IFN_VEC_WIDEN_PLUS
|| code == IFN_VEC_WIDEN_MINUS);
@@ -5086,8 +5084,6 @@ vectorizable_conversion (vec_info *vinfo,
{
gcc_assert (code == WIDEN_MULT_EXPR
|| code == WIDEN_LSHIFT_EXPR
- || code == WIDEN_PLUS_EXPR
- || code == WIDEN_MINUS_EXPR
|| code == IFN_VEC_WIDEN_PLUS
|| code == IFN_VEC_WIDEN_MINUS);
@@ -12211,7 +12207,7 @@ supportable_widening_operation (vec_info *vinfo,
class loop *vect_loop = NULL;
machine_mode vec_mode;
enum insn_code icode1, icode2;
- optab optab1, optab2;
+ optab optab1 = unknown_optab, optab2 = unknown_optab;
tree vectype = vectype_in;
tree wide_vectype = vectype_out;
code_helper c1 = MAX_TREE_CODES, c2 = MAX_TREE_CODES;
@@ -12305,16 +12301,6 @@ supportable_widening_operation (vec_info *vinfo,
c2 = VEC_WIDEN_LSHIFT_HI_EXPR;
break;
- case WIDEN_PLUS_EXPR:
- c1 = VEC_WIDEN_PLUS_LO_EXPR;
- c2 = VEC_WIDEN_PLUS_HI_EXPR;
- break;
-
- case WIDEN_MINUS_EXPR:
- c1 = VEC_WIDEN_MINUS_LO_EXPR;
- c2 = VEC_WIDEN_MINUS_HI_EXPR;
- break;
-
CASE_CONVERT:
c1 = VEC_UNPACK_LO_EXPR;
c2 = VEC_UNPACK_HI_EXPR;
diff --git a/gcc/tree.def b/gcc/tree.def
index ee02754354f015a16737c7e879d89c3e3be0d5aa..a58e608a90078818a7ade9d1173ac7ec84c48c7a 100644
--- a/gcc/tree.def
+++ b/gcc/tree.def
@@ -1374,15 +1374,16 @@ DEFTREECODE (DOT_PROD_EXPR, "dot_prod_expr", tcc_expression, 3)
DEFTREECODE (WIDEN_SUM_EXPR, "widen_sum_expr", tcc_binary, 2)
/* Widening sad (sum of absolute differences).
- The first two arguments are of type t1 which should be integer.
- The third argument and the result are of type t2, such that t2 is at least
- twice the size of t1. Like DOT_PROD_EXPR, SAD_EXPR (arg1,arg2,arg3) is
+ The first two arguments are of type t1 which should be a vector of integers.
+ The third argument and the result are of type t2, such that the size of
+ the elements of t2 is at least twice the size of the elements of t1.
+ Like DOT_PROD_EXPR, SAD_EXPR (arg1,arg2,arg3) is
equivalent to:
- tmp = WIDEN_MINUS_EXPR (arg1, arg2)
+ tmp = IFN_VEC_WIDEN_MINUS_EXPR (arg1, arg2)
tmp2 = ABS_EXPR (tmp)
arg3 = PLUS_EXPR (tmp2, arg3)
or:
- tmp = WIDEN_MINUS_EXPR (arg1, arg2)
+ tmp = IFN_VEC_WIDEN_MINUS_EXPR (arg1, arg2)
tmp2 = ABS_EXPR (tmp)
arg3 = WIDEN_SUM_EXPR (tmp2, arg3)
*/
@@ -1421,8 +1422,6 @@ DEFTREECODE (WIDEN_MULT_MINUS_EXPR, "widen_mult_minus_expr", tcc_expression, 3)
the first argument from type t1 to type t2, and then shifting it
by the second argument. */
DEFTREECODE (WIDEN_LSHIFT_EXPR, "widen_lshift_expr", tcc_binary, 2)
-DEFTREECODE (WIDEN_PLUS_EXPR, "widen_plus_expr", tcc_binary, 2)
-DEFTREECODE (WIDEN_MINUS_EXPR, "widen_minus_expr", tcc_binary, 2)
/* Widening vector multiplication.
The two operands are vectors with N elements of size S. Multiplying the
@@ -1487,10 +1486,6 @@ DEFTREECODE (VEC_PACK_FLOAT_EXPR, "vec_pack_float_expr", tcc_binary, 2)
*/
DEFTREECODE (VEC_WIDEN_LSHIFT_HI_EXPR, "widen_lshift_hi_expr", tcc_binary, 2)
DEFTREECODE (VEC_WIDEN_LSHIFT_LO_EXPR, "widen_lshift_lo_expr", tcc_binary, 2)
-DEFTREECODE (VEC_WIDEN_PLUS_HI_EXPR, "widen_plus_hi_expr", tcc_binary, 2)
-DEFTREECODE (VEC_WIDEN_PLUS_LO_EXPR, "widen_plus_lo_expr", tcc_binary, 2)
-DEFTREECODE (VEC_WIDEN_MINUS_HI_EXPR, "widen_minus_hi_expr", tcc_binary, 2)
-DEFTREECODE (VEC_WIDEN_MINUS_LO_EXPR, "widen_minus_lo_expr", tcc_binary, 2)
/* PREDICT_EXPR. Specify hint for branch prediction. The
PREDICT_EXPR_PREDICTOR specify predictor and PREDICT_EXPR_OUTCOME the
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [ping][vect-patterns] Refactor widen_plus/widen_minus as internal_fns
2023-03-17 10:14 ` Andre Vieira (lists)
@ 2023-03-17 11:52 ` Richard Biener
2023-04-20 13:23 ` Andre Vieira (lists)
0 siblings, 1 reply; 22+ messages in thread
From: Richard Biener @ 2023-03-17 11:52 UTC (permalink / raw)
To: Andre Vieira (lists); +Cc: Richard Sandiford, gcc-patches
On Fri, 17 Mar 2023, Andre Vieira (lists) wrote:
> Hi Richard,
>
> I'm only picking this up now. Just going through your earlier comments and
> stuff and I noticed we didn't address the situation with the gimple::build. Do
> you want me to add overloaded static member functions to cover all
> gimple_build_* functions, or just create one to replace vect_gimple_build and
> we create them as needed? It's more work but I think adding them all would be
> better. I'd even argue that it would be nice to replace the old ones with the
> new ones, but I can imagine you might not want that as it makes backporting
> and the likes a bit annoying...
>
> Let me know what you prefer, I'll go work on your latest comments too.
I think the series was resolved and I approved it. As for
vect_gimple_build the better way forward would be to use
gimple_build () as existing but add a vect_finish_stmt_* handling
a gimple_seq.
Richard.
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [ping][vect-patterns] Refactor widen_plus/widen_minus as internal_fns
2022-07-12 12:32 ` Richard Biener
@ 2023-03-17 10:14 ` Andre Vieira (lists)
2023-03-17 11:52 ` Richard Biener
0 siblings, 1 reply; 22+ messages in thread
From: Andre Vieira (lists) @ 2023-03-17 10:14 UTC (permalink / raw)
To: Richard Biener; +Cc: Richard Sandiford, gcc-patches
Hi Richard,
I'm only picking this up now. Just going through your earlier comments
and stuff and I noticed we didn't address the situation with the
gimple::build. Do you want me to add overloaded static member functions
to cover all gimple_build_* functions, or just create one to replace
vect_gimple_build and we create them as needed? It's more work but I
think adding them all would be better. I'd even argue that it would be
nice to replace the old ones with the new ones, but I can imagine you
might not want that as it makes backporting and the likes a bit annoying...
Let me know what you prefer, I'll go work on your latest comments too.
Cheers,
Andre
^ permalink raw reply [flat|nested] 22+ messages in thread
* RE: [ping][vect-patterns] Refactor widen_plus/widen_minus as internal_fns
2022-06-30 13:20 ` Joel Hutton
@ 2022-07-12 12:32 ` Richard Biener
2023-03-17 10:14 ` Andre Vieira (lists)
0 siblings, 1 reply; 22+ messages in thread
From: Richard Biener @ 2022-07-12 12:32 UTC (permalink / raw)
To: Joel Hutton; +Cc: Richard Sandiford, gcc-patches, Andre Simoes Dias Vieira
On Thu, 30 Jun 2022, Joel Hutton wrote:
> > We can go with a private vect_gimple_build function until we sort out the API
> > issue to unblock Tamar (I'll reply to Richards reply with further thoughts on
> > this)
> >
>
> Done.
>
> > > Similarly are you ok with the use of gimple_extract_op? I would lean
> > towards using it as it is cleaner, but I don't have strong feelings.
> >
> > I don't like using gimple_extract_op here, I think I outlined a variant that is
> > even shorter.
> >
>
> Done.
>
> Updated patches attached, bootstrapped and regression tested on aarch64.
>
> Tomorrow is my last working day at Arm, so it will likely be Andre that commits this/addresses any further comments.
First sorry for the (repeated) delays.
In the first patch I still see ECF_WIDEN, I don't like that, we
use things like associative_binary_fn_p so for widening internal
functions similar predicates should be used.
In the second patch you add vec_widen_{add,sub} optabs
+OPTAB_CD(vec_widen_add_optab, "add$a$b3")
+OPTAB_CD(vec_widen_sub_optab, "sub$a$b3")
but a) the names are that of regular adds which is at least confusing
(if not wrong), b) there's no documentation for them in md.texi which,
c) doesn't explain why they are necessary when we have
vec_widen_[su]{add,sub}l_optab
+ internal_fn ifn = as_internal_fn (code.safe_as_fn_code ());
asks for safe_as_internal_fn () (just complete the API, also with
safe_as_builtin_fn)
+ internal_fn lo, hi;
+ lookup_multi_internal_fn (ifn, &lo, &hi);
+ *code1 = as_combined_fn (lo);
+ *code2 = as_combined_fn (hi);
in fact this probably shows that the guarding condition should
be if (code.is_internal_fn ()) instead of if (code.is_fn_code ()).
+ optab1 = lookup_multi_ifn_optab (lo, !TYPE_UNSIGNED (vectype));
+ optab2 = lookup_multi_ifn_optab (hi, !TYPE_UNSIGNED (vectype));
this shows the two lookup_ APIs are inconsistent in having two vs. one
output, please make them consistent. I'd say give
lookup_multi_internal_fn a enum { LO, HI } argument, returning the
result. Given VEC_WIDEN_MULT has HI, LO, EVEN and ODD variants
that sounds more future proof.
The internal_fn stuff could probably get a 2nd eye from Richard.
In the third patch I see unrelated and wrong changes like
/* Check that the DR_INITs are compile-time constants. */
- if (!tree_fits_shwi_p (DR_INIT (dra))
- || !tree_fits_shwi_p (DR_INIT (drb)))
+ if (TREE_CODE (DR_INIT (dra)) != INTEGER_CST
+ || TREE_CODE (DR_INIT (drb)) != INTEGER_CST)
break;
please strip the patch down to relevant changes.
- tree op = gimple_op (assign, i + 1);
+ tree op;
+ if (is_gimple_assign (stmt))
+ op = gimple_op (stmt, i + 1);
+ else
+ op = gimple_call_arg (stmt, i);
somebody added gimple_arg which can be used here doing
op = gimple_arg (stmt, i);
+ tree lhs = is_gimple_assign (stmt) ? gimple_assign_lhs (stmt):
+ gimple_call_lhs (stmt);
tree lhs = gimple_get_lhs (stmt);
/* Check for an integer operation with the right code. */
- gassign *assign = dyn_cast <gassign *> (stmt_info->stmt);
- if (!assign)
+ gimple* stmt = stmt_info->stmt;
+ if (!(is_gimple_assign (stmt) || is_gimple_call (stmt)))
return 0;
- tree_code rhs_code = gimple_assign_rhs_code (assign);
- if (rhs_code != code && rhs_code != widened_code)
+ code_helper rhs_code;
+ if (is_gimple_assign (stmt))
+ rhs_code = gimple_assign_rhs_code (stmt);
+ else
+ rhs_code = gimple_call_combined_fn (stmt);
+
+ if (rhs_code.safe_as_tree_code () != code
+ && rhs_code.get_rep () != widened_code.get_rep ())
return 0;
that's probably better refactored as
if (is_gimple_assign (stmt))
{
if (code check)
return 0;
}
else if (is_gimple_call (..))
{
..
}
else
return 0;
otherwise the last patch looks reasonable.
Richard.
^ permalink raw reply [flat|nested] 22+ messages in thread
* RE: [ping][vect-patterns] Refactor widen_plus/widen_minus as internal_fns
2022-06-13 9:02 ` Richard Biener
@ 2022-06-30 13:20 ` Joel Hutton
2022-07-12 12:32 ` Richard Biener
0 siblings, 1 reply; 22+ messages in thread
From: Joel Hutton @ 2022-06-30 13:20 UTC (permalink / raw)
To: Richard Biener; +Cc: Richard Sandiford, gcc-patches, Andre Simoes Dias Vieira
[-- Attachment #1: Type: text/plain, Size: 647 bytes --]
> We can go with a private vect_gimple_build function until we sort out the API
> issue to unblock Tamar (I'll reply to Richards reply with further thoughts on
> this)
>
Done.
> > Similarly are you ok with the use of gimple_extract_op? I would lean
> towards using it as it is cleaner, but I don't have strong feelings.
>
> I don't like using gimple_extract_op here, I think I outlined a variant that is
> even shorter.
>
Done.
Updated patches attached, bootstrapped and regression tested on aarch64.
Tomorrow is my last working day at Arm, so it will likely be Andre that commits this/addresses any further comments.
[-- Attachment #2: 0001-Refactor-to-allow-internal_fn-s.patch --]
[-- Type: application/octet-stream, Size: 23390 bytes --]
From f1321c617838e94044cbae357a63db002fbd3edb Mon Sep 17 00:00:00 2001
From: Joel Hutton <joel.hutton@arm.com>
Date: Wed, 25 Aug 2021 14:31:15 +0100
Subject: [PATCH 1/3] Refactor to allow internal_fn's
Hi all,
This refactor allows widening patterns (such as widen_plus/widen_minus) to be represented as
either internal_fns or tree_codes.
[vect-patterns] Refactor as internal_fn's
Refactor vect-patterns to allow patterns to be internal_fns starting
with widening_plus/minus patterns
gcc/ChangeLog:
* tree-core.h (ECF_WIDEN): New flag.
* gimple-match.h (class code_helper): * tree-core.h (ECF_WIDEN): Flag to mark internal_fn as widening.
* tree-vect-patterns.cc (vect_recog_widen_op_pattern): Refactor to
use code_helper.
(vect_gimple_build): New function.
* tree-vect-stmts.cc (vect_gen_widened_results_half): Refactor to
use code_helper.
(vect_create_vectorized_promotion_stmts): Refactor to use
code_helper.
(vectorizable_conversion): Refactor to use code_helper.
gimple_call or gimple_assign.
(supportable_widening_operation): Refactor to use code_helper.
(supportable_narrowing_operation): Refactor to use code_helper.
* tree-vectorizer.h (supportable_widening_operation): Change
prototype to use code_helper.
(supportable_narrowing_operation): change prototype to use
code_helper.
(vect_gimple_build): New function prototype.
* tree.h (code_helper::safe_as_tree_code): New function.
helper functions.
(code_helper::safe_as_fn_code): New function.
---
gcc/tree-core.h | 3 +
gcc/tree-vect-patterns.cc | 34 ++++++-
gcc/tree-vect-stmts.cc | 208 +++++++++++++++++++++++++-------------
gcc/tree-vectorizer.h | 14 +--
gcc/tree.h | 13 +++
5 files changed, 189 insertions(+), 83 deletions(-)
diff --git a/gcc/tree-core.h b/gcc/tree-core.h
index ab5fa01e5cb5fb56c1964b93b014ed55a4aa704a..cff6211080bced0bffb39e98039a6550897acf77 100644
--- a/gcc/tree-core.h
+++ b/gcc/tree-core.h
@@ -96,6 +96,9 @@ struct die_struct;
/* Nonzero if this is a cold function. */
#define ECF_COLD (1 << 15)
+/* Nonzero if this is a widening function. */
+#define ECF_WIDEN (1 << 16)
+
/* Call argument flags. */
/* Nonzero if the argument is not used by the function. */
diff --git a/gcc/tree-vect-patterns.cc b/gcc/tree-vect-patterns.cc
index 8f624863971392c891fde7278949c8818f646576..d892158f024fc045b897aebe76f2e2b66211cf83 100644
--- a/gcc/tree-vect-patterns.cc
+++ b/gcc/tree-vect-patterns.cc
@@ -25,6 +25,8 @@ along with GCC; see the file COPYING3. If not see
#include "rtl.h"
#include "tree.h"
#include "gimple.h"
+#include "gimple-iterator.h"
+#include "gimple-fold.h"
#include "ssa.h"
#include "expmed.h"
#include "optabs-tree.h"
@@ -1348,7 +1350,7 @@ vect_recog_sad_pattern (vec_info *vinfo,
static gimple *
vect_recog_widen_op_pattern (vec_info *vinfo,
stmt_vec_info last_stmt_info, tree *type_out,
- tree_code orig_code, tree_code wide_code,
+ tree_code orig_code, code_helper wide_code,
bool shift_p, const char *name)
{
gimple *last_stmt = last_stmt_info->stmt;
@@ -1391,7 +1393,7 @@ vect_recog_widen_op_pattern (vec_info *vinfo,
vecctype = get_vectype_for_scalar_type (vinfo, ctype);
}
- enum tree_code dummy_code;
+ code_helper dummy_code;
int dummy_int;
auto_vec<tree> dummy_vec;
if (!vectype
@@ -1412,8 +1414,7 @@ vect_recog_widen_op_pattern (vec_info *vinfo,
2, oprnd, half_type, unprom, vectype);
tree var = vect_recog_temp_ssa_var (itype, NULL);
- gimple *pattern_stmt = gimple_build_assign (var, wide_code,
- oprnd[0], oprnd[1]);
+ gimple *pattern_stmt = vect_gimple_build (var, wide_code, oprnd[0], oprnd[1]);
if (vecctype != vecitype)
pattern_stmt = vect_convert_output (vinfo, last_stmt_info, ctype,
@@ -5968,3 +5969,28 @@ vect_pattern_recog (vec_info *vinfo)
/* After this no more add_stmt calls are allowed. */
vinfo->stmt_vec_info_ro = true;
}
+
+/* Build a GIMPLE_ASSIGN or GIMPLE_CALL with the tree_code,
+ or internal_fn contained in ch, respectively. */
+gimple *
+vect_gimple_build (tree lhs, code_helper ch, tree op0, tree op1)
+{
+ if (op0 == NULL_TREE)
+ return NULL;
+ if (ch.is_tree_code ())
+ return op1 == NULL_TREE ? gimple_build_assign (lhs, ch.safe_as_tree_code (),
+ op0) :
+ gimple_build_assign (lhs, ch.safe_as_tree_code (),
+ op0, op1);
+ else
+ {
+ internal_fn fn = as_internal_fn (ch.safe_as_fn_code ());
+ gimple* stmt;
+ if (op1 == NULL_TREE)
+ stmt = gimple_build_call_internal (fn, 1, op0);
+ else
+ stmt = gimple_build_call_internal (fn, 2, op0, op1);
+ gimple_call_set_lhs (stmt, lhs);
+ return stmt;
+ }
+}
diff --git a/gcc/tree-vect-stmts.cc b/gcc/tree-vect-stmts.cc
index 346d8ce280437e00bfeb19a4b4adc59eb96207f9..d6aabb873c86ab8ff0bae41c7f6c3bad34d583c5 100644
--- a/gcc/tree-vect-stmts.cc
+++ b/gcc/tree-vect-stmts.cc
@@ -4636,7 +4636,7 @@ vectorizable_simd_clone_call (vec_info *vinfo, stmt_vec_info stmt_info,
STMT_INFO is the original scalar stmt that we are vectorizing. */
static gimple *
-vect_gen_widened_results_half (vec_info *vinfo, enum tree_code code,
+vect_gen_widened_results_half (vec_info *vinfo, code_helper ch,
tree vec_oprnd0, tree vec_oprnd1, int op_type,
tree vec_dest, gimple_stmt_iterator *gsi,
stmt_vec_info stmt_info)
@@ -4645,12 +4645,11 @@ vect_gen_widened_results_half (vec_info *vinfo, enum tree_code code,
tree new_temp;
/* Generate half of the widened result: */
- gcc_assert (op_type == TREE_CODE_LENGTH (code));
if (op_type != binary_op)
vec_oprnd1 = NULL;
- new_stmt = gimple_build_assign (vec_dest, code, vec_oprnd0, vec_oprnd1);
+ new_stmt = vect_gimple_build (vec_dest, ch, vec_oprnd0, vec_oprnd1);
new_temp = make_ssa_name (vec_dest, new_stmt);
- gimple_assign_set_lhs (new_stmt, new_temp);
+ gimple_set_lhs (new_stmt, new_temp);
vect_finish_stmt_generation (vinfo, stmt_info, new_stmt, gsi);
return new_stmt;
@@ -4729,8 +4728,8 @@ vect_create_vectorized_promotion_stmts (vec_info *vinfo,
vec<tree> *vec_oprnds1,
stmt_vec_info stmt_info, tree vec_dest,
gimple_stmt_iterator *gsi,
- enum tree_code code1,
- enum tree_code code2, int op_type)
+ code_helper ch1,
+ code_helper ch2, int op_type)
{
int i;
tree vop0, vop1, new_tmp1, new_tmp2;
@@ -4746,10 +4745,10 @@ vect_create_vectorized_promotion_stmts (vec_info *vinfo,
vop1 = NULL_TREE;
/* Generate the two halves of promotion operation. */
- new_stmt1 = vect_gen_widened_results_half (vinfo, code1, vop0, vop1,
+ new_stmt1 = vect_gen_widened_results_half (vinfo, ch1, vop0, vop1,
op_type, vec_dest, gsi,
stmt_info);
- new_stmt2 = vect_gen_widened_results_half (vinfo, code2, vop0, vop1,
+ new_stmt2 = vect_gen_widened_results_half (vinfo, ch2, vop0, vop1,
op_type, vec_dest, gsi,
stmt_info);
if (is_gimple_call (new_stmt1))
@@ -4846,8 +4845,9 @@ vectorizable_conversion (vec_info *vinfo,
tree scalar_dest;
tree op0, op1 = NULL_TREE;
loop_vec_info loop_vinfo = dyn_cast <loop_vec_info> (vinfo);
- enum tree_code code, code1 = ERROR_MARK, code2 = ERROR_MARK;
- enum tree_code codecvt1 = ERROR_MARK, codecvt2 = ERROR_MARK;
+ tree_code tc1;
+ code_helper code, code1, code2;
+ code_helper codecvt1 = ERROR_MARK, codecvt2 = ERROR_MARK;
tree new_temp;
enum vect_def_type dt[2] = {vect_unknown_def_type, vect_unknown_def_type};
int ndts = 2;
@@ -4876,31 +4876,43 @@ vectorizable_conversion (vec_info *vinfo,
&& ! vec_stmt)
return false;
- gassign *stmt = dyn_cast <gassign *> (stmt_info->stmt);
- if (!stmt)
+ gimple* stmt = stmt_info->stmt;
+ if (!(is_gimple_assign (stmt) || is_gimple_call (stmt)))
return false;
- if (TREE_CODE (gimple_assign_lhs (stmt)) != SSA_NAME)
+ if (gimple_get_lhs (stmt) == NULL_TREE
+ || TREE_CODE (gimple_get_lhs (stmt)) != SSA_NAME)
return false;
- code = gimple_assign_rhs_code (stmt);
- if (!CONVERT_EXPR_CODE_P (code)
- && code != FIX_TRUNC_EXPR
- && code != FLOAT_EXPR
- && code != WIDEN_PLUS_EXPR
- && code != WIDEN_MINUS_EXPR
- && code != WIDEN_MULT_EXPR
- && code != WIDEN_LSHIFT_EXPR)
+ if (TREE_CODE (gimple_get_lhs (stmt)) != SSA_NAME)
+ return false;
+
+ if (is_gimple_assign (stmt))
+ {
+ code = gimple_assign_rhs_code (stmt);
+ op_type = TREE_CODE_LENGTH (code.safe_as_tree_code ());
+ }
+ else if (gimple_call_internal_p (stmt))
+ {
+ code = gimple_call_internal_fn (stmt);
+ op_type = gimple_call_num_args (stmt);
+ }
+ else
return false;
bool widen_arith = (code == WIDEN_PLUS_EXPR
- || code == WIDEN_MINUS_EXPR
- || code == WIDEN_MULT_EXPR
- || code == WIDEN_LSHIFT_EXPR);
- op_type = TREE_CODE_LENGTH (code);
+ || code == WIDEN_MINUS_EXPR
+ || code == WIDEN_MULT_EXPR
+ || code == WIDEN_LSHIFT_EXPR);
+
+ if (!widen_arith
+ && !CONVERT_EXPR_CODE_P (code)
+ && code != FIX_TRUNC_EXPR
+ && code != FLOAT_EXPR)
+ return false;
/* Check types of lhs and rhs. */
- scalar_dest = gimple_assign_lhs (stmt);
+ scalar_dest = gimple_get_lhs (stmt);
lhs_type = TREE_TYPE (scalar_dest);
vectype_out = STMT_VINFO_VECTYPE (stmt_info);
@@ -4938,10 +4950,14 @@ vectorizable_conversion (vec_info *vinfo,
if (op_type == binary_op)
{
- gcc_assert (code == WIDEN_MULT_EXPR || code == WIDEN_LSHIFT_EXPR
- || code == WIDEN_PLUS_EXPR || code == WIDEN_MINUS_EXPR);
+ gcc_assert (code == WIDEN_MULT_EXPR
+ || code == WIDEN_LSHIFT_EXPR
+ || code == WIDEN_PLUS_EXPR
+ || code == WIDEN_MINUS_EXPR);
- op1 = gimple_assign_rhs2 (stmt);
+
+ op1 = is_gimple_assign (stmt) ? gimple_assign_rhs2 (stmt) :
+ gimple_call_arg (stmt, 0);
tree vectype1_in;
if (!vect_is_simple_use (vinfo, stmt_info, slp_node, 1,
&op1, &slp_op1, &dt[1], &vectype1_in))
@@ -5025,8 +5041,12 @@ vectorizable_conversion (vec_info *vinfo,
&& code != FLOAT_EXPR
&& !CONVERT_EXPR_CODE_P (code))
return false;
- if (supportable_convert_operation (code, vectype_out, vectype_in, &code1))
+ if (supportable_convert_operation (code.safe_as_tree_code (), vectype_out,
+ vectype_in, &tc1))
+ {
+ code1 = tc1;
break;
+ }
/* FALLTHRU */
unsupported:
if (dump_enabled_p ())
@@ -5037,9 +5057,11 @@ vectorizable_conversion (vec_info *vinfo,
case WIDEN:
if (known_eq (nunits_in, nunits_out))
{
- if (!supportable_half_widening_operation (code, vectype_out,
- vectype_in, &code1))
+ if (!supportable_half_widening_operation (code.safe_as_tree_code (),
+ vectype_out, vectype_in,
+ &tc1))
goto unsupported;
+ code1 = tc1;
gcc_assert (!(multi_step_cvt && op_type == binary_op));
break;
}
@@ -5073,14 +5095,17 @@ vectorizable_conversion (vec_info *vinfo,
if (GET_MODE_SIZE (rhs_mode) == fltsz)
{
- if (!supportable_convert_operation (code, vectype_out,
- cvt_type, &codecvt1))
+ tc1 = ERROR_MARK;
+ if (!supportable_convert_operation (code.safe_as_tree_code (),
+ vectype_out,
+ cvt_type, &tc1))
goto unsupported;
+ codecvt1 = tc1;
}
- else if (!supportable_widening_operation (vinfo, code, stmt_info,
- vectype_out, cvt_type,
- &codecvt1, &codecvt2,
- &multi_step_cvt,
+ else if (!supportable_widening_operation (vinfo, code,
+ stmt_info, vectype_out,
+ cvt_type, &codecvt1,
+ &codecvt2, &multi_step_cvt,
&interm_types))
continue;
else
@@ -5088,8 +5113,9 @@ vectorizable_conversion (vec_info *vinfo,
if (supportable_widening_operation (vinfo, NOP_EXPR, stmt_info,
cvt_type,
- vectype_in, &code1, &code2,
- &multi_step_cvt, &interm_types))
+ vectype_in, &code1,
+ &code2, &multi_step_cvt,
+ &interm_types))
{
found_mode = true;
break;
@@ -5111,10 +5137,15 @@ vectorizable_conversion (vec_info *vinfo,
case NARROW:
gcc_assert (op_type == unary_op);
- if (supportable_narrowing_operation (code, vectype_out, vectype_in,
- &code1, &multi_step_cvt,
+ if (supportable_narrowing_operation (code.safe_as_tree_code (),
+ vectype_out,
+ vectype_in,
+ &tc1, &multi_step_cvt,
&interm_types))
- break;
+ {
+ code1 = tc1;
+ break;
+ }
if (code != FIX_TRUNC_EXPR
|| GET_MODE_SIZE (lhs_mode) >= GET_MODE_SIZE (rhs_mode))
@@ -5125,13 +5156,18 @@ vectorizable_conversion (vec_info *vinfo,
cvt_type = get_same_sized_vectype (cvt_type, vectype_in);
if (cvt_type == NULL_TREE)
goto unsupported;
- if (!supportable_convert_operation (code, cvt_type, vectype_in,
- &codecvt1))
+ if (!supportable_convert_operation (code.safe_as_tree_code (), cvt_type,
+ vectype_in,
+ &tc1))
goto unsupported;
+ codecvt1 = tc1;
if (supportable_narrowing_operation (NOP_EXPR, vectype_out, cvt_type,
- &code1, &multi_step_cvt,
+ &tc1, &multi_step_cvt,
&interm_types))
- break;
+ {
+ code1 = tc1;
+ break;
+ }
goto unsupported;
default:
@@ -5245,8 +5281,10 @@ vectorizable_conversion (vec_info *vinfo,
FOR_EACH_VEC_ELT (vec_oprnds0, i, vop0)
{
/* Arguments are ready, create the new vector stmt. */
- gcc_assert (TREE_CODE_LENGTH (code1) == unary_op);
- gassign *new_stmt = gimple_build_assign (vec_dest, code1, vop0);
+ gcc_assert (TREE_CODE_LENGTH ((tree_code) code1) == unary_op);
+ gassign *new_stmt = gimple_build_assign (vec_dest,
+ code1.safe_as_tree_code (),
+ vop0);
new_temp = make_ssa_name (vec_dest, new_stmt);
gimple_assign_set_lhs (new_stmt, new_temp);
vect_finish_stmt_generation (vinfo, stmt_info, new_stmt, gsi);
@@ -5278,7 +5316,7 @@ vectorizable_conversion (vec_info *vinfo,
for (i = multi_step_cvt; i >= 0; i--)
{
tree this_dest = vec_dsts[i];
- enum tree_code c1 = code1, c2 = code2;
+ code_helper c1 = code1, c2 = code2;
if (i == 0 && codecvt2 != ERROR_MARK)
{
c1 = codecvt1;
@@ -5288,7 +5326,8 @@ vectorizable_conversion (vec_info *vinfo,
vect_create_half_widening_stmts (vinfo, &vec_oprnds0,
&vec_oprnds1, stmt_info,
this_dest, gsi,
- c1, op_type);
+ c1.safe_as_tree_code (),
+ op_type);
else
vect_create_vectorized_promotion_stmts (vinfo, &vec_oprnds0,
&vec_oprnds1, stmt_info,
@@ -5301,9 +5340,11 @@ vectorizable_conversion (vec_info *vinfo,
gimple *new_stmt;
if (cvt_type)
{
- gcc_assert (TREE_CODE_LENGTH (codecvt1) == unary_op);
+ gcc_assert (TREE_CODE_LENGTH ((tree_code) codecvt1) == unary_op);
new_temp = make_ssa_name (vec_dest);
- new_stmt = gimple_build_assign (new_temp, codecvt1, vop0);
+ new_stmt = gimple_build_assign (new_temp,
+ codecvt1.safe_as_tree_code (),
+ vop0);
vect_finish_stmt_generation (vinfo, stmt_info, new_stmt, gsi);
}
else
@@ -5327,10 +5368,12 @@ vectorizable_conversion (vec_info *vinfo,
if (cvt_type)
FOR_EACH_VEC_ELT (vec_oprnds0, i, vop0)
{
- gcc_assert (TREE_CODE_LENGTH (codecvt1) == unary_op);
+ gcc_assert (TREE_CODE_LENGTH (((tree_code) codecvt1)) == unary_op);
new_temp = make_ssa_name (vec_dest);
gassign *new_stmt
- = gimple_build_assign (new_temp, codecvt1, vop0);
+ = gimple_build_assign (new_temp,
+ codecvt1.safe_as_tree_code (),
+ vop0);
vect_finish_stmt_generation (vinfo, stmt_info, new_stmt, gsi);
vec_oprnds0[i] = new_temp;
}
@@ -5338,7 +5381,8 @@ vectorizable_conversion (vec_info *vinfo,
vect_create_vectorized_demotion_stmts (vinfo, &vec_oprnds0,
multi_step_cvt,
stmt_info, vec_dsts, gsi,
- slp_node, code1);
+ slp_node,
+ code1.safe_as_tree_code ());
break;
}
if (!slp_node)
@@ -11926,9 +11970,11 @@ vect_maybe_update_slp_op_vectype (slp_tree op, tree vectype)
bool
supportable_widening_operation (vec_info *vinfo,
- enum tree_code code, stmt_vec_info stmt_info,
+ code_helper code,
+ stmt_vec_info stmt_info,
tree vectype_out, tree vectype_in,
- enum tree_code *code1, enum tree_code *code2,
+ code_helper *code1,
+ code_helper *code2,
int *multi_step_cvt,
vec<tree> *interm_types)
{
@@ -11939,7 +11985,7 @@ supportable_widening_operation (vec_info *vinfo,
optab optab1, optab2;
tree vectype = vectype_in;
tree wide_vectype = vectype_out;
- enum tree_code c1, c2;
+ code_helper c1 = MAX_TREE_CODES, c2 = MAX_TREE_CODES;
int i;
tree prev_type, intermediate_type;
machine_mode intermediate_mode, prev_mode;
@@ -11949,7 +11995,7 @@ supportable_widening_operation (vec_info *vinfo,
if (loop_info)
vect_loop = LOOP_VINFO_LOOP (loop_info);
- switch (code)
+ switch (code.safe_as_tree_code ())
{
case WIDEN_MULT_EXPR:
/* The result of a vectorized widening operation usually requires
@@ -11990,8 +12036,9 @@ supportable_widening_operation (vec_info *vinfo,
&& !nested_in_vect_loop_p (vect_loop, stmt_info)
&& supportable_widening_operation (vinfo, VEC_WIDEN_MULT_EVEN_EXPR,
stmt_info, vectype_out,
- vectype_in, code1, code2,
- multi_step_cvt, interm_types))
+ vectype_in, code1,
+ code2, multi_step_cvt,
+ interm_types))
{
/* Elements in a vector with vect_used_by_reduction property cannot
be reordered if the use chain with this property does not have the
@@ -12054,6 +12101,9 @@ supportable_widening_operation (vec_info *vinfo,
c2 = VEC_UNPACK_FIX_TRUNC_HI_EXPR;
break;
+ case MAX_TREE_CODES:
+ break;
+
default:
gcc_unreachable ();
}
@@ -12064,10 +12114,12 @@ supportable_widening_operation (vec_info *vinfo,
if (code == FIX_TRUNC_EXPR)
{
/* The signedness is determined from output operand. */
- optab1 = optab_for_tree_code (c1, vectype_out, optab_default);
- optab2 = optab_for_tree_code (c2, vectype_out, optab_default);
+ optab1 = optab_for_tree_code (c1.safe_as_tree_code (), vectype_out,
+ optab_default);
+ optab2 = optab_for_tree_code (c2.safe_as_tree_code (), vectype_out,
+ optab_default);
}
- else if (CONVERT_EXPR_CODE_P (code)
+ else if (CONVERT_EXPR_CODE_P (code.safe_as_tree_code ())
&& VECTOR_BOOLEAN_TYPE_P (wide_vectype)
&& VECTOR_BOOLEAN_TYPE_P (vectype)
&& TYPE_MODE (wide_vectype) == TYPE_MODE (vectype)
@@ -12080,8 +12132,10 @@ supportable_widening_operation (vec_info *vinfo,
}
else
{
- optab1 = optab_for_tree_code (c1, vectype, optab_default);
- optab2 = optab_for_tree_code (c2, vectype, optab_default);
+ optab1 = optab_for_tree_code (c1.safe_as_tree_code (), vectype,
+ optab_default);
+ optab2 = optab_for_tree_code (c2.safe_as_tree_code (), vectype,
+ optab_default);
}
if (!optab1 || !optab2)
@@ -12092,8 +12146,12 @@ supportable_widening_operation (vec_info *vinfo,
|| (icode2 = optab_handler (optab2, vec_mode)) == CODE_FOR_nothing)
return false;
- *code1 = c1;
- *code2 = c2;
+ if (code.is_tree_code ())
+ {
+ *code1 = c1;
+ *code2 = c2;
+ }
+
if (insn_data[icode1].operand[0].mode == TYPE_MODE (wide_vectype)
&& insn_data[icode2].operand[0].mode == TYPE_MODE (wide_vectype))
@@ -12114,7 +12172,7 @@ supportable_widening_operation (vec_info *vinfo,
prev_type = vectype;
prev_mode = vec_mode;
- if (!CONVERT_EXPR_CODE_P (code))
+ if (!CONVERT_EXPR_CODE_P ((tree_code) code))
return false;
/* We assume here that there will not be more than MAX_INTERM_CVT_STEPS
@@ -12145,8 +12203,12 @@ supportable_widening_operation (vec_info *vinfo,
}
else
{
- optab3 = optab_for_tree_code (c1, intermediate_type, optab_default);
- optab4 = optab_for_tree_code (c2, intermediate_type, optab_default);
+ optab3 = optab_for_tree_code (c1.safe_as_tree_code (),
+ intermediate_type,
+ optab_default);
+ optab4 = optab_for_tree_code (c2.safe_as_tree_code (),
+ intermediate_type,
+ optab_default);
}
if (!optab3 || !optab4
@@ -12205,7 +12267,7 @@ supportable_widening_operation (vec_info *vinfo,
bool
supportable_narrowing_operation (enum tree_code code,
tree vectype_out, tree vectype_in,
- enum tree_code *code1, int *multi_step_cvt,
+ tree_code *code1, int *multi_step_cvt,
vec<tree> *interm_types)
{
machine_mode vec_mode;
diff --git a/gcc/tree-vectorizer.h b/gcc/tree-vectorizer.h
index 642eb0aeb21264cd736a479b1ec25357abef29cd..6f70bd622c4a4dea8c432cd26c96d24af399ef3e 100644
--- a/gcc/tree-vectorizer.h
+++ b/gcc/tree-vectorizer.h
@@ -2120,13 +2120,12 @@ extern bool vect_is_simple_use (vec_info *, stmt_vec_info, slp_tree,
enum vect_def_type *,
tree *, stmt_vec_info * = NULL);
extern bool vect_maybe_update_slp_op_vectype (slp_tree, tree);
-extern bool supportable_widening_operation (vec_info *,
- enum tree_code, stmt_vec_info,
- tree, tree, enum tree_code *,
- enum tree_code *, int *,
- vec<tree> *);
+extern bool supportable_widening_operation (vec_info*, code_helper,
+ stmt_vec_info, tree, tree,
+ code_helper*, code_helper*,
+ int*, vec<tree> *);
extern bool supportable_narrowing_operation (enum tree_code, tree, tree,
- enum tree_code *, int *,
+ tree_code *, int *,
vec<tree> *);
extern unsigned record_stmt_cost (stmt_vector_for_cost *, int,
@@ -2558,4 +2557,7 @@ vect_is_integer_truncation (stmt_vec_info stmt_info)
&& TYPE_PRECISION (lhs_type) < TYPE_PRECISION (rhs_type));
}
+/* Build a GIMPLE_ASSIGN or GIMPLE_CALL with the tree_code,
+ or internal_fn contained in ch, respectively. */
+gimple * vect_gimple_build (tree, code_helper, tree, tree);
#endif /* GCC_TREE_VECTORIZER_H */
diff --git a/gcc/tree.h b/gcc/tree.h
index 6f6ad5a3a5f4dd4173482dfe259acf539ba24000..24b5184122550fe21ab0a5387867b6c65c20bb03 100644
--- a/gcc/tree.h
+++ b/gcc/tree.h
@@ -93,6 +93,8 @@ public:
bool is_internal_fn () const;
bool is_builtin_fn () const;
int get_rep () const { return rep; }
+ enum tree_code safe_as_tree_code () const;
+ combined_fn safe_as_fn_code () const;
bool operator== (const code_helper &other) { return rep == other.rep; }
bool operator!= (const code_helper &other) { return rep != other.rep; }
bool operator== (tree_code c) { return rep == code_helper (c).rep; }
@@ -102,6 +104,17 @@ private:
int rep;
};
+inline enum tree_code
+code_helper::safe_as_tree_code () const
+{
+ return is_tree_code () ? (tree_code)* this : MAX_TREE_CODES;
+}
+
+inline combined_fn
+code_helper::safe_as_fn_code () const {
+ return is_fn_code () ? (combined_fn) *this : CFN_LAST;
+}
+
inline code_helper::operator internal_fn () const
{
return as_internal_fn (combined_fn (*this));
--
2.17.1
[-- Attachment #3: 0002-Refactor-widen_plus-as-internal_fn.patch --]
[-- Type: application/octet-stream, Size: 22832 bytes --]
From 1e8afa697157c3cb520a36304326e14891444226 Mon Sep 17 00:00:00 2001
From: Joel Hutton <joel.hutton@arm.com>
Date: Wed, 26 Jan 2022 14:00:17 +0000
Subject: [PATCH 2/3] Refactor widen_plus as internal_fn
This patch replaces the existing tree_code widen_plus and widen_minus
patterns with internal_fn versions.
DEF_INTERNAL_OPTAB_MULTI_FN is like DEF_INTERNAL_OPTAB_FN except it provides convenience wrappers for defining conversions that require a hi/lo split, like widening and narrowing operations. Each definition for <NAME> will require an optab named <OPTAB> and two other optabs that you specify for signed and unsigned. The hi/lo pair is necessary because the widening operations take n narrow elements as inputs and return n/2 wide elements as outputs. The 'lo' operation operates on the first n/2 elements of input. The 'hi' operation operates on the second n/2 elements of input. Defining an internal_fn along with hi/lo variations allows a single internal function to be returned from a vect_recog function that will later be expanded to hi/lo.
DEF_INTERNAL_OPTAB_MULTI_FN is used in internal-fn.def to register a widening internal_fn. It is defined differently in different places and internal-fn.def is sourced from those places so the parameters given can be reused.
internal-fn.c: defined to expand to hi/lo signed/unsigned optabs, later defined to generate the 'expand_' functions for the hi/lo versions of the fn.
internal-fn.def: defined to invoke DEF_INTERNAL_OPTAB_FN for the original and hi/lo variants of the internal_fn
For example:
IFN_VEC_WIDEN_PLUS -> IFN_VEC_WIDEN_PLUS_HI, IFN_VEC_WIDEN_PLUS_LO
for aarch64: IFN_VEC_WIDEN_PLUS_HI -> vec_widen_<su>addl_hi_<mode> -> (u/s)addl2
IFN_VEC_WIDEN_PLUS_LO -> vec_widen_<su>addl_lo_<mode> -> (u/s)addl
This gives the same functionality as the previous WIDEN_PLUS/WIDEN_MINUS tree codes which are expanded into VEC_WIDEN_PLUS_LO, VEC_WIDEN_PLUS_HI.
gcc/ChangeLog:
2022-04-13 Joel Hutton <joel.hutton@arm.com>
2022-04-13 Tamar Christina <tamar.christina@arm.com>
* internal-fn.cc (INCLUDE_MAP): Include maps for use in optab
lookup.
(DEF_INTERNAL_OPTAB_MULTI_FN): Macro to define an internal_fn that
expands into multiple internal_fns (for widening).
(ifn_cmp): Function to compare ifn's for sorting/searching.
(lookup_multi_ifn_optab): Add lookup function.
(lookup_multi_internal_fn): Add lookup function.
(commutative_binary_fn_p): Add widen_plus fn's.
* internal-fn.def (DEF_INTERNAL_OPTAB_MULTI_FN): Define widening
plus,minus functions.
(VEC_WIDEN_PLUS): Replacement for VEC_WIDEN_PLUS tree code.
(VEC_WIDEN_MINUS): Replacement for VEC_WIDEN_MINUS tree code.
* internal-fn.h (GCC_INTERNAL_FN_H): Add headers.
(lookup_multi_ifn_optab): Add prototype.
(lookup_multi_internal_fn): Add prototype.
* optabs.cc (commutative_optab_p): Add widening plus, minus optabs.
* optabs.def (OPTAB_CD): widen add, sub optabs
* tree-core.h (ECF_MULTI): Flag to indicate if a function decays
into hi/lo parts.
* tree-vect-patterns.cc (vect_recog_widen_op_pattern): Support
patterns with a hi/lo split.
(vect_recog_widen_plus_pattern): Refactor to return
IFN_VECT_WIDEN_PLUS.
(vect_recog_widen_minus_pattern): Refactor to return new
IFN_VEC_WIDEN_MINUS.
* tree-vect-stmts.cc (vectorizable_conversion): Add widen plus/minus
ifn
support.
(supportable_widening_operation): Add widen plus/minus ifn support.
gcc/testsuite/ChangeLog:
* gcc.target/aarch64/vect-widen-add.c: Test that new
IFN_VEC_WIDEN_PLUS is being used.
* gcc.target/aarch64/vect-widen-sub.c: Test that new
IFN_VEC_WIDEN_MINUS is being used.
---
gcc/internal-fn.cc | 107 ++++++++++++++++++
gcc/internal-fn.def | 23 ++++
gcc/internal-fn.h | 7 ++
gcc/optabs.cc | 12 +-
gcc/optabs.def | 2 +
.../gcc.target/aarch64/vect-widen-add.c | 4 +-
.../gcc.target/aarch64/vect-widen-sub.c | 4 +-
gcc/tree-core.h | 4 +
gcc/tree-vect-patterns.cc | 37 ++++--
gcc/tree-vect-stmts.cc | 68 +++++++++--
10 files changed, 249 insertions(+), 19 deletions(-)
diff --git a/gcc/internal-fn.cc b/gcc/internal-fn.cc
index 91588f8bc9f7c3fe2bac17f3c4e6078cddb7b4d2..b2cb3e508027a84e4456d676d78b27b6c04b7b61 100644
--- a/gcc/internal-fn.cc
+++ b/gcc/internal-fn.cc
@@ -17,6 +17,7 @@ You should have received a copy of the GNU General Public License
along with GCC; see the file COPYING3. If not see
<http://www.gnu.org/licenses/>. */
+#define INCLUDE_MAP
#include "config.h"
#include "system.h"
#include "coretypes.h"
@@ -70,6 +71,26 @@ const int internal_fn_flags_array[] = {
0
};
+const enum internal_fn internal_fn_hilo_keys_array[] = {
+#undef DEF_INTERNAL_OPTAB_MULTI_FN
+#define DEF_INTERNAL_OPTAB_MULTI_FN(NAME, FLAGS, OPTAB, SOPTAB, UOPTAB, TYPE) \
+ IFN_##NAME##_LO, \
+ IFN_##NAME##_HI,
+#include "internal-fn.def"
+ IFN_LAST
+#undef DEF_INTERNAL_OPTAB_MULTI_FN
+};
+
+const optab internal_fn_hilo_values_array[] = {
+#undef DEF_INTERNAL_OPTAB_MULTI_FN
+#define DEF_INTERNAL_OPTAB_MULTI_FN(NAME, FLAGS, OPTAB, SOPTAB, UOPTAB, TYPE) \
+ SOPTAB##_lo_optab, UOPTAB##_lo_optab, \
+ SOPTAB##_hi_optab, UOPTAB##_hi_optab,
+#include "internal-fn.def"
+ unknown_optab, unknown_optab
+#undef DEF_INTERNAL_OPTAB_MULTI_FN
+};
+
/* Return the internal function called NAME, or IFN_LAST if there's
no such function. */
@@ -90,6 +111,62 @@ lookup_internal_fn (const char *name)
return entry ? *entry : IFN_LAST;
}
+static int
+ifn_cmp (const void *a_, const void *b_)
+{
+ typedef std::pair<enum internal_fn, unsigned> ifn_pair;
+ auto *a = (const std::pair<ifn_pair, optab> *)a_;
+ auto *b = (const std::pair<ifn_pair, optab> *)b_;
+ return (int) (a->first.first) - (b->first.first);
+}
+
+/* Return the optab belonging to the given internal function NAME for the given
+ SIGN or unknown_optab. */
+
+optab
+lookup_multi_ifn_optab (enum internal_fn fn, unsigned sign)
+{
+ typedef std::pair<enum internal_fn, unsigned> ifn_pair;
+ typedef auto_vec <std::pair<ifn_pair, optab>>fn_to_optab_map_type;
+ static fn_to_optab_map_type *fn_to_optab_map;
+
+ if (!fn_to_optab_map)
+ {
+ unsigned num
+ = sizeof (internal_fn_hilo_keys_array) / sizeof (enum internal_fn);
+ fn_to_optab_map = new fn_to_optab_map_type ();
+ for (unsigned int i = 0; i < num - 1; ++i)
+ {
+ enum internal_fn fn = internal_fn_hilo_keys_array[i];
+ optab v1 = internal_fn_hilo_values_array[2*i];
+ optab v2 = internal_fn_hilo_values_array[2*i + 1];
+ ifn_pair key1 (fn, 0);
+ fn_to_optab_map->safe_push ({key1, v1});
+ ifn_pair key2 (fn, 1);
+ fn_to_optab_map->safe_push ({key2, v2});
+ }
+ fn_to_optab_map->qsort (ifn_cmp);
+ }
+
+ ifn_pair new_pair (fn, sign ? 1 : 0);
+ optab tmp;
+ std::pair<ifn_pair,optab> pair_wrap (new_pair, tmp);
+ auto entry = fn_to_optab_map->bsearch (&pair_wrap, ifn_cmp);
+ return entry != fn_to_optab_map->end () ? entry->second : unknown_optab;
+}
+
+extern void
+lookup_multi_internal_fn (enum internal_fn ifn, enum internal_fn *lo,
+ enum internal_fn *hi)
+{
+ int ecf_flags = internal_fn_flags (ifn);
+ gcc_assert (ecf_flags & ECF_MULTI);
+
+ *lo = internal_fn (ifn + 1);
+ *hi = internal_fn (ifn + 2);
+}
+
+
/* Fnspec of each internal function, indexed by function number. */
const_tree internal_fn_fnspec_array[IFN_LAST + 1];
@@ -3928,6 +4005,9 @@ commutative_binary_fn_p (internal_fn fn)
case IFN_UBSAN_CHECK_MUL:
case IFN_ADD_OVERFLOW:
case IFN_MUL_OVERFLOW:
+ case IFN_VEC_WIDEN_PLUS:
+ case IFN_VEC_WIDEN_PLUS_LO:
+ case IFN_VEC_WIDEN_PLUS_HI:
return true;
default:
@@ -4013,6 +4093,32 @@ set_edom_supported_p (void)
#endif
}
+#undef DEF_INTERNAL_OPTAB_MULTI_FN
+#define DEF_INTERNAL_OPTAB_MULTI_FN(CODE, FLAGS, OPTAB, SOPTAB, UOPTAB, TYPE) \
+ static void \
+ expand_##CODE (internal_fn, gcall *) \
+ { \
+ gcc_unreachable (); \
+ } \
+ static void \
+ expand_##CODE##_LO (internal_fn fn, gcall *stmt) \
+ { \
+ tree ty = TREE_TYPE (gimple_get_lhs (stmt)); \
+ if (!TYPE_UNSIGNED (ty)) \
+ expand_##TYPE##_optab_fn (fn, stmt, SOPTAB##_lo##_optab); \
+ else \
+ expand_##TYPE##_optab_fn (fn, stmt, UOPTAB##_lo##_optab); \
+ } \
+ static void \
+ expand_##CODE##_HI (internal_fn fn, gcall *stmt) \
+ { \
+ tree ty = TREE_TYPE (gimple_get_lhs (stmt)); \
+ if (!TYPE_UNSIGNED (ty)) \
+ expand_##TYPE##_optab_fn (fn, stmt, SOPTAB##_hi##_optab); \
+ else \
+ expand_##TYPE##_optab_fn (fn, stmt, UOPTAB##_hi##_optab); \
+ }
+
#define DEF_INTERNAL_OPTAB_FN(CODE, FLAGS, OPTAB, TYPE) \
static void \
expand_##CODE (internal_fn fn, gcall *stmt) \
@@ -4029,6 +4135,7 @@ set_edom_supported_p (void)
expand_##TYPE##_optab_fn (fn, stmt, which_optab); \
}
#include "internal-fn.def"
+#undef DEF_INTERNAL_OPTAB_MULTI_FN
/* Routines to expand each internal function, indexed by function number.
Each routine has the prototype:
diff --git a/gcc/internal-fn.def b/gcc/internal-fn.def
index d2d550d358606022b1cb44fa842f06e0be507bc3..4635a9c8af9ad27bb05d7510388d0fe2270428e5 100644
--- a/gcc/internal-fn.def
+++ b/gcc/internal-fn.def
@@ -82,6 +82,13 @@ along with GCC; see the file COPYING3. If not see
says that the function extends the C-level BUILT_IN_<NAME>{,L,LL,IMAX}
group of functions to any integral mode (including vector modes).
+ DEF_INTERNAL_OPTAB_MULTI_FN is like DEF_INTERNAL_OPTAB_FN except it
+ provides convenience wrappers for defining conversions that require a
+ hi/lo split, like widening and narrowing operations. Each definition
+ for <NAME> will require an optab named <OPTAB> and two other optabs that
+ you specify for signed and unsigned.
+
+
Each entry must have a corresponding expander of the form:
void expand_NAME (gimple_call stmt)
@@ -120,6 +127,14 @@ along with GCC; see the file COPYING3. If not see
DEF_INTERNAL_OPTAB_FN (NAME, FLAGS, OPTAB, TYPE)
#endif
+#ifndef DEF_INTERNAL_OPTAB_MULTI_FN
+#define DEF_INTERNAL_OPTAB_MULTI_FN(NAME, FLAGS, OPTAB, SOPTAB, UOPTAB, TYPE) \
+ DEF_INTERNAL_OPTAB_FN (NAME, FLAGS | ECF_MULTI, OPTAB, TYPE) \
+ DEF_INTERNAL_OPTAB_FN (NAME ## _LO, FLAGS, unknown, TYPE) \
+ DEF_INTERNAL_OPTAB_FN (NAME ## _HI, FLAGS, unknown, TYPE)
+#endif
+
+
DEF_INTERNAL_OPTAB_FN (MASK_LOAD, ECF_PURE, maskload, mask_load)
DEF_INTERNAL_OPTAB_FN (LOAD_LANES, ECF_CONST, vec_load_lanes, load_lanes)
DEF_INTERNAL_OPTAB_FN (MASK_LOAD_LANES, ECF_PURE,
@@ -292,6 +307,14 @@ DEF_INTERNAL_OPTAB_FN (COMPLEX_ADD_ROT270, ECF_CONST, cadd270, binary)
DEF_INTERNAL_OPTAB_FN (COMPLEX_MUL, ECF_CONST, cmul, binary)
DEF_INTERNAL_OPTAB_FN (COMPLEX_MUL_CONJ, ECF_CONST, cmul_conj, binary)
DEF_INTERNAL_OPTAB_FN (VEC_ADDSUB, ECF_CONST, vec_addsub, binary)
+DEF_INTERNAL_OPTAB_MULTI_FN (VEC_WIDEN_PLUS,
+ ECF_CONST | ECF_WIDEN | ECF_NOTHROW,
+ vec_widen_add, vec_widen_saddl, vec_widen_uaddl,
+ binary)
+DEF_INTERNAL_OPTAB_MULTI_FN (VEC_WIDEN_MINUS,
+ ECF_CONST | ECF_WIDEN | ECF_NOTHROW,
+ vec_widen_sub, vec_widen_ssubl, vec_widen_usubl,
+ binary)
DEF_INTERNAL_OPTAB_FN (VEC_FMADDSUB, ECF_CONST, vec_fmaddsub, ternary)
DEF_INTERNAL_OPTAB_FN (VEC_FMSUBADD, ECF_CONST, vec_fmsubadd, ternary)
diff --git a/gcc/internal-fn.h b/gcc/internal-fn.h
index 23c014a963c4d72da92c763db87ee486a2adb485..b35de19747d251d19dc13de1e0323368bd2ebdf2 100644
--- a/gcc/internal-fn.h
+++ b/gcc/internal-fn.h
@@ -20,6 +20,10 @@ along with GCC; see the file COPYING3. If not see
#ifndef GCC_INTERNAL_FN_H
#define GCC_INTERNAL_FN_H
+#include "insn-codes.h"
+#include "insn-opinit.h"
+
+
/* INTEGER_CST values for IFN_UNIQUE function arg-0.
UNSPEC: Undifferentiated UNIQUE.
@@ -112,6 +116,9 @@ internal_fn_name (enum internal_fn fn)
}
extern internal_fn lookup_internal_fn (const char *);
+extern optab lookup_multi_ifn_optab (enum internal_fn, unsigned);
+extern void lookup_multi_internal_fn (enum internal_fn, enum internal_fn *,
+ enum internal_fn *);
/* Return the ECF_* flags for function FN. */
diff --git a/gcc/optabs.cc b/gcc/optabs.cc
index a50dd798f2a454ac54e247f3e6cbab17577ea304..f9be369a6c5b99de5bbad664a11364d1c2cc4b95 100644
--- a/gcc/optabs.cc
+++ b/gcc/optabs.cc
@@ -1312,7 +1312,17 @@ commutative_optab_p (optab binoptab)
|| binoptab == smul_widen_optab
|| binoptab == umul_widen_optab
|| binoptab == smul_highpart_optab
- || binoptab == umul_highpart_optab);
+ || binoptab == umul_highpart_optab
+ || binoptab == vec_widen_add_optab
+ || binoptab == vec_widen_sub_optab
+ || binoptab == vec_widen_saddl_hi_optab
+ || binoptab == vec_widen_saddl_lo_optab
+ || binoptab == vec_widen_ssubl_hi_optab
+ || binoptab == vec_widen_ssubl_lo_optab
+ || binoptab == vec_widen_uaddl_hi_optab
+ || binoptab == vec_widen_uaddl_lo_optab
+ || binoptab == vec_widen_usubl_hi_optab
+ || binoptab == vec_widen_usubl_lo_optab);
}
/* X is to be used in mode MODE as operand OPN to BINOPTAB. If we're
diff --git a/gcc/optabs.def b/gcc/optabs.def
index 801310ebaa7d469520809bb7efed6820f8eb866b..a7881dcb49e4ef07d8f07aa31214eb3a7a944e2e 100644
--- a/gcc/optabs.def
+++ b/gcc/optabs.def
@@ -78,6 +78,8 @@ OPTAB_CD(smsub_widen_optab, "msub$b$a4")
OPTAB_CD(umsub_widen_optab, "umsub$b$a4")
OPTAB_CD(ssmsub_widen_optab, "ssmsub$b$a4")
OPTAB_CD(usmsub_widen_optab, "usmsub$a$b4")
+OPTAB_CD(vec_widen_add_optab, "add$a$b3")
+OPTAB_CD(vec_widen_sub_optab, "sub$a$b3")
OPTAB_CD(vec_load_lanes_optab, "vec_load_lanes$a$b")
OPTAB_CD(vec_store_lanes_optab, "vec_store_lanes$a$b")
OPTAB_CD(vec_mask_load_lanes_optab, "vec_mask_load_lanes$a$b")
diff --git a/gcc/testsuite/gcc.target/aarch64/vect-widen-add.c b/gcc/testsuite/gcc.target/aarch64/vect-widen-add.c
index 220bd9352a4c7acd2e3713e441d74898d3e92b30..7037673d32bd780e1c9b58a51e58e2bac3b30b7e 100644
--- a/gcc/testsuite/gcc.target/aarch64/vect-widen-add.c
+++ b/gcc/testsuite/gcc.target/aarch64/vect-widen-add.c
@@ -1,5 +1,5 @@
/* { dg-do run } */
-/* { dg-options "-O3 -save-temps" } */
+/* { dg-options "-O3 -save-temps -fdump-tree-vect-all" } */
#include <stdint.h>
#include <string.h>
@@ -86,6 +86,8 @@ main()
return 0;
}
+/* { dg-final { scan-tree-dump "add new stmt.*VEC_WIDEN_PLUS_LO" "vect" } } */
+/* { dg-final { scan-tree-dump "add new stmt.*VEC_WIDEN_PLUS_HI" "vect" } } */
/* { dg-final { scan-assembler-times {\tuaddl\t} 1} } */
/* { dg-final { scan-assembler-times {\tuaddl2\t} 1} } */
/* { dg-final { scan-assembler-times {\tsaddl\t} 1} } */
diff --git a/gcc/testsuite/gcc.target/aarch64/vect-widen-sub.c b/gcc/testsuite/gcc.target/aarch64/vect-widen-sub.c
index a2bed63affbd091977df95a126da1f5b8c1d41d2..83bc1edb6105f47114b665e24a13e6194b2179a2 100644
--- a/gcc/testsuite/gcc.target/aarch64/vect-widen-sub.c
+++ b/gcc/testsuite/gcc.target/aarch64/vect-widen-sub.c
@@ -1,5 +1,5 @@
/* { dg-do run } */
-/* { dg-options "-O3 -save-temps" } */
+/* { dg-options "-O3 -save-temps -fdump-tree-vect-all" } */
#include <stdint.h>
#include <string.h>
@@ -86,6 +86,8 @@ main()
return 0;
}
+/* { dg-final { scan-tree-dump "add new stmt.*VEC_WIDEN_MINUS_LO" "vect" } } */
+/* { dg-final { scan-tree-dump "add new stmt.*VEC_WIDEN_MINUS_HI" "vect" } } */
/* { dg-final { scan-assembler-times {\tusubl\t} 1} } */
/* { dg-final { scan-assembler-times {\tusubl2\t} 1} } */
/* { dg-final { scan-assembler-times {\tssubl\t} 1} } */
diff --git a/gcc/tree-core.h b/gcc/tree-core.h
index cff6211080bced0bffb39e98039a6550897acf77..d0c8b812cfb9c3ac83bf25fff0431b08cb7d823d 100644
--- a/gcc/tree-core.h
+++ b/gcc/tree-core.h
@@ -99,6 +99,10 @@ struct die_struct;
/* Nonzero if this is a widening function. */
#define ECF_WIDEN (1 << 16)
+/* Nonzero if this is a function that decomposes into a lo/hi operation. */
+#define ECF_MULTI (1 << 17)
+
+
/* Call argument flags. */
/* Nonzero if the argument is not used by the function. */
diff --git a/gcc/tree-vect-patterns.cc b/gcc/tree-vect-patterns.cc
index d892158f024fc045b897aebe76f2e2b66211cf83..62ca28d725ed4ac8d7e4d493119e40772a0fbac6 100644
--- a/gcc/tree-vect-patterns.cc
+++ b/gcc/tree-vect-patterns.cc
@@ -1351,14 +1351,16 @@ static gimple *
vect_recog_widen_op_pattern (vec_info *vinfo,
stmt_vec_info last_stmt_info, tree *type_out,
tree_code orig_code, code_helper wide_code,
- bool shift_p, const char *name)
+ bool shift_p, const char *name,
+ enum optab_subtype *subtype = NULL)
{
gimple *last_stmt = last_stmt_info->stmt;
vect_unpromoted_value unprom[2];
tree half_type;
if (!vect_widened_op_tree (vinfo, last_stmt_info, orig_code, orig_code,
- shift_p, 2, unprom, &half_type))
+ shift_p, 2, unprom, &half_type, subtype))
+
return NULL;
/* Pattern detected. */
@@ -1424,6 +1426,20 @@ vect_recog_widen_op_pattern (vec_info *vinfo,
type, pattern_stmt, vecctype);
}
+static gimple *
+vect_recog_widen_op_pattern (vec_info *vinfo,
+ stmt_vec_info last_stmt_info, tree *type_out,
+ tree_code orig_code, internal_fn wide_ifn,
+ bool shift_p, const char *name,
+ enum optab_subtype *subtype = NULL)
+{
+ combined_fn ifn = as_combined_fn (wide_ifn);
+ return vect_recog_widen_op_pattern (vinfo, last_stmt_info, type_out,
+ orig_code, ifn, shift_p, name,
+ subtype);
+}
+
+
/* Try to detect multiplication on widened inputs, converting MULT_EXPR
to WIDEN_MULT_EXPR. See vect_recog_widen_op_pattern for details. */
@@ -1437,26 +1453,30 @@ vect_recog_widen_mult_pattern (vec_info *vinfo, stmt_vec_info last_stmt_info,
}
/* Try to detect addition on widened inputs, converting PLUS_EXPR
- to WIDEN_PLUS_EXPR. See vect_recog_widen_op_pattern for details. */
+ to IFN_VEC_WIDEN_PLUS. See vect_recog_widen_op_pattern for details. */
static gimple *
vect_recog_widen_plus_pattern (vec_info *vinfo, stmt_vec_info last_stmt_info,
tree *type_out)
{
+ enum optab_subtype subtype;
return vect_recog_widen_op_pattern (vinfo, last_stmt_info, type_out,
- PLUS_EXPR, WIDEN_PLUS_EXPR, false,
- "vect_recog_widen_plus_pattern");
+ PLUS_EXPR, IFN_VEC_WIDEN_PLUS,
+ false, "vect_recog_widen_plus_pattern",
+ &subtype);
}
/* Try to detect subtraction on widened inputs, converting MINUS_EXPR
- to WIDEN_MINUS_EXPR. See vect_recog_widen_op_pattern for details. */
+ to IFN_VEC_WIDEN_MINUS. See vect_recog_widen_op_pattern for details. */
static gimple *
vect_recog_widen_minus_pattern (vec_info *vinfo, stmt_vec_info last_stmt_info,
tree *type_out)
{
+ enum optab_subtype subtype;
return vect_recog_widen_op_pattern (vinfo, last_stmt_info, type_out,
- MINUS_EXPR, WIDEN_MINUS_EXPR, false,
- "vect_recog_widen_minus_pattern");
+ MINUS_EXPR, IFN_VEC_WIDEN_MINUS,
+ false, "vect_recog_widen_minus_pattern",
+ &subtype);
}
/* Function vect_recog_popcount_pattern
@@ -5629,6 +5649,7 @@ static vect_recog_func vect_vect_recog_func_ptrs[] = {
{ vect_recog_mask_conversion_pattern, "mask_conversion" },
{ vect_recog_widen_plus_pattern, "widen_plus" },
{ vect_recog_widen_minus_pattern, "widen_minus" },
+ /* These must come after the double widening ones. */
};
const unsigned int NUM_PATTERNS = ARRAY_SIZE (vect_vect_recog_func_ptrs);
diff --git a/gcc/tree-vect-stmts.cc b/gcc/tree-vect-stmts.cc
index d6aabb873c86ab8ff0bae41c7f6c3bad34d583c5..6fa0669fdfc8630842b3f9f32f4b4a253e79bb92 100644
--- a/gcc/tree-vect-stmts.cc
+++ b/gcc/tree-vect-stmts.cc
@@ -4903,7 +4903,9 @@ vectorizable_conversion (vec_info *vinfo,
bool widen_arith = (code == WIDEN_PLUS_EXPR
|| code == WIDEN_MINUS_EXPR
|| code == WIDEN_MULT_EXPR
- || code == WIDEN_LSHIFT_EXPR);
+ || code == WIDEN_LSHIFT_EXPR
+ || code == IFN_VEC_WIDEN_PLUS
+ || code == IFN_VEC_WIDEN_MINUS);
if (!widen_arith
&& !CONVERT_EXPR_CODE_P (code)
@@ -4953,7 +4955,9 @@ vectorizable_conversion (vec_info *vinfo,
gcc_assert (code == WIDEN_MULT_EXPR
|| code == WIDEN_LSHIFT_EXPR
|| code == WIDEN_PLUS_EXPR
- || code == WIDEN_MINUS_EXPR);
+ || code == WIDEN_MINUS_EXPR
+ || code == IFN_VEC_WIDEN_PLUS
+ || code == IFN_VEC_WIDEN_MINUS);
op1 = is_gimple_assign (stmt) ? gimple_assign_rhs2 (stmt) :
@@ -12130,14 +12134,62 @@ supportable_widening_operation (vec_info *vinfo,
optab1 = vec_unpacks_sbool_lo_optab;
optab2 = vec_unpacks_sbool_hi_optab;
}
- else
- {
- optab1 = optab_for_tree_code (c1.safe_as_tree_code (), vectype,
- optab_default);
- optab2 = optab_for_tree_code (c2.safe_as_tree_code (), vectype,
- optab_default);
+
+ if (code.is_fn_code ())
+ {
+ internal_fn ifn = as_internal_fn (code.safe_as_fn_code ());
+ int ecf_flags = internal_fn_flags (ifn);
+ gcc_assert (ecf_flags & ECF_MULTI);
+
+ switch (code.safe_as_fn_code ())
+ {
+ case CFN_VEC_WIDEN_PLUS:
+ break;
+ case CFN_VEC_WIDEN_MINUS:
+ break;
+ case CFN_LAST:
+ default:
+ return false;
+ }
+
+ internal_fn lo, hi;
+ lookup_multi_internal_fn (ifn, &lo, &hi);
+ *code1 = as_combined_fn (lo);
+ *code2 = as_combined_fn (hi);
+ optab1 = lookup_multi_ifn_optab (lo, !TYPE_UNSIGNED (vectype));
+ optab2 = lookup_multi_ifn_optab (hi, !TYPE_UNSIGNED (vectype));
}
+ if (code.is_tree_code ())
+ {
+ if (code == FIX_TRUNC_EXPR)
+ {
+ /* The signedness is determined from output operand. */
+ optab1 = optab_for_tree_code (c1.safe_as_tree_code (), vectype_out,
+ optab_default);
+ optab2 = optab_for_tree_code (c2.safe_as_tree_code (), vectype_out,
+ optab_default);
+ }
+ else if (CONVERT_EXPR_CODE_P (code.safe_as_tree_code ())
+ && VECTOR_BOOLEAN_TYPE_P (wide_vectype)
+ && VECTOR_BOOLEAN_TYPE_P (vectype)
+ && TYPE_MODE (wide_vectype) == TYPE_MODE (vectype)
+ && SCALAR_INT_MODE_P (TYPE_MODE (vectype)))
+ {
+ /* If the input and result modes are the same, a different optab
+ is needed where we pass in the number of units in vectype. */
+ optab1 = vec_unpacks_sbool_lo_optab;
+ optab2 = vec_unpacks_sbool_hi_optab;
+ }
+ else
+ {
+ optab1 = optab_for_tree_code (c1.safe_as_tree_code (), vectype,
+ optab_default);
+ optab2 = optab_for_tree_code (c2.safe_as_tree_code (), vectype,
+ optab_default);
+ }
+ }
+
if (!optab1 || !optab2)
return false;
--
2.17.1
[-- Attachment #4: 0003-Remove-widen_plus-minus_expr-tree-codes.patch --]
[-- Type: application/octet-stream, Size: 19552 bytes --]
From 60664218e6e59510f02fb64b49a236e9e5b26c9f Mon Sep 17 00:00:00 2001
From: Joel Hutton <joel.hutton@arm.com>
Date: Fri, 28 Jan 2022 12:04:44 +0000
Subject: [PATCH 3/3] Remove widen_plus/minus_expr tree codes
This patch removes the old widen plus/minus tree codes which have been
replaced by internal functions.
gcc/ChangeLog:
* doc/generic.texi: Remove old tree codes.
* expr.cc (expand_expr_real_2): Remove old tree code cases.
* gimple-pretty-print.cc (dump_binary_rhs): Remove old tree code
cases.
* optabs-tree.cc (optab_for_tree_code): Remove old tree code cases.
(supportable_half_widening_operation): Remove old tree code cases.
* tree-cfg.cc (verify_gimple_assign_binary): Remove old tree code
cases.
* tree-inline.cc (estimate_operator_cost): Remove old tree code
cases.
* tree-pretty-print.cc (dump_generic_node): Remove tree code definition.
(op_symbol_code): Remove old tree code
cases.
* tree-vect-data-refs.cc (vect_get_smallest_scalar_type): Remove old tree code
cases.
(vect_analyze_data_ref_accesses): Remove old tree code
cases.
* tree-vect-generic.cc (expand_vector_operations_1): Remove old tree code
cases.
* tree-vect-patterns.cc (vect_widened_op_tree): Refactor ot replace
usage in vect_recog_sad_pattern.
(vect_recog_sad_pattern): Replace tree code widening pattern with
internal function.
(vect_recog_average_pattern): Replace tree code widening pattern
with internal function.
* tree-vect-stmts.cc (vectorizable_conversion): Remove old tree code
cases.
(supportable_widening_operation): Remove old tree code
cases.
* tree.def (WIDEN_PLUS_EXPR): Remove tree code definition.
(WIDEN_MINUS_EXPR): Remove tree code definition.
(VEC_WIDEN_PLUS_HI_EXPR): Remove tree code definition.
(VEC_WIDEN_PLUS_LO_EXPR): Remove tree code definition.
(VEC_WIDEN_MINUS_HI_EXPR): Remove tree code definition.
(VEC_WIDEN_MINUS_LO_EXPR): Remove tree code definition.
---
gcc/doc/generic.texi | 31 -------------------------------
gcc/expr.cc | 6 ------
gcc/gimple-pretty-print.cc | 4 ----
gcc/optabs-tree.cc | 24 ------------------------
gcc/tree-cfg.cc | 6 ------
gcc/tree-inline.cc | 6 ------
gcc/tree-pretty-print.cc | 12 ------------
gcc/tree-vect-data-refs.cc | 8 +++-----
gcc/tree-vect-generic.cc | 4 ----
gcc/tree-vect-patterns.cc | 36 +++++++++++++++++++++++++-----------
gcc/tree-vect-stmts.cc | 18 ++----------------
gcc/tree.def | 6 ------
12 files changed, 30 insertions(+), 131 deletions(-)
diff --git a/gcc/doc/generic.texi b/gcc/doc/generic.texi
index e5f9d1be8ea81f3da002ec3bb925590d331a2551..344045efd419b0cc3a11771acf70d2fd279c48ac 100644
--- a/gcc/doc/generic.texi
+++ b/gcc/doc/generic.texi
@@ -1811,10 +1811,6 @@ a value from @code{enum annot_expr_kind}, the third is an @code{INTEGER_CST}.
@tindex VEC_RSHIFT_EXPR
@tindex VEC_WIDEN_MULT_HI_EXPR
@tindex VEC_WIDEN_MULT_LO_EXPR
-@tindex VEC_WIDEN_PLUS_HI_EXPR
-@tindex VEC_WIDEN_PLUS_LO_EXPR
-@tindex VEC_WIDEN_MINUS_HI_EXPR
-@tindex VEC_WIDEN_MINUS_LO_EXPR
@tindex VEC_UNPACK_HI_EXPR
@tindex VEC_UNPACK_LO_EXPR
@tindex VEC_UNPACK_FLOAT_HI_EXPR
@@ -1861,33 +1857,6 @@ vector of @code{N/2} products. In the case of @code{VEC_WIDEN_MULT_LO_EXPR} the
low @code{N/2} elements of the two vector are multiplied to produce the
vector of @code{N/2} products.
-@item VEC_WIDEN_PLUS_HI_EXPR
-@itemx VEC_WIDEN_PLUS_LO_EXPR
-These nodes represent widening vector addition of the high and low parts of
-the two input vectors, respectively. Their operands are vectors that contain
-the same number of elements (@code{N}) of the same integral type. The result
-is a vector that contains half as many elements, of an integral type whose size
-is twice as wide. In the case of @code{VEC_WIDEN_PLUS_HI_EXPR} the high
-@code{N/2} elements of the two vectors are added to produce the vector of
-@code{N/2} products. In the case of @code{VEC_WIDEN_PLUS_LO_EXPR} the low
-@code{N/2} elements of the two vectors are added to produce the vector of
-@code{N/2} products.
-
-@item VEC_WIDEN_MINUS_HI_EXPR
-@itemx VEC_WIDEN_MINUS_LO_EXPR
-These nodes represent widening vector subtraction of the high and low parts of
-the two input vectors, respectively. Their operands are vectors that contain
-the same number of elements (@code{N}) of the same integral type. The high/low
-elements of the second vector are subtracted from the high/low elements of the
-first. The result is a vector that contains half as many elements, of an
-integral type whose size is twice as wide. In the case of
-@code{VEC_WIDEN_MINUS_HI_EXPR} the high @code{N/2} elements of the second
-vector are subtracted from the high @code{N/2} of the first to produce the
-vector of @code{N/2} products. In the case of
-@code{VEC_WIDEN_MINUS_LO_EXPR} the low @code{N/2} elements of the second
-vector are subtracted from the low @code{N/2} of the first to produce the
-vector of @code{N/2} products.
-
@item VEC_UNPACK_HI_EXPR
@itemx VEC_UNPACK_LO_EXPR
These nodes represent unpacking of the high and low parts of the input vector,
diff --git a/gcc/expr.cc b/gcc/expr.cc
index 5d66c9f21f0ccd2eafb322eb9001f0dc873e35b4..b80385d51ba22172750d94535e04c82f75661255 100644
--- a/gcc/expr.cc
+++ b/gcc/expr.cc
@@ -9386,8 +9386,6 @@ expand_expr_real_2 (sepops ops, rtx target, machine_mode tmode,
target, unsignedp);
return target;
- case WIDEN_PLUS_EXPR:
- case WIDEN_MINUS_EXPR:
case WIDEN_MULT_EXPR:
/* If first operand is constant, swap them.
Thus the following special case checks need only
@@ -10165,10 +10163,6 @@ expand_expr_real_2 (sepops ops, rtx target, machine_mode tmode,
return temp;
}
- case VEC_WIDEN_PLUS_HI_EXPR:
- case VEC_WIDEN_PLUS_LO_EXPR:
- case VEC_WIDEN_MINUS_HI_EXPR:
- case VEC_WIDEN_MINUS_LO_EXPR:
case VEC_WIDEN_MULT_HI_EXPR:
case VEC_WIDEN_MULT_LO_EXPR:
case VEC_WIDEN_MULT_EVEN_EXPR:
diff --git a/gcc/gimple-pretty-print.cc b/gcc/gimple-pretty-print.cc
index ebd87b20a0adc080c4a8f9429e75f49b96e72f9a..2a1a5b7f811ca341e8ee7e85a9701d3a37ff80bf 100644
--- a/gcc/gimple-pretty-print.cc
+++ b/gcc/gimple-pretty-print.cc
@@ -459,10 +459,6 @@ dump_binary_rhs (pretty_printer *buffer, const gassign *gs, int spc,
case VEC_PACK_FLOAT_EXPR:
case VEC_WIDEN_LSHIFT_HI_EXPR:
case VEC_WIDEN_LSHIFT_LO_EXPR:
- case VEC_WIDEN_PLUS_HI_EXPR:
- case VEC_WIDEN_PLUS_LO_EXPR:
- case VEC_WIDEN_MINUS_HI_EXPR:
- case VEC_WIDEN_MINUS_LO_EXPR:
case VEC_SERIES_EXPR:
for (p = get_tree_code_name (code); *p; p++)
pp_character (buffer, TOUPPER (*p));
diff --git a/gcc/optabs-tree.cc b/gcc/optabs-tree.cc
index 8383fe820b080f6d66f83dcf3b77d3c9f869f4bc..2f5f93dc6624f86f6b5618cf6e7aa2b508053a64 100644
--- a/gcc/optabs-tree.cc
+++ b/gcc/optabs-tree.cc
@@ -190,22 +190,6 @@ optab_for_tree_code (enum tree_code code, const_tree type,
return (TYPE_UNSIGNED (type)
? vec_widen_ushiftl_lo_optab : vec_widen_sshiftl_lo_optab);
- case VEC_WIDEN_PLUS_LO_EXPR:
- return (TYPE_UNSIGNED (type)
- ? vec_widen_uaddl_lo_optab : vec_widen_saddl_lo_optab);
-
- case VEC_WIDEN_PLUS_HI_EXPR:
- return (TYPE_UNSIGNED (type)
- ? vec_widen_uaddl_hi_optab : vec_widen_saddl_hi_optab);
-
- case VEC_WIDEN_MINUS_LO_EXPR:
- return (TYPE_UNSIGNED (type)
- ? vec_widen_usubl_lo_optab : vec_widen_ssubl_lo_optab);
-
- case VEC_WIDEN_MINUS_HI_EXPR:
- return (TYPE_UNSIGNED (type)
- ? vec_widen_usubl_hi_optab : vec_widen_ssubl_hi_optab);
-
case VEC_UNPACK_HI_EXPR:
return (TYPE_UNSIGNED (type)
? vec_unpacku_hi_optab : vec_unpacks_hi_optab);
@@ -312,8 +296,6 @@ optab_for_tree_code (enum tree_code code, const_tree type,
'hi'/'lo' pair using codes such as VEC_WIDEN_MINUS_HI/LO.
Supported widening operations:
- WIDEN_MINUS_EXPR
- WIDEN_PLUS_EXPR
WIDEN_MULT_EXPR
WIDEN_LSHIFT_EXPR
@@ -345,12 +327,6 @@ supportable_half_widening_operation (enum tree_code code, tree vectype_out,
case WIDEN_LSHIFT_EXPR:
*code1 = LSHIFT_EXPR;
break;
- case WIDEN_MINUS_EXPR:
- *code1 = MINUS_EXPR;
- break;
- case WIDEN_PLUS_EXPR:
- *code1 = PLUS_EXPR;
- break;
case WIDEN_MULT_EXPR:
*code1 = MULT_EXPR;
break;
diff --git a/gcc/tree-cfg.cc b/gcc/tree-cfg.cc
index bfcb1425f7e2e46e3d525808adda11560041dd68..757c6c73e351c13bc6695699d9f449530546f70f 100644
--- a/gcc/tree-cfg.cc
+++ b/gcc/tree-cfg.cc
@@ -3951,8 +3951,6 @@ verify_gimple_assign_binary (gassign *stmt)
return false;
}
- case WIDEN_PLUS_EXPR:
- case WIDEN_MINUS_EXPR:
case PLUS_EXPR:
case MINUS_EXPR:
{
@@ -4073,10 +4071,6 @@ verify_gimple_assign_binary (gassign *stmt)
return false;
}
- case VEC_WIDEN_MINUS_HI_EXPR:
- case VEC_WIDEN_MINUS_LO_EXPR:
- case VEC_WIDEN_PLUS_HI_EXPR:
- case VEC_WIDEN_PLUS_LO_EXPR:
case VEC_WIDEN_MULT_HI_EXPR:
case VEC_WIDEN_MULT_LO_EXPR:
case VEC_WIDEN_MULT_EVEN_EXPR:
diff --git a/gcc/tree-inline.cc b/gcc/tree-inline.cc
index 043e1d5987a4c4b0159109dafb85a805ca828c1e..c0bebb7f4de36838341ed62389ad0e2b79f03034 100644
--- a/gcc/tree-inline.cc
+++ b/gcc/tree-inline.cc
@@ -4288,8 +4288,6 @@ estimate_operator_cost (enum tree_code code, eni_weights *weights,
case REALIGN_LOAD_EXPR:
- case WIDEN_PLUS_EXPR:
- case WIDEN_MINUS_EXPR:
case WIDEN_SUM_EXPR:
case WIDEN_MULT_EXPR:
case DOT_PROD_EXPR:
@@ -4298,10 +4296,6 @@ estimate_operator_cost (enum tree_code code, eni_weights *weights,
case WIDEN_MULT_MINUS_EXPR:
case WIDEN_LSHIFT_EXPR:
- case VEC_WIDEN_PLUS_HI_EXPR:
- case VEC_WIDEN_PLUS_LO_EXPR:
- case VEC_WIDEN_MINUS_HI_EXPR:
- case VEC_WIDEN_MINUS_LO_EXPR:
case VEC_WIDEN_MULT_HI_EXPR:
case VEC_WIDEN_MULT_LO_EXPR:
case VEC_WIDEN_MULT_EVEN_EXPR:
diff --git a/gcc/tree-pretty-print.cc b/gcc/tree-pretty-print.cc
index bfabe9e76279d7c3383b684ed61cc92228de4500..0ca8802576656f098e60cb77fa4312d1375ff3f0 100644
--- a/gcc/tree-pretty-print.cc
+++ b/gcc/tree-pretty-print.cc
@@ -2846,8 +2846,6 @@ dump_generic_node (pretty_printer *pp, tree node, int spc, dump_flags_t flags,
break;
/* Binary arithmetic and logic expressions. */
- case WIDEN_PLUS_EXPR:
- case WIDEN_MINUS_EXPR:
case WIDEN_SUM_EXPR:
case WIDEN_MULT_EXPR:
case MULT_EXPR:
@@ -3811,10 +3809,6 @@ dump_generic_node (pretty_printer *pp, tree node, int spc, dump_flags_t flags,
case VEC_SERIES_EXPR:
case VEC_WIDEN_MULT_HI_EXPR:
case VEC_WIDEN_MULT_LO_EXPR:
- case VEC_WIDEN_PLUS_HI_EXPR:
- case VEC_WIDEN_PLUS_LO_EXPR:
- case VEC_WIDEN_MINUS_HI_EXPR:
- case VEC_WIDEN_MINUS_LO_EXPR:
case VEC_WIDEN_MULT_EVEN_EXPR:
case VEC_WIDEN_MULT_ODD_EXPR:
case VEC_WIDEN_LSHIFT_HI_EXPR:
@@ -4332,12 +4326,6 @@ op_symbol_code (enum tree_code code)
case WIDEN_LSHIFT_EXPR:
return "w<<";
- case WIDEN_PLUS_EXPR:
- return "w+";
-
- case WIDEN_MINUS_EXPR:
- return "w-";
-
case POINTER_PLUS_EXPR:
return "+";
diff --git a/gcc/tree-vect-data-refs.cc b/gcc/tree-vect-data-refs.cc
index d20a10a1524164eef788ab4b88ba57c7a09c3387..98dd56ff022233ccead36a1f5a5e896e352f9f5b 100644
--- a/gcc/tree-vect-data-refs.cc
+++ b/gcc/tree-vect-data-refs.cc
@@ -136,8 +136,6 @@ vect_get_smallest_scalar_type (stmt_vec_info stmt_info, tree scalar_type)
|| gimple_assign_rhs_code (assign) == WIDEN_SUM_EXPR
|| gimple_assign_rhs_code (assign) == WIDEN_MULT_EXPR
|| gimple_assign_rhs_code (assign) == WIDEN_LSHIFT_EXPR
- || gimple_assign_rhs_code (assign) == WIDEN_PLUS_EXPR
- || gimple_assign_rhs_code (assign) == WIDEN_MINUS_EXPR
|| gimple_assign_rhs_code (assign) == FLOAT_EXPR)
{
tree rhs_type = TREE_TYPE (gimple_assign_rhs1 (assign));
@@ -3172,8 +3170,8 @@ vect_analyze_data_ref_accesses (vec_info *vinfo,
break;
/* Check that the DR_INITs are compile-time constants. */
- if (!tree_fits_shwi_p (DR_INIT (dra))
- || !tree_fits_shwi_p (DR_INIT (drb)))
+ if (TREE_CODE (DR_INIT (dra)) != INTEGER_CST
+ || TREE_CODE (DR_INIT (drb)) != INTEGER_CST)
break;
/* Different .GOMP_SIMD_LANE calls still give the same lane,
@@ -3225,7 +3223,7 @@ vect_analyze_data_ref_accesses (vec_info *vinfo,
unsigned HOST_WIDE_INT step
= absu_hwi (tree_to_shwi (DR_STEP (dra)));
if (step != 0
- && step <= ((unsigned HOST_WIDE_INT)init_b - init_a))
+ && step <= (unsigned HOST_WIDE_INT)(init_b - init_a))
break;
}
}
diff --git a/gcc/tree-vect-generic.cc b/gcc/tree-vect-generic.cc
index 350129555a0c71c0896c4f1003163f3b3557c11b..066f05873118c2288c90604e6287c91ef9aed72b 100644
--- a/gcc/tree-vect-generic.cc
+++ b/gcc/tree-vect-generic.cc
@@ -2209,10 +2209,6 @@ expand_vector_operations_1 (gimple_stmt_iterator *gsi,
arguments, not the widened result. VEC_UNPACK_FLOAT_*_EXPR is
calculated in the same way above. */
if (code == WIDEN_SUM_EXPR
- || code == VEC_WIDEN_PLUS_HI_EXPR
- || code == VEC_WIDEN_PLUS_LO_EXPR
- || code == VEC_WIDEN_MINUS_HI_EXPR
- || code == VEC_WIDEN_MINUS_LO_EXPR
|| code == VEC_WIDEN_MULT_HI_EXPR
|| code == VEC_WIDEN_MULT_LO_EXPR
|| code == VEC_WIDEN_MULT_EVEN_EXPR
diff --git a/gcc/tree-vect-patterns.cc b/gcc/tree-vect-patterns.cc
index 62ca28d725ed4ac8d7e4d493119e40772a0fbac6..9cd3989656c024b1d0394b2fcde6f6d774dff74e 100644
--- a/gcc/tree-vect-patterns.cc
+++ b/gcc/tree-vect-patterns.cc
@@ -559,21 +559,29 @@ vect_joust_widened_type (tree type, tree new_type, tree *common_type)
static unsigned int
vect_widened_op_tree (vec_info *vinfo, stmt_vec_info stmt_info, tree_code code,
- tree_code widened_code, bool shift_p,
+ code_helper widened_code, bool shift_p,
unsigned int max_nops,
vect_unpromoted_value *unprom, tree *common_type,
enum optab_subtype *subtype = NULL)
{
/* Check for an integer operation with the right code. */
- gassign *assign = dyn_cast <gassign *> (stmt_info->stmt);
- if (!assign)
+ gimple* stmt = stmt_info->stmt;
+ if (!(is_gimple_assign (stmt) || is_gimple_call (stmt)))
return 0;
- tree_code rhs_code = gimple_assign_rhs_code (assign);
- if (rhs_code != code && rhs_code != widened_code)
+ code_helper rhs_code;
+ if (is_gimple_assign (stmt))
+ rhs_code = gimple_assign_rhs_code (stmt);
+ else
+ rhs_code = gimple_call_combined_fn (stmt);
+
+ if (rhs_code.safe_as_tree_code () != code
+ && rhs_code.get_rep () != widened_code.get_rep ())
return 0;
- tree type = TREE_TYPE (gimple_assign_lhs (assign));
+ tree lhs = is_gimple_assign (stmt) ? gimple_assign_lhs (stmt):
+ gimple_call_lhs (stmt);
+ tree type = TREE_TYPE (lhs);
if (!INTEGRAL_TYPE_P (type))
return 0;
@@ -586,7 +594,11 @@ vect_widened_op_tree (vec_info *vinfo, stmt_vec_info stmt_info, tree_code code,
{
vect_unpromoted_value *this_unprom = &unprom[next_op];
unsigned int nops = 1;
- tree op = gimple_op (assign, i + 1);
+ tree op;
+ if (is_gimple_assign (stmt))
+ op = gimple_op (stmt, i + 1);
+ else
+ op = gimple_call_arg (stmt, i);
if (i == 1 && TREE_CODE (op) == INTEGER_CST)
{
/* We already have a common type from earlier operands.
@@ -1299,8 +1311,9 @@ vect_recog_sad_pattern (vec_info *vinfo,
/* FORNOW. Can continue analyzing the def-use chain when this stmt in a phi
inside the loop (in case we are analyzing an outer-loop). */
vect_unpromoted_value unprom[2];
- if (!vect_widened_op_tree (vinfo, diff_stmt_vinfo, MINUS_EXPR, WIDEN_MINUS_EXPR,
- false, 2, unprom, &half_type))
+ if (!vect_widened_op_tree (vinfo, diff_stmt_vinfo, MINUS_EXPR,
+ CFN_VEC_WIDEN_MINUS, false, 2, unprom,
+ &half_type))
return NULL;
vect_pattern_detected ("vect_recog_sad_pattern", last_stmt);
@@ -2337,9 +2350,10 @@ vect_recog_average_pattern (vec_info *vinfo,
internal_fn ifn = IFN_AVG_FLOOR;
vect_unpromoted_value unprom[3];
tree new_type;
+ enum optab_subtype subtype;
unsigned int nops = vect_widened_op_tree (vinfo, plus_stmt_info, PLUS_EXPR,
- WIDEN_PLUS_EXPR, false, 3,
- unprom, &new_type);
+ CFN_VEC_WIDEN_PLUS, false, 3,
+ unprom, &new_type, &subtype);
if (nops == 0)
return NULL;
if (nops == 3)
diff --git a/gcc/tree-vect-stmts.cc b/gcc/tree-vect-stmts.cc
index 6fa0669fdfc8630842b3f9f32f4b4a253e79bb92..92b17e6d0ec18ce3d90290dba9efec5d1968264c 100644
--- a/gcc/tree-vect-stmts.cc
+++ b/gcc/tree-vect-stmts.cc
@@ -4900,9 +4900,7 @@ vectorizable_conversion (vec_info *vinfo,
else
return false;
- bool widen_arith = (code == WIDEN_PLUS_EXPR
- || code == WIDEN_MINUS_EXPR
- || code == WIDEN_MULT_EXPR
+ bool widen_arith = (code == WIDEN_MULT_EXPR
|| code == WIDEN_LSHIFT_EXPR
|| code == IFN_VEC_WIDEN_PLUS
|| code == IFN_VEC_WIDEN_MINUS);
@@ -4954,8 +4952,6 @@ vectorizable_conversion (vec_info *vinfo,
{
gcc_assert (code == WIDEN_MULT_EXPR
|| code == WIDEN_LSHIFT_EXPR
- || code == WIDEN_PLUS_EXPR
- || code == WIDEN_MINUS_EXPR
|| code == IFN_VEC_WIDEN_PLUS
|| code == IFN_VEC_WIDEN_MINUS);
@@ -11986,7 +11982,7 @@ supportable_widening_operation (vec_info *vinfo,
class loop *vect_loop = NULL;
machine_mode vec_mode;
enum insn_code icode1, icode2;
- optab optab1, optab2;
+ optab optab1 = unknown_optab, optab2 = unknown_optab;
tree vectype = vectype_in;
tree wide_vectype = vectype_out;
code_helper c1 = MAX_TREE_CODES, c2 = MAX_TREE_CODES;
@@ -12080,16 +12076,6 @@ supportable_widening_operation (vec_info *vinfo,
c2 = VEC_WIDEN_LSHIFT_HI_EXPR;
break;
- case WIDEN_PLUS_EXPR:
- c1 = VEC_WIDEN_PLUS_LO_EXPR;
- c2 = VEC_WIDEN_PLUS_HI_EXPR;
- break;
-
- case WIDEN_MINUS_EXPR:
- c1 = VEC_WIDEN_MINUS_LO_EXPR;
- c2 = VEC_WIDEN_MINUS_HI_EXPR;
- break;
-
CASE_CONVERT:
c1 = VEC_UNPACK_LO_EXPR;
c2 = VEC_UNPACK_HI_EXPR;
diff --git a/gcc/tree.def b/gcc/tree.def
index 62650b6934b337c5d56e5393dc114173d72c9aa9..9b2dce3576440c445d3240b9ed937fe67c9a5992 100644
--- a/gcc/tree.def
+++ b/gcc/tree.def
@@ -1383,8 +1383,6 @@ DEFTREECODE (WIDEN_MULT_MINUS_EXPR, "widen_mult_minus_expr", tcc_expression, 3)
the first argument from type t1 to type t2, and then shifting it
by the second argument. */
DEFTREECODE (WIDEN_LSHIFT_EXPR, "widen_lshift_expr", tcc_binary, 2)
-DEFTREECODE (WIDEN_PLUS_EXPR, "widen_plus_expr", tcc_binary, 2)
-DEFTREECODE (WIDEN_MINUS_EXPR, "widen_minus_expr", tcc_binary, 2)
/* Widening vector multiplication.
The two operands are vectors with N elements of size S. Multiplying the
@@ -1449,10 +1447,6 @@ DEFTREECODE (VEC_PACK_FLOAT_EXPR, "vec_pack_float_expr", tcc_binary, 2)
*/
DEFTREECODE (VEC_WIDEN_LSHIFT_HI_EXPR, "widen_lshift_hi_expr", tcc_binary, 2)
DEFTREECODE (VEC_WIDEN_LSHIFT_LO_EXPR, "widen_lshift_lo_expr", tcc_binary, 2)
-DEFTREECODE (VEC_WIDEN_PLUS_HI_EXPR, "widen_plus_hi_expr", tcc_binary, 2)
-DEFTREECODE (VEC_WIDEN_PLUS_LO_EXPR, "widen_plus_lo_expr", tcc_binary, 2)
-DEFTREECODE (VEC_WIDEN_MINUS_HI_EXPR, "widen_minus_hi_expr", tcc_binary, 2)
-DEFTREECODE (VEC_WIDEN_MINUS_LO_EXPR, "widen_minus_lo_expr", tcc_binary, 2)
/* PREDICT_EXPR. Specify hint for branch prediction. The
PREDICT_EXPR_PREDICTOR specify predictor and PREDICT_EXPR_OUTCOME the
--
2.17.1
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [ping][vect-patterns] Refactor widen_plus/widen_minus as internal_fns
2022-06-07 8:18 ` Richard Sandiford
2022-06-07 9:01 ` Joel Hutton
@ 2022-06-13 9:18 ` Richard Biener
1 sibling, 0 replies; 22+ messages in thread
From: Richard Biener @ 2022-06-13 9:18 UTC (permalink / raw)
To: Richard Sandiford; +Cc: Joel Hutton, gcc-patches
On Tue, 7 Jun 2022, Richard Sandiford wrote:
> Joel Hutton <Joel.Hutton@arm.com> writes:
> >> > Patches attached. They already incorporated the .cc rename, now
> >> > rebased to be after the change to tree.h
> >>
> >> @@ -1412,8 +1412,7 @@ vect_recog_widen_op_pattern (vec_info *vinfo,
> >> 2, oprnd, half_type, unprom, vectype);
> >>
> >> tree var = vect_recog_temp_ssa_var (itype, NULL);
> >> - gimple *pattern_stmt = gimple_build_assign (var, wide_code,
> >> - oprnd[0], oprnd[1]);
> >> + gimple *pattern_stmt = gimple_build (var, wide_code, oprnd[0],
> >> oprnd[1]);
> >>
> >>
> >> you should be able to do without the new gimple_build overload
> >> by using
> >>
> >> gimple_seq stmts = NULL;
> >> gimple_build (&stmts, wide_code, itype, oprnd[0], oprnd[1]);
> >> gimple *pattern_stmt = gimple_seq_last_stmt (stmts);
> >>
> >> because 'gimple_build' is an existing API.
> >
> > Done.
> >
> > The gimple_build overload was at the request of Richard Sandiford, I assume removing it is ok with you Richard S?
> > From Richard Sandiford:
> >> For example, I think we should hide this inside a new:
> >>
> >> gimple_build (var, wide_code, oprnd[0], oprnd[1]);
> >>
> >> that works directly on code_helper, similarly to the new code_helper
> >> gimple_build interfaces.
>
> I thought the potential problem with the above is that gimple_build
> is a folding interface, so in principle it's allowed to return an
> existing SSA_NAME set by an existing statement (or even a constant).
> I think in this context we do need to force a new statement to be
> created.
Yes, that's due to how we use vect_finish_stmt_generation (only?).
It might be useful to add an overload that takes a gimple_seq
instead of a single gimple * for the vectorized stmt and leave
all the magic to that. Now - we have the additional issue
that we have STMT_VINFO_VEC_STMTS instead of STMT_VINFO_VEC_DEFS
(in the end we'll only ever need the defs, never the stmts I think).
I do think that we eventually want to 'enhance' the gimple.h
non-folding stmt building API, unfortunately I took the 'gimple_build'
name for the folding one, so alternatively we can unify assign/call
with gimple_build_assign_or_call (...). I don't really like the
idea of having folding and non-folding APIs being overloads :/
Maybe the non-folding API should be CTORs (guess GTY won't like
that) or static member functions:
gimple *gimple::build (tree, code_helper, tree, tree);
and in the long run the gimple_build API should be (for some uses?)
off a class as well, like instead of
gimple_seq seq = NULL;
op = gimple_build (&seq, ...);
do
gimple_builder b (location); // location defaulted to UNKNOWN
op = b.build (...);
So - writing the above I somewhat like the idea of static
member functions in 'gimple' (yes, at the root of the class
hierarchy, definitely not at gimple_statement_with_memory_ops_base,
not sure if we want gassign::build for assigns
and only the code_helper 'overloads' at the class root).
Richard.
--
Richard Biener <rguenther@suse.de>
SUSE Software Solutions Germany GmbH, Frankenstraße 146, 90461 Nuernberg,
Germany; GF: Ivo Totev, Andrew Myers, Andrew McDonald, Boudien Moerman;
HRB 36809 (AG Nuernberg)
^ permalink raw reply [flat|nested] 22+ messages in thread
* RE: [ping][vect-patterns] Refactor widen_plus/widen_minus as internal_fns
2022-06-07 9:01 ` Joel Hutton
2022-06-09 14:03 ` Joel Hutton
@ 2022-06-13 9:02 ` Richard Biener
2022-06-30 13:20 ` Joel Hutton
1 sibling, 1 reply; 22+ messages in thread
From: Richard Biener @ 2022-06-13 9:02 UTC (permalink / raw)
To: Joel Hutton; +Cc: Richard Sandiford, gcc-patches
On Tue, 7 Jun 2022, Joel Hutton wrote:
> Thanks Richard,
>
> > I thought the potential problem with the above is that gimple_build is a
> > folding interface, so in principle it's allowed to return an existing SSA_NAME
> > set by an existing statement (or even a constant).
> > I think in this context we do need to force a new statement to be created.
>
> Before I make any changes, I'd like to check we're all on the same page.
>
> richi, are you ok with the gimple_build function, perhaps with a
> different name if you are concerned with overloading? we could use
> gimple_ch_build or gimple_code_helper_build?
We can go with a private vect_gimple_build function until we sort out
the API issue to unblock Tamar (I'll reply to Richards reply with further
thoughts on this)
> Similarly are you ok with the use of gimple_extract_op? I would lean towards using it as it is cleaner, but I don't have strong feelings.
I don't like using gimple_extract_op here, I think I outlined a variant
that is even shorter.
Richard.
> Joel
>
> > -----Original Message-----
> > From: Richard Sandiford <richard.sandiford@arm.com>
> > Sent: 07 June 2022 09:18
> > To: Joel Hutton <Joel.Hutton@arm.com>
> > Cc: Richard Biener <rguenther@suse.de>; gcc-patches@gcc.gnu.org
> > Subject: Re: [ping][vect-patterns] Refactor widen_plus/widen_minus as
> > internal_fns
> >
> > Joel Hutton <Joel.Hutton@arm.com> writes:
> > >> > Patches attached. They already incorporated the .cc rename, now
> > >> > rebased to be after the change to tree.h
> > >>
> > >> @@ -1412,8 +1412,7 @@ vect_recog_widen_op_pattern (vec_info *vinfo,
> > >> 2, oprnd, half_type, unprom, vectype);
> > >>
> > >> tree var = vect_recog_temp_ssa_var (itype, NULL);
> > >> - gimple *pattern_stmt = gimple_build_assign (var, wide_code,
> > >> - oprnd[0], oprnd[1]);
> > >> + gimple *pattern_stmt = gimple_build (var, wide_code, oprnd[0],
> > >> oprnd[1]);
> > >>
> > >>
> > >> you should be able to do without the new gimple_build overload by
> > >> using
> > >>
> > >> gimple_seq stmts = NULL;
> > >> gimple_build (&stmts, wide_code, itype, oprnd[0], oprnd[1]);
> > >> gimple *pattern_stmt = gimple_seq_last_stmt (stmts);
> > >>
> > >> because 'gimple_build' is an existing API.
> > >
> > > Done.
> > >
> > > The gimple_build overload was at the request of Richard Sandiford, I
> > assume removing it is ok with you Richard S?
> > > From Richard Sandiford:
> > >> For example, I think we should hide this inside a new:
> > >>
> > >> gimple_build (var, wide_code, oprnd[0], oprnd[1]);
> > >>
> > >> that works directly on code_helper, similarly to the new code_helper
> > >> gimple_build interfaces.
> >
> > I thought the potential problem with the above is that gimple_build is a
> > folding interface, so in principle it's allowed to return an existing SSA_NAME
> > set by an existing statement (or even a constant).
> > I think in this context we do need to force a new statement to be created.
> >
> > Of course, the hope is that there wouldn't still be such folding opportunities
> > at this stage, but I don't think it's guaranteed (especially with options
> > fuzzing).
> >
> > Sind I was mentioned :-) ...
> >
> > Could you run the patch through contrib/check_GNU_style.py?
> > There seem to be a few long lines.
> >
> > > + if (res_op.code.is_tree_code ())
> >
> > Do you need this is_tree_code ()? These comparisons…
> >
> > > + {
> > > + widen_arith = (code == WIDEN_PLUS_EXPR
> > > + || code == WIDEN_MINUS_EXPR
> > > + || code == WIDEN_MULT_EXPR
> > > + || code == WIDEN_LSHIFT_EXPR);
> >
> > …ought to be safe unconditionally.
> >
> > > + }
> > > + else
> > > + widen_arith = false;
> > > +
> > > + if (!widen_arith
> > > + && !CONVERT_EXPR_CODE_P (code)
> > > + && code != FIX_TRUNC_EXPR
> > > + && code != FLOAT_EXPR)
> > > + return false;
> > >
> > > /* Check types of lhs and rhs. */
> > > - scalar_dest = gimple_assign_lhs (stmt);
> > > + scalar_dest = gimple_get_lhs (stmt);
> > > lhs_type = TREE_TYPE (scalar_dest);
> > > vectype_out = STMT_VINFO_VECTYPE (stmt_info);
> > >
> > > @@ -4938,10 +4951,14 @@ vectorizable_conversion (vec_info *vinfo,
> > >
> > > if (op_type == binary_op)
> > > {
> > > - gcc_assert (code == WIDEN_MULT_EXPR || code ==
> > WIDEN_LSHIFT_EXPR
> > > - || code == WIDEN_PLUS_EXPR || code ==
> > WIDEN_MINUS_EXPR);
> > > + gcc_assert (code == WIDEN_MULT_EXPR
> > > + || code == WIDEN_LSHIFT_EXPR
> > > + || code == WIDEN_PLUS_EXPR
> > > + || code == WIDEN_MINUS_EXPR);
> > >
> > > - op1 = gimple_assign_rhs2 (stmt);
> > > +
> > > + op1 = is_gimple_assign (stmt) ? gimple_assign_rhs2 (stmt) :
> > > + gimple_call_arg (stmt, 0);
> > > tree vectype1_in;
> > > if (!vect_is_simple_use (vinfo, stmt_info, slp_node, 1,
> > > &op1, &slp_op1, &dt[1], &vectype1_in)) […] @@
> > -12181,7
> > > +12235,6 @@ supportable_widening_operation (vec_info *vinfo,
> > > return false;
> > > }
> > >
> > > -
> > > /* Function supportable_narrowing_operation
> > >
> > > Check whether an operation represented by the code CODE is a
> >
> > Seems like a spurious change.
> >
> > > @@ -12205,7 +12258,7 @@ supportable_widening_operation (vec_info
> > > *vinfo, bool supportable_narrowing_operation (enum tree_code code,
> > > tree vectype_out, tree vectype_in,
> > > - enum tree_code *code1, int *multi_step_cvt,
> > > + tree_code* _code1, int *multi_step_cvt,
> >
> > The original formatting (space before the “*”) was correct.
> > Names beginning with _ are reserved, so I think we need a different
> > name here. Also, the name in the comment should stay in sync with
> > the name in the code.
> >
> > That said though, I'm not sure…
> >
> > > vec<tree> *interm_types)
> > > {
> > > machine_mode vec_mode;
> > > @@ -12217,8 +12270,8 @@ supportable_narrowing_operation (enum
> > tree_code code,
> > > tree intermediate_type, prev_type;
> > > machine_mode intermediate_mode, prev_mode;
> > > int i;
> > > - unsigned HOST_WIDE_INT n_elts;
> > > bool uns;
> > > + tree_code * code1 = (tree_code*) _code1;
> >
> > …the combination of these two changes makes sense on their own.
> >
> > >
> > > *multi_step_cvt = 0;
> > > switch (code)
> > > @@ -12227,9 +12280,8 @@ supportable_narrowing_operation (enum
> > tree_code code,
> > > c1 = VEC_PACK_TRUNC_EXPR;
> > > if (VECTOR_BOOLEAN_TYPE_P (narrow_vectype)
> > > && VECTOR_BOOLEAN_TYPE_P (vectype)
> > > - && SCALAR_INT_MODE_P (TYPE_MODE (vectype))
> > > - && TYPE_VECTOR_SUBPARTS (vectype).is_constant (&n_elts)
> > > - && n_elts < BITS_PER_UNIT)
> > > + && TYPE_MODE (narrow_vectype) == TYPE_MODE (vectype)
> > > + && SCALAR_INT_MODE_P (TYPE_MODE (vectype)))
> > > optab1 = vec_pack_sbool_trunc_optab;
> > > else
> > > optab1 = optab_for_tree_code (c1, vectype, optab_default);
> > > @@ -12320,9 +12372,8 @@ supportable_narrowing_operation (enum
> > tree_code code,
> > > = lang_hooks.types.type_for_mode (intermediate_mode, uns);
> > > if (VECTOR_BOOLEAN_TYPE_P (intermediate_type)
> > > && VECTOR_BOOLEAN_TYPE_P (prev_type)
> > > - && SCALAR_INT_MODE_P (prev_mode)
> > > - && TYPE_VECTOR_SUBPARTS (intermediate_type).is_constant
> > (&n_elts)
> > > - && n_elts < BITS_PER_UNIT)
> > > + && intermediate_mode == prev_mode
> > > + && SCALAR_INT_MODE_P (prev_mode))
> > > interm_optab = vec_pack_sbool_trunc_optab;
> > > else
> > > interm_optab
> >
> > This part looks like a behavioural change, so I think it should be part
> > of a separate patch.
> >
> > > diff --git a/gcc/tree-vectorizer.h b/gcc/tree-vectorizer.h
> > > index
> > 642eb0aeb21264cd736a479b1ec25357abef29cd..50ff8eeac1e6b9859302bd78
> > 4f10ee3d8ff4b4dc 100644
> > > --- a/gcc/tree-vectorizer.h
> > > +++ b/gcc/tree-vectorizer.h
> > > @@ -2120,13 +2120,12 @@ extern bool vect_is_simple_use (vec_info *,
> > stmt_vec_info, slp_tree,
> > > enum vect_def_type *,
> > > tree *, stmt_vec_info * = NULL);
> > > extern bool vect_maybe_update_slp_op_vectype (slp_tree, tree);
> > > -extern bool supportable_widening_operation (vec_info *,
> > > - enum tree_code, stmt_vec_info,
> > > - tree, tree, enum tree_code *,
> > > - enum tree_code *, int *,
> > > - vec<tree> *);
> > > +extern bool supportable_widening_operation (vec_info*, code_helper,
> > > + stmt_vec_info, tree, tree,
> > > + code_helper*, code_helper*,
> > > + int*, vec<tree> *);
> > > extern bool supportable_narrowing_operation (enum tree_code, tree,
> > tree,
> > > - enum tree_code *, int *,
> > > + tree_code *, int *,
> > > vec<tree> *);
> > >
> > > extern unsigned record_stmt_cost (stmt_vector_for_cost *, int,
> > > diff --git a/gcc/tree.h b/gcc/tree.h
> > > index
> > f84958933d51144bb6ce7cc41eca5f7f06814550..00b0e4d1c696633fe38baad5
> > 295b1f90398d53fc 100644
> > > --- a/gcc/tree.h
> > > +++ b/gcc/tree.h
> > > @@ -92,6 +92,10 @@ public:
> > > bool is_fn_code () const { return rep < 0; }
> > > bool is_internal_fn () const;
> > > bool is_builtin_fn () const;
> > > + enum tree_code safe_as_tree_code () const { return is_tree_code () ?
> > > + (tree_code)* this : MAX_TREE_CODES; }
> > > + combined_fn safe_as_fn_code () const { return is_fn_code () ?
> > (combined_fn) *this
> > > + : CFN_LAST;}
> >
> > Since these don't fit on a line, the coding convention says that they
> > should be defined outside of the class.
> >
> > Thanks,
> > Richard
> >
> > > int get_rep () const { return rep; }
> > > bool operator== (const code_helper &other) { return rep == other.rep; }
> > > bool operator!= (const code_helper &other) { return rep != other.rep; }
>
--
Richard Biener <rguenther@suse.de>
SUSE Software Solutions Germany GmbH, Frankenstraße 146, 90461 Nuernberg,
Germany; GF: Ivo Totev, Andrew Myers, Andrew McDonald, Boudien Moerman;
HRB 36809 (AG Nuernberg)
^ permalink raw reply [flat|nested] 22+ messages in thread
* [ping][vect-patterns] Refactor widen_plus/widen_minus as internal_fns
2022-06-07 9:01 ` Joel Hutton
@ 2022-06-09 14:03 ` Joel Hutton
2022-06-13 9:02 ` Richard Biener
1 sibling, 0 replies; 22+ messages in thread
From: Joel Hutton @ 2022-06-09 14:03 UTC (permalink / raw)
To: Richard Sandiford; +Cc: Richard Biener, gcc-patches
> Before I make any changes, I'd like to check we're all on the same page.
>
> richi, are you ok with the gimple_build function, perhaps with a different
> name if you are concerned with overloading? we could use gimple_ch_build
> or gimple_code_helper_build?
>
> Similarly are you ok with the use of gimple_extract_op? I would lean towards
> using it as it is cleaner, but I don't have strong feelings.
>
> Joel
Ping. Just looking for some confirmation before I rework this patch. It would be good to get some agreement on this as Tamar is blocked on this patch.
Joel
> -----Original Message-----
> From: Joel Hutton
> Sent: 07 June 2022 10:02
> To: Richard Sandiford <richard.sandiford@arm.com>
> Cc: Richard Biener <rguenther@suse.de>; gcc-patches@gcc.gnu.org
> Subject: RE: [ping][vect-patterns] Refactor widen_plus/widen_minus as
> internal_fns
>
> Thanks Richard,
>
> > I thought the potential problem with the above is that gimple_build is
> > a folding interface, so in principle it's allowed to return an
> > existing SSA_NAME set by an existing statement (or even a constant).
> > I think in this context we do need to force a new statement to be created.
>
> Before I make any changes, I'd like to check we're all on the same page.
>
> richi, are you ok with the gimple_build function, perhaps with a different
> name if you are concerned with overloading? we could use gimple_ch_build
> or gimple_code_helper_build?
>
> Similarly are you ok with the use of gimple_extract_op? I would lean towards
> using it as it is cleaner, but I don't have strong feelings.
>
> Joel
>
> > -----Original Message-----
> > From: Richard Sandiford <richard.sandiford@arm.com>
> > Sent: 07 June 2022 09:18
> > To: Joel Hutton <Joel.Hutton@arm.com>
> > Cc: Richard Biener <rguenther@suse.de>; gcc-patches@gcc.gnu.org
> > Subject: Re: [ping][vect-patterns] Refactor widen_plus/widen_minus as
> > internal_fns
> >
> > Joel Hutton <Joel.Hutton@arm.com> writes:
> > >> > Patches attached. They already incorporated the .cc rename, now
> > >> > rebased to be after the change to tree.h
> > >>
> > >> @@ -1412,8 +1412,7 @@ vect_recog_widen_op_pattern (vec_info
> *vinfo,
> > >> 2, oprnd, half_type, unprom, vectype);
> > >>
> > >> tree var = vect_recog_temp_ssa_var (itype, NULL);
> > >> - gimple *pattern_stmt = gimple_build_assign (var, wide_code,
> > >> - oprnd[0], oprnd[1]);
> > >> + gimple *pattern_stmt = gimple_build (var, wide_code, oprnd[0],
> > >> oprnd[1]);
> > >>
> > >>
> > >> you should be able to do without the new gimple_build overload by
> > >> using
> > >>
> > >> gimple_seq stmts = NULL;
> > >> gimple_build (&stmts, wide_code, itype, oprnd[0], oprnd[1]);
> > >> gimple *pattern_stmt = gimple_seq_last_stmt (stmts);
> > >>
> > >> because 'gimple_build' is an existing API.
> > >
> > > Done.
> > >
> > > The gimple_build overload was at the request of Richard Sandiford, I
> > assume removing it is ok with you Richard S?
> > > From Richard Sandiford:
> > >> For example, I think we should hide this inside a new:
> > >>
> > >> gimple_build (var, wide_code, oprnd[0], oprnd[1]);
> > >>
> > >> that works directly on code_helper, similarly to the new
> > >> code_helper gimple_build interfaces.
> >
> > I thought the potential problem with the above is that gimple_build is
> > a folding interface, so in principle it's allowed to return an
> > existing SSA_NAME set by an existing statement (or even a constant).
> > I think in this context we do need to force a new statement to be created.
> >
> > Of course, the hope is that there wouldn't still be such folding
> > opportunities at this stage, but I don't think it's guaranteed
> > (especially with options fuzzing).
> >
> > Sind I was mentioned :-) ...
> >
> > Could you run the patch through contrib/check_GNU_style.py?
> > There seem to be a few long lines.
> >
> > > + if (res_op.code.is_tree_code ())
> >
> > Do you need this is_tree_code ()? These comparisons…
> >
> > > + {
> > > + widen_arith = (code == WIDEN_PLUS_EXPR
> > > + || code == WIDEN_MINUS_EXPR
> > > + || code == WIDEN_MULT_EXPR
> > > + || code == WIDEN_LSHIFT_EXPR);
> >
> > …ought to be safe unconditionally.
> >
> > > + }
> > > + else
> > > + widen_arith = false;
> > > +
> > > + if (!widen_arith
> > > + && !CONVERT_EXPR_CODE_P (code)
> > > + && code != FIX_TRUNC_EXPR
> > > + && code != FLOAT_EXPR)
> > > + return false;
> > >
> > > /* Check types of lhs and rhs. */
> > > - scalar_dest = gimple_assign_lhs (stmt);
> > > + scalar_dest = gimple_get_lhs (stmt);
> > > lhs_type = TREE_TYPE (scalar_dest);
> > > vectype_out = STMT_VINFO_VECTYPE (stmt_info);
> > >
> > > @@ -4938,10 +4951,14 @@ vectorizable_conversion (vec_info *vinfo,
> > >
> > > if (op_type == binary_op)
> > > {
> > > - gcc_assert (code == WIDEN_MULT_EXPR || code ==
> > WIDEN_LSHIFT_EXPR
> > > - || code == WIDEN_PLUS_EXPR || code ==
> > WIDEN_MINUS_EXPR);
> > > + gcc_assert (code == WIDEN_MULT_EXPR
> > > + || code == WIDEN_LSHIFT_EXPR
> > > + || code == WIDEN_PLUS_EXPR
> > > + || code == WIDEN_MINUS_EXPR);
> > >
> > > - op1 = gimple_assign_rhs2 (stmt);
> > > +
> > > + op1 = is_gimple_assign (stmt) ? gimple_assign_rhs2 (stmt) :
> > > + gimple_call_arg (stmt, 0);
> > > tree vectype1_in;
> > > if (!vect_is_simple_use (vinfo, stmt_info, slp_node, 1,
> > > &op1, &slp_op1, &dt[1], &vectype1_in)) […] @@
> > -12181,7
> > > +12235,6 @@ supportable_widening_operation (vec_info *vinfo,
> > > return false;
> > > }
> > >
> > > -
> > > /* Function supportable_narrowing_operation
> > >
> > > Check whether an operation represented by the code CODE is a
> >
> > Seems like a spurious change.
> >
> > > @@ -12205,7 +12258,7 @@ supportable_widening_operation (vec_info
> > > *vinfo, bool supportable_narrowing_operation (enum tree_code code,
> > > tree vectype_out, tree vectype_in,
> > > - enum tree_code *code1, int *multi_step_cvt,
> > > + tree_code* _code1, int *multi_step_cvt,
> >
> > The original formatting (space before the “*”) was correct.
> > Names beginning with _ are reserved, so I think we need a different
> > name here. Also, the name in the comment should stay in sync with the
> > name in the code.
> >
> > That said though, I'm not sure…
> >
> > > vec<tree> *interm_types) {
> > > machine_mode vec_mode;
> > > @@ -12217,8 +12270,8 @@ supportable_narrowing_operation (enum
> > tree_code code,
> > > tree intermediate_type, prev_type;
> > > machine_mode intermediate_mode, prev_mode;
> > > int i;
> > > - unsigned HOST_WIDE_INT n_elts;
> > > bool uns;
> > > + tree_code * code1 = (tree_code*) _code1;
> >
> > …the combination of these two changes makes sense on their own.
> >
> > >
> > > *multi_step_cvt = 0;
> > > switch (code)
> > > @@ -12227,9 +12280,8 @@ supportable_narrowing_operation (enum
> > tree_code code,
> > > c1 = VEC_PACK_TRUNC_EXPR;
> > > if (VECTOR_BOOLEAN_TYPE_P (narrow_vectype)
> > > && VECTOR_BOOLEAN_TYPE_P (vectype)
> > > - && SCALAR_INT_MODE_P (TYPE_MODE (vectype))
> > > - && TYPE_VECTOR_SUBPARTS (vectype).is_constant (&n_elts)
> > > - && n_elts < BITS_PER_UNIT)
> > > + && TYPE_MODE (narrow_vectype) == TYPE_MODE (vectype)
> > > + && SCALAR_INT_MODE_P (TYPE_MODE (vectype)))
> > > optab1 = vec_pack_sbool_trunc_optab;
> > > else
> > > optab1 = optab_for_tree_code (c1, vectype, optab_default); @@
> > > -12320,9 +12372,8 @@ supportable_narrowing_operation (enum
> > tree_code code,
> > > = lang_hooks.types.type_for_mode (intermediate_mode, uns);
> > > if (VECTOR_BOOLEAN_TYPE_P (intermediate_type)
> > > && VECTOR_BOOLEAN_TYPE_P (prev_type)
> > > - && SCALAR_INT_MODE_P (prev_mode)
> > > - && TYPE_VECTOR_SUBPARTS (intermediate_type).is_constant
> > (&n_elts)
> > > - && n_elts < BITS_PER_UNIT)
> > > + && intermediate_mode == prev_mode
> > > + && SCALAR_INT_MODE_P (prev_mode))
> > > interm_optab = vec_pack_sbool_trunc_optab;
> > > else
> > > interm_optab
> >
> > This part looks like a behavioural change, so I think it should be
> > part of a separate patch.
> >
> > > diff --git a/gcc/tree-vectorizer.h b/gcc/tree-vectorizer.h index
> >
> 642eb0aeb21264cd736a479b1ec25357abef29cd..50ff8eeac1e6b9859302bd78
> > 4f10ee3d8ff4b4dc 100644
> > > --- a/gcc/tree-vectorizer.h
> > > +++ b/gcc/tree-vectorizer.h
> > > @@ -2120,13 +2120,12 @@ extern bool vect_is_simple_use (vec_info *,
> > stmt_vec_info, slp_tree,
> > > enum vect_def_type *,
> > > tree *, stmt_vec_info * = NULL); extern bool
> > > vect_maybe_update_slp_op_vectype (slp_tree, tree); -extern bool
> > > supportable_widening_operation (vec_info *,
> > > - enum tree_code, stmt_vec_info,
> > > - tree, tree, enum tree_code *,
> > > - enum tree_code *, int *,
> > > - vec<tree> *);
> > > +extern bool supportable_widening_operation (vec_info*, code_helper,
> > > + stmt_vec_info, tree, tree,
> > > + code_helper*, code_helper*,
> > > + int*, vec<tree> *);
> > > extern bool supportable_narrowing_operation (enum tree_code, tree,
> > tree,
> > > - enum tree_code *, int *,
> > > + tree_code *, int *,
> > > vec<tree> *);
> > >
> > > extern unsigned record_stmt_cost (stmt_vector_for_cost *, int, diff
> > > --git a/gcc/tree.h b/gcc/tree.h index
> >
> f84958933d51144bb6ce7cc41eca5f7f06814550..00b0e4d1c696633fe38baad5
> > 295b1f90398d53fc 100644
> > > --- a/gcc/tree.h
> > > +++ b/gcc/tree.h
> > > @@ -92,6 +92,10 @@ public:
> > > bool is_fn_code () const { return rep < 0; }
> > > bool is_internal_fn () const;
> > > bool is_builtin_fn () const;
> > > + enum tree_code safe_as_tree_code () const { return is_tree_code () ?
> > > + (tree_code)* this : MAX_TREE_CODES; } combined_fn
> > > + safe_as_fn_code () const { return is_fn_code () ?
> > (combined_fn) *this
> > > + : CFN_LAST;}
> >
> > Since these don't fit on a line, the coding convention says that they
> > should be defined outside of the class.
> >
> > Thanks,
> > Richard
> >
> > > int get_rep () const { return rep; }
> > > bool operator== (const code_helper &other) { return rep == other.rep; }
> > > bool operator!= (const code_helper &other) { return rep !=
> > > other.rep; }
^ permalink raw reply [flat|nested] 22+ messages in thread
* RE: [ping][vect-patterns] Refactor widen_plus/widen_minus as internal_fns
2022-06-07 8:18 ` Richard Sandiford
@ 2022-06-07 9:01 ` Joel Hutton
2022-06-09 14:03 ` Joel Hutton
2022-06-13 9:02 ` Richard Biener
2022-06-13 9:18 ` Richard Biener
1 sibling, 2 replies; 22+ messages in thread
From: Joel Hutton @ 2022-06-07 9:01 UTC (permalink / raw)
To: Richard Sandiford; +Cc: Richard Biener, gcc-patches
Thanks Richard,
> I thought the potential problem with the above is that gimple_build is a
> folding interface, so in principle it's allowed to return an existing SSA_NAME
> set by an existing statement (or even a constant).
> I think in this context we do need to force a new statement to be created.
Before I make any changes, I'd like to check we're all on the same page.
richi, are you ok with the gimple_build function, perhaps with a different name if you are concerned with overloading? we could use gimple_ch_build or gimple_code_helper_build?
Similarly are you ok with the use of gimple_extract_op? I would lean towards using it as it is cleaner, but I don't have strong feelings.
Joel
> -----Original Message-----
> From: Richard Sandiford <richard.sandiford@arm.com>
> Sent: 07 June 2022 09:18
> To: Joel Hutton <Joel.Hutton@arm.com>
> Cc: Richard Biener <rguenther@suse.de>; gcc-patches@gcc.gnu.org
> Subject: Re: [ping][vect-patterns] Refactor widen_plus/widen_minus as
> internal_fns
>
> Joel Hutton <Joel.Hutton@arm.com> writes:
> >> > Patches attached. They already incorporated the .cc rename, now
> >> > rebased to be after the change to tree.h
> >>
> >> @@ -1412,8 +1412,7 @@ vect_recog_widen_op_pattern (vec_info *vinfo,
> >> 2, oprnd, half_type, unprom, vectype);
> >>
> >> tree var = vect_recog_temp_ssa_var (itype, NULL);
> >> - gimple *pattern_stmt = gimple_build_assign (var, wide_code,
> >> - oprnd[0], oprnd[1]);
> >> + gimple *pattern_stmt = gimple_build (var, wide_code, oprnd[0],
> >> oprnd[1]);
> >>
> >>
> >> you should be able to do without the new gimple_build overload by
> >> using
> >>
> >> gimple_seq stmts = NULL;
> >> gimple_build (&stmts, wide_code, itype, oprnd[0], oprnd[1]);
> >> gimple *pattern_stmt = gimple_seq_last_stmt (stmts);
> >>
> >> because 'gimple_build' is an existing API.
> >
> > Done.
> >
> > The gimple_build overload was at the request of Richard Sandiford, I
> assume removing it is ok with you Richard S?
> > From Richard Sandiford:
> >> For example, I think we should hide this inside a new:
> >>
> >> gimple_build (var, wide_code, oprnd[0], oprnd[1]);
> >>
> >> that works directly on code_helper, similarly to the new code_helper
> >> gimple_build interfaces.
>
> I thought the potential problem with the above is that gimple_build is a
> folding interface, so in principle it's allowed to return an existing SSA_NAME
> set by an existing statement (or even a constant).
> I think in this context we do need to force a new statement to be created.
>
> Of course, the hope is that there wouldn't still be such folding opportunities
> at this stage, but I don't think it's guaranteed (especially with options
> fuzzing).
>
> Sind I was mentioned :-) ...
>
> Could you run the patch through contrib/check_GNU_style.py?
> There seem to be a few long lines.
>
> > + if (res_op.code.is_tree_code ())
>
> Do you need this is_tree_code ()? These comparisons…
>
> > + {
> > + widen_arith = (code == WIDEN_PLUS_EXPR
> > + || code == WIDEN_MINUS_EXPR
> > + || code == WIDEN_MULT_EXPR
> > + || code == WIDEN_LSHIFT_EXPR);
>
> …ought to be safe unconditionally.
>
> > + }
> > + else
> > + widen_arith = false;
> > +
> > + if (!widen_arith
> > + && !CONVERT_EXPR_CODE_P (code)
> > + && code != FIX_TRUNC_EXPR
> > + && code != FLOAT_EXPR)
> > + return false;
> >
> > /* Check types of lhs and rhs. */
> > - scalar_dest = gimple_assign_lhs (stmt);
> > + scalar_dest = gimple_get_lhs (stmt);
> > lhs_type = TREE_TYPE (scalar_dest);
> > vectype_out = STMT_VINFO_VECTYPE (stmt_info);
> >
> > @@ -4938,10 +4951,14 @@ vectorizable_conversion (vec_info *vinfo,
> >
> > if (op_type == binary_op)
> > {
> > - gcc_assert (code == WIDEN_MULT_EXPR || code ==
> WIDEN_LSHIFT_EXPR
> > - || code == WIDEN_PLUS_EXPR || code ==
> WIDEN_MINUS_EXPR);
> > + gcc_assert (code == WIDEN_MULT_EXPR
> > + || code == WIDEN_LSHIFT_EXPR
> > + || code == WIDEN_PLUS_EXPR
> > + || code == WIDEN_MINUS_EXPR);
> >
> > - op1 = gimple_assign_rhs2 (stmt);
> > +
> > + op1 = is_gimple_assign (stmt) ? gimple_assign_rhs2 (stmt) :
> > + gimple_call_arg (stmt, 0);
> > tree vectype1_in;
> > if (!vect_is_simple_use (vinfo, stmt_info, slp_node, 1,
> > &op1, &slp_op1, &dt[1], &vectype1_in)) […] @@
> -12181,7
> > +12235,6 @@ supportable_widening_operation (vec_info *vinfo,
> > return false;
> > }
> >
> > -
> > /* Function supportable_narrowing_operation
> >
> > Check whether an operation represented by the code CODE is a
>
> Seems like a spurious change.
>
> > @@ -12205,7 +12258,7 @@ supportable_widening_operation (vec_info
> > *vinfo, bool supportable_narrowing_operation (enum tree_code code,
> > tree vectype_out, tree vectype_in,
> > - enum tree_code *code1, int *multi_step_cvt,
> > + tree_code* _code1, int *multi_step_cvt,
>
> The original formatting (space before the “*”) was correct.
> Names beginning with _ are reserved, so I think we need a different
> name here. Also, the name in the comment should stay in sync with
> the name in the code.
>
> That said though, I'm not sure…
>
> > vec<tree> *interm_types)
> > {
> > machine_mode vec_mode;
> > @@ -12217,8 +12270,8 @@ supportable_narrowing_operation (enum
> tree_code code,
> > tree intermediate_type, prev_type;
> > machine_mode intermediate_mode, prev_mode;
> > int i;
> > - unsigned HOST_WIDE_INT n_elts;
> > bool uns;
> > + tree_code * code1 = (tree_code*) _code1;
>
> …the combination of these two changes makes sense on their own.
>
> >
> > *multi_step_cvt = 0;
> > switch (code)
> > @@ -12227,9 +12280,8 @@ supportable_narrowing_operation (enum
> tree_code code,
> > c1 = VEC_PACK_TRUNC_EXPR;
> > if (VECTOR_BOOLEAN_TYPE_P (narrow_vectype)
> > && VECTOR_BOOLEAN_TYPE_P (vectype)
> > - && SCALAR_INT_MODE_P (TYPE_MODE (vectype))
> > - && TYPE_VECTOR_SUBPARTS (vectype).is_constant (&n_elts)
> > - && n_elts < BITS_PER_UNIT)
> > + && TYPE_MODE (narrow_vectype) == TYPE_MODE (vectype)
> > + && SCALAR_INT_MODE_P (TYPE_MODE (vectype)))
> > optab1 = vec_pack_sbool_trunc_optab;
> > else
> > optab1 = optab_for_tree_code (c1, vectype, optab_default);
> > @@ -12320,9 +12372,8 @@ supportable_narrowing_operation (enum
> tree_code code,
> > = lang_hooks.types.type_for_mode (intermediate_mode, uns);
> > if (VECTOR_BOOLEAN_TYPE_P (intermediate_type)
> > && VECTOR_BOOLEAN_TYPE_P (prev_type)
> > - && SCALAR_INT_MODE_P (prev_mode)
> > - && TYPE_VECTOR_SUBPARTS (intermediate_type).is_constant
> (&n_elts)
> > - && n_elts < BITS_PER_UNIT)
> > + && intermediate_mode == prev_mode
> > + && SCALAR_INT_MODE_P (prev_mode))
> > interm_optab = vec_pack_sbool_trunc_optab;
> > else
> > interm_optab
>
> This part looks like a behavioural change, so I think it should be part
> of a separate patch.
>
> > diff --git a/gcc/tree-vectorizer.h b/gcc/tree-vectorizer.h
> > index
> 642eb0aeb21264cd736a479b1ec25357abef29cd..50ff8eeac1e6b9859302bd78
> 4f10ee3d8ff4b4dc 100644
> > --- a/gcc/tree-vectorizer.h
> > +++ b/gcc/tree-vectorizer.h
> > @@ -2120,13 +2120,12 @@ extern bool vect_is_simple_use (vec_info *,
> stmt_vec_info, slp_tree,
> > enum vect_def_type *,
> > tree *, stmt_vec_info * = NULL);
> > extern bool vect_maybe_update_slp_op_vectype (slp_tree, tree);
> > -extern bool supportable_widening_operation (vec_info *,
> > - enum tree_code, stmt_vec_info,
> > - tree, tree, enum tree_code *,
> > - enum tree_code *, int *,
> > - vec<tree> *);
> > +extern bool supportable_widening_operation (vec_info*, code_helper,
> > + stmt_vec_info, tree, tree,
> > + code_helper*, code_helper*,
> > + int*, vec<tree> *);
> > extern bool supportable_narrowing_operation (enum tree_code, tree,
> tree,
> > - enum tree_code *, int *,
> > + tree_code *, int *,
> > vec<tree> *);
> >
> > extern unsigned record_stmt_cost (stmt_vector_for_cost *, int,
> > diff --git a/gcc/tree.h b/gcc/tree.h
> > index
> f84958933d51144bb6ce7cc41eca5f7f06814550..00b0e4d1c696633fe38baad5
> 295b1f90398d53fc 100644
> > --- a/gcc/tree.h
> > +++ b/gcc/tree.h
> > @@ -92,6 +92,10 @@ public:
> > bool is_fn_code () const { return rep < 0; }
> > bool is_internal_fn () const;
> > bool is_builtin_fn () const;
> > + enum tree_code safe_as_tree_code () const { return is_tree_code () ?
> > + (tree_code)* this : MAX_TREE_CODES; }
> > + combined_fn safe_as_fn_code () const { return is_fn_code () ?
> (combined_fn) *this
> > + : CFN_LAST;}
>
> Since these don't fit on a line, the coding convention says that they
> should be defined outside of the class.
>
> Thanks,
> Richard
>
> > int get_rep () const { return rep; }
> > bool operator== (const code_helper &other) { return rep == other.rep; }
> > bool operator!= (const code_helper &other) { return rep != other.rep; }
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [ping][vect-patterns] Refactor widen_plus/widen_minus as internal_fns
2022-06-06 17:20 ` Joel Hutton
@ 2022-06-07 8:18 ` Richard Sandiford
2022-06-07 9:01 ` Joel Hutton
2022-06-13 9:18 ` Richard Biener
0 siblings, 2 replies; 22+ messages in thread
From: Richard Sandiford @ 2022-06-07 8:18 UTC (permalink / raw)
To: Joel Hutton; +Cc: Richard Biener, gcc-patches
Joel Hutton <Joel.Hutton@arm.com> writes:
>> > Patches attached. They already incorporated the .cc rename, now
>> > rebased to be after the change to tree.h
>>
>> @@ -1412,8 +1412,7 @@ vect_recog_widen_op_pattern (vec_info *vinfo,
>> 2, oprnd, half_type, unprom, vectype);
>>
>> tree var = vect_recog_temp_ssa_var (itype, NULL);
>> - gimple *pattern_stmt = gimple_build_assign (var, wide_code,
>> - oprnd[0], oprnd[1]);
>> + gimple *pattern_stmt = gimple_build (var, wide_code, oprnd[0],
>> oprnd[1]);
>>
>>
>> you should be able to do without the new gimple_build overload
>> by using
>>
>> gimple_seq stmts = NULL;
>> gimple_build (&stmts, wide_code, itype, oprnd[0], oprnd[1]);
>> gimple *pattern_stmt = gimple_seq_last_stmt (stmts);
>>
>> because 'gimple_build' is an existing API.
>
> Done.
>
> The gimple_build overload was at the request of Richard Sandiford, I assume removing it is ok with you Richard S?
> From Richard Sandiford:
>> For example, I think we should hide this inside a new:
>>
>> gimple_build (var, wide_code, oprnd[0], oprnd[1]);
>>
>> that works directly on code_helper, similarly to the new code_helper
>> gimple_build interfaces.
I thought the potential problem with the above is that gimple_build
is a folding interface, so in principle it's allowed to return an
existing SSA_NAME set by an existing statement (or even a constant).
I think in this context we do need to force a new statement to be
created.
Of course, the hope is that there wouldn't still be such folding
opportunities at this stage, but I don't think it's guaranteed
(especially with options fuzzing).
Sind I was mentioned :-) ...
Could you run the patch through contrib/check_GNU_style.py?
There seem to be a few long lines.
> + if (res_op.code.is_tree_code ())
Do you need this is_tree_code ()? These comparisons…
> + {
> + widen_arith = (code == WIDEN_PLUS_EXPR
> + || code == WIDEN_MINUS_EXPR
> + || code == WIDEN_MULT_EXPR
> + || code == WIDEN_LSHIFT_EXPR);
…ought to be safe unconditionally.
> + }
> + else
> + widen_arith = false;
> +
> + if (!widen_arith
> + && !CONVERT_EXPR_CODE_P (code)
> + && code != FIX_TRUNC_EXPR
> + && code != FLOAT_EXPR)
> + return false;
>
> /* Check types of lhs and rhs. */
> - scalar_dest = gimple_assign_lhs (stmt);
> + scalar_dest = gimple_get_lhs (stmt);
> lhs_type = TREE_TYPE (scalar_dest);
> vectype_out = STMT_VINFO_VECTYPE (stmt_info);
>
> @@ -4938,10 +4951,14 @@ vectorizable_conversion (vec_info *vinfo,
>
> if (op_type == binary_op)
> {
> - gcc_assert (code == WIDEN_MULT_EXPR || code == WIDEN_LSHIFT_EXPR
> - || code == WIDEN_PLUS_EXPR || code == WIDEN_MINUS_EXPR);
> + gcc_assert (code == WIDEN_MULT_EXPR
> + || code == WIDEN_LSHIFT_EXPR
> + || code == WIDEN_PLUS_EXPR
> + || code == WIDEN_MINUS_EXPR);
>
> - op1 = gimple_assign_rhs2 (stmt);
> +
> + op1 = is_gimple_assign (stmt) ? gimple_assign_rhs2 (stmt) :
> + gimple_call_arg (stmt, 0);
> tree vectype1_in;
> if (!vect_is_simple_use (vinfo, stmt_info, slp_node, 1,
> &op1, &slp_op1, &dt[1], &vectype1_in))
> […]
> @@ -12181,7 +12235,6 @@ supportable_widening_operation (vec_info *vinfo,
> return false;
> }
>
> -
> /* Function supportable_narrowing_operation
>
> Check whether an operation represented by the code CODE is a
Seems like a spurious change.
> @@ -12205,7 +12258,7 @@ supportable_widening_operation (vec_info *vinfo,
> bool
> supportable_narrowing_operation (enum tree_code code,
> tree vectype_out, tree vectype_in,
> - enum tree_code *code1, int *multi_step_cvt,
> + tree_code* _code1, int *multi_step_cvt,
The original formatting (space before the “*”) was correct.
Names beginning with _ are reserved, so I think we need a different
name here. Also, the name in the comment should stay in sync with
the name in the code.
That said though, I'm not sure…
> vec<tree> *interm_types)
> {
> machine_mode vec_mode;
> @@ -12217,8 +12270,8 @@ supportable_narrowing_operation (enum tree_code code,
> tree intermediate_type, prev_type;
> machine_mode intermediate_mode, prev_mode;
> int i;
> - unsigned HOST_WIDE_INT n_elts;
> bool uns;
> + tree_code * code1 = (tree_code*) _code1;
…the combination of these two changes makes sense on their own.
>
> *multi_step_cvt = 0;
> switch (code)
> @@ -12227,9 +12280,8 @@ supportable_narrowing_operation (enum tree_code code,
> c1 = VEC_PACK_TRUNC_EXPR;
> if (VECTOR_BOOLEAN_TYPE_P (narrow_vectype)
> && VECTOR_BOOLEAN_TYPE_P (vectype)
> - && SCALAR_INT_MODE_P (TYPE_MODE (vectype))
> - && TYPE_VECTOR_SUBPARTS (vectype).is_constant (&n_elts)
> - && n_elts < BITS_PER_UNIT)
> + && TYPE_MODE (narrow_vectype) == TYPE_MODE (vectype)
> + && SCALAR_INT_MODE_P (TYPE_MODE (vectype)))
> optab1 = vec_pack_sbool_trunc_optab;
> else
> optab1 = optab_for_tree_code (c1, vectype, optab_default);
> @@ -12320,9 +12372,8 @@ supportable_narrowing_operation (enum tree_code code,
> = lang_hooks.types.type_for_mode (intermediate_mode, uns);
> if (VECTOR_BOOLEAN_TYPE_P (intermediate_type)
> && VECTOR_BOOLEAN_TYPE_P (prev_type)
> - && SCALAR_INT_MODE_P (prev_mode)
> - && TYPE_VECTOR_SUBPARTS (intermediate_type).is_constant (&n_elts)
> - && n_elts < BITS_PER_UNIT)
> + && intermediate_mode == prev_mode
> + && SCALAR_INT_MODE_P (prev_mode))
> interm_optab = vec_pack_sbool_trunc_optab;
> else
> interm_optab
This part looks like a behavioural change, so I think it should be part
of a separate patch.
> diff --git a/gcc/tree-vectorizer.h b/gcc/tree-vectorizer.h
> index 642eb0aeb21264cd736a479b1ec25357abef29cd..50ff8eeac1e6b9859302bd784f10ee3d8ff4b4dc 100644
> --- a/gcc/tree-vectorizer.h
> +++ b/gcc/tree-vectorizer.h
> @@ -2120,13 +2120,12 @@ extern bool vect_is_simple_use (vec_info *, stmt_vec_info, slp_tree,
> enum vect_def_type *,
> tree *, stmt_vec_info * = NULL);
> extern bool vect_maybe_update_slp_op_vectype (slp_tree, tree);
> -extern bool supportable_widening_operation (vec_info *,
> - enum tree_code, stmt_vec_info,
> - tree, tree, enum tree_code *,
> - enum tree_code *, int *,
> - vec<tree> *);
> +extern bool supportable_widening_operation (vec_info*, code_helper,
> + stmt_vec_info, tree, tree,
> + code_helper*, code_helper*,
> + int*, vec<tree> *);
> extern bool supportable_narrowing_operation (enum tree_code, tree, tree,
> - enum tree_code *, int *,
> + tree_code *, int *,
> vec<tree> *);
>
> extern unsigned record_stmt_cost (stmt_vector_for_cost *, int,
> diff --git a/gcc/tree.h b/gcc/tree.h
> index f84958933d51144bb6ce7cc41eca5f7f06814550..00b0e4d1c696633fe38baad5295b1f90398d53fc 100644
> --- a/gcc/tree.h
> +++ b/gcc/tree.h
> @@ -92,6 +92,10 @@ public:
> bool is_fn_code () const { return rep < 0; }
> bool is_internal_fn () const;
> bool is_builtin_fn () const;
> + enum tree_code safe_as_tree_code () const { return is_tree_code () ?
> + (tree_code)* this : MAX_TREE_CODES; }
> + combined_fn safe_as_fn_code () const { return is_fn_code () ? (combined_fn) *this
> + : CFN_LAST;}
Since these don't fit on a line, the coding convention says that they
should be defined outside of the class.
Thanks,
Richard
> int get_rep () const { return rep; }
> bool operator== (const code_helper &other) { return rep == other.rep; }
> bool operator!= (const code_helper &other) { return rep != other.rep; }
^ permalink raw reply [flat|nested] 22+ messages in thread
* RE: [ping][vect-patterns] Refactor widen_plus/widen_minus as internal_fns
2022-06-01 10:11 ` Richard Biener
@ 2022-06-06 17:20 ` Joel Hutton
2022-06-07 8:18 ` Richard Sandiford
0 siblings, 1 reply; 22+ messages in thread
From: Joel Hutton @ 2022-06-06 17:20 UTC (permalink / raw)
To: Richard Biener; +Cc: Richard Sandiford, gcc-patches
[-- Attachment #1: Type: text/plain, Size: 5524 bytes --]
> > Patches attached. They already incorporated the .cc rename, now
> > rebased to be after the change to tree.h
>
> @@ -1412,8 +1412,7 @@ vect_recog_widen_op_pattern (vec_info *vinfo,
> 2, oprnd, half_type, unprom, vectype);
>
> tree var = vect_recog_temp_ssa_var (itype, NULL);
> - gimple *pattern_stmt = gimple_build_assign (var, wide_code,
> - oprnd[0], oprnd[1]);
> + gimple *pattern_stmt = gimple_build (var, wide_code, oprnd[0],
> oprnd[1]);
>
>
> you should be able to do without the new gimple_build overload
> by using
>
> gimple_seq stmts = NULL;
> gimple_build (&stmts, wide_code, itype, oprnd[0], oprnd[1]);
> gimple *pattern_stmt = gimple_seq_last_stmt (stmts);
>
> because 'gimple_build' is an existing API.
Done.
The gimple_build overload was at the request of Richard Sandiford, I assume removing it is ok with you Richard S?
From Richard Sandiford:
> For example, I think we should hide this inside a new:
>
> gimple_build (var, wide_code, oprnd[0], oprnd[1]);
>
> that works directly on code_helper, similarly to the new code_helper
> gimple_build interfaces.
>
>
> - if (TREE_CODE (gimple_assign_lhs (stmt)) != SSA_NAME)
> + if (gimple_get_lhs (stmt) == NULL_TREE ||
> + TREE_CODE(gimple_get_lhs (stmt)) != SSA_NAME)
> return false;
>
> || go to the next line, space after TREE_CODE
>
Done.
> + bool widen_arith = false;
> + gimple_match_op res_op;
> + if (!gimple_extract_op (stmt, &res_op))
> + return false;
> + code = res_op.code;
> + op_type = res_op.num_ops;
> +
> + if (is_gimple_assign (stmt))
> + {
> + widen_arith = (code == WIDEN_PLUS_EXPR
> + || code == WIDEN_MINUS_EXPR
> + || code == WIDEN_MULT_EXPR
> + || code == WIDEN_LSHIFT_EXPR);
> + }
> + else
> + widen_arith = gimple_call_flags (stmt) & ECF_WIDEN;
>
> there seem to be formatting issues. Also shouldn't you check
> if (res_op.code.is_tree_code ()) instead if is_gimple_assign?
> I also don't like the ECF_WIDEN "trick", just do as with the
> tree codes and explicitely enumerate widening ifns here.
>
Done. I've set widen_arith to False for the first patch as the second patch introduces the widening ifns.
> gimple_extract_op is a bit heavy-weight as well, so maybe
> instead simply do
>
> if (is_gimple_assign (stmt))
> {
> code = gimple_assign_rhs_code (stmt);
> ...
> }
> else if (gimple_call_internal_p (stmt))
> {
> code = gimple_call_internal_fn (stmt);
> ...
> }
> else
> return false;
The patch was originally written as above, it was changed to use gimple_extract_op at the request of Richard Sandiford. I prefer gimple_extract_op as it's more compact, but I don't have strong feelings. If the Richards can agree on either version I'm happy.
From Richard Sandiford:
> > + if (is_gimple_assign (stmt))
> > + {
> > + code_or_ifn = gimple_assign_rhs_code (stmt); } else
> > + code_or_ifn = gimple_call_combined_fn (stmt);
>
> It might be possible to use gimple_extract_op here (only recently added).
> This would also provide the number of operands directly, instead of
> needing "op_type". It would also provide an array of operands.
>
> + code_helper c1=MAX_TREE_CODES, c2=MAX_TREE_CODES;
>
> spaces before/after '='
>
Done.
> @@ -12061,13 +12105,16 @@ supportable_widening_operation (vec_info
> *vinfo,
> if (BYTES_BIG_ENDIAN && c1 != VEC_WIDEN_MULT_EVEN_EXPR)
> std::swap (c1, c2);
>
> +
> if (code == FIX_TRUNC_EXPR)
> {
>
> unnecessary whitespace change.
>
Fixed.
> diff --git a/gcc/tree.h b/gcc/tree.h
> index
> f84958933d51144bb6ce7cc41eca5f7f06814550..e51e34c051d9b91d1c02a4b2
> fefdb2b15606a36f
> 100644
> --- a/gcc/tree.h
> +++ b/gcc/tree.h
> @@ -92,6 +92,10 @@ public:
> bool is_fn_code () const { return rep < 0; }
> bool is_internal_fn () const;
> bool is_builtin_fn () const;
> + enum tree_code as_tree_code () const { return is_tree_code () ?
> + (tree_code)* this : MAX_TREE_CODES; }
> + combined_fn as_fn_code () const { return is_fn_code () ? (combined_fn)
> *this
> + : CFN_LAST;}
>
> hmm, the other as_* functions we have are not member functions.
> Also this semantically differs from the tree_code () conversion
> operator (that one was supposed to be "cheap"). The existing
> as_internal_fn for example is documented as being valid only if
> the code is actually an internal fn. I see you are introducing
> the new function as convenience to get a "safe" not-a-X value,
> so maybe they should be called safe_as_tree_code () instead?
>
SGTM. Done
>
> int get_rep () const { return rep; }
> bool operator== (const code_helper &other) { return rep == other.rep; }
> bool operator!= (const code_helper &other) { return rep != other.rep; }
> @@ -6657,6 +6661,54 @@ extern unsigned fndecl_dealloc_argno (tree);
> if nonnull, set the second argument to the referenced enclosing
> object or pointer. Otherwise return null. */
> extern tree get_attr_nonstring_decl (tree, tree * = NULL);
> +/* Helper to transparently allow tree codes and builtin function codes
> + exist in one storage entity. */
> +class code_helper
> +{
>
> duplicate add of code_helper.
Fixed.
Tests are being re-run.
Ok, with changes?
[-- Attachment #2: 0001-Refactor-to-allow-internal_fn-s.patch --]
[-- Type: application/octet-stream, Size: 23246 bytes --]
From 58d1f19224bd6501b5238916871cf2c0f3ba8bd0 Mon Sep 17 00:00:00 2001
From: Joel Hutton <joel.hutton@arm.com>
Date: Wed, 25 Aug 2021 14:31:15 +0100
Subject: [PATCH 1/3] Refactor to allow internal_fn's
Hi all,
This refactor allows widening patterns (such as widen_plus/widen_minus) to be represented as
either internal_fns or tree_codes.
[vect-patterns] Refactor as internal_fn's
Refactor vect-patterns to allow patterns to be internal_fns starting
with widening_plus/minus patterns
gcc/ChangeLog:
* gimple-match.h (class code_helper): Add safe_as_internal_fn, safe_as_tree_code
helper functions.
* tree-core.h (ECF_WIDEN): Flag to mark internal_fn as widening.
* tree-vect-patterns.cc (vect_recog_widen_op_pattern): Refactor to
use code_helper.
* tree-vect-stmts.cc (vect_gen_widened_results_half): Refactor to
use code_helper.
(vect_create_vectorized_promotion_stmts): Refactor to use
code_helper.
(vectorizable_conversion): Refactor to use code_helper.
gimple_call or gimple_assign.
(supportable_widening_operation): Refactor to use code_helper.
(supportable_narrowing_operation): Refactor to use code_helper.
* tree-vectorizer.h (supportable_widening_operation): Change
prototype to use code_helper.
(supportable_narrowing_operation): change prototype to use
code_helper.
---
gcc/tree-core.h | 3 +
gcc/tree-vect-patterns.cc | 11 +-
gcc/tree-vect-stmts.cc | 217 +++++++++++++++++++++++---------------
gcc/tree-vectorizer.h | 11 +-
gcc/tree.h | 4 +
5 files changed, 153 insertions(+), 93 deletions(-)
diff --git a/gcc/tree-core.h b/gcc/tree-core.h
index ab5fa01e5cb5fb56c1964b93b014ed55a4aa704a..cff6211080bced0bffb39e98039a6550897acf77 100644
--- a/gcc/tree-core.h
+++ b/gcc/tree-core.h
@@ -96,6 +96,9 @@ struct die_struct;
/* Nonzero if this is a cold function. */
#define ECF_COLD (1 << 15)
+/* Nonzero if this is a widening function. */
+#define ECF_WIDEN (1 << 16)
+
/* Call argument flags. */
/* Nonzero if the argument is not used by the function. */
diff --git a/gcc/tree-vect-patterns.cc b/gcc/tree-vect-patterns.cc
index 0fad4dbd0945c6c176f3457b751e812f17fcd148..c011b8ede3c266b59f731e316efbec7d98e91068 100644
--- a/gcc/tree-vect-patterns.cc
+++ b/gcc/tree-vect-patterns.cc
@@ -25,6 +25,8 @@ along with GCC; see the file COPYING3. If not see
#include "rtl.h"
#include "tree.h"
#include "gimple.h"
+#include "gimple-iterator.h"
+#include "gimple-fold.h"
#include "ssa.h"
#include "expmed.h"
#include "optabs-tree.h"
@@ -1348,7 +1350,7 @@ vect_recog_sad_pattern (vec_info *vinfo,
static gimple *
vect_recog_widen_op_pattern (vec_info *vinfo,
stmt_vec_info last_stmt_info, tree *type_out,
- tree_code orig_code, tree_code wide_code,
+ tree_code orig_code, code_helper wide_code,
bool shift_p, const char *name)
{
gimple *last_stmt = last_stmt_info->stmt;
@@ -1391,7 +1393,7 @@ vect_recog_widen_op_pattern (vec_info *vinfo,
vecctype = get_vectype_for_scalar_type (vinfo, ctype);
}
- enum tree_code dummy_code;
+ code_helper dummy_code;
int dummy_int;
auto_vec<tree> dummy_vec;
if (!vectype
@@ -1412,8 +1414,9 @@ vect_recog_widen_op_pattern (vec_info *vinfo,
2, oprnd, half_type, unprom, vectype);
tree var = vect_recog_temp_ssa_var (itype, NULL);
- gimple *pattern_stmt = gimple_build_assign (var, wide_code,
- oprnd[0], oprnd[1]);
+ gimple_seq stmts = NULL;
+ gimple_build (&stmts, wide_code, itype, oprnd[0], oprnd[1]);
+ gimple *pattern_stmt = gimple_seq_last_stmt (stmts);
if (vecctype != vecitype)
pattern_stmt = vect_convert_output (vinfo, last_stmt_info, ctype,
diff --git a/gcc/tree-vect-stmts.cc b/gcc/tree-vect-stmts.cc
index 346d8ce280437e00bfeb19a4b4adc59eb96207f9..9b31425352689d409b8c0aa0c1d5c69e72db869a 100644
--- a/gcc/tree-vect-stmts.cc
+++ b/gcc/tree-vect-stmts.cc
@@ -4636,7 +4636,7 @@ vectorizable_simd_clone_call (vec_info *vinfo, stmt_vec_info stmt_info,
STMT_INFO is the original scalar stmt that we are vectorizing. */
static gimple *
-vect_gen_widened_results_half (vec_info *vinfo, enum tree_code code,
+vect_gen_widened_results_half (vec_info *vinfo, code_helper ch,
tree vec_oprnd0, tree vec_oprnd1, int op_type,
tree vec_dest, gimple_stmt_iterator *gsi,
stmt_vec_info stmt_info)
@@ -4645,14 +4645,15 @@ vect_gen_widened_results_half (vec_info *vinfo, enum tree_code code,
tree new_temp;
/* Generate half of the widened result: */
- gcc_assert (op_type == TREE_CODE_LENGTH (code));
if (op_type != binary_op)
vec_oprnd1 = NULL;
- new_stmt = gimple_build_assign (vec_dest, code, vec_oprnd0, vec_oprnd1);
+
+ gimple_seq stmts = NULL;
+ gimple_build (&stmts, ch, vec_oprnd0, vec_oprnd1);
+ new_stmt = gimple_seq_last_stmt (stmts);
new_temp = make_ssa_name (vec_dest, new_stmt);
- gimple_assign_set_lhs (new_stmt, new_temp);
+ gimple_set_lhs (new_stmt, new_temp);
vect_finish_stmt_generation (vinfo, stmt_info, new_stmt, gsi);
-
return new_stmt;
}
@@ -4729,8 +4730,8 @@ vect_create_vectorized_promotion_stmts (vec_info *vinfo,
vec<tree> *vec_oprnds1,
stmt_vec_info stmt_info, tree vec_dest,
gimple_stmt_iterator *gsi,
- enum tree_code code1,
- enum tree_code code2, int op_type)
+ code_helper ch1,
+ code_helper ch2, int op_type)
{
int i;
tree vop0, vop1, new_tmp1, new_tmp2;
@@ -4746,10 +4747,10 @@ vect_create_vectorized_promotion_stmts (vec_info *vinfo,
vop1 = NULL_TREE;
/* Generate the two halves of promotion operation. */
- new_stmt1 = vect_gen_widened_results_half (vinfo, code1, vop0, vop1,
+ new_stmt1 = vect_gen_widened_results_half (vinfo, ch1, vop0, vop1,
op_type, vec_dest, gsi,
stmt_info);
- new_stmt2 = vect_gen_widened_results_half (vinfo, code2, vop0, vop1,
+ new_stmt2 = vect_gen_widened_results_half (vinfo, ch2, vop0, vop1,
op_type, vec_dest, gsi,
stmt_info);
if (is_gimple_call (new_stmt1))
@@ -4846,8 +4847,9 @@ vectorizable_conversion (vec_info *vinfo,
tree scalar_dest;
tree op0, op1 = NULL_TREE;
loop_vec_info loop_vinfo = dyn_cast <loop_vec_info> (vinfo);
- enum tree_code code, code1 = ERROR_MARK, code2 = ERROR_MARK;
- enum tree_code codecvt1 = ERROR_MARK, codecvt2 = ERROR_MARK;
+ tree_code tc1;
+ code_helper code, code1, code2;
+ code_helper codecvt1 = ERROR_MARK, codecvt2 = ERROR_MARK;
tree new_temp;
enum vect_def_type dt[2] = {vect_unknown_def_type, vect_unknown_def_type};
int ndts = 2;
@@ -4876,31 +4878,42 @@ vectorizable_conversion (vec_info *vinfo,
&& ! vec_stmt)
return false;
- gassign *stmt = dyn_cast <gassign *> (stmt_info->stmt);
- if (!stmt)
+ gimple* stmt = stmt_info->stmt;
+ if (!(is_gimple_assign (stmt) || is_gimple_call (stmt)))
return false;
- if (TREE_CODE (gimple_assign_lhs (stmt)) != SSA_NAME)
+ if (gimple_get_lhs (stmt) == NULL_TREE
+ || TREE_CODE (gimple_get_lhs (stmt)) != SSA_NAME)
return false;
- code = gimple_assign_rhs_code (stmt);
- if (!CONVERT_EXPR_CODE_P (code)
- && code != FIX_TRUNC_EXPR
- && code != FLOAT_EXPR
- && code != WIDEN_PLUS_EXPR
- && code != WIDEN_MINUS_EXPR
- && code != WIDEN_MULT_EXPR
- && code != WIDEN_LSHIFT_EXPR)
+ if (TREE_CODE (gimple_get_lhs (stmt)) != SSA_NAME)
return false;
- bool widen_arith = (code == WIDEN_PLUS_EXPR
- || code == WIDEN_MINUS_EXPR
- || code == WIDEN_MULT_EXPR
- || code == WIDEN_LSHIFT_EXPR);
- op_type = TREE_CODE_LENGTH (code);
+ bool widen_arith = false;
+ gimple_match_op res_op;
+ if (!gimple_extract_op (stmt, &res_op))
+ return false;
+ code = res_op.code;
+ op_type = res_op.num_ops;
+
+ if (res_op.code.is_tree_code ())
+ {
+ widen_arith = (code == WIDEN_PLUS_EXPR
+ || code == WIDEN_MINUS_EXPR
+ || code == WIDEN_MULT_EXPR
+ || code == WIDEN_LSHIFT_EXPR);
+ }
+ else
+ widen_arith = false;
+
+ if (!widen_arith
+ && !CONVERT_EXPR_CODE_P (code)
+ && code != FIX_TRUNC_EXPR
+ && code != FLOAT_EXPR)
+ return false;
/* Check types of lhs and rhs. */
- scalar_dest = gimple_assign_lhs (stmt);
+ scalar_dest = gimple_get_lhs (stmt);
lhs_type = TREE_TYPE (scalar_dest);
vectype_out = STMT_VINFO_VECTYPE (stmt_info);
@@ -4938,10 +4951,14 @@ vectorizable_conversion (vec_info *vinfo,
if (op_type == binary_op)
{
- gcc_assert (code == WIDEN_MULT_EXPR || code == WIDEN_LSHIFT_EXPR
- || code == WIDEN_PLUS_EXPR || code == WIDEN_MINUS_EXPR);
+ gcc_assert (code == WIDEN_MULT_EXPR
+ || code == WIDEN_LSHIFT_EXPR
+ || code == WIDEN_PLUS_EXPR
+ || code == WIDEN_MINUS_EXPR);
- op1 = gimple_assign_rhs2 (stmt);
+
+ op1 = is_gimple_assign (stmt) ? gimple_assign_rhs2 (stmt) :
+ gimple_call_arg (stmt, 0);
tree vectype1_in;
if (!vect_is_simple_use (vinfo, stmt_info, slp_node, 1,
&op1, &slp_op1, &dt[1], &vectype1_in))
@@ -5025,8 +5042,12 @@ vectorizable_conversion (vec_info *vinfo,
&& code != FLOAT_EXPR
&& !CONVERT_EXPR_CODE_P (code))
return false;
- if (supportable_convert_operation (code, vectype_out, vectype_in, &code1))
+ if (supportable_convert_operation (code.safe_as_tree_code (), vectype_out,
+ vectype_in, &tc1))
+ {
+ code1 = tc1;
break;
+ }
/* FALLTHRU */
unsupported:
if (dump_enabled_p ())
@@ -5037,9 +5058,11 @@ vectorizable_conversion (vec_info *vinfo,
case WIDEN:
if (known_eq (nunits_in, nunits_out))
{
- if (!supportable_half_widening_operation (code, vectype_out,
- vectype_in, &code1))
+ if (!supportable_half_widening_operation (code.safe_as_tree_code (),
+ vectype_out, vectype_in,
+ &tc1))
goto unsupported;
+ code1 = tc1;
gcc_assert (!(multi_step_cvt && op_type == binary_op));
break;
}
@@ -5073,14 +5096,17 @@ vectorizable_conversion (vec_info *vinfo,
if (GET_MODE_SIZE (rhs_mode) == fltsz)
{
- if (!supportable_convert_operation (code, vectype_out,
- cvt_type, &codecvt1))
+ tc1 = ERROR_MARK;
+ if (!supportable_convert_operation (code.safe_as_tree_code (),
+ vectype_out,
+ cvt_type, &tc1))
goto unsupported;
+ codecvt1 = tc1;
}
- else if (!supportable_widening_operation (vinfo, code, stmt_info,
- vectype_out, cvt_type,
- &codecvt1, &codecvt2,
- &multi_step_cvt,
+ else if (!supportable_widening_operation (vinfo, code,
+ stmt_info, vectype_out,
+ cvt_type, &codecvt1,
+ &codecvt2, &multi_step_cvt,
&interm_types))
continue;
else
@@ -5088,8 +5114,9 @@ vectorizable_conversion (vec_info *vinfo,
if (supportable_widening_operation (vinfo, NOP_EXPR, stmt_info,
cvt_type,
- vectype_in, &code1, &code2,
- &multi_step_cvt, &interm_types))
+ vectype_in, &code1,
+ &code2, &multi_step_cvt,
+ &interm_types))
{
found_mode = true;
break;
@@ -5111,10 +5138,14 @@ vectorizable_conversion (vec_info *vinfo,
case NARROW:
gcc_assert (op_type == unary_op);
- if (supportable_narrowing_operation (code, vectype_out, vectype_in,
- &code1, &multi_step_cvt,
+ if (supportable_narrowing_operation (code.safe_as_tree_code (), vectype_out,
+ vectype_in,
+ &tc1, &multi_step_cvt,
&interm_types))
- break;
+ {
+ code1 = tc1;
+ break;
+ }
if (code != FIX_TRUNC_EXPR
|| GET_MODE_SIZE (lhs_mode) >= GET_MODE_SIZE (rhs_mode))
@@ -5125,13 +5156,18 @@ vectorizable_conversion (vec_info *vinfo,
cvt_type = get_same_sized_vectype (cvt_type, vectype_in);
if (cvt_type == NULL_TREE)
goto unsupported;
- if (!supportable_convert_operation (code, cvt_type, vectype_in,
- &codecvt1))
+ if (!supportable_convert_operation (code.safe_as_tree_code (), cvt_type,
+ vectype_in,
+ &tc1))
goto unsupported;
+ codecvt1 = tc1;
if (supportable_narrowing_operation (NOP_EXPR, vectype_out, cvt_type,
- &code1, &multi_step_cvt,
+ &tc1, &multi_step_cvt,
&interm_types))
- break;
+ {
+ code1 = tc1;
+ break;
+ }
goto unsupported;
default:
@@ -5245,8 +5281,9 @@ vectorizable_conversion (vec_info *vinfo,
FOR_EACH_VEC_ELT (vec_oprnds0, i, vop0)
{
/* Arguments are ready, create the new vector stmt. */
- gcc_assert (TREE_CODE_LENGTH (code1) == unary_op);
- gassign *new_stmt = gimple_build_assign (vec_dest, code1, vop0);
+ gcc_assert (TREE_CODE_LENGTH ((tree_code) code1) == unary_op);
+ gassign *new_stmt = gimple_build_assign (vec_dest,
+ code1.safe_as_tree_code (), vop0);
new_temp = make_ssa_name (vec_dest, new_stmt);
gimple_assign_set_lhs (new_stmt, new_temp);
vect_finish_stmt_generation (vinfo, stmt_info, new_stmt, gsi);
@@ -5278,7 +5315,7 @@ vectorizable_conversion (vec_info *vinfo,
for (i = multi_step_cvt; i >= 0; i--)
{
tree this_dest = vec_dsts[i];
- enum tree_code c1 = code1, c2 = code2;
+ code_helper c1 = code1, c2 = code2;
if (i == 0 && codecvt2 != ERROR_MARK)
{
c1 = codecvt1;
@@ -5288,7 +5325,8 @@ vectorizable_conversion (vec_info *vinfo,
vect_create_half_widening_stmts (vinfo, &vec_oprnds0,
&vec_oprnds1, stmt_info,
this_dest, gsi,
- c1, op_type);
+ c1.safe_as_tree_code (),
+ op_type);
else
vect_create_vectorized_promotion_stmts (vinfo, &vec_oprnds0,
&vec_oprnds1, stmt_info,
@@ -5301,9 +5339,11 @@ vectorizable_conversion (vec_info *vinfo,
gimple *new_stmt;
if (cvt_type)
{
- gcc_assert (TREE_CODE_LENGTH (codecvt1) == unary_op);
+ gcc_assert (TREE_CODE_LENGTH ((tree_code) codecvt1) == unary_op);
new_temp = make_ssa_name (vec_dest);
- new_stmt = gimple_build_assign (new_temp, codecvt1, vop0);
+ new_stmt = gimple_build_assign (new_temp,
+ codecvt1.safe_as_tree_code (),
+ vop0);
vect_finish_stmt_generation (vinfo, stmt_info, new_stmt, gsi);
}
else
@@ -5327,10 +5367,10 @@ vectorizable_conversion (vec_info *vinfo,
if (cvt_type)
FOR_EACH_VEC_ELT (vec_oprnds0, i, vop0)
{
- gcc_assert (TREE_CODE_LENGTH (codecvt1) == unary_op);
+ gcc_assert (TREE_CODE_LENGTH (((tree_code) codecvt1)) == unary_op);
new_temp = make_ssa_name (vec_dest);
gassign *new_stmt
- = gimple_build_assign (new_temp, codecvt1, vop0);
+ = gimple_build_assign (new_temp, codecvt1.safe_as_tree_code (), vop0);
vect_finish_stmt_generation (vinfo, stmt_info, new_stmt, gsi);
vec_oprnds0[i] = new_temp;
}
@@ -5338,7 +5378,7 @@ vectorizable_conversion (vec_info *vinfo,
vect_create_vectorized_demotion_stmts (vinfo, &vec_oprnds0,
multi_step_cvt,
stmt_info, vec_dsts, gsi,
- slp_node, code1);
+ slp_node, code1.safe_as_tree_code ());
break;
}
if (!slp_node)
@@ -11926,9 +11966,11 @@ vect_maybe_update_slp_op_vectype (slp_tree op, tree vectype)
bool
supportable_widening_operation (vec_info *vinfo,
- enum tree_code code, stmt_vec_info stmt_info,
+ code_helper code,
+ stmt_vec_info stmt_info,
tree vectype_out, tree vectype_in,
- enum tree_code *code1, enum tree_code *code2,
+ code_helper *code1,
+ code_helper *code2,
int *multi_step_cvt,
vec<tree> *interm_types)
{
@@ -11939,7 +11981,7 @@ supportable_widening_operation (vec_info *vinfo,
optab optab1, optab2;
tree vectype = vectype_in;
tree wide_vectype = vectype_out;
- enum tree_code c1, c2;
+ code_helper c1 = MAX_TREE_CODES, c2 = MAX_TREE_CODES;
int i;
tree prev_type, intermediate_type;
machine_mode intermediate_mode, prev_mode;
@@ -11949,7 +11991,7 @@ supportable_widening_operation (vec_info *vinfo,
if (loop_info)
vect_loop = LOOP_VINFO_LOOP (loop_info);
- switch (code)
+ switch (code.safe_as_tree_code ())
{
case WIDEN_MULT_EXPR:
/* The result of a vectorized widening operation usually requires
@@ -11990,8 +12032,9 @@ supportable_widening_operation (vec_info *vinfo,
&& !nested_in_vect_loop_p (vect_loop, stmt_info)
&& supportable_widening_operation (vinfo, VEC_WIDEN_MULT_EVEN_EXPR,
stmt_info, vectype_out,
- vectype_in, code1, code2,
- multi_step_cvt, interm_types))
+ vectype_in, code1,
+ code2, multi_step_cvt,
+ interm_types))
{
/* Elements in a vector with vect_used_by_reduction property cannot
be reordered if the use chain with this property does not have the
@@ -12054,6 +12097,9 @@ supportable_widening_operation (vec_info *vinfo,
c2 = VEC_UNPACK_FIX_TRUNC_HI_EXPR;
break;
+ case MAX_TREE_CODES:
+ break;
+
default:
gcc_unreachable ();
}
@@ -12064,10 +12110,12 @@ supportable_widening_operation (vec_info *vinfo,
if (code == FIX_TRUNC_EXPR)
{
/* The signedness is determined from output operand. */
- optab1 = optab_for_tree_code (c1, vectype_out, optab_default);
- optab2 = optab_for_tree_code (c2, vectype_out, optab_default);
+ optab1 = optab_for_tree_code (c1.safe_as_tree_code (), vectype_out,
+ optab_default);
+ optab2 = optab_for_tree_code (c2.safe_as_tree_code (), vectype_out,
+ optab_default);
}
- else if (CONVERT_EXPR_CODE_P (code)
+ else if (CONVERT_EXPR_CODE_P (code.safe_as_tree_code ())
&& VECTOR_BOOLEAN_TYPE_P (wide_vectype)
&& VECTOR_BOOLEAN_TYPE_P (vectype)
&& TYPE_MODE (wide_vectype) == TYPE_MODE (vectype)
@@ -12080,8 +12128,8 @@ supportable_widening_operation (vec_info *vinfo,
}
else
{
- optab1 = optab_for_tree_code (c1, vectype, optab_default);
- optab2 = optab_for_tree_code (c2, vectype, optab_default);
+ optab1 = optab_for_tree_code (c1.safe_as_tree_code (), vectype, optab_default);
+ optab2 = optab_for_tree_code (c2.safe_as_tree_code (), vectype, optab_default);
}
if (!optab1 || !optab2)
@@ -12092,8 +12140,12 @@ supportable_widening_operation (vec_info *vinfo,
|| (icode2 = optab_handler (optab2, vec_mode)) == CODE_FOR_nothing)
return false;
- *code1 = c1;
- *code2 = c2;
+ if (code.is_tree_code ())
+ {
+ *code1 = c1;
+ *code2 = c2;
+ }
+
if (insn_data[icode1].operand[0].mode == TYPE_MODE (wide_vectype)
&& insn_data[icode2].operand[0].mode == TYPE_MODE (wide_vectype))
@@ -12114,7 +12166,7 @@ supportable_widening_operation (vec_info *vinfo,
prev_type = vectype;
prev_mode = vec_mode;
- if (!CONVERT_EXPR_CODE_P (code))
+ if (!CONVERT_EXPR_CODE_P ((tree_code) code))
return false;
/* We assume here that there will not be more than MAX_INTERM_CVT_STEPS
@@ -12145,8 +12197,10 @@ supportable_widening_operation (vec_info *vinfo,
}
else
{
- optab3 = optab_for_tree_code (c1, intermediate_type, optab_default);
- optab4 = optab_for_tree_code (c2, intermediate_type, optab_default);
+ optab3 = optab_for_tree_code (c1.safe_as_tree_code (), intermediate_type,
+ optab_default);
+ optab4 = optab_for_tree_code (c2.safe_as_tree_code (), intermediate_type,
+ optab_default);
}
if (!optab3 || !optab4
@@ -12181,7 +12235,6 @@ supportable_widening_operation (vec_info *vinfo,
return false;
}
-
/* Function supportable_narrowing_operation
Check whether an operation represented by the code CODE is a
@@ -12205,7 +12258,7 @@ supportable_widening_operation (vec_info *vinfo,
bool
supportable_narrowing_operation (enum tree_code code,
tree vectype_out, tree vectype_in,
- enum tree_code *code1, int *multi_step_cvt,
+ tree_code* _code1, int *multi_step_cvt,
vec<tree> *interm_types)
{
machine_mode vec_mode;
@@ -12217,8 +12270,8 @@ supportable_narrowing_operation (enum tree_code code,
tree intermediate_type, prev_type;
machine_mode intermediate_mode, prev_mode;
int i;
- unsigned HOST_WIDE_INT n_elts;
bool uns;
+ tree_code * code1 = (tree_code*) _code1;
*multi_step_cvt = 0;
switch (code)
@@ -12227,9 +12280,8 @@ supportable_narrowing_operation (enum tree_code code,
c1 = VEC_PACK_TRUNC_EXPR;
if (VECTOR_BOOLEAN_TYPE_P (narrow_vectype)
&& VECTOR_BOOLEAN_TYPE_P (vectype)
- && SCALAR_INT_MODE_P (TYPE_MODE (vectype))
- && TYPE_VECTOR_SUBPARTS (vectype).is_constant (&n_elts)
- && n_elts < BITS_PER_UNIT)
+ && TYPE_MODE (narrow_vectype) == TYPE_MODE (vectype)
+ && SCALAR_INT_MODE_P (TYPE_MODE (vectype)))
optab1 = vec_pack_sbool_trunc_optab;
else
optab1 = optab_for_tree_code (c1, vectype, optab_default);
@@ -12320,9 +12372,8 @@ supportable_narrowing_operation (enum tree_code code,
= lang_hooks.types.type_for_mode (intermediate_mode, uns);
if (VECTOR_BOOLEAN_TYPE_P (intermediate_type)
&& VECTOR_BOOLEAN_TYPE_P (prev_type)
- && SCALAR_INT_MODE_P (prev_mode)
- && TYPE_VECTOR_SUBPARTS (intermediate_type).is_constant (&n_elts)
- && n_elts < BITS_PER_UNIT)
+ && intermediate_mode == prev_mode
+ && SCALAR_INT_MODE_P (prev_mode))
interm_optab = vec_pack_sbool_trunc_optab;
else
interm_optab
diff --git a/gcc/tree-vectorizer.h b/gcc/tree-vectorizer.h
index 642eb0aeb21264cd736a479b1ec25357abef29cd..50ff8eeac1e6b9859302bd784f10ee3d8ff4b4dc 100644
--- a/gcc/tree-vectorizer.h
+++ b/gcc/tree-vectorizer.h
@@ -2120,13 +2120,12 @@ extern bool vect_is_simple_use (vec_info *, stmt_vec_info, slp_tree,
enum vect_def_type *,
tree *, stmt_vec_info * = NULL);
extern bool vect_maybe_update_slp_op_vectype (slp_tree, tree);
-extern bool supportable_widening_operation (vec_info *,
- enum tree_code, stmt_vec_info,
- tree, tree, enum tree_code *,
- enum tree_code *, int *,
- vec<tree> *);
+extern bool supportable_widening_operation (vec_info*, code_helper,
+ stmt_vec_info, tree, tree,
+ code_helper*, code_helper*,
+ int*, vec<tree> *);
extern bool supportable_narrowing_operation (enum tree_code, tree, tree,
- enum tree_code *, int *,
+ tree_code *, int *,
vec<tree> *);
extern unsigned record_stmt_cost (stmt_vector_for_cost *, int,
diff --git a/gcc/tree.h b/gcc/tree.h
index f84958933d51144bb6ce7cc41eca5f7f06814550..00b0e4d1c696633fe38baad5295b1f90398d53fc 100644
--- a/gcc/tree.h
+++ b/gcc/tree.h
@@ -92,6 +92,10 @@ public:
bool is_fn_code () const { return rep < 0; }
bool is_internal_fn () const;
bool is_builtin_fn () const;
+ enum tree_code safe_as_tree_code () const { return is_tree_code () ?
+ (tree_code)* this : MAX_TREE_CODES; }
+ combined_fn safe_as_fn_code () const { return is_fn_code () ? (combined_fn) *this
+ : CFN_LAST;}
int get_rep () const { return rep; }
bool operator== (const code_helper &other) { return rep == other.rep; }
bool operator!= (const code_helper &other) { return rep != other.rep; }
--
2.17.1
[-- Attachment #3: 0002-Refactor-widen_plus-as-internal_fn.patch --]
[-- Type: application/octet-stream, Size: 23132 bytes --]
From 233a24f2a4eeced2fd4e99578e6ea81ec8622192 Mon Sep 17 00:00:00 2001
From: Joel Hutton <joel.hutton@arm.com>
Date: Wed, 26 Jan 2022 14:00:17 +0000
Subject: [PATCH 2/3] Refactor widen_plus as internal_fn
This patch replaces the existing tree_code widen_plus and widen_minus
patterns with internal_fn versions.
DEF_INTERNAL_OPTAB_MULTI_FN is like DEF_INTERNAL_OPTAB_FN except it provides convenience wrappers for defining conversions that require a hi/lo split, like widening and narrowing operations. Each definition for <NAME> will require an optab named <OPTAB> and two other optabs that you specify for signed and unsigned. The hi/lo pair is necessary because the widening operations take n narrow elements as inputs and return n/2 wide elements as outputs. The 'lo' operation operates on the first n/2 elements of input. The 'hi' operation operates on the second n/2 elements of input. Defining an internal_fn along with hi/lo variations allows a single internal function to be returned from a vect_recog function that will later be expanded to hi/lo.
DEF_INTERNAL_OPTAB_MULTI_FN is used in internal-fn.def to register a widening internal_fn. It is defined differently in different places and internal-fn.def is sourced from those places so the parameters given can be reused.
internal-fn.c: defined to expand to hi/lo signed/unsigned optabs, later defined to generate the 'expand_' functions for the hi/lo versions of the fn.
internal-fn.def: defined to invoke DEF_INTERNAL_OPTAB_FN for the original and hi/lo variants of the internal_fn
For example:
IFN_VEC_WIDEN_PLUS -> IFN_VEC_WIDEN_PLUS_HI, IFN_VEC_WIDEN_PLUS_LO
for aarch64: IFN_VEC_WIDEN_PLUS_HI -> vec_widen_<su>addl_hi_<mode> -> (u/s)addl2
IFN_VEC_WIDEN_PLUS_LO -> vec_widen_<su>addl_lo_<mode> -> (u/s)addl
This gives the same functionality as the previous WIDEN_PLUS/WIDEN_MINUS tree codes which are expanded into VEC_WIDEN_PLUS_LO, VEC_WIDEN_PLUS_HI.
gcc/ChangeLog:
2022-04-13 Joel Hutton <joel.hutton@arm.com>
2022-04-13 Tamar Christina <tamar.christina@arm.com>
* internal-fn.cc (INCLUDE_MAP): Include maps for use in optab
lookup.
(DEF_INTERNAL_OPTAB_MULTI_FN): Macro to define an internal_fn that
expands into multiple internal_fns (for widening).
(ifn_cmp): Function to compare ifn's for sorting/searching.
(lookup_multi_ifn_optab): Add lookup function.
(lookup_multi_internal_fn): Add lookup function.
(commutative_binary_fn_p): Add widen_plus fn's.
* internal-fn.def (DEF_INTERNAL_OPTAB_MULTI_FN): Define widening
plus,minus functions.
(VEC_WIDEN_PLUS): Replacement for VEC_WIDEN_PLUS tree code.
(VEC_WIDEN_MINUS): Replacement for VEC_WIDEN_MINUS tree code.
* internal-fn.h (GCC_INTERNAL_FN_H): Add headers.
(lookup_multi_ifn_optab): Add prototype.
(lookup_multi_internal_fn): Add prototype.
* optabs.cc (commutative_optab_p): Add widening plus, minus optabs.
* optabs.def (OPTAB_CD): widen add, sub optabs
* tree-core.h (ECF_MULTI): Flag to indicate if a function decays
into hi/lo parts.
* tree-vect-patterns.cc (vect_recog_widen_op_pattern): Support
patterns with a hi/lo split.
(vect_recog_widen_plus_pattern): Refactor to return
IFN_VECT_WIDEN_PLUS.
(vect_recog_widen_minus_pattern): Refactor to return new
IFN_VEC_WIDEN_MINUS.
* tree-vect-stmts.cc (vectorizable_conversion): Add widen plus/minus
ifn
support.
(supportable_widening_operation): Add widen plus/minus ifn support.
gcc/testsuite/ChangeLog:
* gcc.target/aarch64/vect-widen-add.c: Test that new
IFN_VEC_WIDEN_PLUS is being used.
* gcc.target/aarch64/vect-widen-sub.c: Test that new
IFN_VEC_WIDEN_MINUS is being used.
---
gcc/internal-fn.cc | 107 ++++++++++++++++++
gcc/internal-fn.def | 23 ++++
gcc/internal-fn.h | 7 ++
gcc/optabs.cc | 12 +-
gcc/optabs.def | 2 +
.../gcc.target/aarch64/vect-widen-add.c | 4 +-
.../gcc.target/aarch64/vect-widen-sub.c | 4 +-
gcc/tree-core.h | 4 +
gcc/tree-vect-patterns.cc | 37 ++++--
gcc/tree-vect-stmts.cc | 65 ++++++++++-
10 files changed, 248 insertions(+), 17 deletions(-)
diff --git a/gcc/internal-fn.cc b/gcc/internal-fn.cc
index 8b1733e20c4455e4e8c383c92fe859f4256cae69..e95b13af884f67990ad43c286990a351e2bd641b 100644
--- a/gcc/internal-fn.cc
+++ b/gcc/internal-fn.cc
@@ -17,6 +17,7 @@ You should have received a copy of the GNU General Public License
along with GCC; see the file COPYING3. If not see
<http://www.gnu.org/licenses/>. */
+#define INCLUDE_MAP
#include "config.h"
#include "system.h"
#include "coretypes.h"
@@ -70,6 +71,26 @@ const int internal_fn_flags_array[] = {
0
};
+const enum internal_fn internal_fn_hilo_keys_array[] = {
+#undef DEF_INTERNAL_OPTAB_MULTI_FN
+#define DEF_INTERNAL_OPTAB_MULTI_FN(NAME, FLAGS, OPTAB, SOPTAB, UOPTAB, TYPE) \
+ IFN_##NAME##_LO, \
+ IFN_##NAME##_HI,
+#include "internal-fn.def"
+ IFN_LAST
+#undef DEF_INTERNAL_OPTAB_MULTI_FN
+};
+
+const optab internal_fn_hilo_values_array[] = {
+#undef DEF_INTERNAL_OPTAB_MULTI_FN
+#define DEF_INTERNAL_OPTAB_MULTI_FN(NAME, FLAGS, OPTAB, SOPTAB, UOPTAB, TYPE) \
+ SOPTAB##_lo_optab, UOPTAB##_lo_optab, \
+ SOPTAB##_hi_optab, UOPTAB##_hi_optab,
+#include "internal-fn.def"
+ unknown_optab, unknown_optab
+#undef DEF_INTERNAL_OPTAB_MULTI_FN
+};
+
/* Return the internal function called NAME, or IFN_LAST if there's
no such function. */
@@ -90,6 +111,62 @@ lookup_internal_fn (const char *name)
return entry ? *entry : IFN_LAST;
}
+static int
+ifn_cmp (const void *a_, const void *b_)
+{
+ typedef std::pair<enum internal_fn, unsigned> ifn_pair;
+ auto *a = (const std::pair<ifn_pair, optab> *)a_;
+ auto *b = (const std::pair<ifn_pair, optab> *)b_;
+ return (int) (a->first.first) - (b->first.first);
+}
+
+/* Return the optab belonging to the given internal function NAME for the given
+ SIGN or unknown_optab. */
+
+optab
+lookup_multi_ifn_optab (enum internal_fn fn, unsigned sign)
+{
+ typedef std::pair<enum internal_fn, unsigned> ifn_pair;
+ typedef auto_vec <std::pair<ifn_pair, optab>>fn_to_optab_map_type;
+ static fn_to_optab_map_type *fn_to_optab_map;
+
+ if (!fn_to_optab_map)
+ {
+ unsigned num
+ = sizeof (internal_fn_hilo_keys_array) / sizeof (enum internal_fn);
+ fn_to_optab_map = new fn_to_optab_map_type ();
+ for (unsigned int i = 0; i < num - 1; ++i)
+ {
+ enum internal_fn fn = internal_fn_hilo_keys_array[i];
+ optab v1 = internal_fn_hilo_values_array[2*i];
+ optab v2 = internal_fn_hilo_values_array[2*i + 1];
+ ifn_pair key1 (fn, 0);
+ fn_to_optab_map->safe_push ({key1, v1});
+ ifn_pair key2 (fn, 1);
+ fn_to_optab_map->safe_push ({key2, v2});
+ }
+ fn_to_optab_map->qsort(ifn_cmp);
+ }
+
+ ifn_pair new_pair (fn, sign ? 1 : 0);
+ optab tmp;
+ std::pair<ifn_pair,optab> pair_wrap (new_pair, tmp);
+ auto entry = fn_to_optab_map->bsearch (&pair_wrap, ifn_cmp);
+ return entry != fn_to_optab_map->end () ? entry->second : unknown_optab;
+}
+
+extern void
+lookup_multi_internal_fn (enum internal_fn ifn, enum internal_fn *lo,
+ enum internal_fn *hi)
+{
+ int ecf_flags = internal_fn_flags (ifn);
+ gcc_assert (ecf_flags & ECF_MULTI);
+
+ *lo = internal_fn (ifn + 1);
+ *hi = internal_fn (ifn + 2);
+}
+
+
/* Fnspec of each internal function, indexed by function number. */
const_tree internal_fn_fnspec_array[IFN_LAST + 1];
@@ -3906,6 +3983,9 @@ commutative_binary_fn_p (internal_fn fn)
case IFN_UBSAN_CHECK_MUL:
case IFN_ADD_OVERFLOW:
case IFN_MUL_OVERFLOW:
+ case IFN_VEC_WIDEN_PLUS:
+ case IFN_VEC_WIDEN_PLUS_LO:
+ case IFN_VEC_WIDEN_PLUS_HI:
return true;
default:
@@ -3991,6 +4071,32 @@ set_edom_supported_p (void)
#endif
}
+#undef DEF_INTERNAL_OPTAB_MULTI_FN
+#define DEF_INTERNAL_OPTAB_MULTI_FN(CODE, FLAGS, OPTAB, SOPTAB, UOPTAB, TYPE) \
+ static void \
+ expand_##CODE (internal_fn, gcall *) \
+ { \
+ gcc_unreachable (); \
+ } \
+ static void \
+ expand_##CODE##_LO (internal_fn fn, gcall *stmt) \
+ { \
+ tree ty = TREE_TYPE (gimple_get_lhs (stmt)); \
+ if (!TYPE_UNSIGNED (ty)) \
+ expand_##TYPE##_optab_fn (fn, stmt, SOPTAB##_lo##_optab); \
+ else \
+ expand_##TYPE##_optab_fn (fn, stmt, UOPTAB##_lo##_optab); \
+ } \
+ static void \
+ expand_##CODE##_HI (internal_fn fn, gcall *stmt) \
+ { \
+ tree ty = TREE_TYPE (gimple_get_lhs (stmt)); \
+ if (!TYPE_UNSIGNED (ty)) \
+ expand_##TYPE##_optab_fn (fn, stmt, SOPTAB##_hi##_optab); \
+ else \
+ expand_##TYPE##_optab_fn (fn, stmt, UOPTAB##_hi##_optab); \
+ }
+
#define DEF_INTERNAL_OPTAB_FN(CODE, FLAGS, OPTAB, TYPE) \
static void \
expand_##CODE (internal_fn fn, gcall *stmt) \
@@ -4007,6 +4113,7 @@ set_edom_supported_p (void)
expand_##TYPE##_optab_fn (fn, stmt, which_optab); \
}
#include "internal-fn.def"
+#undef DEF_INTERNAL_OPTAB_MULTI_FN
/* Routines to expand each internal function, indexed by function number.
Each routine has the prototype:
diff --git a/gcc/internal-fn.def b/gcc/internal-fn.def
index d2d550d358606022b1cb44fa842f06e0be507bc3..4635a9c8af9ad27bb05d7510388d0fe2270428e5 100644
--- a/gcc/internal-fn.def
+++ b/gcc/internal-fn.def
@@ -82,6 +82,13 @@ along with GCC; see the file COPYING3. If not see
says that the function extends the C-level BUILT_IN_<NAME>{,L,LL,IMAX}
group of functions to any integral mode (including vector modes).
+ DEF_INTERNAL_OPTAB_MULTI_FN is like DEF_INTERNAL_OPTAB_FN except it
+ provides convenience wrappers for defining conversions that require a
+ hi/lo split, like widening and narrowing operations. Each definition
+ for <NAME> will require an optab named <OPTAB> and two other optabs that
+ you specify for signed and unsigned.
+
+
Each entry must have a corresponding expander of the form:
void expand_NAME (gimple_call stmt)
@@ -120,6 +127,14 @@ along with GCC; see the file COPYING3. If not see
DEF_INTERNAL_OPTAB_FN (NAME, FLAGS, OPTAB, TYPE)
#endif
+#ifndef DEF_INTERNAL_OPTAB_MULTI_FN
+#define DEF_INTERNAL_OPTAB_MULTI_FN(NAME, FLAGS, OPTAB, SOPTAB, UOPTAB, TYPE) \
+ DEF_INTERNAL_OPTAB_FN (NAME, FLAGS | ECF_MULTI, OPTAB, TYPE) \
+ DEF_INTERNAL_OPTAB_FN (NAME ## _LO, FLAGS, unknown, TYPE) \
+ DEF_INTERNAL_OPTAB_FN (NAME ## _HI, FLAGS, unknown, TYPE)
+#endif
+
+
DEF_INTERNAL_OPTAB_FN (MASK_LOAD, ECF_PURE, maskload, mask_load)
DEF_INTERNAL_OPTAB_FN (LOAD_LANES, ECF_CONST, vec_load_lanes, load_lanes)
DEF_INTERNAL_OPTAB_FN (MASK_LOAD_LANES, ECF_PURE,
@@ -292,6 +307,14 @@ DEF_INTERNAL_OPTAB_FN (COMPLEX_ADD_ROT270, ECF_CONST, cadd270, binary)
DEF_INTERNAL_OPTAB_FN (COMPLEX_MUL, ECF_CONST, cmul, binary)
DEF_INTERNAL_OPTAB_FN (COMPLEX_MUL_CONJ, ECF_CONST, cmul_conj, binary)
DEF_INTERNAL_OPTAB_FN (VEC_ADDSUB, ECF_CONST, vec_addsub, binary)
+DEF_INTERNAL_OPTAB_MULTI_FN (VEC_WIDEN_PLUS,
+ ECF_CONST | ECF_WIDEN | ECF_NOTHROW,
+ vec_widen_add, vec_widen_saddl, vec_widen_uaddl,
+ binary)
+DEF_INTERNAL_OPTAB_MULTI_FN (VEC_WIDEN_MINUS,
+ ECF_CONST | ECF_WIDEN | ECF_NOTHROW,
+ vec_widen_sub, vec_widen_ssubl, vec_widen_usubl,
+ binary)
DEF_INTERNAL_OPTAB_FN (VEC_FMADDSUB, ECF_CONST, vec_fmaddsub, ternary)
DEF_INTERNAL_OPTAB_FN (VEC_FMSUBADD, ECF_CONST, vec_fmsubadd, ternary)
diff --git a/gcc/internal-fn.h b/gcc/internal-fn.h
index 23c014a963c4d72da92c763db87ee486a2adb485..b35de19747d251d19dc13de1e0323368bd2ebdf2 100644
--- a/gcc/internal-fn.h
+++ b/gcc/internal-fn.h
@@ -20,6 +20,10 @@ along with GCC; see the file COPYING3. If not see
#ifndef GCC_INTERNAL_FN_H
#define GCC_INTERNAL_FN_H
+#include "insn-codes.h"
+#include "insn-opinit.h"
+
+
/* INTEGER_CST values for IFN_UNIQUE function arg-0.
UNSPEC: Undifferentiated UNIQUE.
@@ -112,6 +116,9 @@ internal_fn_name (enum internal_fn fn)
}
extern internal_fn lookup_internal_fn (const char *);
+extern optab lookup_multi_ifn_optab (enum internal_fn, unsigned);
+extern void lookup_multi_internal_fn (enum internal_fn, enum internal_fn *,
+ enum internal_fn *);
/* Return the ECF_* flags for function FN. */
diff --git a/gcc/optabs.cc b/gcc/optabs.cc
index c0a68471d2ddf08bc0e6a3fd592ebb9f05e516c1..7e904b3e154d018779bb1a36de74e6997f70e193 100644
--- a/gcc/optabs.cc
+++ b/gcc/optabs.cc
@@ -1314,7 +1314,17 @@ commutative_optab_p (optab binoptab)
|| binoptab == smul_widen_optab
|| binoptab == umul_widen_optab
|| binoptab == smul_highpart_optab
- || binoptab == umul_highpart_optab);
+ || binoptab == umul_highpart_optab
+ || binoptab == vec_widen_add_optab
+ || binoptab == vec_widen_sub_optab
+ || binoptab == vec_widen_saddl_hi_optab
+ || binoptab == vec_widen_saddl_lo_optab
+ || binoptab == vec_widen_ssubl_hi_optab
+ || binoptab == vec_widen_ssubl_lo_optab
+ || binoptab == vec_widen_uaddl_hi_optab
+ || binoptab == vec_widen_uaddl_lo_optab
+ || binoptab == vec_widen_usubl_hi_optab
+ || binoptab == vec_widen_usubl_lo_optab);
}
/* X is to be used in mode MODE as operand OPN to BINOPTAB. If we're
diff --git a/gcc/optabs.def b/gcc/optabs.def
index 801310ebaa7d469520809bb7efed6820f8eb866b..a7881dcb49e4ef07d8f07aa31214eb3a7a944e2e 100644
--- a/gcc/optabs.def
+++ b/gcc/optabs.def
@@ -78,6 +78,8 @@ OPTAB_CD(smsub_widen_optab, "msub$b$a4")
OPTAB_CD(umsub_widen_optab, "umsub$b$a4")
OPTAB_CD(ssmsub_widen_optab, "ssmsub$b$a4")
OPTAB_CD(usmsub_widen_optab, "usmsub$a$b4")
+OPTAB_CD(vec_widen_add_optab, "add$a$b3")
+OPTAB_CD(vec_widen_sub_optab, "sub$a$b3")
OPTAB_CD(vec_load_lanes_optab, "vec_load_lanes$a$b")
OPTAB_CD(vec_store_lanes_optab, "vec_store_lanes$a$b")
OPTAB_CD(vec_mask_load_lanes_optab, "vec_mask_load_lanes$a$b")
diff --git a/gcc/testsuite/gcc.target/aarch64/vect-widen-add.c b/gcc/testsuite/gcc.target/aarch64/vect-widen-add.c
index 220bd9352a4c7acd2e3713e441d74898d3e92b30..7037673d32bd780e1c9b58a51e58e2bac3b30b7e 100644
--- a/gcc/testsuite/gcc.target/aarch64/vect-widen-add.c
+++ b/gcc/testsuite/gcc.target/aarch64/vect-widen-add.c
@@ -1,5 +1,5 @@
/* { dg-do run } */
-/* { dg-options "-O3 -save-temps" } */
+/* { dg-options "-O3 -save-temps -fdump-tree-vect-all" } */
#include <stdint.h>
#include <string.h>
@@ -86,6 +86,8 @@ main()
return 0;
}
+/* { dg-final { scan-tree-dump "add new stmt.*VEC_WIDEN_PLUS_LO" "vect" } } */
+/* { dg-final { scan-tree-dump "add new stmt.*VEC_WIDEN_PLUS_HI" "vect" } } */
/* { dg-final { scan-assembler-times {\tuaddl\t} 1} } */
/* { dg-final { scan-assembler-times {\tuaddl2\t} 1} } */
/* { dg-final { scan-assembler-times {\tsaddl\t} 1} } */
diff --git a/gcc/testsuite/gcc.target/aarch64/vect-widen-sub.c b/gcc/testsuite/gcc.target/aarch64/vect-widen-sub.c
index a2bed63affbd091977df95a126da1f5b8c1d41d2..83bc1edb6105f47114b665e24a13e6194b2179a2 100644
--- a/gcc/testsuite/gcc.target/aarch64/vect-widen-sub.c
+++ b/gcc/testsuite/gcc.target/aarch64/vect-widen-sub.c
@@ -1,5 +1,5 @@
/* { dg-do run } */
-/* { dg-options "-O3 -save-temps" } */
+/* { dg-options "-O3 -save-temps -fdump-tree-vect-all" } */
#include <stdint.h>
#include <string.h>
@@ -86,6 +86,8 @@ main()
return 0;
}
+/* { dg-final { scan-tree-dump "add new stmt.*VEC_WIDEN_MINUS_LO" "vect" } } */
+/* { dg-final { scan-tree-dump "add new stmt.*VEC_WIDEN_MINUS_HI" "vect" } } */
/* { dg-final { scan-assembler-times {\tusubl\t} 1} } */
/* { dg-final { scan-assembler-times {\tusubl2\t} 1} } */
/* { dg-final { scan-assembler-times {\tssubl\t} 1} } */
diff --git a/gcc/tree-core.h b/gcc/tree-core.h
index cff6211080bced0bffb39e98039a6550897acf77..d0c8b812cfb9c3ac83bf25fff0431b08cb7d823d 100644
--- a/gcc/tree-core.h
+++ b/gcc/tree-core.h
@@ -99,6 +99,10 @@ struct die_struct;
/* Nonzero if this is a widening function. */
#define ECF_WIDEN (1 << 16)
+/* Nonzero if this is a function that decomposes into a lo/hi operation. */
+#define ECF_MULTI (1 << 17)
+
+
/* Call argument flags. */
/* Nonzero if the argument is not used by the function. */
diff --git a/gcc/tree-vect-patterns.cc b/gcc/tree-vect-patterns.cc
index c011b8ede3c266b59f731e316efbec7d98e91068..268f5402fcdd5ec5bfb806db8c410e701c771275 100644
--- a/gcc/tree-vect-patterns.cc
+++ b/gcc/tree-vect-patterns.cc
@@ -1351,14 +1351,16 @@ static gimple *
vect_recog_widen_op_pattern (vec_info *vinfo,
stmt_vec_info last_stmt_info, tree *type_out,
tree_code orig_code, code_helper wide_code,
- bool shift_p, const char *name)
+ bool shift_p, const char *name,
+ enum optab_subtype *subtype = NULL)
{
gimple *last_stmt = last_stmt_info->stmt;
vect_unpromoted_value unprom[2];
tree half_type;
if (!vect_widened_op_tree (vinfo, last_stmt_info, orig_code, orig_code,
- shift_p, 2, unprom, &half_type))
+ shift_p, 2, unprom, &half_type, subtype))
+
return NULL;
/* Pattern detected. */
@@ -1426,6 +1428,20 @@ vect_recog_widen_op_pattern (vec_info *vinfo,
type, pattern_stmt, vecctype);
}
+static gimple *
+vect_recog_widen_op_pattern (vec_info *vinfo,
+ stmt_vec_info last_stmt_info, tree *type_out,
+ tree_code orig_code, internal_fn wide_ifn,
+ bool shift_p, const char *name,
+ enum optab_subtype *subtype = NULL)
+{
+ combined_fn ifn = as_combined_fn (wide_ifn);
+ return vect_recog_widen_op_pattern (vinfo, last_stmt_info, type_out,
+ orig_code, ifn, shift_p, name,
+ subtype);
+}
+
+
/* Try to detect multiplication on widened inputs, converting MULT_EXPR
to WIDEN_MULT_EXPR. See vect_recog_widen_op_pattern for details. */
@@ -1439,26 +1455,30 @@ vect_recog_widen_mult_pattern (vec_info *vinfo, stmt_vec_info last_stmt_info,
}
/* Try to detect addition on widened inputs, converting PLUS_EXPR
- to WIDEN_PLUS_EXPR. See vect_recog_widen_op_pattern for details. */
+ to IFN_VEC_WIDEN_PLUS. See vect_recog_widen_op_pattern for details. */
static gimple *
vect_recog_widen_plus_pattern (vec_info *vinfo, stmt_vec_info last_stmt_info,
tree *type_out)
{
+ enum optab_subtype subtype;
return vect_recog_widen_op_pattern (vinfo, last_stmt_info, type_out,
- PLUS_EXPR, WIDEN_PLUS_EXPR, false,
- "vect_recog_widen_plus_pattern");
+ PLUS_EXPR, IFN_VEC_WIDEN_PLUS,
+ false, "vect_recog_widen_plus_pattern",
+ &subtype);
}
/* Try to detect subtraction on widened inputs, converting MINUS_EXPR
- to WIDEN_MINUS_EXPR. See vect_recog_widen_op_pattern for details. */
+ to IFN_VEC_WIDEN_MINUS. See vect_recog_widen_op_pattern for details. */
static gimple *
vect_recog_widen_minus_pattern (vec_info *vinfo, stmt_vec_info last_stmt_info,
tree *type_out)
{
+ enum optab_subtype subtype;
return vect_recog_widen_op_pattern (vinfo, last_stmt_info, type_out,
- MINUS_EXPR, WIDEN_MINUS_EXPR, false,
- "vect_recog_widen_minus_pattern");
+ MINUS_EXPR, IFN_VEC_WIDEN_MINUS,
+ false, "vect_recog_widen_minus_pattern",
+ &subtype);
}
/* Function vect_recog_popcount_pattern
@@ -5622,6 +5642,7 @@ static vect_recog_func vect_vect_recog_func_ptrs[] = {
{ vect_recog_mask_conversion_pattern, "mask_conversion" },
{ vect_recog_widen_plus_pattern, "widen_plus" },
{ vect_recog_widen_minus_pattern, "widen_minus" },
+ /* These must come after the double widening ones. */
};
const unsigned int NUM_PATTERNS = ARRAY_SIZE (vect_vect_recog_func_ptrs);
diff --git a/gcc/tree-vect-stmts.cc b/gcc/tree-vect-stmts.cc
index 9b31425352689d409b8c0aa0c1d5c69e72db869a..9af0d107fdafb959db10d87e4e0ba5fda4e47bd7 100644
--- a/gcc/tree-vect-stmts.cc
+++ b/gcc/tree-vect-stmts.cc
@@ -4904,7 +4904,8 @@ vectorizable_conversion (vec_info *vinfo,
|| code == WIDEN_LSHIFT_EXPR);
}
else
- widen_arith = false;
+ widen_arith = (code == IFN_VEC_WIDEN_PLUS
+ || code == IFN_VEC_WIDEN_MINUS);
if (!widen_arith
&& !CONVERT_EXPR_CODE_P (code)
@@ -4954,7 +4955,9 @@ vectorizable_conversion (vec_info *vinfo,
gcc_assert (code == WIDEN_MULT_EXPR
|| code == WIDEN_LSHIFT_EXPR
|| code == WIDEN_PLUS_EXPR
- || code == WIDEN_MINUS_EXPR);
+ || code == WIDEN_MINUS_EXPR
+ || code == IFN_VEC_WIDEN_PLUS
+ || code == IFN_VEC_WIDEN_MINUS);
op1 = is_gimple_assign (stmt) ? gimple_assign_rhs2 (stmt) :
@@ -12126,12 +12129,62 @@ supportable_widening_operation (vec_info *vinfo,
optab1 = vec_unpacks_sbool_lo_optab;
optab2 = vec_unpacks_sbool_hi_optab;
}
- else
- {
- optab1 = optab_for_tree_code (c1.safe_as_tree_code (), vectype, optab_default);
- optab2 = optab_for_tree_code (c2.safe_as_tree_code (), vectype, optab_default);
+
+ if (code.is_fn_code ())
+ {
+ internal_fn ifn = as_internal_fn (code.safe_as_fn_code ());
+ int ecf_flags = internal_fn_flags (ifn);
+ gcc_assert (ecf_flags & ECF_MULTI);
+
+ switch (code.safe_as_fn_code ())
+ {
+ case CFN_VEC_WIDEN_PLUS:
+ break;
+ case CFN_VEC_WIDEN_MINUS:
+ break;
+ case CFN_LAST:
+ default:
+ return false;
+ }
+
+ internal_fn lo, hi;
+ lookup_multi_internal_fn (ifn, &lo, &hi);
+ *code1 = as_combined_fn (lo);
+ *code2 = as_combined_fn (hi);
+ optab1 = lookup_multi_ifn_optab (lo, !TYPE_UNSIGNED (vectype));
+ optab2 = lookup_multi_ifn_optab (hi, !TYPE_UNSIGNED (vectype));
}
+ if (code.is_tree_code ())
+ {
+ if (code == FIX_TRUNC_EXPR)
+ {
+ /* The signedness is determined from output operand. */
+ optab1 = optab_for_tree_code (c1.safe_as_tree_code (), vectype_out,
+ optab_default);
+ optab2 = optab_for_tree_code (c2.safe_as_tree_code (), vectype_out,
+ optab_default);
+ }
+ else if (CONVERT_EXPR_CODE_P (code.safe_as_tree_code ())
+ && VECTOR_BOOLEAN_TYPE_P (wide_vectype)
+ && VECTOR_BOOLEAN_TYPE_P (vectype)
+ && TYPE_MODE (wide_vectype) == TYPE_MODE (vectype)
+ && SCALAR_INT_MODE_P (TYPE_MODE (vectype)))
+ {
+ /* If the input and result modes are the same, a different optab
+ is needed where we pass in the number of units in vectype. */
+ optab1 = vec_unpacks_sbool_lo_optab;
+ optab2 = vec_unpacks_sbool_hi_optab;
+ }
+ else
+ {
+ optab1 = optab_for_tree_code (c1.safe_as_tree_code (), vectype,
+ optab_default);
+ optab2 = optab_for_tree_code (c2.safe_as_tree_code (), vectype,
+ optab_default);
+ }
+ }
+
if (!optab1 || !optab2)
return false;
--
2.17.1
[-- Attachment #4: 0003-Remove-widen_plus-minus_expr-tree-codes.patch --]
[-- Type: application/octet-stream, Size: 19519 bytes --]
From 061e11fa3c76c42640ab6467858e057e3067a6d3 Mon Sep 17 00:00:00 2001
From: Joel Hutton <joel.hutton@arm.com>
Date: Fri, 28 Jan 2022 12:04:44 +0000
Subject: [PATCH 3/3] Remove widen_plus/minus_expr tree codes
This patch removes the old widen plus/minus tree codes which have been
replaced by internal functions.
gcc/ChangeLog:
* doc/generic.texi: Remove old tree codes.
* expr.cc (expand_expr_real_2): Remove old tree code cases.
* gimple-pretty-print.cc (dump_binary_rhs): Remove old tree code
cases.
* optabs-tree.cc (optab_for_tree_code): Remove old tree code cases.
(supportable_half_widening_operation): Remove old tree code cases.
* tree-cfg.cc (verify_gimple_assign_binary): Remove old tree code
cases.
* tree-inline.cc (estimate_operator_cost): Remove old tree code
cases.
* tree-pretty-print.cc (dump_generic_node): Remove tree code definition.
(op_symbol_code): Remove old tree code
cases.
* tree-vect-data-refs.cc (vect_get_smallest_scalar_type): Remove old tree code
cases.
(vect_analyze_data_ref_accesses): Remove old tree code
cases.
* tree-vect-generic.cc (expand_vector_operations_1): Remove old tree code
cases.
* tree-vect-patterns.cc (vect_widened_op_tree): Refactor ot replace
usage in vect_recog_sad_pattern.
(vect_recog_sad_pattern): Replace tree code widening pattern with
internal function.
(vect_recog_average_pattern): Replace tree code widening pattern
with internal function.
* tree-vect-stmts.cc (vectorizable_conversion): Remove old tree code
cases.
(supportable_widening_operation): Remove old tree code
cases.
* tree.def (WIDEN_PLUS_EXPR): Remove tree code definition.
(WIDEN_MINUS_EXPR): Remove tree code definition.
(VEC_WIDEN_PLUS_HI_EXPR): Remove tree code definition.
(VEC_WIDEN_PLUS_LO_EXPR): Remove tree code definition.
(VEC_WIDEN_MINUS_HI_EXPR): Remove tree code definition.
(VEC_WIDEN_MINUS_LO_EXPR): Remove tree code definition.
---
gcc/doc/generic.texi | 31 -------------------------------
gcc/expr.cc | 6 ------
gcc/gimple-pretty-print.cc | 4 ----
gcc/optabs-tree.cc | 24 ------------------------
gcc/tree-cfg.cc | 6 ------
gcc/tree-inline.cc | 6 ------
gcc/tree-pretty-print.cc | 12 ------------
gcc/tree-vect-data-refs.cc | 8 +++-----
gcc/tree-vect-generic.cc | 4 ----
gcc/tree-vect-patterns.cc | 36 +++++++++++++++++++++++++-----------
gcc/tree-vect-stmts.cc | 18 ++----------------
gcc/tree.def | 6 ------
12 files changed, 30 insertions(+), 131 deletions(-)
diff --git a/gcc/doc/generic.texi b/gcc/doc/generic.texi
index e5f9d1be8ea81f3da002ec3bb925590d331a2551..344045efd419b0cc3a11771acf70d2fd279c48ac 100644
--- a/gcc/doc/generic.texi
+++ b/gcc/doc/generic.texi
@@ -1811,10 +1811,6 @@ a value from @code{enum annot_expr_kind}, the third is an @code{INTEGER_CST}.
@tindex VEC_RSHIFT_EXPR
@tindex VEC_WIDEN_MULT_HI_EXPR
@tindex VEC_WIDEN_MULT_LO_EXPR
-@tindex VEC_WIDEN_PLUS_HI_EXPR
-@tindex VEC_WIDEN_PLUS_LO_EXPR
-@tindex VEC_WIDEN_MINUS_HI_EXPR
-@tindex VEC_WIDEN_MINUS_LO_EXPR
@tindex VEC_UNPACK_HI_EXPR
@tindex VEC_UNPACK_LO_EXPR
@tindex VEC_UNPACK_FLOAT_HI_EXPR
@@ -1861,33 +1857,6 @@ vector of @code{N/2} products. In the case of @code{VEC_WIDEN_MULT_LO_EXPR} the
low @code{N/2} elements of the two vector are multiplied to produce the
vector of @code{N/2} products.
-@item VEC_WIDEN_PLUS_HI_EXPR
-@itemx VEC_WIDEN_PLUS_LO_EXPR
-These nodes represent widening vector addition of the high and low parts of
-the two input vectors, respectively. Their operands are vectors that contain
-the same number of elements (@code{N}) of the same integral type. The result
-is a vector that contains half as many elements, of an integral type whose size
-is twice as wide. In the case of @code{VEC_WIDEN_PLUS_HI_EXPR} the high
-@code{N/2} elements of the two vectors are added to produce the vector of
-@code{N/2} products. In the case of @code{VEC_WIDEN_PLUS_LO_EXPR} the low
-@code{N/2} elements of the two vectors are added to produce the vector of
-@code{N/2} products.
-
-@item VEC_WIDEN_MINUS_HI_EXPR
-@itemx VEC_WIDEN_MINUS_LO_EXPR
-These nodes represent widening vector subtraction of the high and low parts of
-the two input vectors, respectively. Their operands are vectors that contain
-the same number of elements (@code{N}) of the same integral type. The high/low
-elements of the second vector are subtracted from the high/low elements of the
-first. The result is a vector that contains half as many elements, of an
-integral type whose size is twice as wide. In the case of
-@code{VEC_WIDEN_MINUS_HI_EXPR} the high @code{N/2} elements of the second
-vector are subtracted from the high @code{N/2} of the first to produce the
-vector of @code{N/2} products. In the case of
-@code{VEC_WIDEN_MINUS_LO_EXPR} the low @code{N/2} elements of the second
-vector are subtracted from the low @code{N/2} of the first to produce the
-vector of @code{N/2} products.
-
@item VEC_UNPACK_HI_EXPR
@itemx VEC_UNPACK_LO_EXPR
These nodes represent unpacking of the high and low parts of the input vector,
diff --git a/gcc/expr.cc b/gcc/expr.cc
index fb062dc847577ec9dc2c951330f4cfadcc869325..4e3655070400cee086c2fdc6ac5bbe08d303de5e 100644
--- a/gcc/expr.cc
+++ b/gcc/expr.cc
@@ -9371,8 +9371,6 @@ expand_expr_real_2 (sepops ops, rtx target, machine_mode tmode,
target, unsignedp);
return target;
- case WIDEN_PLUS_EXPR:
- case WIDEN_MINUS_EXPR:
case WIDEN_MULT_EXPR:
/* If first operand is constant, swap them.
Thus the following special case checks need only
@@ -10150,10 +10148,6 @@ expand_expr_real_2 (sepops ops, rtx target, machine_mode tmode,
return temp;
}
- case VEC_WIDEN_PLUS_HI_EXPR:
- case VEC_WIDEN_PLUS_LO_EXPR:
- case VEC_WIDEN_MINUS_HI_EXPR:
- case VEC_WIDEN_MINUS_LO_EXPR:
case VEC_WIDEN_MULT_HI_EXPR:
case VEC_WIDEN_MULT_LO_EXPR:
case VEC_WIDEN_MULT_EVEN_EXPR:
diff --git a/gcc/gimple-pretty-print.cc b/gcc/gimple-pretty-print.cc
index ebd87b20a0adc080c4a8f9429e75f49b96e72f9a..2a1a5b7f811ca341e8ee7e85a9701d3a37ff80bf 100644
--- a/gcc/gimple-pretty-print.cc
+++ b/gcc/gimple-pretty-print.cc
@@ -459,10 +459,6 @@ dump_binary_rhs (pretty_printer *buffer, const gassign *gs, int spc,
case VEC_PACK_FLOAT_EXPR:
case VEC_WIDEN_LSHIFT_HI_EXPR:
case VEC_WIDEN_LSHIFT_LO_EXPR:
- case VEC_WIDEN_PLUS_HI_EXPR:
- case VEC_WIDEN_PLUS_LO_EXPR:
- case VEC_WIDEN_MINUS_HI_EXPR:
- case VEC_WIDEN_MINUS_LO_EXPR:
case VEC_SERIES_EXPR:
for (p = get_tree_code_name (code); *p; p++)
pp_character (buffer, TOUPPER (*p));
diff --git a/gcc/optabs-tree.cc b/gcc/optabs-tree.cc
index 8383fe820b080f6d66f83dcf3b77d3c9f869f4bc..2f5f93dc6624f86f6b5618cf6e7aa2b508053a64 100644
--- a/gcc/optabs-tree.cc
+++ b/gcc/optabs-tree.cc
@@ -190,22 +190,6 @@ optab_for_tree_code (enum tree_code code, const_tree type,
return (TYPE_UNSIGNED (type)
? vec_widen_ushiftl_lo_optab : vec_widen_sshiftl_lo_optab);
- case VEC_WIDEN_PLUS_LO_EXPR:
- return (TYPE_UNSIGNED (type)
- ? vec_widen_uaddl_lo_optab : vec_widen_saddl_lo_optab);
-
- case VEC_WIDEN_PLUS_HI_EXPR:
- return (TYPE_UNSIGNED (type)
- ? vec_widen_uaddl_hi_optab : vec_widen_saddl_hi_optab);
-
- case VEC_WIDEN_MINUS_LO_EXPR:
- return (TYPE_UNSIGNED (type)
- ? vec_widen_usubl_lo_optab : vec_widen_ssubl_lo_optab);
-
- case VEC_WIDEN_MINUS_HI_EXPR:
- return (TYPE_UNSIGNED (type)
- ? vec_widen_usubl_hi_optab : vec_widen_ssubl_hi_optab);
-
case VEC_UNPACK_HI_EXPR:
return (TYPE_UNSIGNED (type)
? vec_unpacku_hi_optab : vec_unpacks_hi_optab);
@@ -312,8 +296,6 @@ optab_for_tree_code (enum tree_code code, const_tree type,
'hi'/'lo' pair using codes such as VEC_WIDEN_MINUS_HI/LO.
Supported widening operations:
- WIDEN_MINUS_EXPR
- WIDEN_PLUS_EXPR
WIDEN_MULT_EXPR
WIDEN_LSHIFT_EXPR
@@ -345,12 +327,6 @@ supportable_half_widening_operation (enum tree_code code, tree vectype_out,
case WIDEN_LSHIFT_EXPR:
*code1 = LSHIFT_EXPR;
break;
- case WIDEN_MINUS_EXPR:
- *code1 = MINUS_EXPR;
- break;
- case WIDEN_PLUS_EXPR:
- *code1 = PLUS_EXPR;
- break;
case WIDEN_MULT_EXPR:
*code1 = MULT_EXPR;
break;
diff --git a/gcc/tree-cfg.cc b/gcc/tree-cfg.cc
index 8de1b144a426776bf464765477c71ee8f2e52b81..46eed1e1f22052fc077f2fc25e5be627bce541b6 100644
--- a/gcc/tree-cfg.cc
+++ b/gcc/tree-cfg.cc
@@ -3948,8 +3948,6 @@ verify_gimple_assign_binary (gassign *stmt)
return false;
}
- case WIDEN_PLUS_EXPR:
- case WIDEN_MINUS_EXPR:
case PLUS_EXPR:
case MINUS_EXPR:
{
@@ -4070,10 +4068,6 @@ verify_gimple_assign_binary (gassign *stmt)
return false;
}
- case VEC_WIDEN_MINUS_HI_EXPR:
- case VEC_WIDEN_MINUS_LO_EXPR:
- case VEC_WIDEN_PLUS_HI_EXPR:
- case VEC_WIDEN_PLUS_LO_EXPR:
case VEC_WIDEN_MULT_HI_EXPR:
case VEC_WIDEN_MULT_LO_EXPR:
case VEC_WIDEN_MULT_EVEN_EXPR:
diff --git a/gcc/tree-inline.cc b/gcc/tree-inline.cc
index 043e1d5987a4c4b0159109dafb85a805ca828c1e..c0bebb7f4de36838341ed62389ad0e2b79f03034 100644
--- a/gcc/tree-inline.cc
+++ b/gcc/tree-inline.cc
@@ -4288,8 +4288,6 @@ estimate_operator_cost (enum tree_code code, eni_weights *weights,
case REALIGN_LOAD_EXPR:
- case WIDEN_PLUS_EXPR:
- case WIDEN_MINUS_EXPR:
case WIDEN_SUM_EXPR:
case WIDEN_MULT_EXPR:
case DOT_PROD_EXPR:
@@ -4298,10 +4296,6 @@ estimate_operator_cost (enum tree_code code, eni_weights *weights,
case WIDEN_MULT_MINUS_EXPR:
case WIDEN_LSHIFT_EXPR:
- case VEC_WIDEN_PLUS_HI_EXPR:
- case VEC_WIDEN_PLUS_LO_EXPR:
- case VEC_WIDEN_MINUS_HI_EXPR:
- case VEC_WIDEN_MINUS_LO_EXPR:
case VEC_WIDEN_MULT_HI_EXPR:
case VEC_WIDEN_MULT_LO_EXPR:
case VEC_WIDEN_MULT_EVEN_EXPR:
diff --git a/gcc/tree-pretty-print.cc b/gcc/tree-pretty-print.cc
index 6acd394a0790ad2ad989f195a3288f0f0a8cc489..53ca62dc1a6873ae9365f199061bde9edd486196 100644
--- a/gcc/tree-pretty-print.cc
+++ b/gcc/tree-pretty-print.cc
@@ -2825,8 +2825,6 @@ dump_generic_node (pretty_printer *pp, tree node, int spc, dump_flags_t flags,
break;
/* Binary arithmetic and logic expressions. */
- case WIDEN_PLUS_EXPR:
- case WIDEN_MINUS_EXPR:
case WIDEN_SUM_EXPR:
case WIDEN_MULT_EXPR:
case MULT_EXPR:
@@ -3790,10 +3788,6 @@ dump_generic_node (pretty_printer *pp, tree node, int spc, dump_flags_t flags,
case VEC_SERIES_EXPR:
case VEC_WIDEN_MULT_HI_EXPR:
case VEC_WIDEN_MULT_LO_EXPR:
- case VEC_WIDEN_PLUS_HI_EXPR:
- case VEC_WIDEN_PLUS_LO_EXPR:
- case VEC_WIDEN_MINUS_HI_EXPR:
- case VEC_WIDEN_MINUS_LO_EXPR:
case VEC_WIDEN_MULT_EVEN_EXPR:
case VEC_WIDEN_MULT_ODD_EXPR:
case VEC_WIDEN_LSHIFT_HI_EXPR:
@@ -4311,12 +4305,6 @@ op_symbol_code (enum tree_code code)
case WIDEN_LSHIFT_EXPR:
return "w<<";
- case WIDEN_PLUS_EXPR:
- return "w+";
-
- case WIDEN_MINUS_EXPR:
- return "w-";
-
case POINTER_PLUS_EXPR:
return "+";
diff --git a/gcc/tree-vect-data-refs.cc b/gcc/tree-vect-data-refs.cc
index d20a10a1524164eef788ab4b88ba57c7a09c3387..98dd56ff022233ccead36a1f5a5e896e352f9f5b 100644
--- a/gcc/tree-vect-data-refs.cc
+++ b/gcc/tree-vect-data-refs.cc
@@ -136,8 +136,6 @@ vect_get_smallest_scalar_type (stmt_vec_info stmt_info, tree scalar_type)
|| gimple_assign_rhs_code (assign) == WIDEN_SUM_EXPR
|| gimple_assign_rhs_code (assign) == WIDEN_MULT_EXPR
|| gimple_assign_rhs_code (assign) == WIDEN_LSHIFT_EXPR
- || gimple_assign_rhs_code (assign) == WIDEN_PLUS_EXPR
- || gimple_assign_rhs_code (assign) == WIDEN_MINUS_EXPR
|| gimple_assign_rhs_code (assign) == FLOAT_EXPR)
{
tree rhs_type = TREE_TYPE (gimple_assign_rhs1 (assign));
@@ -3172,8 +3170,8 @@ vect_analyze_data_ref_accesses (vec_info *vinfo,
break;
/* Check that the DR_INITs are compile-time constants. */
- if (!tree_fits_shwi_p (DR_INIT (dra))
- || !tree_fits_shwi_p (DR_INIT (drb)))
+ if (TREE_CODE (DR_INIT (dra)) != INTEGER_CST
+ || TREE_CODE (DR_INIT (drb)) != INTEGER_CST)
break;
/* Different .GOMP_SIMD_LANE calls still give the same lane,
@@ -3225,7 +3223,7 @@ vect_analyze_data_ref_accesses (vec_info *vinfo,
unsigned HOST_WIDE_INT step
= absu_hwi (tree_to_shwi (DR_STEP (dra)));
if (step != 0
- && step <= ((unsigned HOST_WIDE_INT)init_b - init_a))
+ && step <= (unsigned HOST_WIDE_INT)(init_b - init_a))
break;
}
}
diff --git a/gcc/tree-vect-generic.cc b/gcc/tree-vect-generic.cc
index 92aba5d4af61dd478ec3f1b94854e4ad84166774..5823b08baf70b89b22ecc148b0702a84671ad084 100644
--- a/gcc/tree-vect-generic.cc
+++ b/gcc/tree-vect-generic.cc
@@ -2209,10 +2209,6 @@ expand_vector_operations_1 (gimple_stmt_iterator *gsi,
arguments, not the widened result. VEC_UNPACK_FLOAT_*_EXPR is
calculated in the same way above. */
if (code == WIDEN_SUM_EXPR
- || code == VEC_WIDEN_PLUS_HI_EXPR
- || code == VEC_WIDEN_PLUS_LO_EXPR
- || code == VEC_WIDEN_MINUS_HI_EXPR
- || code == VEC_WIDEN_MINUS_LO_EXPR
|| code == VEC_WIDEN_MULT_HI_EXPR
|| code == VEC_WIDEN_MULT_LO_EXPR
|| code == VEC_WIDEN_MULT_EVEN_EXPR
diff --git a/gcc/tree-vect-patterns.cc b/gcc/tree-vect-patterns.cc
index 268f5402fcdd5ec5bfb806db8c410e701c771275..64a0bde05bfcf62d2dc4bd18a9b6f1cb5f8698b5 100644
--- a/gcc/tree-vect-patterns.cc
+++ b/gcc/tree-vect-patterns.cc
@@ -559,21 +559,29 @@ vect_joust_widened_type (tree type, tree new_type, tree *common_type)
static unsigned int
vect_widened_op_tree (vec_info *vinfo, stmt_vec_info stmt_info, tree_code code,
- tree_code widened_code, bool shift_p,
+ code_helper widened_code, bool shift_p,
unsigned int max_nops,
vect_unpromoted_value *unprom, tree *common_type,
enum optab_subtype *subtype = NULL)
{
/* Check for an integer operation with the right code. */
- gassign *assign = dyn_cast <gassign *> (stmt_info->stmt);
- if (!assign)
+ gimple* stmt = stmt_info->stmt;
+ if (!(is_gimple_assign (stmt) || is_gimple_call (stmt)))
return 0;
- tree_code rhs_code = gimple_assign_rhs_code (assign);
- if (rhs_code != code && rhs_code != widened_code)
+ code_helper rhs_code;
+ if (is_gimple_assign (stmt))
+ rhs_code = gimple_assign_rhs_code (stmt);
+ else
+ rhs_code = gimple_call_combined_fn (stmt);
+
+ if (rhs_code.safe_as_tree_code () != code
+ && rhs_code.get_rep () != widened_code.get_rep ())
return 0;
- tree type = TREE_TYPE (gimple_assign_lhs (assign));
+ tree lhs = is_gimple_assign (stmt) ? gimple_assign_lhs (stmt):
+ gimple_call_lhs (stmt);
+ tree type = TREE_TYPE (lhs);
if (!INTEGRAL_TYPE_P (type))
return 0;
@@ -586,7 +594,11 @@ vect_widened_op_tree (vec_info *vinfo, stmt_vec_info stmt_info, tree_code code,
{
vect_unpromoted_value *this_unprom = &unprom[next_op];
unsigned int nops = 1;
- tree op = gimple_op (assign, i + 1);
+ tree op;
+ if (is_gimple_assign (stmt))
+ op = gimple_op (stmt, i + 1);
+ else
+ op = gimple_call_arg (stmt, i);
if (i == 1 && TREE_CODE (op) == INTEGER_CST)
{
/* We already have a common type from earlier operands.
@@ -1299,8 +1311,9 @@ vect_recog_sad_pattern (vec_info *vinfo,
/* FORNOW. Can continue analyzing the def-use chain when this stmt in a phi
inside the loop (in case we are analyzing an outer-loop). */
vect_unpromoted_value unprom[2];
- if (!vect_widened_op_tree (vinfo, diff_stmt_vinfo, MINUS_EXPR, WIDEN_MINUS_EXPR,
- false, 2, unprom, &half_type))
+ if (!vect_widened_op_tree (vinfo, diff_stmt_vinfo, MINUS_EXPR,
+ CFN_VEC_WIDEN_MINUS, false, 2, unprom,
+ &half_type))
return NULL;
vect_pattern_detected ("vect_recog_sad_pattern", last_stmt);
@@ -2339,9 +2352,10 @@ vect_recog_average_pattern (vec_info *vinfo,
internal_fn ifn = IFN_AVG_FLOOR;
vect_unpromoted_value unprom[3];
tree new_type;
+ enum optab_subtype subtype;
unsigned int nops = vect_widened_op_tree (vinfo, plus_stmt_info, PLUS_EXPR,
- WIDEN_PLUS_EXPR, false, 3,
- unprom, &new_type);
+ CFN_VEC_WIDEN_PLUS, false, 3,
+ unprom, &new_type, &subtype);
if (nops == 0)
return NULL;
if (nops == 3)
diff --git a/gcc/tree-vect-stmts.cc b/gcc/tree-vect-stmts.cc
index 9af0d107fdafb959db10d87e4e0ba5fda4e47bd7..a2c0947864a64aea7c19a9c2bc53961821838e00 100644
--- a/gcc/tree-vect-stmts.cc
+++ b/gcc/tree-vect-stmts.cc
@@ -4898,9 +4898,7 @@ vectorizable_conversion (vec_info *vinfo,
if (res_op.code.is_tree_code ())
{
- widen_arith = (code == WIDEN_PLUS_EXPR
- || code == WIDEN_MINUS_EXPR
- || code == WIDEN_MULT_EXPR
+ widen_arith = (code == WIDEN_MULT_EXPR
|| code == WIDEN_LSHIFT_EXPR);
}
else
@@ -4954,8 +4952,6 @@ vectorizable_conversion (vec_info *vinfo,
{
gcc_assert (code == WIDEN_MULT_EXPR
|| code == WIDEN_LSHIFT_EXPR
- || code == WIDEN_PLUS_EXPR
- || code == WIDEN_MINUS_EXPR
|| code == IFN_VEC_WIDEN_PLUS
|| code == IFN_VEC_WIDEN_MINUS);
@@ -11981,7 +11977,7 @@ supportable_widening_operation (vec_info *vinfo,
class loop *vect_loop = NULL;
machine_mode vec_mode;
enum insn_code icode1, icode2;
- optab optab1, optab2;
+ optab optab1 = unknown_optab, optab2 = unknown_optab;
tree vectype = vectype_in;
tree wide_vectype = vectype_out;
code_helper c1 = MAX_TREE_CODES, c2 = MAX_TREE_CODES;
@@ -12075,16 +12071,6 @@ supportable_widening_operation (vec_info *vinfo,
c2 = VEC_WIDEN_LSHIFT_HI_EXPR;
break;
- case WIDEN_PLUS_EXPR:
- c1 = VEC_WIDEN_PLUS_LO_EXPR;
- c2 = VEC_WIDEN_PLUS_HI_EXPR;
- break;
-
- case WIDEN_MINUS_EXPR:
- c1 = VEC_WIDEN_MINUS_LO_EXPR;
- c2 = VEC_WIDEN_MINUS_HI_EXPR;
- break;
-
CASE_CONVERT:
c1 = VEC_UNPACK_LO_EXPR;
c2 = VEC_UNPACK_HI_EXPR;
diff --git a/gcc/tree.def b/gcc/tree.def
index 62650b6934b337c5d56e5393dc114173d72c9aa9..9b2dce3576440c445d3240b9ed937fe67c9a5992 100644
--- a/gcc/tree.def
+++ b/gcc/tree.def
@@ -1383,8 +1383,6 @@ DEFTREECODE (WIDEN_MULT_MINUS_EXPR, "widen_mult_minus_expr", tcc_expression, 3)
the first argument from type t1 to type t2, and then shifting it
by the second argument. */
DEFTREECODE (WIDEN_LSHIFT_EXPR, "widen_lshift_expr", tcc_binary, 2)
-DEFTREECODE (WIDEN_PLUS_EXPR, "widen_plus_expr", tcc_binary, 2)
-DEFTREECODE (WIDEN_MINUS_EXPR, "widen_minus_expr", tcc_binary, 2)
/* Widening vector multiplication.
The two operands are vectors with N elements of size S. Multiplying the
@@ -1449,10 +1447,6 @@ DEFTREECODE (VEC_PACK_FLOAT_EXPR, "vec_pack_float_expr", tcc_binary, 2)
*/
DEFTREECODE (VEC_WIDEN_LSHIFT_HI_EXPR, "widen_lshift_hi_expr", tcc_binary, 2)
DEFTREECODE (VEC_WIDEN_LSHIFT_LO_EXPR, "widen_lshift_lo_expr", tcc_binary, 2)
-DEFTREECODE (VEC_WIDEN_PLUS_HI_EXPR, "widen_plus_hi_expr", tcc_binary, 2)
-DEFTREECODE (VEC_WIDEN_PLUS_LO_EXPR, "widen_plus_lo_expr", tcc_binary, 2)
-DEFTREECODE (VEC_WIDEN_MINUS_HI_EXPR, "widen_minus_hi_expr", tcc_binary, 2)
-DEFTREECODE (VEC_WIDEN_MINUS_LO_EXPR, "widen_minus_lo_expr", tcc_binary, 2)
/* PREDICT_EXPR. Specify hint for branch prediction. The
PREDICT_EXPR_PREDICTOR specify predictor and PREDICT_EXPR_OUTCOME the
--
2.17.1
^ permalink raw reply [flat|nested] 22+ messages in thread
* RE: [ping][vect-patterns] Refactor widen_plus/widen_minus as internal_fns
2022-05-31 10:07 ` Joel Hutton
2022-05-31 16:46 ` Tamar Christina
@ 2022-06-01 10:11 ` Richard Biener
2022-06-06 17:20 ` Joel Hutton
1 sibling, 1 reply; 22+ messages in thread
From: Richard Biener @ 2022-06-01 10:11 UTC (permalink / raw)
To: Joel Hutton; +Cc: Richard Sandiford, gcc-patches
On Tue, 31 May 2022, Joel Hutton wrote:
> > Can you post an updated patch (after the .cc renaming, and code_helper
> > now already moved to tree.h).
> >
> > Thanks,
> > Richard.
>
> Patches attached. They already incorporated the .cc rename, now rebased to be after the change to tree.h
@@ -1412,8 +1412,7 @@ vect_recog_widen_op_pattern (vec_info *vinfo,
2, oprnd, half_type, unprom, vectype);
tree var = vect_recog_temp_ssa_var (itype, NULL);
- gimple *pattern_stmt = gimple_build_assign (var, wide_code,
- oprnd[0], oprnd[1]);
+ gimple *pattern_stmt = gimple_build (var, wide_code, oprnd[0],
oprnd[1]);
you should be able to do without the new gimple_build overload
by using
gimple_seq stmts = NULL;
gimple_build (&stmts, wide_code, itype, oprnd[0], oprnd[1]);
gimple *pattern_stmt = gimple_seq_last_stmt (stmts);
because 'gimple_build' is an existing API.
- if (TREE_CODE (gimple_assign_lhs (stmt)) != SSA_NAME)
+ if (gimple_get_lhs (stmt) == NULL_TREE ||
+ TREE_CODE(gimple_get_lhs (stmt)) != SSA_NAME)
return false;
|| go to the next line, space after TREE_CODE
+ bool widen_arith = false;
+ gimple_match_op res_op;
+ if (!gimple_extract_op (stmt, &res_op))
+ return false;
+ code = res_op.code;
+ op_type = res_op.num_ops;
+
+ if (is_gimple_assign (stmt))
+ {
+ widen_arith = (code == WIDEN_PLUS_EXPR
+ || code == WIDEN_MINUS_EXPR
+ || code == WIDEN_MULT_EXPR
+ || code == WIDEN_LSHIFT_EXPR);
+ }
+ else
+ widen_arith = gimple_call_flags (stmt) & ECF_WIDEN;
there seem to be formatting issues. Also shouldn't you check
if (res_op.code.is_tree_code ()) instead if is_gimple_assign?
I also don't like the ECF_WIDEN "trick", just do as with the
tree codes and explicitely enumerate widening ifns here.
gimple_extract_op is a bit heavy-weight as well, so maybe
instead simply do
if (is_gimple_assign (stmt))
{
code = gimple_assign_rhs_code (stmt);
...
}
else if (gimple_call_internal_p (stmt))
{
code = gimple_call_internal_fn (stmt);
...
}
else
return false;
+ code_helper c1=MAX_TREE_CODES, c2=MAX_TREE_CODES;
spaces before/after '='
@@ -12061,13 +12105,16 @@ supportable_widening_operation (vec_info *vinfo,
if (BYTES_BIG_ENDIAN && c1 != VEC_WIDEN_MULT_EVEN_EXPR)
std::swap (c1, c2);
+
if (code == FIX_TRUNC_EXPR)
{
unnecessary whitespace change.
diff --git a/gcc/tree.h b/gcc/tree.h
index
f84958933d51144bb6ce7cc41eca5f7f06814550..e51e34c051d9b91d1c02a4b2fefdb2b15606a36f
100644
--- a/gcc/tree.h
+++ b/gcc/tree.h
@@ -92,6 +92,10 @@ public:
bool is_fn_code () const { return rep < 0; }
bool is_internal_fn () const;
bool is_builtin_fn () const;
+ enum tree_code as_tree_code () const { return is_tree_code () ?
+ (tree_code)* this : MAX_TREE_CODES; }
+ combined_fn as_fn_code () const { return is_fn_code () ? (combined_fn)
*this
+ : CFN_LAST;}
hmm, the other as_* functions we have are not member functions.
Also this semantically differs from the tree_code () conversion
operator (that one was supposed to be "cheap"). The existing
as_internal_fn for example is documented as being valid only if
the code is actually an internal fn. I see you are introducing
the new function as convenience to get a "safe" not-a-X value,
so maybe they should be called safe_as_tree_code () instead?
int get_rep () const { return rep; }
bool operator== (const code_helper &other) { return rep == other.rep; }
bool operator!= (const code_helper &other) { return rep != other.rep; }
@@ -6657,6 +6661,54 @@ extern unsigned fndecl_dealloc_argno (tree);
if nonnull, set the second argument to the referenced enclosing
object or pointer. Otherwise return null. */
extern tree get_attr_nonstring_decl (tree, tree * = NULL);
+/* Helper to transparently allow tree codes and builtin function codes
+ exist in one storage entity. */
+class code_helper
+{
duplicate add of code_helper.
Sorry to raise these issues so late.
Richard.
^ permalink raw reply [flat|nested] 22+ messages in thread
* RE: [ping][vect-patterns] Refactor widen_plus/widen_minus as internal_fns
2022-05-31 10:07 ` Joel Hutton
@ 2022-05-31 16:46 ` Tamar Christina
2022-06-01 10:11 ` Richard Biener
1 sibling, 0 replies; 22+ messages in thread
From: Tamar Christina @ 2022-05-31 16:46 UTC (permalink / raw)
To: Joel Hutton, Richard Biener; +Cc: Richard Sandiford, gcc-patches
> Just checking there is still interest in this
Definitely, I am waiting for this to be able to send a new patch upstream 😊
Cheers,
Tamar.
> -----Original Message-----
> From: Gcc-patches <gcc-patches-
> bounces+tamar.christina=arm.com@gcc.gnu.org> On Behalf Of Joel Hutton
> via Gcc-patches
> Sent: Tuesday, May 31, 2022 11:08 AM
> To: Richard Biener <rguenther@suse.de>
> Cc: Richard Sandiford <Richard.Sandiford@arm.com>; gcc-
> patches@gcc.gnu.org
> Subject: RE: [ping][vect-patterns] Refactor widen_plus/widen_minus as
> internal_fns
>
> > Can you post an updated patch (after the .cc renaming, and code_helper
> > now already moved to tree.h).
> >
> > Thanks,
> > Richard.
>
> Patches attached. They already incorporated the .cc rename, now rebased to
> be after the change to tree.h
>
> Joel
^ permalink raw reply [flat|nested] 22+ messages in thread
* RE: [ping][vect-patterns] Refactor widen_plus/widen_minus as internal_fns
2022-05-27 13:23 ` Richard Biener
@ 2022-05-31 10:07 ` Joel Hutton
2022-05-31 16:46 ` Tamar Christina
2022-06-01 10:11 ` Richard Biener
0 siblings, 2 replies; 22+ messages in thread
From: Joel Hutton @ 2022-05-31 10:07 UTC (permalink / raw)
To: Richard Biener; +Cc: Richard Sandiford, gcc-patches
[-- Attachment #1: Type: text/plain, Size: 250 bytes --]
> Can you post an updated patch (after the .cc renaming, and code_helper
> now already moved to tree.h).
>
> Thanks,
> Richard.
Patches attached. They already incorporated the .cc rename, now rebased to be after the change to tree.h
Joel
[-- Attachment #2: 0001-Refactor-to-allow-internal_fn-s.patch --]
[-- Type: application/octet-stream, Size: 26740 bytes --]
From 3467bf531402d83c6427716954b9fab933f858ef Mon Sep 17 00:00:00 2001
From: Joel Hutton <joel.hutton@arm.com>
Date: Wed, 25 Aug 2021 14:31:15 +0100
Subject: [PATCH 1/3] Refactor to allow internal_fn's
Hi all,
This refactor allows widening patterns (such as widen_plus/widen_minus) to be represented as
either internal_fns or tree_codes.
[vect-patterns] Refactor as internal_fn's
Refactor vect-patterns to allow patterns to be internal_fns starting
with widening_plus/minus patterns
gcc/ChangeLog:
* gimple-match.h (class code_helper): Add as_internal_fn, as_tree_code
helper functions.
* gimple.cc (gimple_build): Function to build a GIMPLE_CALL or
GIMPLE_ASSIGN as appropriate, given a code_helper.
* gimple.h (gimple_build): Function prototype.
* tree-core.h (ECF_WIDEN): Flag to mark internal_fn as widening.
* tree-vect-patterns.cc (vect_recog_widen_op_pattern): Refactor to
use code_helper.
* tree-vect-stmts.cc (vect_gen_widened_results_half): Refactor to
use code_helper.
(vect_create_vectorized_promotion_stmts): Refactor to use
code_helper.
(vectorizable_conversion): Refactor to use code_helper.
gimple_call or gimple_assign.
(supportable_widening_operation): Refactor to use code_helper.
(supportable_narrowing_operation): Refactor to use code_helper.
* tree-vectorizer.h (supportable_widening_operation): Change
prototype to use code_helper.
(supportable_narrowing_operation): change prototype to use
code_helper.
---
gcc/gimple.cc | 24 +++++
gcc/gimple.h | 1 +
gcc/tree-core.h | 3 +
gcc/tree-vect-patterns.cc | 7 +-
gcc/tree-vect-stmts.cc | 216 +++++++++++++++++++++++---------------
gcc/tree-vectorizer.h | 11 +-
gcc/tree.h | 52 +++++++++
7 files changed, 221 insertions(+), 93 deletions(-)
diff --git a/gcc/gimple.cc b/gcc/gimple.cc
index b70ab4d25230374f0c90f93d77f9caf8d57587ee..bd3210cd35298daf7c74276e38658b5151d5cc1f 100644
--- a/gcc/gimple.cc
+++ b/gcc/gimple.cc
@@ -502,6 +502,30 @@ gimple_build_assign (tree lhs, enum tree_code subcode, tree op1 MEM_STAT_DECL)
PASS_MEM_STAT);
}
+/* Build a GIMPLE_ASSIGN or GIMPLE_CALL with the tree_code,
+ or internal_fn contained in ch, respectively. */
+gimple *
+gimple_build (tree lhs, code_helper ch, tree op0, tree op1)
+{
+ if (op0 == NULL_TREE)
+ return NULL;
+ if (ch.is_tree_code ())
+ return op1 == NULL_TREE ? gimple_build_assign (lhs, ch.as_tree_code (),
+ op0) :
+ gimple_build_assign (lhs, ch.as_tree_code (), op0,
+ op1);
+ else
+ {
+ internal_fn fn = as_internal_fn (ch.as_fn_code ());
+ gimple* stmt;
+ if (op1 == NULL_TREE)
+ stmt = gimple_build_call_internal (fn, 1, op0);
+ else
+ stmt = gimple_build_call_internal (fn, 2, op0, op1);
+ gimple_call_set_lhs (stmt, lhs);
+ return stmt;
+ }
+}
/* Build a GIMPLE_COND statement.
diff --git a/gcc/gimple.h b/gcc/gimple.h
index 6b1e89ad74e6b22dd534ff48e48fef688032f844..5350f14f6f29bd8e011b79c2aa79c2bfaef8c58f 100644
--- a/gcc/gimple.h
+++ b/gcc/gimple.h
@@ -1523,6 +1523,7 @@ gcall *gimple_build_call_valist (tree, unsigned, va_list);
gcall *gimple_build_call_internal (enum internal_fn, unsigned, ...);
gcall *gimple_build_call_internal_vec (enum internal_fn, const vec<tree> &);
gcall *gimple_build_call_from_tree (tree, tree);
+gimple* gimple_build (tree, code_helper, tree, tree);
gassign *gimple_build_assign (tree, tree CXX_MEM_STAT_INFO);
gassign *gimple_build_assign (tree, enum tree_code,
tree, tree, tree CXX_MEM_STAT_INFO);
diff --git a/gcc/tree-core.h b/gcc/tree-core.h
index ab5fa01e5cb5fb56c1964b93b014ed55a4aa704a..cff6211080bced0bffb39e98039a6550897acf77 100644
--- a/gcc/tree-core.h
+++ b/gcc/tree-core.h
@@ -96,6 +96,9 @@ struct die_struct;
/* Nonzero if this is a cold function. */
#define ECF_COLD (1 << 15)
+/* Nonzero if this is a widening function. */
+#define ECF_WIDEN (1 << 16)
+
/* Call argument flags. */
/* Nonzero if the argument is not used by the function. */
diff --git a/gcc/tree-vect-patterns.cc b/gcc/tree-vect-patterns.cc
index 0fad4dbd0945c6c176f3457b751e812f17fcd148..36e362e1daf3f946c6074600a6a322b3bda67755 100644
--- a/gcc/tree-vect-patterns.cc
+++ b/gcc/tree-vect-patterns.cc
@@ -1348,7 +1348,7 @@ vect_recog_sad_pattern (vec_info *vinfo,
static gimple *
vect_recog_widen_op_pattern (vec_info *vinfo,
stmt_vec_info last_stmt_info, tree *type_out,
- tree_code orig_code, tree_code wide_code,
+ tree_code orig_code, code_helper wide_code,
bool shift_p, const char *name)
{
gimple *last_stmt = last_stmt_info->stmt;
@@ -1391,7 +1391,7 @@ vect_recog_widen_op_pattern (vec_info *vinfo,
vecctype = get_vectype_for_scalar_type (vinfo, ctype);
}
- enum tree_code dummy_code;
+ code_helper dummy_code;
int dummy_int;
auto_vec<tree> dummy_vec;
if (!vectype
@@ -1412,8 +1412,7 @@ vect_recog_widen_op_pattern (vec_info *vinfo,
2, oprnd, half_type, unprom, vectype);
tree var = vect_recog_temp_ssa_var (itype, NULL);
- gimple *pattern_stmt = gimple_build_assign (var, wide_code,
- oprnd[0], oprnd[1]);
+ gimple *pattern_stmt = gimple_build (var, wide_code, oprnd[0], oprnd[1]);
if (vecctype != vecitype)
pattern_stmt = vect_convert_output (vinfo, last_stmt_info, ctype,
diff --git a/gcc/tree-vect-stmts.cc b/gcc/tree-vect-stmts.cc
index 346d8ce280437e00bfeb19a4b4adc59eb96207f9..61b51a29f99bcdf0ff6b4ead4a69163ebf8ed383 100644
--- a/gcc/tree-vect-stmts.cc
+++ b/gcc/tree-vect-stmts.cc
@@ -4636,7 +4636,7 @@ vectorizable_simd_clone_call (vec_info *vinfo, stmt_vec_info stmt_info,
STMT_INFO is the original scalar stmt that we are vectorizing. */
static gimple *
-vect_gen_widened_results_half (vec_info *vinfo, enum tree_code code,
+vect_gen_widened_results_half (vec_info *vinfo, code_helper ch,
tree vec_oprnd0, tree vec_oprnd1, int op_type,
tree vec_dest, gimple_stmt_iterator *gsi,
stmt_vec_info stmt_info)
@@ -4645,14 +4645,12 @@ vect_gen_widened_results_half (vec_info *vinfo, enum tree_code code,
tree new_temp;
/* Generate half of the widened result: */
- gcc_assert (op_type == TREE_CODE_LENGTH (code));
if (op_type != binary_op)
vec_oprnd1 = NULL;
- new_stmt = gimple_build_assign (vec_dest, code, vec_oprnd0, vec_oprnd1);
+ new_stmt = gimple_build (vec_dest, ch, vec_oprnd0, vec_oprnd1);
new_temp = make_ssa_name (vec_dest, new_stmt);
- gimple_assign_set_lhs (new_stmt, new_temp);
+ gimple_set_lhs (new_stmt, new_temp);
vect_finish_stmt_generation (vinfo, stmt_info, new_stmt, gsi);
-
return new_stmt;
}
@@ -4729,8 +4727,8 @@ vect_create_vectorized_promotion_stmts (vec_info *vinfo,
vec<tree> *vec_oprnds1,
stmt_vec_info stmt_info, tree vec_dest,
gimple_stmt_iterator *gsi,
- enum tree_code code1,
- enum tree_code code2, int op_type)
+ code_helper ch1,
+ code_helper ch2, int op_type)
{
int i;
tree vop0, vop1, new_tmp1, new_tmp2;
@@ -4746,10 +4744,10 @@ vect_create_vectorized_promotion_stmts (vec_info *vinfo,
vop1 = NULL_TREE;
/* Generate the two halves of promotion operation. */
- new_stmt1 = vect_gen_widened_results_half (vinfo, code1, vop0, vop1,
+ new_stmt1 = vect_gen_widened_results_half (vinfo, ch1, vop0, vop1,
op_type, vec_dest, gsi,
stmt_info);
- new_stmt2 = vect_gen_widened_results_half (vinfo, code2, vop0, vop1,
+ new_stmt2 = vect_gen_widened_results_half (vinfo, ch2, vop0, vop1,
op_type, vec_dest, gsi,
stmt_info);
if (is_gimple_call (new_stmt1))
@@ -4846,8 +4844,9 @@ vectorizable_conversion (vec_info *vinfo,
tree scalar_dest;
tree op0, op1 = NULL_TREE;
loop_vec_info loop_vinfo = dyn_cast <loop_vec_info> (vinfo);
- enum tree_code code, code1 = ERROR_MARK, code2 = ERROR_MARK;
- enum tree_code codecvt1 = ERROR_MARK, codecvt2 = ERROR_MARK;
+ tree_code tc1;
+ code_helper code, code1, code2;
+ code_helper codecvt1 = ERROR_MARK, codecvt2 = ERROR_MARK;
tree new_temp;
enum vect_def_type dt[2] = {vect_unknown_def_type, vect_unknown_def_type};
int ndts = 2;
@@ -4876,31 +4875,42 @@ vectorizable_conversion (vec_info *vinfo,
&& ! vec_stmt)
return false;
- gassign *stmt = dyn_cast <gassign *> (stmt_info->stmt);
- if (!stmt)
+ gimple* stmt = stmt_info->stmt;
+ if (!(is_gimple_assign (stmt) || is_gimple_call (stmt)))
return false;
- if (TREE_CODE (gimple_assign_lhs (stmt)) != SSA_NAME)
+ if (gimple_get_lhs (stmt) == NULL_TREE ||
+ TREE_CODE(gimple_get_lhs (stmt)) != SSA_NAME)
return false;
- code = gimple_assign_rhs_code (stmt);
- if (!CONVERT_EXPR_CODE_P (code)
- && code != FIX_TRUNC_EXPR
- && code != FLOAT_EXPR
- && code != WIDEN_PLUS_EXPR
- && code != WIDEN_MINUS_EXPR
- && code != WIDEN_MULT_EXPR
- && code != WIDEN_LSHIFT_EXPR)
+ if (TREE_CODE (gimple_get_lhs (stmt)) != SSA_NAME)
return false;
- bool widen_arith = (code == WIDEN_PLUS_EXPR
- || code == WIDEN_MINUS_EXPR
- || code == WIDEN_MULT_EXPR
- || code == WIDEN_LSHIFT_EXPR);
- op_type = TREE_CODE_LENGTH (code);
+ bool widen_arith = false;
+ gimple_match_op res_op;
+ if (!gimple_extract_op (stmt, &res_op))
+ return false;
+ code = res_op.code;
+ op_type = res_op.num_ops;
+
+ if (is_gimple_assign (stmt))
+ {
+ widen_arith = (code == WIDEN_PLUS_EXPR
+ || code == WIDEN_MINUS_EXPR
+ || code == WIDEN_MULT_EXPR
+ || code == WIDEN_LSHIFT_EXPR);
+ }
+ else
+ widen_arith = gimple_call_flags (stmt) & ECF_WIDEN;
+
+ if (!widen_arith
+ && !CONVERT_EXPR_CODE_P (code)
+ && code != FIX_TRUNC_EXPR
+ && code != FLOAT_EXPR)
+ return false;
/* Check types of lhs and rhs. */
- scalar_dest = gimple_assign_lhs (stmt);
+ scalar_dest = gimple_get_lhs (stmt);
lhs_type = TREE_TYPE (scalar_dest);
vectype_out = STMT_VINFO_VECTYPE (stmt_info);
@@ -4938,10 +4948,15 @@ vectorizable_conversion (vec_info *vinfo,
if (op_type == binary_op)
{
- gcc_assert (code == WIDEN_MULT_EXPR || code == WIDEN_LSHIFT_EXPR
- || code == WIDEN_PLUS_EXPR || code == WIDEN_MINUS_EXPR);
+ gcc_assert (code == WIDEN_MULT_EXPR
+ || code == WIDEN_LSHIFT_EXPR
+ || code == WIDEN_PLUS_EXPR
+ || code == WIDEN_MINUS_EXPR
+ || widen_arith);
+
- op1 = gimple_assign_rhs2 (stmt);
+ op1 = is_gimple_assign (stmt) ? gimple_assign_rhs2 (stmt) :
+ gimple_call_arg (stmt, 0);
tree vectype1_in;
if (!vect_is_simple_use (vinfo, stmt_info, slp_node, 1,
&op1, &slp_op1, &dt[1], &vectype1_in))
@@ -5025,8 +5040,12 @@ vectorizable_conversion (vec_info *vinfo,
&& code != FLOAT_EXPR
&& !CONVERT_EXPR_CODE_P (code))
return false;
- if (supportable_convert_operation (code, vectype_out, vectype_in, &code1))
+ if (supportable_convert_operation (code.as_tree_code (), vectype_out,
+ vectype_in, &tc1))
+ {
+ code1 = tc1;
break;
+ }
/* FALLTHRU */
unsupported:
if (dump_enabled_p ())
@@ -5037,9 +5056,11 @@ vectorizable_conversion (vec_info *vinfo,
case WIDEN:
if (known_eq (nunits_in, nunits_out))
{
- if (!supportable_half_widening_operation (code, vectype_out,
- vectype_in, &code1))
+ if (!supportable_half_widening_operation (code.as_tree_code (),
+ vectype_out, vectype_in,
+ &tc1))
goto unsupported;
+ code1 = tc1;
gcc_assert (!(multi_step_cvt && op_type == binary_op));
break;
}
@@ -5073,14 +5094,17 @@ vectorizable_conversion (vec_info *vinfo,
if (GET_MODE_SIZE (rhs_mode) == fltsz)
{
- if (!supportable_convert_operation (code, vectype_out,
- cvt_type, &codecvt1))
+ tc1 = ERROR_MARK;
+ if (!supportable_convert_operation (code.as_tree_code (),
+ vectype_out,
+ cvt_type, &tc1))
goto unsupported;
+ codecvt1 = tc1;
}
- else if (!supportable_widening_operation (vinfo, code, stmt_info,
- vectype_out, cvt_type,
- &codecvt1, &codecvt2,
- &multi_step_cvt,
+ else if (!supportable_widening_operation (vinfo, code,
+ stmt_info, vectype_out,
+ cvt_type, &codecvt1,
+ &codecvt2, &multi_step_cvt,
&interm_types))
continue;
else
@@ -5088,8 +5112,9 @@ vectorizable_conversion (vec_info *vinfo,
if (supportable_widening_operation (vinfo, NOP_EXPR, stmt_info,
cvt_type,
- vectype_in, &code1, &code2,
- &multi_step_cvt, &interm_types))
+ vectype_in, &code1,
+ &code2, &multi_step_cvt,
+ &interm_types))
{
found_mode = true;
break;
@@ -5111,10 +5136,14 @@ vectorizable_conversion (vec_info *vinfo,
case NARROW:
gcc_assert (op_type == unary_op);
- if (supportable_narrowing_operation (code, vectype_out, vectype_in,
- &code1, &multi_step_cvt,
+ if (supportable_narrowing_operation (code.as_tree_code (), vectype_out,
+ vectype_in,
+ &tc1, &multi_step_cvt,
&interm_types))
- break;
+ {
+ code1 = tc1;
+ break;
+ }
if (code != FIX_TRUNC_EXPR
|| GET_MODE_SIZE (lhs_mode) >= GET_MODE_SIZE (rhs_mode))
@@ -5125,13 +5154,18 @@ vectorizable_conversion (vec_info *vinfo,
cvt_type = get_same_sized_vectype (cvt_type, vectype_in);
if (cvt_type == NULL_TREE)
goto unsupported;
- if (!supportable_convert_operation (code, cvt_type, vectype_in,
- &codecvt1))
+ if (!supportable_convert_operation (code.as_tree_code (), cvt_type,
+ vectype_in,
+ &tc1))
goto unsupported;
+ codecvt1 = tc1;
if (supportable_narrowing_operation (NOP_EXPR, vectype_out, cvt_type,
- &code1, &multi_step_cvt,
+ &tc1, &multi_step_cvt,
&interm_types))
- break;
+ {
+ code1 = tc1;
+ break;
+ }
goto unsupported;
default:
@@ -5245,8 +5279,9 @@ vectorizable_conversion (vec_info *vinfo,
FOR_EACH_VEC_ELT (vec_oprnds0, i, vop0)
{
/* Arguments are ready, create the new vector stmt. */
- gcc_assert (TREE_CODE_LENGTH (code1) == unary_op);
- gassign *new_stmt = gimple_build_assign (vec_dest, code1, vop0);
+ gcc_assert (TREE_CODE_LENGTH ((tree_code) code1) == unary_op);
+ gassign *new_stmt = gimple_build_assign (vec_dest,
+ code1.as_tree_code (), vop0);
new_temp = make_ssa_name (vec_dest, new_stmt);
gimple_assign_set_lhs (new_stmt, new_temp);
vect_finish_stmt_generation (vinfo, stmt_info, new_stmt, gsi);
@@ -5278,7 +5313,7 @@ vectorizable_conversion (vec_info *vinfo,
for (i = multi_step_cvt; i >= 0; i--)
{
tree this_dest = vec_dsts[i];
- enum tree_code c1 = code1, c2 = code2;
+ code_helper c1 = code1, c2 = code2;
if (i == 0 && codecvt2 != ERROR_MARK)
{
c1 = codecvt1;
@@ -5288,7 +5323,8 @@ vectorizable_conversion (vec_info *vinfo,
vect_create_half_widening_stmts (vinfo, &vec_oprnds0,
&vec_oprnds1, stmt_info,
this_dest, gsi,
- c1, op_type);
+ c1.as_tree_code (),
+ op_type);
else
vect_create_vectorized_promotion_stmts (vinfo, &vec_oprnds0,
&vec_oprnds1, stmt_info,
@@ -5301,9 +5337,11 @@ vectorizable_conversion (vec_info *vinfo,
gimple *new_stmt;
if (cvt_type)
{
- gcc_assert (TREE_CODE_LENGTH (codecvt1) == unary_op);
+ gcc_assert (TREE_CODE_LENGTH ((tree_code) codecvt1) == unary_op);
new_temp = make_ssa_name (vec_dest);
- new_stmt = gimple_build_assign (new_temp, codecvt1, vop0);
+ new_stmt = gimple_build_assign (new_temp,
+ codecvt1.as_tree_code (),
+ vop0);
vect_finish_stmt_generation (vinfo, stmt_info, new_stmt, gsi);
}
else
@@ -5327,10 +5365,10 @@ vectorizable_conversion (vec_info *vinfo,
if (cvt_type)
FOR_EACH_VEC_ELT (vec_oprnds0, i, vop0)
{
- gcc_assert (TREE_CODE_LENGTH (codecvt1) == unary_op);
+ gcc_assert (TREE_CODE_LENGTH (((tree_code) codecvt1)) == unary_op);
new_temp = make_ssa_name (vec_dest);
gassign *new_stmt
- = gimple_build_assign (new_temp, codecvt1, vop0);
+ = gimple_build_assign (new_temp, codecvt1.as_tree_code (), vop0);
vect_finish_stmt_generation (vinfo, stmt_info, new_stmt, gsi);
vec_oprnds0[i] = new_temp;
}
@@ -5338,7 +5376,7 @@ vectorizable_conversion (vec_info *vinfo,
vect_create_vectorized_demotion_stmts (vinfo, &vec_oprnds0,
multi_step_cvt,
stmt_info, vec_dsts, gsi,
- slp_node, code1);
+ slp_node, code1.as_tree_code ());
break;
}
if (!slp_node)
@@ -11926,9 +11964,11 @@ vect_maybe_update_slp_op_vectype (slp_tree op, tree vectype)
bool
supportable_widening_operation (vec_info *vinfo,
- enum tree_code code, stmt_vec_info stmt_info,
+ code_helper code,
+ stmt_vec_info stmt_info,
tree vectype_out, tree vectype_in,
- enum tree_code *code1, enum tree_code *code2,
+ code_helper *code1,
+ code_helper *code2,
int *multi_step_cvt,
vec<tree> *interm_types)
{
@@ -11939,7 +11979,7 @@ supportable_widening_operation (vec_info *vinfo,
optab optab1, optab2;
tree vectype = vectype_in;
tree wide_vectype = vectype_out;
- enum tree_code c1, c2;
+ code_helper c1=MAX_TREE_CODES, c2=MAX_TREE_CODES;
int i;
tree prev_type, intermediate_type;
machine_mode intermediate_mode, prev_mode;
@@ -11949,7 +11989,7 @@ supportable_widening_operation (vec_info *vinfo,
if (loop_info)
vect_loop = LOOP_VINFO_LOOP (loop_info);
- switch (code)
+ switch (code.as_tree_code ())
{
case WIDEN_MULT_EXPR:
/* The result of a vectorized widening operation usually requires
@@ -11990,8 +12030,9 @@ supportable_widening_operation (vec_info *vinfo,
&& !nested_in_vect_loop_p (vect_loop, stmt_info)
&& supportable_widening_operation (vinfo, VEC_WIDEN_MULT_EVEN_EXPR,
stmt_info, vectype_out,
- vectype_in, code1, code2,
- multi_step_cvt, interm_types))
+ vectype_in, code1,
+ code2, multi_step_cvt,
+ interm_types))
{
/* Elements in a vector with vect_used_by_reduction property cannot
be reordered if the use chain with this property does not have the
@@ -12054,6 +12095,9 @@ supportable_widening_operation (vec_info *vinfo,
c2 = VEC_UNPACK_FIX_TRUNC_HI_EXPR;
break;
+ case MAX_TREE_CODES:
+ break;
+
default:
gcc_unreachable ();
}
@@ -12061,13 +12105,16 @@ supportable_widening_operation (vec_info *vinfo,
if (BYTES_BIG_ENDIAN && c1 != VEC_WIDEN_MULT_EVEN_EXPR)
std::swap (c1, c2);
+
if (code == FIX_TRUNC_EXPR)
{
/* The signedness is determined from output operand. */
- optab1 = optab_for_tree_code (c1, vectype_out, optab_default);
- optab2 = optab_for_tree_code (c2, vectype_out, optab_default);
+ optab1 = optab_for_tree_code (c1.as_tree_code (), vectype_out,
+ optab_default);
+ optab2 = optab_for_tree_code (c2.as_tree_code (), vectype_out,
+ optab_default);
}
- else if (CONVERT_EXPR_CODE_P (code)
+ else if (CONVERT_EXPR_CODE_P (code.as_tree_code ())
&& VECTOR_BOOLEAN_TYPE_P (wide_vectype)
&& VECTOR_BOOLEAN_TYPE_P (vectype)
&& TYPE_MODE (wide_vectype) == TYPE_MODE (vectype)
@@ -12080,8 +12127,8 @@ supportable_widening_operation (vec_info *vinfo,
}
else
{
- optab1 = optab_for_tree_code (c1, vectype, optab_default);
- optab2 = optab_for_tree_code (c2, vectype, optab_default);
+ optab1 = optab_for_tree_code (c1.as_tree_code (), vectype, optab_default);
+ optab2 = optab_for_tree_code (c2.as_tree_code (), vectype, optab_default);
}
if (!optab1 || !optab2)
@@ -12092,8 +12139,12 @@ supportable_widening_operation (vec_info *vinfo,
|| (icode2 = optab_handler (optab2, vec_mode)) == CODE_FOR_nothing)
return false;
- *code1 = c1;
- *code2 = c2;
+ if (code.is_tree_code ())
+ {
+ *code1 = c1;
+ *code2 = c2;
+ }
+
if (insn_data[icode1].operand[0].mode == TYPE_MODE (wide_vectype)
&& insn_data[icode2].operand[0].mode == TYPE_MODE (wide_vectype))
@@ -12114,7 +12165,7 @@ supportable_widening_operation (vec_info *vinfo,
prev_type = vectype;
prev_mode = vec_mode;
- if (!CONVERT_EXPR_CODE_P (code))
+ if (!CONVERT_EXPR_CODE_P ((tree_code) code))
return false;
/* We assume here that there will not be more than MAX_INTERM_CVT_STEPS
@@ -12145,8 +12196,10 @@ supportable_widening_operation (vec_info *vinfo,
}
else
{
- optab3 = optab_for_tree_code (c1, intermediate_type, optab_default);
- optab4 = optab_for_tree_code (c2, intermediate_type, optab_default);
+ optab3 = optab_for_tree_code (c1.as_tree_code (), intermediate_type,
+ optab_default);
+ optab4 = optab_for_tree_code (c2.as_tree_code (), intermediate_type,
+ optab_default);
}
if (!optab3 || !optab4
@@ -12181,7 +12234,6 @@ supportable_widening_operation (vec_info *vinfo,
return false;
}
-
/* Function supportable_narrowing_operation
Check whether an operation represented by the code CODE is a
@@ -12205,7 +12257,7 @@ supportable_widening_operation (vec_info *vinfo,
bool
supportable_narrowing_operation (enum tree_code code,
tree vectype_out, tree vectype_in,
- enum tree_code *code1, int *multi_step_cvt,
+ tree_code* _code1, int *multi_step_cvt,
vec<tree> *interm_types)
{
machine_mode vec_mode;
@@ -12217,8 +12269,8 @@ supportable_narrowing_operation (enum tree_code code,
tree intermediate_type, prev_type;
machine_mode intermediate_mode, prev_mode;
int i;
- unsigned HOST_WIDE_INT n_elts;
bool uns;
+ tree_code * code1 = (tree_code*) _code1;
*multi_step_cvt = 0;
switch (code)
@@ -12227,9 +12279,8 @@ supportable_narrowing_operation (enum tree_code code,
c1 = VEC_PACK_TRUNC_EXPR;
if (VECTOR_BOOLEAN_TYPE_P (narrow_vectype)
&& VECTOR_BOOLEAN_TYPE_P (vectype)
- && SCALAR_INT_MODE_P (TYPE_MODE (vectype))
- && TYPE_VECTOR_SUBPARTS (vectype).is_constant (&n_elts)
- && n_elts < BITS_PER_UNIT)
+ && TYPE_MODE (narrow_vectype) == TYPE_MODE (vectype)
+ && SCALAR_INT_MODE_P (TYPE_MODE (vectype)))
optab1 = vec_pack_sbool_trunc_optab;
else
optab1 = optab_for_tree_code (c1, vectype, optab_default);
@@ -12320,9 +12371,8 @@ supportable_narrowing_operation (enum tree_code code,
= lang_hooks.types.type_for_mode (intermediate_mode, uns);
if (VECTOR_BOOLEAN_TYPE_P (intermediate_type)
&& VECTOR_BOOLEAN_TYPE_P (prev_type)
- && SCALAR_INT_MODE_P (prev_mode)
- && TYPE_VECTOR_SUBPARTS (intermediate_type).is_constant (&n_elts)
- && n_elts < BITS_PER_UNIT)
+ && intermediate_mode == prev_mode
+ && SCALAR_INT_MODE_P (prev_mode))
interm_optab = vec_pack_sbool_trunc_optab;
else
interm_optab
diff --git a/gcc/tree-vectorizer.h b/gcc/tree-vectorizer.h
index 642eb0aeb21264cd736a479b1ec25357abef29cd..50ff8eeac1e6b9859302bd784f10ee3d8ff4b4dc 100644
--- a/gcc/tree-vectorizer.h
+++ b/gcc/tree-vectorizer.h
@@ -2120,13 +2120,12 @@ extern bool vect_is_simple_use (vec_info *, stmt_vec_info, slp_tree,
enum vect_def_type *,
tree *, stmt_vec_info * = NULL);
extern bool vect_maybe_update_slp_op_vectype (slp_tree, tree);
-extern bool supportable_widening_operation (vec_info *,
- enum tree_code, stmt_vec_info,
- tree, tree, enum tree_code *,
- enum tree_code *, int *,
- vec<tree> *);
+extern bool supportable_widening_operation (vec_info*, code_helper,
+ stmt_vec_info, tree, tree,
+ code_helper*, code_helper*,
+ int*, vec<tree> *);
extern bool supportable_narrowing_operation (enum tree_code, tree, tree,
- enum tree_code *, int *,
+ tree_code *, int *,
vec<tree> *);
extern unsigned record_stmt_cost (stmt_vector_for_cost *, int,
diff --git a/gcc/tree.h b/gcc/tree.h
index f84958933d51144bb6ce7cc41eca5f7f06814550..e51e34c051d9b91d1c02a4b2fefdb2b15606a36f 100644
--- a/gcc/tree.h
+++ b/gcc/tree.h
@@ -92,6 +92,10 @@ public:
bool is_fn_code () const { return rep < 0; }
bool is_internal_fn () const;
bool is_builtin_fn () const;
+ enum tree_code as_tree_code () const { return is_tree_code () ?
+ (tree_code)* this : MAX_TREE_CODES; }
+ combined_fn as_fn_code () const { return is_fn_code () ? (combined_fn) *this
+ : CFN_LAST;}
int get_rep () const { return rep; }
bool operator== (const code_helper &other) { return rep == other.rep; }
bool operator!= (const code_helper &other) { return rep != other.rep; }
@@ -6657,6 +6661,54 @@ extern unsigned fndecl_dealloc_argno (tree);
if nonnull, set the second argument to the referenced enclosing
object or pointer. Otherwise return null. */
extern tree get_attr_nonstring_decl (tree, tree * = NULL);
+/* Helper to transparently allow tree codes and builtin function codes
+ exist in one storage entity. */
+class code_helper
+{
+public:
+ code_helper () {}
+ code_helper (tree_code code) : rep ((int) code) {}
+ code_helper (combined_fn fn) : rep (-(int) fn) {}
+ code_helper (internal_fn fn) : rep (-(int) as_combined_fn (fn)) {}
+ explicit operator tree_code () const { return (tree_code) rep; }
+ explicit operator combined_fn () const { return (combined_fn) -rep; }
+ explicit operator internal_fn () const;
+ explicit operator built_in_function () const;
+ bool is_tree_code () const { return rep > 0; }
+ bool is_fn_code () const { return rep < 0; }
+ bool is_internal_fn () const;
+ bool is_builtin_fn () const;
+ int get_rep () const { return rep; }
+ bool operator== (const code_helper &other) { return rep == other.rep; }
+ bool operator!= (const code_helper &other) { return rep != other.rep; }
+ bool operator== (tree_code c) { return rep == code_helper (c).rep; }
+ bool operator!= (tree_code c) { return rep != code_helper (c).rep; }
+
+private:
+ int rep;
+};
+
+inline code_helper::operator internal_fn () const
+{
+ return as_internal_fn (combined_fn (*this));
+}
+
+inline code_helper::operator built_in_function () const
+{
+ return as_builtin_fn (combined_fn (*this));
+}
+
+inline bool
+code_helper::is_internal_fn () const
+{
+ return is_fn_code () && internal_fn_p (combined_fn (*this));
+}
+
+inline bool
+code_helper::is_builtin_fn () const
+{
+ return is_fn_code () && builtin_fn_p (combined_fn (*this));
+}
extern int get_target_clone_attr_len (tree);
--
2.17.1
[-- Attachment #3: 0002-Refactor-widen_plus-as-internal_fn.patch --]
[-- Type: application/octet-stream, Size: 22720 bytes --]
From 70fa5fc3e8282a73b973bdd79fccd3450d5b312c Mon Sep 17 00:00:00 2001
From: Joel Hutton <joel.hutton@arm.com>
Date: Wed, 26 Jan 2022 14:00:17 +0000
Subject: [PATCH 2/3] Refactor widen_plus as internal_fn
This patch replaces the existing tree_code widen_plus and widen_minus
patterns with internal_fn versions.
DEF_INTERNAL_OPTAB_MULTI_FN is like DEF_INTERNAL_OPTAB_FN except it provides convenience wrappers for defining conversions that require a hi/lo split, like widening and narrowing operations. Each definition for <NAME> will require an optab named <OPTAB> and two other optabs that you specify for signed and unsigned. The hi/lo pair is necessary because the widening operations take n narrow elements as inputs and return n/2 wide elements as outputs. The 'lo' operation operates on the first n/2 elements of input. The 'hi' operation operates on the second n/2 elements of input. Defining an internal_fn along with hi/lo variations allows a single internal function to be returned from a vect_recog function that will later be expanded to hi/lo.
DEF_INTERNAL_OPTAB_MULTI_FN is used in internal-fn.def to register a widening internal_fn. It is defined differently in different places and internal-fn.def is sourced from those places so the parameters given can be reused.
internal-fn.c: defined to expand to hi/lo signed/unsigned optabs, later defined to generate the 'expand_' functions for the hi/lo versions of the fn.
internal-fn.def: defined to invoke DEF_INTERNAL_OPTAB_FN for the original and hi/lo variants of the internal_fn
For example:
IFN_VEC_WIDEN_PLUS -> IFN_VEC_WIDEN_PLUS_HI, IFN_VEC_WIDEN_PLUS_LO
for aarch64: IFN_VEC_WIDEN_PLUS_HI -> vec_widen_<su>addl_hi_<mode> -> (u/s)addl2
IFN_VEC_WIDEN_PLUS_LO -> vec_widen_<su>addl_lo_<mode> -> (u/s)addl
This gives the same functionality as the previous WIDEN_PLUS/WIDEN_MINUS tree codes which are expanded into VEC_WIDEN_PLUS_LO, VEC_WIDEN_PLUS_HI.
gcc/ChangeLog:
2022-04-13 Joel Hutton <joel.hutton@arm.com>
2022-04-13 Tamar Christina <tamar.christina@arm.com>
* internal-fn.cc (INCLUDE_MAP): Include maps for use in optab
lookup.
(DEF_INTERNAL_OPTAB_MULTI_FN): Macro to define an internal_fn that
expands into multiple internal_fns (for widening).
(ifn_cmp): Function to compare ifn's for sorting/searching.
(lookup_multi_ifn_optab): Add lookup function.
(lookup_multi_internal_fn): Add lookup function.
(commutative_binary_fn_p): Add widen_plus fn's.
* internal-fn.def (DEF_INTERNAL_OPTAB_MULTI_FN): Define widening
plus,minus functions.
(VEC_WIDEN_PLUS): Replacement for VEC_WIDEN_PLUS tree code.
(VEC_WIDEN_MINUS): Replacement for VEC_WIDEN_MINUS tree code.
* internal-fn.h (GCC_INTERNAL_FN_H): Add headers.
(lookup_multi_ifn_optab): Add prototype.
(lookup_multi_internal_fn): Add prototype.
* optabs.cc (commutative_optab_p): Add widening plus, minus optabs.
* optabs.def (OPTAB_CD): widen add, sub optabs
* tree-core.h (ECF_MULTI): Flag to indicate if a function decays
into hi/lo parts.
* tree-vect-patterns.cc (vect_recog_widen_op_pattern): Support
patterns with a hi/lo split.
(vect_recog_widen_plus_pattern): Refactor to return
IFN_VECT_WIDEN_PLUS.
(vect_recog_widen_minus_pattern): Refactor to return new
IFN_VEC_WIDEN_MINUS.
* tree-vect-stmts.cc (vectorizable_conversion): Add widen plus/minus
ifn
support.
(supportable_widening_operation): Add widen plus/minus ifn support.
gcc/testsuite/ChangeLog:
* gcc.target/aarch64/vect-widen-add.c: Test that new
IFN_VEC_WIDEN_PLUS is being used.
* gcc.target/aarch64/vect-widen-sub.c: Test that new
IFN_VEC_WIDEN_MINUS is being used.
---
gcc/internal-fn.cc | 107 ++++++++++++++++++
gcc/internal-fn.def | 23 ++++
gcc/internal-fn.h | 7 ++
gcc/optabs.cc | 12 +-
gcc/optabs.def | 2 +
.../gcc.target/aarch64/vect-widen-add.c | 4 +-
.../gcc.target/aarch64/vect-widen-sub.c | 4 +-
gcc/tree-core.h | 4 +
gcc/tree-vect-patterns.cc | 37 ++++--
gcc/tree-vect-stmts.cc | 60 +++++++++-
10 files changed, 244 insertions(+), 16 deletions(-)
diff --git a/gcc/internal-fn.cc b/gcc/internal-fn.cc
index 8b1733e20c4455e4e8c383c92fe859f4256cae69..e95b13af884f67990ad43c286990a351e2bd641b 100644
--- a/gcc/internal-fn.cc
+++ b/gcc/internal-fn.cc
@@ -17,6 +17,7 @@ You should have received a copy of the GNU General Public License
along with GCC; see the file COPYING3. If not see
<http://www.gnu.org/licenses/>. */
+#define INCLUDE_MAP
#include "config.h"
#include "system.h"
#include "coretypes.h"
@@ -70,6 +71,26 @@ const int internal_fn_flags_array[] = {
0
};
+const enum internal_fn internal_fn_hilo_keys_array[] = {
+#undef DEF_INTERNAL_OPTAB_MULTI_FN
+#define DEF_INTERNAL_OPTAB_MULTI_FN(NAME, FLAGS, OPTAB, SOPTAB, UOPTAB, TYPE) \
+ IFN_##NAME##_LO, \
+ IFN_##NAME##_HI,
+#include "internal-fn.def"
+ IFN_LAST
+#undef DEF_INTERNAL_OPTAB_MULTI_FN
+};
+
+const optab internal_fn_hilo_values_array[] = {
+#undef DEF_INTERNAL_OPTAB_MULTI_FN
+#define DEF_INTERNAL_OPTAB_MULTI_FN(NAME, FLAGS, OPTAB, SOPTAB, UOPTAB, TYPE) \
+ SOPTAB##_lo_optab, UOPTAB##_lo_optab, \
+ SOPTAB##_hi_optab, UOPTAB##_hi_optab,
+#include "internal-fn.def"
+ unknown_optab, unknown_optab
+#undef DEF_INTERNAL_OPTAB_MULTI_FN
+};
+
/* Return the internal function called NAME, or IFN_LAST if there's
no such function. */
@@ -90,6 +111,62 @@ lookup_internal_fn (const char *name)
return entry ? *entry : IFN_LAST;
}
+static int
+ifn_cmp (const void *a_, const void *b_)
+{
+ typedef std::pair<enum internal_fn, unsigned> ifn_pair;
+ auto *a = (const std::pair<ifn_pair, optab> *)a_;
+ auto *b = (const std::pair<ifn_pair, optab> *)b_;
+ return (int) (a->first.first) - (b->first.first);
+}
+
+/* Return the optab belonging to the given internal function NAME for the given
+ SIGN or unknown_optab. */
+
+optab
+lookup_multi_ifn_optab (enum internal_fn fn, unsigned sign)
+{
+ typedef std::pair<enum internal_fn, unsigned> ifn_pair;
+ typedef auto_vec <std::pair<ifn_pair, optab>>fn_to_optab_map_type;
+ static fn_to_optab_map_type *fn_to_optab_map;
+
+ if (!fn_to_optab_map)
+ {
+ unsigned num
+ = sizeof (internal_fn_hilo_keys_array) / sizeof (enum internal_fn);
+ fn_to_optab_map = new fn_to_optab_map_type ();
+ for (unsigned int i = 0; i < num - 1; ++i)
+ {
+ enum internal_fn fn = internal_fn_hilo_keys_array[i];
+ optab v1 = internal_fn_hilo_values_array[2*i];
+ optab v2 = internal_fn_hilo_values_array[2*i + 1];
+ ifn_pair key1 (fn, 0);
+ fn_to_optab_map->safe_push ({key1, v1});
+ ifn_pair key2 (fn, 1);
+ fn_to_optab_map->safe_push ({key2, v2});
+ }
+ fn_to_optab_map->qsort(ifn_cmp);
+ }
+
+ ifn_pair new_pair (fn, sign ? 1 : 0);
+ optab tmp;
+ std::pair<ifn_pair,optab> pair_wrap (new_pair, tmp);
+ auto entry = fn_to_optab_map->bsearch (&pair_wrap, ifn_cmp);
+ return entry != fn_to_optab_map->end () ? entry->second : unknown_optab;
+}
+
+extern void
+lookup_multi_internal_fn (enum internal_fn ifn, enum internal_fn *lo,
+ enum internal_fn *hi)
+{
+ int ecf_flags = internal_fn_flags (ifn);
+ gcc_assert (ecf_flags & ECF_MULTI);
+
+ *lo = internal_fn (ifn + 1);
+ *hi = internal_fn (ifn + 2);
+}
+
+
/* Fnspec of each internal function, indexed by function number. */
const_tree internal_fn_fnspec_array[IFN_LAST + 1];
@@ -3906,6 +3983,9 @@ commutative_binary_fn_p (internal_fn fn)
case IFN_UBSAN_CHECK_MUL:
case IFN_ADD_OVERFLOW:
case IFN_MUL_OVERFLOW:
+ case IFN_VEC_WIDEN_PLUS:
+ case IFN_VEC_WIDEN_PLUS_LO:
+ case IFN_VEC_WIDEN_PLUS_HI:
return true;
default:
@@ -3991,6 +4071,32 @@ set_edom_supported_p (void)
#endif
}
+#undef DEF_INTERNAL_OPTAB_MULTI_FN
+#define DEF_INTERNAL_OPTAB_MULTI_FN(CODE, FLAGS, OPTAB, SOPTAB, UOPTAB, TYPE) \
+ static void \
+ expand_##CODE (internal_fn, gcall *) \
+ { \
+ gcc_unreachable (); \
+ } \
+ static void \
+ expand_##CODE##_LO (internal_fn fn, gcall *stmt) \
+ { \
+ tree ty = TREE_TYPE (gimple_get_lhs (stmt)); \
+ if (!TYPE_UNSIGNED (ty)) \
+ expand_##TYPE##_optab_fn (fn, stmt, SOPTAB##_lo##_optab); \
+ else \
+ expand_##TYPE##_optab_fn (fn, stmt, UOPTAB##_lo##_optab); \
+ } \
+ static void \
+ expand_##CODE##_HI (internal_fn fn, gcall *stmt) \
+ { \
+ tree ty = TREE_TYPE (gimple_get_lhs (stmt)); \
+ if (!TYPE_UNSIGNED (ty)) \
+ expand_##TYPE##_optab_fn (fn, stmt, SOPTAB##_hi##_optab); \
+ else \
+ expand_##TYPE##_optab_fn (fn, stmt, UOPTAB##_hi##_optab); \
+ }
+
#define DEF_INTERNAL_OPTAB_FN(CODE, FLAGS, OPTAB, TYPE) \
static void \
expand_##CODE (internal_fn fn, gcall *stmt) \
@@ -4007,6 +4113,7 @@ set_edom_supported_p (void)
expand_##TYPE##_optab_fn (fn, stmt, which_optab); \
}
#include "internal-fn.def"
+#undef DEF_INTERNAL_OPTAB_MULTI_FN
/* Routines to expand each internal function, indexed by function number.
Each routine has the prototype:
diff --git a/gcc/internal-fn.def b/gcc/internal-fn.def
index d2d550d358606022b1cb44fa842f06e0be507bc3..4635a9c8af9ad27bb05d7510388d0fe2270428e5 100644
--- a/gcc/internal-fn.def
+++ b/gcc/internal-fn.def
@@ -82,6 +82,13 @@ along with GCC; see the file COPYING3. If not see
says that the function extends the C-level BUILT_IN_<NAME>{,L,LL,IMAX}
group of functions to any integral mode (including vector modes).
+ DEF_INTERNAL_OPTAB_MULTI_FN is like DEF_INTERNAL_OPTAB_FN except it
+ provides convenience wrappers for defining conversions that require a
+ hi/lo split, like widening and narrowing operations. Each definition
+ for <NAME> will require an optab named <OPTAB> and two other optabs that
+ you specify for signed and unsigned.
+
+
Each entry must have a corresponding expander of the form:
void expand_NAME (gimple_call stmt)
@@ -120,6 +127,14 @@ along with GCC; see the file COPYING3. If not see
DEF_INTERNAL_OPTAB_FN (NAME, FLAGS, OPTAB, TYPE)
#endif
+#ifndef DEF_INTERNAL_OPTAB_MULTI_FN
+#define DEF_INTERNAL_OPTAB_MULTI_FN(NAME, FLAGS, OPTAB, SOPTAB, UOPTAB, TYPE) \
+ DEF_INTERNAL_OPTAB_FN (NAME, FLAGS | ECF_MULTI, OPTAB, TYPE) \
+ DEF_INTERNAL_OPTAB_FN (NAME ## _LO, FLAGS, unknown, TYPE) \
+ DEF_INTERNAL_OPTAB_FN (NAME ## _HI, FLAGS, unknown, TYPE)
+#endif
+
+
DEF_INTERNAL_OPTAB_FN (MASK_LOAD, ECF_PURE, maskload, mask_load)
DEF_INTERNAL_OPTAB_FN (LOAD_LANES, ECF_CONST, vec_load_lanes, load_lanes)
DEF_INTERNAL_OPTAB_FN (MASK_LOAD_LANES, ECF_PURE,
@@ -292,6 +307,14 @@ DEF_INTERNAL_OPTAB_FN (COMPLEX_ADD_ROT270, ECF_CONST, cadd270, binary)
DEF_INTERNAL_OPTAB_FN (COMPLEX_MUL, ECF_CONST, cmul, binary)
DEF_INTERNAL_OPTAB_FN (COMPLEX_MUL_CONJ, ECF_CONST, cmul_conj, binary)
DEF_INTERNAL_OPTAB_FN (VEC_ADDSUB, ECF_CONST, vec_addsub, binary)
+DEF_INTERNAL_OPTAB_MULTI_FN (VEC_WIDEN_PLUS,
+ ECF_CONST | ECF_WIDEN | ECF_NOTHROW,
+ vec_widen_add, vec_widen_saddl, vec_widen_uaddl,
+ binary)
+DEF_INTERNAL_OPTAB_MULTI_FN (VEC_WIDEN_MINUS,
+ ECF_CONST | ECF_WIDEN | ECF_NOTHROW,
+ vec_widen_sub, vec_widen_ssubl, vec_widen_usubl,
+ binary)
DEF_INTERNAL_OPTAB_FN (VEC_FMADDSUB, ECF_CONST, vec_fmaddsub, ternary)
DEF_INTERNAL_OPTAB_FN (VEC_FMSUBADD, ECF_CONST, vec_fmsubadd, ternary)
diff --git a/gcc/internal-fn.h b/gcc/internal-fn.h
index 23c014a963c4d72da92c763db87ee486a2adb485..b35de19747d251d19dc13de1e0323368bd2ebdf2 100644
--- a/gcc/internal-fn.h
+++ b/gcc/internal-fn.h
@@ -20,6 +20,10 @@ along with GCC; see the file COPYING3. If not see
#ifndef GCC_INTERNAL_FN_H
#define GCC_INTERNAL_FN_H
+#include "insn-codes.h"
+#include "insn-opinit.h"
+
+
/* INTEGER_CST values for IFN_UNIQUE function arg-0.
UNSPEC: Undifferentiated UNIQUE.
@@ -112,6 +116,9 @@ internal_fn_name (enum internal_fn fn)
}
extern internal_fn lookup_internal_fn (const char *);
+extern optab lookup_multi_ifn_optab (enum internal_fn, unsigned);
+extern void lookup_multi_internal_fn (enum internal_fn, enum internal_fn *,
+ enum internal_fn *);
/* Return the ECF_* flags for function FN. */
diff --git a/gcc/optabs.cc b/gcc/optabs.cc
index c0a68471d2ddf08bc0e6a3fd592ebb9f05e516c1..7e904b3e154d018779bb1a36de74e6997f70e193 100644
--- a/gcc/optabs.cc
+++ b/gcc/optabs.cc
@@ -1314,7 +1314,17 @@ commutative_optab_p (optab binoptab)
|| binoptab == smul_widen_optab
|| binoptab == umul_widen_optab
|| binoptab == smul_highpart_optab
- || binoptab == umul_highpart_optab);
+ || binoptab == umul_highpart_optab
+ || binoptab == vec_widen_add_optab
+ || binoptab == vec_widen_sub_optab
+ || binoptab == vec_widen_saddl_hi_optab
+ || binoptab == vec_widen_saddl_lo_optab
+ || binoptab == vec_widen_ssubl_hi_optab
+ || binoptab == vec_widen_ssubl_lo_optab
+ || binoptab == vec_widen_uaddl_hi_optab
+ || binoptab == vec_widen_uaddl_lo_optab
+ || binoptab == vec_widen_usubl_hi_optab
+ || binoptab == vec_widen_usubl_lo_optab);
}
/* X is to be used in mode MODE as operand OPN to BINOPTAB. If we're
diff --git a/gcc/optabs.def b/gcc/optabs.def
index 801310ebaa7d469520809bb7efed6820f8eb866b..a7881dcb49e4ef07d8f07aa31214eb3a7a944e2e 100644
--- a/gcc/optabs.def
+++ b/gcc/optabs.def
@@ -78,6 +78,8 @@ OPTAB_CD(smsub_widen_optab, "msub$b$a4")
OPTAB_CD(umsub_widen_optab, "umsub$b$a4")
OPTAB_CD(ssmsub_widen_optab, "ssmsub$b$a4")
OPTAB_CD(usmsub_widen_optab, "usmsub$a$b4")
+OPTAB_CD(vec_widen_add_optab, "add$a$b3")
+OPTAB_CD(vec_widen_sub_optab, "sub$a$b3")
OPTAB_CD(vec_load_lanes_optab, "vec_load_lanes$a$b")
OPTAB_CD(vec_store_lanes_optab, "vec_store_lanes$a$b")
OPTAB_CD(vec_mask_load_lanes_optab, "vec_mask_load_lanes$a$b")
diff --git a/gcc/testsuite/gcc.target/aarch64/vect-widen-add.c b/gcc/testsuite/gcc.target/aarch64/vect-widen-add.c
index 220bd9352a4c7acd2e3713e441d74898d3e92b30..7037673d32bd780e1c9b58a51e58e2bac3b30b7e 100644
--- a/gcc/testsuite/gcc.target/aarch64/vect-widen-add.c
+++ b/gcc/testsuite/gcc.target/aarch64/vect-widen-add.c
@@ -1,5 +1,5 @@
/* { dg-do run } */
-/* { dg-options "-O3 -save-temps" } */
+/* { dg-options "-O3 -save-temps -fdump-tree-vect-all" } */
#include <stdint.h>
#include <string.h>
@@ -86,6 +86,8 @@ main()
return 0;
}
+/* { dg-final { scan-tree-dump "add new stmt.*VEC_WIDEN_PLUS_LO" "vect" } } */
+/* { dg-final { scan-tree-dump "add new stmt.*VEC_WIDEN_PLUS_HI" "vect" } } */
/* { dg-final { scan-assembler-times {\tuaddl\t} 1} } */
/* { dg-final { scan-assembler-times {\tuaddl2\t} 1} } */
/* { dg-final { scan-assembler-times {\tsaddl\t} 1} } */
diff --git a/gcc/testsuite/gcc.target/aarch64/vect-widen-sub.c b/gcc/testsuite/gcc.target/aarch64/vect-widen-sub.c
index a2bed63affbd091977df95a126da1f5b8c1d41d2..83bc1edb6105f47114b665e24a13e6194b2179a2 100644
--- a/gcc/testsuite/gcc.target/aarch64/vect-widen-sub.c
+++ b/gcc/testsuite/gcc.target/aarch64/vect-widen-sub.c
@@ -1,5 +1,5 @@
/* { dg-do run } */
-/* { dg-options "-O3 -save-temps" } */
+/* { dg-options "-O3 -save-temps -fdump-tree-vect-all" } */
#include <stdint.h>
#include <string.h>
@@ -86,6 +86,8 @@ main()
return 0;
}
+/* { dg-final { scan-tree-dump "add new stmt.*VEC_WIDEN_MINUS_LO" "vect" } } */
+/* { dg-final { scan-tree-dump "add new stmt.*VEC_WIDEN_MINUS_HI" "vect" } } */
/* { dg-final { scan-assembler-times {\tusubl\t} 1} } */
/* { dg-final { scan-assembler-times {\tusubl2\t} 1} } */
/* { dg-final { scan-assembler-times {\tssubl\t} 1} } */
diff --git a/gcc/tree-core.h b/gcc/tree-core.h
index cff6211080bced0bffb39e98039a6550897acf77..d0c8b812cfb9c3ac83bf25fff0431b08cb7d823d 100644
--- a/gcc/tree-core.h
+++ b/gcc/tree-core.h
@@ -99,6 +99,10 @@ struct die_struct;
/* Nonzero if this is a widening function. */
#define ECF_WIDEN (1 << 16)
+/* Nonzero if this is a function that decomposes into a lo/hi operation. */
+#define ECF_MULTI (1 << 17)
+
+
/* Call argument flags. */
/* Nonzero if the argument is not used by the function. */
diff --git a/gcc/tree-vect-patterns.cc b/gcc/tree-vect-patterns.cc
index 36e362e1daf3f946c6074600a6a322b3bda67755..0fd587da12b4a17b238327ae60f5a2a7a0efc514 100644
--- a/gcc/tree-vect-patterns.cc
+++ b/gcc/tree-vect-patterns.cc
@@ -1349,14 +1349,16 @@ static gimple *
vect_recog_widen_op_pattern (vec_info *vinfo,
stmt_vec_info last_stmt_info, tree *type_out,
tree_code orig_code, code_helper wide_code,
- bool shift_p, const char *name)
+ bool shift_p, const char *name,
+ enum optab_subtype *subtype = NULL)
{
gimple *last_stmt = last_stmt_info->stmt;
vect_unpromoted_value unprom[2];
tree half_type;
if (!vect_widened_op_tree (vinfo, last_stmt_info, orig_code, orig_code,
- shift_p, 2, unprom, &half_type))
+ shift_p, 2, unprom, &half_type, subtype))
+
return NULL;
/* Pattern detected. */
@@ -1422,6 +1424,20 @@ vect_recog_widen_op_pattern (vec_info *vinfo,
type, pattern_stmt, vecctype);
}
+static gimple *
+vect_recog_widen_op_pattern (vec_info *vinfo,
+ stmt_vec_info last_stmt_info, tree *type_out,
+ tree_code orig_code, internal_fn wide_ifn,
+ bool shift_p, const char *name,
+ enum optab_subtype *subtype = NULL)
+{
+ combined_fn ifn = as_combined_fn (wide_ifn);
+ return vect_recog_widen_op_pattern (vinfo, last_stmt_info, type_out,
+ orig_code, ifn, shift_p, name,
+ subtype);
+}
+
+
/* Try to detect multiplication on widened inputs, converting MULT_EXPR
to WIDEN_MULT_EXPR. See vect_recog_widen_op_pattern for details. */
@@ -1435,26 +1451,30 @@ vect_recog_widen_mult_pattern (vec_info *vinfo, stmt_vec_info last_stmt_info,
}
/* Try to detect addition on widened inputs, converting PLUS_EXPR
- to WIDEN_PLUS_EXPR. See vect_recog_widen_op_pattern for details. */
+ to IFN_VEC_WIDEN_PLUS. See vect_recog_widen_op_pattern for details. */
static gimple *
vect_recog_widen_plus_pattern (vec_info *vinfo, stmt_vec_info last_stmt_info,
tree *type_out)
{
+ enum optab_subtype subtype;
return vect_recog_widen_op_pattern (vinfo, last_stmt_info, type_out,
- PLUS_EXPR, WIDEN_PLUS_EXPR, false,
- "vect_recog_widen_plus_pattern");
+ PLUS_EXPR, IFN_VEC_WIDEN_PLUS,
+ false, "vect_recog_widen_plus_pattern",
+ &subtype);
}
/* Try to detect subtraction on widened inputs, converting MINUS_EXPR
- to WIDEN_MINUS_EXPR. See vect_recog_widen_op_pattern for details. */
+ to IFN_VEC_WIDEN_MINUS. See vect_recog_widen_op_pattern for details. */
static gimple *
vect_recog_widen_minus_pattern (vec_info *vinfo, stmt_vec_info last_stmt_info,
tree *type_out)
{
+ enum optab_subtype subtype;
return vect_recog_widen_op_pattern (vinfo, last_stmt_info, type_out,
- MINUS_EXPR, WIDEN_MINUS_EXPR, false,
- "vect_recog_widen_minus_pattern");
+ MINUS_EXPR, IFN_VEC_WIDEN_MINUS,
+ false, "vect_recog_widen_minus_pattern",
+ &subtype);
}
/* Function vect_recog_popcount_pattern
@@ -5618,6 +5638,7 @@ static vect_recog_func vect_vect_recog_func_ptrs[] = {
{ vect_recog_mask_conversion_pattern, "mask_conversion" },
{ vect_recog_widen_plus_pattern, "widen_plus" },
{ vect_recog_widen_minus_pattern, "widen_minus" },
+ /* These must come after the double widening ones. */
};
const unsigned int NUM_PATTERNS = ARRAY_SIZE (vect_vect_recog_func_ptrs);
diff --git a/gcc/tree-vect-stmts.cc b/gcc/tree-vect-stmts.cc
index 61b51a29f99bcdf0ff6b4ead4a69163ebf8ed383..c31831df723eeae8ea4fca2790a18b562106c889 100644
--- a/gcc/tree-vect-stmts.cc
+++ b/gcc/tree-vect-stmts.cc
@@ -4880,7 +4880,7 @@ vectorizable_conversion (vec_info *vinfo,
return false;
if (gimple_get_lhs (stmt) == NULL_TREE ||
- TREE_CODE(gimple_get_lhs (stmt)) != SSA_NAME)
+ TREE_CODE (gimple_get_lhs (stmt)) != SSA_NAME)
return false;
if (TREE_CODE (gimple_get_lhs (stmt)) != SSA_NAME)
@@ -12125,12 +12125,62 @@ supportable_widening_operation (vec_info *vinfo,
optab1 = vec_unpacks_sbool_lo_optab;
optab2 = vec_unpacks_sbool_hi_optab;
}
- else
- {
- optab1 = optab_for_tree_code (c1.as_tree_code (), vectype, optab_default);
- optab2 = optab_for_tree_code (c2.as_tree_code (), vectype, optab_default);
+
+ if (code.is_fn_code ())
+ {
+ internal_fn ifn = as_internal_fn (code.as_fn_code ());
+ int ecf_flags = internal_fn_flags (ifn);
+ gcc_assert (ecf_flags & ECF_MULTI);
+
+ switch (code.as_fn_code ())
+ {
+ case CFN_VEC_WIDEN_PLUS:
+ break;
+ case CFN_VEC_WIDEN_MINUS:
+ break;
+ case CFN_LAST:
+ default:
+ return false;
+ }
+
+ internal_fn lo, hi;
+ lookup_multi_internal_fn (ifn, &lo, &hi);
+ *code1 = as_combined_fn (lo);
+ *code2 = as_combined_fn (hi);
+ optab1 = lookup_multi_ifn_optab (lo, !TYPE_UNSIGNED (vectype));
+ optab2 = lookup_multi_ifn_optab (hi, !TYPE_UNSIGNED (vectype));
}
+ if (code.is_tree_code ())
+ {
+ if (code == FIX_TRUNC_EXPR)
+ {
+ /* The signedness is determined from output operand. */
+ optab1 = optab_for_tree_code (c1.as_tree_code (), vectype_out,
+ optab_default);
+ optab2 = optab_for_tree_code (c2.as_tree_code (), vectype_out,
+ optab_default);
+ }
+ else if (CONVERT_EXPR_CODE_P (code.as_tree_code ())
+ && VECTOR_BOOLEAN_TYPE_P (wide_vectype)
+ && VECTOR_BOOLEAN_TYPE_P (vectype)
+ && TYPE_MODE (wide_vectype) == TYPE_MODE (vectype)
+ && SCALAR_INT_MODE_P (TYPE_MODE (vectype)))
+ {
+ /* If the input and result modes are the same, a different optab
+ is needed where we pass in the number of units in vectype. */
+ optab1 = vec_unpacks_sbool_lo_optab;
+ optab2 = vec_unpacks_sbool_hi_optab;
+ }
+ else
+ {
+ optab1 = optab_for_tree_code (c1.as_tree_code (), vectype,
+ optab_default);
+ optab2 = optab_for_tree_code (c2.as_tree_code (), vectype,
+ optab_default);
+ }
+ }
+
if (!optab1 || !optab2)
return false;
--
2.17.1
[-- Attachment #4: 0003-Remove-widen_plus-minus_expr-tree-codes.patch --]
[-- Type: application/octet-stream, Size: 19457 bytes --]
From 40224aa09ccd4f44aa6bd843f6c7ce0dbb3b6970 Mon Sep 17 00:00:00 2001
From: Joel Hutton <joel.hutton@arm.com>
Date: Fri, 28 Jan 2022 12:04:44 +0000
Subject: [PATCH 3/3] Remove widen_plus/minus_expr tree codes
This patch removes the old widen plus/minus tree codes which have been
replaced by internal functions.
gcc/ChangeLog:
* doc/generic.texi: Remove old tree codes.
* expr.cc (expand_expr_real_2): Remove old tree code cases.
* gimple-pretty-print.cc (dump_binary_rhs): Remove old tree code
cases.
* optabs-tree.cc (optab_for_tree_code): Remove old tree code cases.
(supportable_half_widening_operation): Remove old tree code cases.
* tree-cfg.cc (verify_gimple_assign_binary): Remove old tree code
cases.
* tree-inline.cc (estimate_operator_cost): Remove old tree code
cases.
* tree-pretty-print.cc (dump_generic_node): Remove tree code definition.
(op_symbol_code): Remove old tree code
cases.
* tree-vect-data-refs.cc (vect_get_smallest_scalar_type): Remove old tree code
cases.
(vect_analyze_data_ref_accesses): Remove old tree code
cases.
* tree-vect-generic.cc (expand_vector_operations_1): Remove old tree code
cases.
* tree-vect-patterns.cc (vect_widened_op_tree): Refactor ot replace
usage in vect_recog_sad_pattern.
(vect_recog_sad_pattern): Replace tree code widening pattern with
internal function.
(vect_recog_average_pattern): Replace tree code widening pattern
with internal function.
* tree-vect-stmts.cc (vectorizable_conversion): Remove old tree code
cases.
(supportable_widening_operation): Remove old tree code
cases.
* tree.def (WIDEN_PLUS_EXPR): Remove tree code definition.
(WIDEN_MINUS_EXPR): Remove tree code definition.
(VEC_WIDEN_PLUS_HI_EXPR): Remove tree code definition.
(VEC_WIDEN_PLUS_LO_EXPR): Remove tree code definition.
(VEC_WIDEN_MINUS_HI_EXPR): Remove tree code definition.
(VEC_WIDEN_MINUS_LO_EXPR): Remove tree code definition.
---
gcc/doc/generic.texi | 31 -------------------------------
gcc/expr.cc | 6 ------
gcc/gimple-pretty-print.cc | 4 ----
gcc/optabs-tree.cc | 24 ------------------------
gcc/tree-cfg.cc | 6 ------
gcc/tree-inline.cc | 6 ------
gcc/tree-pretty-print.cc | 12 ------------
gcc/tree-vect-data-refs.cc | 8 +++-----
gcc/tree-vect-generic.cc | 4 ----
gcc/tree-vect-patterns.cc | 36 +++++++++++++++++++++++++-----------
gcc/tree-vect-stmts.cc | 18 ++----------------
gcc/tree.def | 6 ------
12 files changed, 30 insertions(+), 131 deletions(-)
diff --git a/gcc/doc/generic.texi b/gcc/doc/generic.texi
index e5f9d1be8ea81f3da002ec3bb925590d331a2551..344045efd419b0cc3a11771acf70d2fd279c48ac 100644
--- a/gcc/doc/generic.texi
+++ b/gcc/doc/generic.texi
@@ -1811,10 +1811,6 @@ a value from @code{enum annot_expr_kind}, the third is an @code{INTEGER_CST}.
@tindex VEC_RSHIFT_EXPR
@tindex VEC_WIDEN_MULT_HI_EXPR
@tindex VEC_WIDEN_MULT_LO_EXPR
-@tindex VEC_WIDEN_PLUS_HI_EXPR
-@tindex VEC_WIDEN_PLUS_LO_EXPR
-@tindex VEC_WIDEN_MINUS_HI_EXPR
-@tindex VEC_WIDEN_MINUS_LO_EXPR
@tindex VEC_UNPACK_HI_EXPR
@tindex VEC_UNPACK_LO_EXPR
@tindex VEC_UNPACK_FLOAT_HI_EXPR
@@ -1861,33 +1857,6 @@ vector of @code{N/2} products. In the case of @code{VEC_WIDEN_MULT_LO_EXPR} the
low @code{N/2} elements of the two vector are multiplied to produce the
vector of @code{N/2} products.
-@item VEC_WIDEN_PLUS_HI_EXPR
-@itemx VEC_WIDEN_PLUS_LO_EXPR
-These nodes represent widening vector addition of the high and low parts of
-the two input vectors, respectively. Their operands are vectors that contain
-the same number of elements (@code{N}) of the same integral type. The result
-is a vector that contains half as many elements, of an integral type whose size
-is twice as wide. In the case of @code{VEC_WIDEN_PLUS_HI_EXPR} the high
-@code{N/2} elements of the two vectors are added to produce the vector of
-@code{N/2} products. In the case of @code{VEC_WIDEN_PLUS_LO_EXPR} the low
-@code{N/2} elements of the two vectors are added to produce the vector of
-@code{N/2} products.
-
-@item VEC_WIDEN_MINUS_HI_EXPR
-@itemx VEC_WIDEN_MINUS_LO_EXPR
-These nodes represent widening vector subtraction of the high and low parts of
-the two input vectors, respectively. Their operands are vectors that contain
-the same number of elements (@code{N}) of the same integral type. The high/low
-elements of the second vector are subtracted from the high/low elements of the
-first. The result is a vector that contains half as many elements, of an
-integral type whose size is twice as wide. In the case of
-@code{VEC_WIDEN_MINUS_HI_EXPR} the high @code{N/2} elements of the second
-vector are subtracted from the high @code{N/2} of the first to produce the
-vector of @code{N/2} products. In the case of
-@code{VEC_WIDEN_MINUS_LO_EXPR} the low @code{N/2} elements of the second
-vector are subtracted from the low @code{N/2} of the first to produce the
-vector of @code{N/2} products.
-
@item VEC_UNPACK_HI_EXPR
@itemx VEC_UNPACK_LO_EXPR
These nodes represent unpacking of the high and low parts of the input vector,
diff --git a/gcc/expr.cc b/gcc/expr.cc
index 7197996cec7d24dd43d60928d5618b32b77677a1..1f941efc9e995c7f6a35ff93aaa6bd3c35faaa1f 100644
--- a/gcc/expr.cc
+++ b/gcc/expr.cc
@@ -9337,8 +9337,6 @@ expand_expr_real_2 (sepops ops, rtx target, machine_mode tmode,
target, unsignedp);
return target;
- case WIDEN_PLUS_EXPR:
- case WIDEN_MINUS_EXPR:
case WIDEN_MULT_EXPR:
/* If first operand is constant, swap them.
Thus the following special case checks need only
@@ -10116,10 +10114,6 @@ expand_expr_real_2 (sepops ops, rtx target, machine_mode tmode,
return temp;
}
- case VEC_WIDEN_PLUS_HI_EXPR:
- case VEC_WIDEN_PLUS_LO_EXPR:
- case VEC_WIDEN_MINUS_HI_EXPR:
- case VEC_WIDEN_MINUS_LO_EXPR:
case VEC_WIDEN_MULT_HI_EXPR:
case VEC_WIDEN_MULT_LO_EXPR:
case VEC_WIDEN_MULT_EVEN_EXPR:
diff --git a/gcc/gimple-pretty-print.cc b/gcc/gimple-pretty-print.cc
index ebd87b20a0adc080c4a8f9429e75f49b96e72f9a..2a1a5b7f811ca341e8ee7e85a9701d3a37ff80bf 100644
--- a/gcc/gimple-pretty-print.cc
+++ b/gcc/gimple-pretty-print.cc
@@ -459,10 +459,6 @@ dump_binary_rhs (pretty_printer *buffer, const gassign *gs, int spc,
case VEC_PACK_FLOAT_EXPR:
case VEC_WIDEN_LSHIFT_HI_EXPR:
case VEC_WIDEN_LSHIFT_LO_EXPR:
- case VEC_WIDEN_PLUS_HI_EXPR:
- case VEC_WIDEN_PLUS_LO_EXPR:
- case VEC_WIDEN_MINUS_HI_EXPR:
- case VEC_WIDEN_MINUS_LO_EXPR:
case VEC_SERIES_EXPR:
for (p = get_tree_code_name (code); *p; p++)
pp_character (buffer, TOUPPER (*p));
diff --git a/gcc/optabs-tree.cc b/gcc/optabs-tree.cc
index 8383fe820b080f6d66f83dcf3b77d3c9f869f4bc..2f5f93dc6624f86f6b5618cf6e7aa2b508053a64 100644
--- a/gcc/optabs-tree.cc
+++ b/gcc/optabs-tree.cc
@@ -190,22 +190,6 @@ optab_for_tree_code (enum tree_code code, const_tree type,
return (TYPE_UNSIGNED (type)
? vec_widen_ushiftl_lo_optab : vec_widen_sshiftl_lo_optab);
- case VEC_WIDEN_PLUS_LO_EXPR:
- return (TYPE_UNSIGNED (type)
- ? vec_widen_uaddl_lo_optab : vec_widen_saddl_lo_optab);
-
- case VEC_WIDEN_PLUS_HI_EXPR:
- return (TYPE_UNSIGNED (type)
- ? vec_widen_uaddl_hi_optab : vec_widen_saddl_hi_optab);
-
- case VEC_WIDEN_MINUS_LO_EXPR:
- return (TYPE_UNSIGNED (type)
- ? vec_widen_usubl_lo_optab : vec_widen_ssubl_lo_optab);
-
- case VEC_WIDEN_MINUS_HI_EXPR:
- return (TYPE_UNSIGNED (type)
- ? vec_widen_usubl_hi_optab : vec_widen_ssubl_hi_optab);
-
case VEC_UNPACK_HI_EXPR:
return (TYPE_UNSIGNED (type)
? vec_unpacku_hi_optab : vec_unpacks_hi_optab);
@@ -312,8 +296,6 @@ optab_for_tree_code (enum tree_code code, const_tree type,
'hi'/'lo' pair using codes such as VEC_WIDEN_MINUS_HI/LO.
Supported widening operations:
- WIDEN_MINUS_EXPR
- WIDEN_PLUS_EXPR
WIDEN_MULT_EXPR
WIDEN_LSHIFT_EXPR
@@ -345,12 +327,6 @@ supportable_half_widening_operation (enum tree_code code, tree vectype_out,
case WIDEN_LSHIFT_EXPR:
*code1 = LSHIFT_EXPR;
break;
- case WIDEN_MINUS_EXPR:
- *code1 = MINUS_EXPR;
- break;
- case WIDEN_PLUS_EXPR:
- *code1 = PLUS_EXPR;
- break;
case WIDEN_MULT_EXPR:
*code1 = MULT_EXPR;
break;
diff --git a/gcc/tree-cfg.cc b/gcc/tree-cfg.cc
index 8de1b144a426776bf464765477c71ee8f2e52b81..46eed1e1f22052fc077f2fc25e5be627bce541b6 100644
--- a/gcc/tree-cfg.cc
+++ b/gcc/tree-cfg.cc
@@ -3948,8 +3948,6 @@ verify_gimple_assign_binary (gassign *stmt)
return false;
}
- case WIDEN_PLUS_EXPR:
- case WIDEN_MINUS_EXPR:
case PLUS_EXPR:
case MINUS_EXPR:
{
@@ -4070,10 +4068,6 @@ verify_gimple_assign_binary (gassign *stmt)
return false;
}
- case VEC_WIDEN_MINUS_HI_EXPR:
- case VEC_WIDEN_MINUS_LO_EXPR:
- case VEC_WIDEN_PLUS_HI_EXPR:
- case VEC_WIDEN_PLUS_LO_EXPR:
case VEC_WIDEN_MULT_HI_EXPR:
case VEC_WIDEN_MULT_LO_EXPR:
case VEC_WIDEN_MULT_EVEN_EXPR:
diff --git a/gcc/tree-inline.cc b/gcc/tree-inline.cc
index 043e1d5987a4c4b0159109dafb85a805ca828c1e..c0bebb7f4de36838341ed62389ad0e2b79f03034 100644
--- a/gcc/tree-inline.cc
+++ b/gcc/tree-inline.cc
@@ -4288,8 +4288,6 @@ estimate_operator_cost (enum tree_code code, eni_weights *weights,
case REALIGN_LOAD_EXPR:
- case WIDEN_PLUS_EXPR:
- case WIDEN_MINUS_EXPR:
case WIDEN_SUM_EXPR:
case WIDEN_MULT_EXPR:
case DOT_PROD_EXPR:
@@ -4298,10 +4296,6 @@ estimate_operator_cost (enum tree_code code, eni_weights *weights,
case WIDEN_MULT_MINUS_EXPR:
case WIDEN_LSHIFT_EXPR:
- case VEC_WIDEN_PLUS_HI_EXPR:
- case VEC_WIDEN_PLUS_LO_EXPR:
- case VEC_WIDEN_MINUS_HI_EXPR:
- case VEC_WIDEN_MINUS_LO_EXPR:
case VEC_WIDEN_MULT_HI_EXPR:
case VEC_WIDEN_MULT_LO_EXPR:
case VEC_WIDEN_MULT_EVEN_EXPR:
diff --git a/gcc/tree-pretty-print.cc b/gcc/tree-pretty-print.cc
index 6acd394a0790ad2ad989f195a3288f0f0a8cc489..53ca62dc1a6873ae9365f199061bde9edd486196 100644
--- a/gcc/tree-pretty-print.cc
+++ b/gcc/tree-pretty-print.cc
@@ -2825,8 +2825,6 @@ dump_generic_node (pretty_printer *pp, tree node, int spc, dump_flags_t flags,
break;
/* Binary arithmetic and logic expressions. */
- case WIDEN_PLUS_EXPR:
- case WIDEN_MINUS_EXPR:
case WIDEN_SUM_EXPR:
case WIDEN_MULT_EXPR:
case MULT_EXPR:
@@ -3790,10 +3788,6 @@ dump_generic_node (pretty_printer *pp, tree node, int spc, dump_flags_t flags,
case VEC_SERIES_EXPR:
case VEC_WIDEN_MULT_HI_EXPR:
case VEC_WIDEN_MULT_LO_EXPR:
- case VEC_WIDEN_PLUS_HI_EXPR:
- case VEC_WIDEN_PLUS_LO_EXPR:
- case VEC_WIDEN_MINUS_HI_EXPR:
- case VEC_WIDEN_MINUS_LO_EXPR:
case VEC_WIDEN_MULT_EVEN_EXPR:
case VEC_WIDEN_MULT_ODD_EXPR:
case VEC_WIDEN_LSHIFT_HI_EXPR:
@@ -4311,12 +4305,6 @@ op_symbol_code (enum tree_code code)
case WIDEN_LSHIFT_EXPR:
return "w<<";
- case WIDEN_PLUS_EXPR:
- return "w+";
-
- case WIDEN_MINUS_EXPR:
- return "w-";
-
case POINTER_PLUS_EXPR:
return "+";
diff --git a/gcc/tree-vect-data-refs.cc b/gcc/tree-vect-data-refs.cc
index d20a10a1524164eef788ab4b88ba57c7a09c3387..98dd56ff022233ccead36a1f5a5e896e352f9f5b 100644
--- a/gcc/tree-vect-data-refs.cc
+++ b/gcc/tree-vect-data-refs.cc
@@ -136,8 +136,6 @@ vect_get_smallest_scalar_type (stmt_vec_info stmt_info, tree scalar_type)
|| gimple_assign_rhs_code (assign) == WIDEN_SUM_EXPR
|| gimple_assign_rhs_code (assign) == WIDEN_MULT_EXPR
|| gimple_assign_rhs_code (assign) == WIDEN_LSHIFT_EXPR
- || gimple_assign_rhs_code (assign) == WIDEN_PLUS_EXPR
- || gimple_assign_rhs_code (assign) == WIDEN_MINUS_EXPR
|| gimple_assign_rhs_code (assign) == FLOAT_EXPR)
{
tree rhs_type = TREE_TYPE (gimple_assign_rhs1 (assign));
@@ -3172,8 +3170,8 @@ vect_analyze_data_ref_accesses (vec_info *vinfo,
break;
/* Check that the DR_INITs are compile-time constants. */
- if (!tree_fits_shwi_p (DR_INIT (dra))
- || !tree_fits_shwi_p (DR_INIT (drb)))
+ if (TREE_CODE (DR_INIT (dra)) != INTEGER_CST
+ || TREE_CODE (DR_INIT (drb)) != INTEGER_CST)
break;
/* Different .GOMP_SIMD_LANE calls still give the same lane,
@@ -3225,7 +3223,7 @@ vect_analyze_data_ref_accesses (vec_info *vinfo,
unsigned HOST_WIDE_INT step
= absu_hwi (tree_to_shwi (DR_STEP (dra)));
if (step != 0
- && step <= ((unsigned HOST_WIDE_INT)init_b - init_a))
+ && step <= (unsigned HOST_WIDE_INT)(init_b - init_a))
break;
}
}
diff --git a/gcc/tree-vect-generic.cc b/gcc/tree-vect-generic.cc
index 92aba5d4af61dd478ec3f1b94854e4ad84166774..5823b08baf70b89b22ecc148b0702a84671ad084 100644
--- a/gcc/tree-vect-generic.cc
+++ b/gcc/tree-vect-generic.cc
@@ -2209,10 +2209,6 @@ expand_vector_operations_1 (gimple_stmt_iterator *gsi,
arguments, not the widened result. VEC_UNPACK_FLOAT_*_EXPR is
calculated in the same way above. */
if (code == WIDEN_SUM_EXPR
- || code == VEC_WIDEN_PLUS_HI_EXPR
- || code == VEC_WIDEN_PLUS_LO_EXPR
- || code == VEC_WIDEN_MINUS_HI_EXPR
- || code == VEC_WIDEN_MINUS_LO_EXPR
|| code == VEC_WIDEN_MULT_HI_EXPR
|| code == VEC_WIDEN_MULT_LO_EXPR
|| code == VEC_WIDEN_MULT_EVEN_EXPR
diff --git a/gcc/tree-vect-patterns.cc b/gcc/tree-vect-patterns.cc
index 0fd587da12b4a17b238327ae60f5a2a7a0efc514..0d821413b971d983c6562c3e8fbe60e2c3d0cb94 100644
--- a/gcc/tree-vect-patterns.cc
+++ b/gcc/tree-vect-patterns.cc
@@ -557,21 +557,29 @@ vect_joust_widened_type (tree type, tree new_type, tree *common_type)
static unsigned int
vect_widened_op_tree (vec_info *vinfo, stmt_vec_info stmt_info, tree_code code,
- tree_code widened_code, bool shift_p,
+ code_helper widened_code, bool shift_p,
unsigned int max_nops,
vect_unpromoted_value *unprom, tree *common_type,
enum optab_subtype *subtype = NULL)
{
/* Check for an integer operation with the right code. */
- gassign *assign = dyn_cast <gassign *> (stmt_info->stmt);
- if (!assign)
+ gimple* stmt = stmt_info->stmt;
+ if (!(is_gimple_assign (stmt) || is_gimple_call (stmt)))
return 0;
- tree_code rhs_code = gimple_assign_rhs_code (assign);
- if (rhs_code != code && rhs_code != widened_code)
+ code_helper rhs_code;
+ if (is_gimple_assign (stmt))
+ rhs_code = gimple_assign_rhs_code (stmt);
+ else
+ rhs_code = gimple_call_combined_fn (stmt);
+
+ if (rhs_code.as_tree_code () != code
+ && rhs_code.get_rep () != widened_code.get_rep ())
return 0;
- tree type = TREE_TYPE (gimple_assign_lhs (assign));
+ tree lhs = is_gimple_assign (stmt) ? gimple_assign_lhs (stmt):
+ gimple_call_lhs (stmt);
+ tree type = TREE_TYPE (lhs);
if (!INTEGRAL_TYPE_P (type))
return 0;
@@ -584,7 +592,11 @@ vect_widened_op_tree (vec_info *vinfo, stmt_vec_info stmt_info, tree_code code,
{
vect_unpromoted_value *this_unprom = &unprom[next_op];
unsigned int nops = 1;
- tree op = gimple_op (assign, i + 1);
+ tree op;
+ if (is_gimple_assign (stmt))
+ op = gimple_op (stmt, i + 1);
+ else
+ op = gimple_call_arg (stmt, i);
if (i == 1 && TREE_CODE (op) == INTEGER_CST)
{
/* We already have a common type from earlier operands.
@@ -1297,8 +1309,9 @@ vect_recog_sad_pattern (vec_info *vinfo,
/* FORNOW. Can continue analyzing the def-use chain when this stmt in a phi
inside the loop (in case we are analyzing an outer-loop). */
vect_unpromoted_value unprom[2];
- if (!vect_widened_op_tree (vinfo, diff_stmt_vinfo, MINUS_EXPR, WIDEN_MINUS_EXPR,
- false, 2, unprom, &half_type))
+ if (!vect_widened_op_tree (vinfo, diff_stmt_vinfo, MINUS_EXPR,
+ CFN_VEC_WIDEN_MINUS, false, 2, unprom,
+ &half_type))
return NULL;
vect_pattern_detected ("vect_recog_sad_pattern", last_stmt);
@@ -2335,9 +2348,10 @@ vect_recog_average_pattern (vec_info *vinfo,
internal_fn ifn = IFN_AVG_FLOOR;
vect_unpromoted_value unprom[3];
tree new_type;
+ enum optab_subtype subtype;
unsigned int nops = vect_widened_op_tree (vinfo, plus_stmt_info, PLUS_EXPR,
- WIDEN_PLUS_EXPR, false, 3,
- unprom, &new_type);
+ CFN_VEC_WIDEN_PLUS, false, 3,
+ unprom, &new_type, &subtype);
if (nops == 0)
return NULL;
if (nops == 3)
diff --git a/gcc/tree-vect-stmts.cc b/gcc/tree-vect-stmts.cc
index c31831df723eeae8ea4fca2790a18b562106c889..9adbb9fbf116ef316d5bed2c84a7074722055717 100644
--- a/gcc/tree-vect-stmts.cc
+++ b/gcc/tree-vect-stmts.cc
@@ -4895,9 +4895,7 @@ vectorizable_conversion (vec_info *vinfo,
if (is_gimple_assign (stmt))
{
- widen_arith = (code == WIDEN_PLUS_EXPR
- || code == WIDEN_MINUS_EXPR
- || code == WIDEN_MULT_EXPR
+ widen_arith = (code == WIDEN_MULT_EXPR
|| code == WIDEN_LSHIFT_EXPR);
}
else
@@ -4950,8 +4948,6 @@ vectorizable_conversion (vec_info *vinfo,
{
gcc_assert (code == WIDEN_MULT_EXPR
|| code == WIDEN_LSHIFT_EXPR
- || code == WIDEN_PLUS_EXPR
- || code == WIDEN_MINUS_EXPR
|| widen_arith);
@@ -11976,7 +11972,7 @@ supportable_widening_operation (vec_info *vinfo,
class loop *vect_loop = NULL;
machine_mode vec_mode;
enum insn_code icode1, icode2;
- optab optab1, optab2;
+ optab optab1 = unknown_optab, optab2 = unknown_optab;
tree vectype = vectype_in;
tree wide_vectype = vectype_out;
code_helper c1=MAX_TREE_CODES, c2=MAX_TREE_CODES;
@@ -12070,16 +12066,6 @@ supportable_widening_operation (vec_info *vinfo,
c2 = VEC_WIDEN_LSHIFT_HI_EXPR;
break;
- case WIDEN_PLUS_EXPR:
- c1 = VEC_WIDEN_PLUS_LO_EXPR;
- c2 = VEC_WIDEN_PLUS_HI_EXPR;
- break;
-
- case WIDEN_MINUS_EXPR:
- c1 = VEC_WIDEN_MINUS_LO_EXPR;
- c2 = VEC_WIDEN_MINUS_HI_EXPR;
- break;
-
CASE_CONVERT:
c1 = VEC_UNPACK_LO_EXPR;
c2 = VEC_UNPACK_HI_EXPR;
diff --git a/gcc/tree.def b/gcc/tree.def
index 62650b6934b337c5d56e5393dc114173d72c9aa9..9b2dce3576440c445d3240b9ed937fe67c9a5992 100644
--- a/gcc/tree.def
+++ b/gcc/tree.def
@@ -1383,8 +1383,6 @@ DEFTREECODE (WIDEN_MULT_MINUS_EXPR, "widen_mult_minus_expr", tcc_expression, 3)
the first argument from type t1 to type t2, and then shifting it
by the second argument. */
DEFTREECODE (WIDEN_LSHIFT_EXPR, "widen_lshift_expr", tcc_binary, 2)
-DEFTREECODE (WIDEN_PLUS_EXPR, "widen_plus_expr", tcc_binary, 2)
-DEFTREECODE (WIDEN_MINUS_EXPR, "widen_minus_expr", tcc_binary, 2)
/* Widening vector multiplication.
The two operands are vectors with N elements of size S. Multiplying the
@@ -1449,10 +1447,6 @@ DEFTREECODE (VEC_PACK_FLOAT_EXPR, "vec_pack_float_expr", tcc_binary, 2)
*/
DEFTREECODE (VEC_WIDEN_LSHIFT_HI_EXPR, "widen_lshift_hi_expr", tcc_binary, 2)
DEFTREECODE (VEC_WIDEN_LSHIFT_LO_EXPR, "widen_lshift_lo_expr", tcc_binary, 2)
-DEFTREECODE (VEC_WIDEN_PLUS_HI_EXPR, "widen_plus_hi_expr", tcc_binary, 2)
-DEFTREECODE (VEC_WIDEN_PLUS_LO_EXPR, "widen_plus_lo_expr", tcc_binary, 2)
-DEFTREECODE (VEC_WIDEN_MINUS_HI_EXPR, "widen_minus_hi_expr", tcc_binary, 2)
-DEFTREECODE (VEC_WIDEN_MINUS_LO_EXPR, "widen_minus_lo_expr", tcc_binary, 2)
/* PREDICT_EXPR. Specify hint for branch prediction. The
PREDICT_EXPR_PREDICTOR specify predictor and PREDICT_EXPR_OUTCOME the
--
2.17.1
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [ping][vect-patterns] Refactor widen_plus/widen_minus as internal_fns
2022-05-25 9:11 Joel Hutton
@ 2022-05-27 13:23 ` Richard Biener
2022-05-31 10:07 ` Joel Hutton
0 siblings, 1 reply; 22+ messages in thread
From: Richard Biener @ 2022-05-27 13:23 UTC (permalink / raw)
To: Joel Hutton; +Cc: Richard Sandiford, gcc-patches
On Wed, 25 May 2022, Joel Hutton wrote:
> Ping!
>
> Just checking there is still interest in this. I'm assuming you've been
> busy with release.
Can you post an updated patch (after the .cc renaming, and code_helper
now already moved to tree.h).
Thanks,
Richard.
> Joel
>
> > -----Original Message-----
> > From: Joel Hutton
> > Sent: 13 April 2022 16:53
> > To: Richard Sandiford <richard.sandiford@arm.com>
> > Cc: Richard Biener <rguenther@suse.de>; gcc-patches@gcc.gnu.org
> > Subject: [vect-patterns] Refactor widen_plus/widen_minus as internal_fns
> >
> > Hi all,
> >
> > These patches refactor the widening patterns in vect-patterns to use
> > internal_fn instead of tree_codes.
> >
> > Sorry about the delay, some changes to master made it a bit messier.
> >
> > Bootstrapped and regression tested on aarch64.
> >
> > Joel
> >
> > > > diff --git a/gcc/tree-vect-patterns.c b/gcc/tree-vect-patterns.c
> > > > index 854cbcff390..4a8ea67e62f 100644
> > > > --- a/gcc/tree-vect-patterns.c
> > > > +++ b/gcc/tree-vect-patterns.c
> > > > @@ -1245,7 +1245,7 @@ vect_recog_sad_pattern (vec_info *vinfo,
> > > > static gimple * vect_recog_widen_op_pattern (vec_info *vinfo,
> > > > stmt_vec_info last_stmt_info, tree *type_out,
> > > > - tree_code orig_code, tree_code wide_code,
> > > > + tree_code orig_code, code_helper
> > > wide_code_or_ifn,
> > >
> > > I think it'd be better to keep the original ?wide_code? name and try
> > > to remove as many places as possible in which switching based on
> > > tree_code or internal_fn is necessary. The recent gimple-match.h
> > > patches should help with that, but more routines might be needed.
> >
> > Done.
> >
> > > > @@ -1309,8 +1310,16 @@ vect_recog_widen_op_pattern (vec_info
> > *vinfo,
> > > > 2, oprnd, half_type, unprom, vectype);
> > > >
> > > > tree var = vect_recog_temp_ssa_var (itype, NULL);
> > > > - gimple *pattern_stmt = gimple_build_assign (var, wide_code,
> > > > - oprnd[0], oprnd[1]);
> > > > + gimple *pattern_stmt;
> > > > + if (wide_code_or_ifn.is_tree_code ())
> > > > + pattern_stmt = gimple_build_assign (var, wide_code_or_ifn,
> > > > + oprnd[0], oprnd[1]);
> > > > + else
> > > > + {
> > > > + internal_fn fn = as_internal_fn ((combined_fn) wide_code_or_ifn);
> > > > + pattern_stmt = gimple_build_call_internal (fn, 2, oprnd[0], oprnd[1]);
> > > > + gimple_call_set_lhs (pattern_stmt, var);
> > > > + }
> > >
> > > For example, I think we should hide this inside a new:
> > >
> > > gimple_build (var, wide_code, oprnd[0], oprnd[1]);
> > >
> > > that works directly on code_helper, similarly to the new code_helper
> > > gimple_build interfaces.
> >
> > Done.
> >
> > > > @@ -4513,14 +4513,16 @@ vect_gen_widened_results_half (vec_info
> > > *vinfo, enum tree_code code,
> > > > tree new_temp;
> > > >
> > > > /* Generate half of the widened result: */
> > > > - gcc_assert (op_type == TREE_CODE_LENGTH (code));
> > > > if (op_type != binary_op)
> > > > vec_oprnd1 = NULL;
> > > > - new_stmt = gimple_build_assign (vec_dest, code, vec_oprnd0,
> > > vec_oprnd1);
> > > > + if (ch.is_tree_code ())
> > > > + new_stmt = gimple_build_assign (vec_dest, ch, vec_oprnd0,
> > > vec_oprnd1);
> > > > + else
> > > > + new_stmt = gimple_build_call_internal (as_internal_fn
> > > > + ((combined_fn)
> > > ch),
> > > > + 2, vec_oprnd0, vec_oprnd1);
> > >
> > > Similarly here. I guess the combined_fn/internal_fn path will also
> > > need to cope with null trailing operands, for consistency with the tree_code
> > one.
> > >
> >
> > Done.
> >
> > > > @@ -4744,31 +4747,28 @@ vectorizable_conversion (vec_info *vinfo,
> > > > && ! vec_stmt)
> > > > return false;
> > > >
> > > > - gassign *stmt = dyn_cast <gassign *> (stmt_info->stmt);
> > > > - if (!stmt)
> > > > + gimple* stmt = stmt_info->stmt;
> > > > + if (!(is_gimple_assign (stmt) || is_gimple_call (stmt)))
> > > > return false;
> > > >
> > > > - if (TREE_CODE (gimple_assign_lhs (stmt)) != SSA_NAME)
> > > > - return false;
> > > > + if (is_gimple_assign (stmt))
> > > > + {
> > > > + code_or_ifn = gimple_assign_rhs_code (stmt); } else
> > > > + code_or_ifn = gimple_call_combined_fn (stmt);
> > >
> > > It might be possible to use gimple_extract_op here (only recently added).
> > > This would also provide the number of operands directly, instead of
> > > needing ?op_type?. It would also provide an array of operands.
> > >
> >
> > Done.
> >
> > > > - code = gimple_assign_rhs_code (stmt);
> > > > - if (!CONVERT_EXPR_CODE_P (code)
> > > > - && code != FIX_TRUNC_EXPR
> > > > - && code != FLOAT_EXPR
> > > > - && code != WIDEN_PLUS_EXPR
> > > > - && code != WIDEN_MINUS_EXPR
> > > > - && code != WIDEN_MULT_EXPR
> > > > - && code != WIDEN_LSHIFT_EXPR)
> > >
> > > Is it safe to drop this check independently of parts 2 and 3?
> > > (Genuine question, haven't checked in detail.)
> >
> > It requires the parts 2 and 3. I've moved that change into this first patch.
> >
> > > > @@ -4784,7 +4784,8 @@ vectorizable_conversion (vec_info *vinfo,
> > > > }
> > > >
> > > > rhs_type = TREE_TYPE (op0);
> > > > - if ((code != FIX_TRUNC_EXPR && code != FLOAT_EXPR)
> > > > + if ((code_or_ifn.is_tree_code () && code_or_ifn != FIX_TRUNC_EXPR
> > > > + && code_or_ifn != FLOAT_EXPR)
> > >
> > > I don't think we want the is_tree_code condition here. The existing
> > > != should work.
> > >
> >
> > Done.
> >
> > > > @@ -11856,13 +11888,13 @@ supportable_widening_operation
> > (vec_info
> > > *vinfo,
> > > > if (BYTES_BIG_ENDIAN && c1 != VEC_WIDEN_MULT_EVEN_EXPR)
> > > > std::swap (c1, c2);
> > > >
> > > > - if (code == FIX_TRUNC_EXPR)
> > > > + if (code_or_ifn == FIX_TRUNC_EXPR)
> > > > {
> > > > /* The signedness is determined from output operand. */
> > > > optab1 = optab_for_tree_code (c1, vectype_out, optab_default);
> > > > optab2 = optab_for_tree_code (c2, vectype_out, optab_default);
> > > > }
> > > > - else if (CONVERT_EXPR_CODE_P (code)
> > > > + else if (CONVERT_EXPR_CODE_P ((tree_code) code_or_ifn)
> > >
> > > I think this should be as_tree_code (), so that it's safe for internal
> > > functions if (tree_code) ever becomes a checked convrsion in future.
> > > Same for other instances.
> > >
> >
> > Done.
> >
> > > > && VECTOR_BOOLEAN_TYPE_P (wide_vectype)
> > > > && VECTOR_BOOLEAN_TYPE_P (vectype)
> > > > && TYPE_MODE (wide_vectype) == TYPE_MODE (vectype) [?] @@
> > > > -12000,7 +12031,7 @@ supportable_widening_operation (vec_info
> > > *vinfo,
> > > > bool
> > > > supportable_narrowing_operation (enum tree_code code,
> > > > tree vectype_out, tree vectype_in,
> > > > - enum tree_code *code1, int *multi_step_cvt,
> > > > + void* _code1, int *multi_step_cvt,
> > >
> > > This might be rehashing an old conversation, sorry, but why does this
> > > need to be void?
> > >
> >
> > Reworked to avoid using void*.
> >
> > > > vec<tree> *interm_types) {
> > > > machine_mode vec_mode;
> > > > @@ -12013,6 +12044,7 @@ supportable_narrowing_operation (enum
> > > tree_code code,
> > > > machine_mode intermediate_mode, prev_mode;
> > > > int i;
> > > > bool uns;
> > > > + tree_code * code1 = (tree_code*) _code1;
> > > >
> > > > *multi_step_cvt = 0;
> > > > switch (code)
> > > > diff --git a/gcc/tree-vectorizer.h b/gcc/tree-vectorizer.h index
> > > > bd6f334d15f..70c06264c11 100644
> > > > --- a/gcc/tree-vectorizer.h
> > > > +++ b/gcc/tree-vectorizer.h
> > > > @@ -2030,13 +2030,16 @@ extern bool vect_is_simple_use (vec_info *,
> > > stmt_vec_info, slp_tree,
> > > > enum vect_def_type *,
> > > > tree *, stmt_vec_info * = NULL); extern bool
> > > > vect_maybe_update_slp_op_vectype (slp_tree, tree); -extern bool
> > > > supportable_widening_operation (vec_info *,
> > > > - enum tree_code, stmt_vec_info,
> > > > - tree, tree, enum tree_code *,
> > > > - enum tree_code *, int *,
> > > > - vec<tree> *);
> > > > +extern bool supportable_widening_operation (vec_info *vinfo,
> > > > + code_helper code_or_ifn,
> > > > + stmt_vec_info stmt_info,
> > > > + tree vectype_out, tree vectype_in,
> > > > + code_helper *code_or_ifn1,
> > > > + code_helper *code_or_ifn2,
> > > > + int *multi_step_cvt,
> > > > + vec<tree> *interm_types);
> > >
> > > Normal style is to keep the variable names out of the header.
> > > The documentation lives in the .c file, so in practice, anyone who
> > > wants to add a new caller will need to look there anyway.
> > >
> > > Thanks,
> > > Richard
> > >
> > > > extern bool supportable_narrowing_operation (enum tree_code, tree,
> > > tree,
> > > > - enum tree_code *, int *,
> > > > + void *, int *,
> > > > vec<tree> *);
> > > >
> > > > extern unsigned record_stmt_cost (stmt_vector_for_cost *, int, diff
> > > > --git a/gcc/tree.h b/gcc/tree.h index f62c00bc870..346565f84ce
> > > > 100644
> > > > --- a/gcc/tree.h
> > > > +++ b/gcc/tree.h
> > > > @@ -6546,5 +6546,31 @@ extern unsigned fndecl_dealloc_argno (tree);
> > > > if nonnull, set the second argument to the referenced enclosing
> > > > object or pointer. Otherwise return null. */ extern tree
> > > > get_attr_nonstring_decl (tree, tree * = NULL);
> > > > +/* Helper to transparently allow tree codes and builtin function codes
> > > > + exist in one storage entity. */ class code_helper {
> > > > +public:
> > > > + code_helper () {}
> > > > + code_helper (tree_code code) : rep ((int) code) {}
> > > > + code_helper (combined_fn fn) : rep (-(int) fn) {}
> > > > + operator tree_code () const { return is_tree_code () ?
> > > > + (tree_code) rep :
> > > > + ERROR_MARK; }
> > > > + operator combined_fn () const { return is_fn_code () ?
> > > > + (combined_fn) -rep:
> > > > + CFN_LAST; }
> > > > + bool is_tree_code () const { return rep > 0; }
> > > > + bool is_fn_code () const { return rep < 0; }
> > > > + int get_rep () const { return rep; }
> > > > +
> > > > + enum tree_code as_tree_code () const { return is_tree_code () ?
> > > > + (tree_code)* this : MAX_TREE_CODES; } combined_fn as_fn_code
> > > > + () const { return is_fn_code () ? (combined_fn)
> > > *this
> > > > + : CFN_LAST;}
> > > > +
> > > > +private:
> > > > + int rep;
> > > > +};
> > > >
> > > > #endif /* GCC_TREE_H */
>
--
Richard Biener <rguenther@suse.de>
SUSE Software Solutions Germany GmbH, Maxfeldstrasse 5, 90409 Nuernberg,
Germany; GF: Ivo Totev, Andrew Myers, Andrew McDonald, Boudien Moerman;
HRB 36809 (AG Nuernberg)
^ permalink raw reply [flat|nested] 22+ messages in thread
* [ping][vect-patterns] Refactor widen_plus/widen_minus as internal_fns
@ 2022-05-25 9:11 Joel Hutton
2022-05-27 13:23 ` Richard Biener
0 siblings, 1 reply; 22+ messages in thread
From: Joel Hutton @ 2022-05-25 9:11 UTC (permalink / raw)
To: Richard Sandiford; +Cc: Richard Biener, gcc-patches
Ping!
Just checking there is still interest in this. I'm assuming you've been busy with release.
Joel
> -----Original Message-----
> From: Joel Hutton
> Sent: 13 April 2022 16:53
> To: Richard Sandiford <richard.sandiford@arm.com>
> Cc: Richard Biener <rguenther@suse.de>; gcc-patches@gcc.gnu.org
> Subject: [vect-patterns] Refactor widen_plus/widen_minus as internal_fns
>
> Hi all,
>
> These patches refactor the widening patterns in vect-patterns to use
> internal_fn instead of tree_codes.
>
> Sorry about the delay, some changes to master made it a bit messier.
>
> Bootstrapped and regression tested on aarch64.
>
> Joel
>
> > > diff --git a/gcc/tree-vect-patterns.c b/gcc/tree-vect-patterns.c
> > > index 854cbcff390..4a8ea67e62f 100644
> > > --- a/gcc/tree-vect-patterns.c
> > > +++ b/gcc/tree-vect-patterns.c
> > > @@ -1245,7 +1245,7 @@ vect_recog_sad_pattern (vec_info *vinfo,
> > > static gimple * vect_recog_widen_op_pattern (vec_info *vinfo,
> > > stmt_vec_info last_stmt_info, tree *type_out,
> > > - tree_code orig_code, tree_code wide_code,
> > > + tree_code orig_code, code_helper
> > wide_code_or_ifn,
> >
> > I think it'd be better to keep the original “wide_code” name and try
> > to remove as many places as possible in which switching based on
> > tree_code or internal_fn is necessary. The recent gimple-match.h
> > patches should help with that, but more routines might be needed.
>
> Done.
>
> > > @@ -1309,8 +1310,16 @@ vect_recog_widen_op_pattern (vec_info
> *vinfo,
> > > 2, oprnd, half_type, unprom, vectype);
> > >
> > > tree var = vect_recog_temp_ssa_var (itype, NULL);
> > > - gimple *pattern_stmt = gimple_build_assign (var, wide_code,
> > > - oprnd[0], oprnd[1]);
> > > + gimple *pattern_stmt;
> > > + if (wide_code_or_ifn.is_tree_code ())
> > > + pattern_stmt = gimple_build_assign (var, wide_code_or_ifn,
> > > + oprnd[0], oprnd[1]);
> > > + else
> > > + {
> > > + internal_fn fn = as_internal_fn ((combined_fn) wide_code_or_ifn);
> > > + pattern_stmt = gimple_build_call_internal (fn, 2, oprnd[0], oprnd[1]);
> > > + gimple_call_set_lhs (pattern_stmt, var);
> > > + }
> >
> > For example, I think we should hide this inside a new:
> >
> > gimple_build (var, wide_code, oprnd[0], oprnd[1]);
> >
> > that works directly on code_helper, similarly to the new code_helper
> > gimple_build interfaces.
>
> Done.
>
> > > @@ -4513,14 +4513,16 @@ vect_gen_widened_results_half (vec_info
> > *vinfo, enum tree_code code,
> > > tree new_temp;
> > >
> > > /* Generate half of the widened result: */
> > > - gcc_assert (op_type == TREE_CODE_LENGTH (code));
> > > if (op_type != binary_op)
> > > vec_oprnd1 = NULL;
> > > - new_stmt = gimple_build_assign (vec_dest, code, vec_oprnd0,
> > vec_oprnd1);
> > > + if (ch.is_tree_code ())
> > > + new_stmt = gimple_build_assign (vec_dest, ch, vec_oprnd0,
> > vec_oprnd1);
> > > + else
> > > + new_stmt = gimple_build_call_internal (as_internal_fn
> > > + ((combined_fn)
> > ch),
> > > + 2, vec_oprnd0, vec_oprnd1);
> >
> > Similarly here. I guess the combined_fn/internal_fn path will also
> > need to cope with null trailing operands, for consistency with the tree_code
> one.
> >
>
> Done.
>
> > > @@ -4744,31 +4747,28 @@ vectorizable_conversion (vec_info *vinfo,
> > > && ! vec_stmt)
> > > return false;
> > >
> > > - gassign *stmt = dyn_cast <gassign *> (stmt_info->stmt);
> > > - if (!stmt)
> > > + gimple* stmt = stmt_info->stmt;
> > > + if (!(is_gimple_assign (stmt) || is_gimple_call (stmt)))
> > > return false;
> > >
> > > - if (TREE_CODE (gimple_assign_lhs (stmt)) != SSA_NAME)
> > > - return false;
> > > + if (is_gimple_assign (stmt))
> > > + {
> > > + code_or_ifn = gimple_assign_rhs_code (stmt); } else
> > > + code_or_ifn = gimple_call_combined_fn (stmt);
> >
> > It might be possible to use gimple_extract_op here (only recently added).
> > This would also provide the number of operands directly, instead of
> > needing “op_type”. It would also provide an array of operands.
> >
>
> Done.
>
> > > - code = gimple_assign_rhs_code (stmt);
> > > - if (!CONVERT_EXPR_CODE_P (code)
> > > - && code != FIX_TRUNC_EXPR
> > > - && code != FLOAT_EXPR
> > > - && code != WIDEN_PLUS_EXPR
> > > - && code != WIDEN_MINUS_EXPR
> > > - && code != WIDEN_MULT_EXPR
> > > - && code != WIDEN_LSHIFT_EXPR)
> >
> > Is it safe to drop this check independently of parts 2 and 3?
> > (Genuine question, haven't checked in detail.)
>
> It requires the parts 2 and 3. I've moved that change into this first patch.
>
> > > @@ -4784,7 +4784,8 @@ vectorizable_conversion (vec_info *vinfo,
> > > }
> > >
> > > rhs_type = TREE_TYPE (op0);
> > > - if ((code != FIX_TRUNC_EXPR && code != FLOAT_EXPR)
> > > + if ((code_or_ifn.is_tree_code () && code_or_ifn != FIX_TRUNC_EXPR
> > > + && code_or_ifn != FLOAT_EXPR)
> >
> > I don't think we want the is_tree_code condition here. The existing
> > != should work.
> >
>
> Done.
>
> > > @@ -11856,13 +11888,13 @@ supportable_widening_operation
> (vec_info
> > *vinfo,
> > > if (BYTES_BIG_ENDIAN && c1 != VEC_WIDEN_MULT_EVEN_EXPR)
> > > std::swap (c1, c2);
> > >
> > > - if (code == FIX_TRUNC_EXPR)
> > > + if (code_or_ifn == FIX_TRUNC_EXPR)
> > > {
> > > /* The signedness is determined from output operand. */
> > > optab1 = optab_for_tree_code (c1, vectype_out, optab_default);
> > > optab2 = optab_for_tree_code (c2, vectype_out, optab_default);
> > > }
> > > - else if (CONVERT_EXPR_CODE_P (code)
> > > + else if (CONVERT_EXPR_CODE_P ((tree_code) code_or_ifn)
> >
> > I think this should be as_tree_code (), so that it's safe for internal
> > functions if (tree_code) ever becomes a checked convrsion in future.
> > Same for other instances.
> >
>
> Done.
>
> > > && VECTOR_BOOLEAN_TYPE_P (wide_vectype)
> > > && VECTOR_BOOLEAN_TYPE_P (vectype)
> > > && TYPE_MODE (wide_vectype) == TYPE_MODE (vectype) […] @@
> > > -12000,7 +12031,7 @@ supportable_widening_operation (vec_info
> > *vinfo,
> > > bool
> > > supportable_narrowing_operation (enum tree_code code,
> > > tree vectype_out, tree vectype_in,
> > > - enum tree_code *code1, int *multi_step_cvt,
> > > + void* _code1, int *multi_step_cvt,
> >
> > This might be rehashing an old conversation, sorry, but why does this
> > need to be void?
> >
>
> Reworked to avoid using void*.
>
> > > vec<tree> *interm_types) {
> > > machine_mode vec_mode;
> > > @@ -12013,6 +12044,7 @@ supportable_narrowing_operation (enum
> > tree_code code,
> > > machine_mode intermediate_mode, prev_mode;
> > > int i;
> > > bool uns;
> > > + tree_code * code1 = (tree_code*) _code1;
> > >
> > > *multi_step_cvt = 0;
> > > switch (code)
> > > diff --git a/gcc/tree-vectorizer.h b/gcc/tree-vectorizer.h index
> > > bd6f334d15f..70c06264c11 100644
> > > --- a/gcc/tree-vectorizer.h
> > > +++ b/gcc/tree-vectorizer.h
> > > @@ -2030,13 +2030,16 @@ extern bool vect_is_simple_use (vec_info *,
> > stmt_vec_info, slp_tree,
> > > enum vect_def_type *,
> > > tree *, stmt_vec_info * = NULL); extern bool
> > > vect_maybe_update_slp_op_vectype (slp_tree, tree); -extern bool
> > > supportable_widening_operation (vec_info *,
> > > - enum tree_code, stmt_vec_info,
> > > - tree, tree, enum tree_code *,
> > > - enum tree_code *, int *,
> > > - vec<tree> *);
> > > +extern bool supportable_widening_operation (vec_info *vinfo,
> > > + code_helper code_or_ifn,
> > > + stmt_vec_info stmt_info,
> > > + tree vectype_out, tree vectype_in,
> > > + code_helper *code_or_ifn1,
> > > + code_helper *code_or_ifn2,
> > > + int *multi_step_cvt,
> > > + vec<tree> *interm_types);
> >
> > Normal style is to keep the variable names out of the header.
> > The documentation lives in the .c file, so in practice, anyone who
> > wants to add a new caller will need to look there anyway.
> >
> > Thanks,
> > Richard
> >
> > > extern bool supportable_narrowing_operation (enum tree_code, tree,
> > tree,
> > > - enum tree_code *, int *,
> > > + void *, int *,
> > > vec<tree> *);
> > >
> > > extern unsigned record_stmt_cost (stmt_vector_for_cost *, int, diff
> > > --git a/gcc/tree.h b/gcc/tree.h index f62c00bc870..346565f84ce
> > > 100644
> > > --- a/gcc/tree.h
> > > +++ b/gcc/tree.h
> > > @@ -6546,5 +6546,31 @@ extern unsigned fndecl_dealloc_argno (tree);
> > > if nonnull, set the second argument to the referenced enclosing
> > > object or pointer. Otherwise return null. */ extern tree
> > > get_attr_nonstring_decl (tree, tree * = NULL);
> > > +/* Helper to transparently allow tree codes and builtin function codes
> > > + exist in one storage entity. */ class code_helper {
> > > +public:
> > > + code_helper () {}
> > > + code_helper (tree_code code) : rep ((int) code) {}
> > > + code_helper (combined_fn fn) : rep (-(int) fn) {}
> > > + operator tree_code () const { return is_tree_code () ?
> > > + (tree_code) rep :
> > > + ERROR_MARK; }
> > > + operator combined_fn () const { return is_fn_code () ?
> > > + (combined_fn) -rep:
> > > + CFN_LAST; }
> > > + bool is_tree_code () const { return rep > 0; }
> > > + bool is_fn_code () const { return rep < 0; }
> > > + int get_rep () const { return rep; }
> > > +
> > > + enum tree_code as_tree_code () const { return is_tree_code () ?
> > > + (tree_code)* this : MAX_TREE_CODES; } combined_fn as_fn_code
> > > + () const { return is_fn_code () ? (combined_fn)
> > *this
> > > + : CFN_LAST;}
> > > +
> > > +private:
> > > + int rep;
> > > +};
> > >
> > > #endif /* GCC_TREE_H */
^ permalink raw reply [flat|nested] 22+ messages in thread
end of thread, other threads:[~2023-04-28 16:06 UTC | newest]
Thread overview: 22+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-11-25 10:08 [ping][vect-patterns] Refactor widen_plus/widen_minus as internal_fns Joel Hutton
2022-05-25 9:11 Joel Hutton
2022-05-27 13:23 ` Richard Biener
2022-05-31 10:07 ` Joel Hutton
2022-05-31 16:46 ` Tamar Christina
2022-06-01 10:11 ` Richard Biener
2022-06-06 17:20 ` Joel Hutton
2022-06-07 8:18 ` Richard Sandiford
2022-06-07 9:01 ` Joel Hutton
2022-06-09 14:03 ` Joel Hutton
2022-06-13 9:02 ` Richard Biener
2022-06-30 13:20 ` Joel Hutton
2022-07-12 12:32 ` Richard Biener
2023-03-17 10:14 ` Andre Vieira (lists)
2023-03-17 11:52 ` Richard Biener
2023-04-20 13:23 ` Andre Vieira (lists)
2023-04-24 11:57 ` Richard Biener
2023-04-24 13:01 ` Richard Sandiford
2023-04-25 12:30 ` Richard Biener
2023-04-28 16:06 ` Andre Vieira (lists)
2023-04-25 9:55 ` Andre Vieira (lists)
2022-06-13 9:18 ` Richard Biener
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).