From: Richard Biener <rguenther@suse.de>
To: gcc-patches@gcc.gnu.org
Subject: [PATCH] tree-optimization/107247 - reduce SLP reduction accumulator
Date: Thu, 13 Oct 2022 15:19:56 +0200 (CEST) [thread overview]
Message-ID: <20221013131957.8C67013AAA@imap2.suse-dmz.suse.de> (raw)
The following makes sure to reduce a multi-vector SLP reduction
accumulator to a single vector using vector operations if
easily possible (if the number of lanes in the vector type is
a multiple of the number of scalar accumulators).
Bootstrapped on x86_64-unknown-linux-gnu, testing in progress.
Richard.
PR tree-optimization/107247
* tree-vect-loop.cc (vect_create_epilog_for_reduction):
Reduce multi vector SLP reduction accumulators. Check
the adjusted number of accumulator vectors against
one for the re-use in the epilogue.
---
gcc/tree-vect-loop.cc | 14 +++++++++++++-
1 file changed, 13 insertions(+), 1 deletion(-)
diff --git a/gcc/tree-vect-loop.cc b/gcc/tree-vect-loop.cc
index b1442a93581..98a943d8a4b 100644
--- a/gcc/tree-vect-loop.cc
+++ b/gcc/tree-vect-loop.cc
@@ -5642,9 +5642,21 @@ vect_create_epilog_for_reduction (loop_vec_info loop_vinfo,
we may end up with more than one vector result. Here we reduce them
to one vector.
+ The same is true for a SLP reduction, e.g.,
+ # a1 = phi <a2, a0>
+ # b1 = phi <b2, b0>
+ a2 = operation (a1)
+ b2 = operation (a2),
+
+ where we can end up with more than one vector as well. We can
+ easily accumulate vectors when the number of vector elements is
+ a multiple of the SLP group size.
+
The same is true if we couldn't use a single defuse cycle. */
if (REDUC_GROUP_FIRST_ELEMENT (stmt_info)
|| direct_slp_reduc
+ || (slp_reduc
+ && constant_multiple_p (TYPE_VECTOR_SUBPARTS (vectype), group_size))
|| ncopies > 1)
{
gimple_seq stmts = NULL;
@@ -6233,7 +6245,7 @@ vect_create_epilog_for_reduction (loop_vec_info loop_vinfo,
/* Record this operation if it could be reused by the epilogue loop. */
if (STMT_VINFO_REDUC_TYPE (reduc_info) == TREE_CODE_REDUCTION
- && vec_num == 1)
+ && reduc_inputs.length () == 1)
loop_vinfo->reusable_accumulators.put (scalar_results[0],
{ orig_reduc_input, reduc_info });
--
2.35.3
reply other threads:[~2022-10-13 13:19 UTC|newest]
Thread overview: [no followups] expand[flat|nested] mbox.gz Atom feed
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20221013131957.8C67013AAA@imap2.suse-dmz.suse.de \
--to=rguenther@suse.de \
--cc=gcc-patches@gcc.gnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).