From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: by sourceware.org (Postfix, from userid 1666) id 1B0023858D1E; Tue, 24 Jan 2023 15:22:40 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 1B0023858D1E DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gcc.gnu.org; s=default; t=1674573760; bh=FvgUzN1IqcfcT1BjjrkWpEbsiRNFFDe136KqKt4LJVY=; h=From:To:Subject:Date:From; b=oKDMcBKXszbLu9ZPUxeHmPXAXfWg5yoTrBC6TejHxbGx18uUwi0tkjVkCBk73vFok 7ZNdAbwvuVo4jVrUcQZ0MBMQ7Bz3ufPjAbVk/byBqqkDbWU3XkfGZm/Lmoxq8piWgM 2w5i03cq83MHGGj1z+9kbb8FUTf9JCZnKEZdoxsE= MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="utf-8" From: Richard Biener To: gcc-cvs@gcc.gnu.org Subject: [gcc r11-10480] tree-optimization/107212 - SLP reduction of reduction paths X-Act-Checkin: gcc X-Git-Author: Richard Biener X-Git-Refname: refs/heads/releases/gcc-11 X-Git-Oldrev: 9d21cc4edd94f8f2b1a3241fab5cf75649003226 X-Git-Newrev: 2461fa40fc24a403dc8149338f44b0e4aef4f173 Message-Id: <20230124152240.1B0023858D1E@sourceware.org> Date: Tue, 24 Jan 2023 15:22:40 +0000 (GMT) List-Id: https://gcc.gnu.org/g:2461fa40fc24a403dc8149338f44b0e4aef4f173 commit r11-10480-g2461fa40fc24a403dc8149338f44b0e4aef4f173 Author: Richard Biener Date: Tue Oct 11 11:34:55 2022 +0200 tree-optimization/107212 - SLP reduction of reduction paths The following fixes an issue with how we handle epilogue generation for SLP reductions of reduction paths where the actual live lanes are not "canonical". We need to make sure to identify all live lanes as reductions and thus have to iterate over all participating SLP lanes when walking the reduction SSA use-def chain. Also the previous attempt likely to mitigate such issue in vectorizable_live_operation is misguided and has to be removed. PR tree-optimization/107212 * tree-vect-loop.c (vectorizable_reduction): Make sure to set STMT_VINFO_REDUC_DEF for all live lanes in a SLP reduction. (vectorizable_live_operation): Do not pun to the SLP node representative for reduction epilogue generation. * gcc.dg/vect/pr107212-1.c: New testcase. * gcc.dg/vect/pr107212-2.c: Likewise. (cherry picked from commit ee467644c53ee2f7d633a8e1f53603feafab4351) Diff: --- gcc/testsuite/gcc.dg/vect/pr107212-1.c | 27 +++++++++++++++++++++++++++ gcc/testsuite/gcc.dg/vect/pr107212-2.c | 23 +++++++++++++++++++++++ gcc/tree-vect-loop.c | 20 +++++++++++++------- 3 files changed, 63 insertions(+), 7 deletions(-) diff --git a/gcc/testsuite/gcc.dg/vect/pr107212-1.c b/gcc/testsuite/gcc.dg/vect/pr107212-1.c new file mode 100644 index 00000000000..5343f9b6b23 --- /dev/null +++ b/gcc/testsuite/gcc.dg/vect/pr107212-1.c @@ -0,0 +1,27 @@ +/* { dg-do run } */ + +#include "tree-vect.h" + +int main() +{ + check_vect (); + + unsigned int tab[6][2] = { {69, 73}, {36, 40}, {24, 16}, + {16, 11}, {4, 5}, {3, 1} }; + + int sum_0 = 0; + int sum_1 = 0; + + for(int t=0; t<6; t++) { + sum_0 += tab[t][0]; + sum_1 += tab[t][1]; + } + + int x1 = (sum_0 < 100); + int x2 = (sum_0 > 200); + + if (x1 || x2 || sum_1 != 146) + __builtin_abort (); + + return 0; +} diff --git a/gcc/testsuite/gcc.dg/vect/pr107212-2.c b/gcc/testsuite/gcc.dg/vect/pr107212-2.c new file mode 100644 index 00000000000..109c2b991a6 --- /dev/null +++ b/gcc/testsuite/gcc.dg/vect/pr107212-2.c @@ -0,0 +1,23 @@ +/* { dg-do run } */ + +#include "tree-vect.h" + +int sum_1 = 0; + +int main() +{ + check_vect (); + + unsigned int tab[6][2] = {{150, 0}, {0, 0}, {0, 0}, {0, 0}, {0, 0}, {0, 0}}; + + int sum_0 = 0; + + for (int t = 0; t < 6; t++) { + sum_0 += tab[t][0]; + sum_1 += tab[t][0]; + } + + if (sum_0 < 100 || sum_0 > 200) + __builtin_abort(); + return 0; +} diff --git a/gcc/tree-vect-loop.c b/gcc/tree-vect-loop.c index 968087a9008..2c07fa38ff0 100644 --- a/gcc/tree-vect-loop.c +++ b/gcc/tree-vect-loop.c @@ -6589,10 +6589,20 @@ vectorizable_reduction (loop_vec_info loop_vinfo, } if (!REDUC_GROUP_FIRST_ELEMENT (vdef)) only_slp_reduc_chain = false; - /* ??? For epilogue generation live members of the chain need + /* For epilogue generation live members of the chain need to point back to the PHI via their original stmt for - info_for_reduction to work. */ - if (STMT_VINFO_LIVE_P (vdef)) + info_for_reduction to work. For SLP we need to look at + all lanes here - even though we only will vectorize from + the SLP node with live lane zero the other live lanes also + need to be identified as part of a reduction to be able + to skip code generation for them. */ + if (slp_for_stmt_info) + { + for (auto s : SLP_TREE_SCALAR_STMTS (slp_for_stmt_info)) + if (STMT_VINFO_LIVE_P (s)) + STMT_VINFO_REDUC_DEF (vect_orig_stmt (s)) = phi_info; + } + else if (STMT_VINFO_LIVE_P (vdef)) STMT_VINFO_REDUC_DEF (def) = phi_info; gassign *assign = dyn_cast (vdef->stmt); if (!assign) @@ -8578,10 +8588,6 @@ vectorizable_live_operation (vec_info *vinfo, all involved stmts together. */ else if (slp_index != 0) return true; - else - /* For SLP reductions the meta-info is attached to - the representative. */ - stmt_info = SLP_TREE_REPRESENTATIVE (slp_node); } stmt_vec_info reduc_info = info_for_reduction (loop_vinfo, stmt_info); gcc_assert (reduc_info->is_reduc_info);