From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: by sourceware.org (Postfix, from userid 1666) id 29BA3385829D; Fri, 23 Feb 2024 12:01:03 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 29BA3385829D DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gcc.gnu.org; s=default; t=1708689663; bh=xaskso8quZGkKV8kbtirvblYzxLT93t7eoEAwhSP+2c=; h=From:To:Subject:Date:From; b=iyZ1qBaHRQ1YMXXtI29/tNj6jZ9vZVuru1fs1VICrGUjydhQgnVZAzn1s96HMvvFE 3gPfQa1LUz5nF2kvEW3GVHH4Iletj2CoHLR1EEGkJ5au0d1O03Zwv0kTchrqFaDhLN /K9pfPh0AgMRq+Kfd6Wmxp6aiqqZcCgdgGTmmK2E= Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: Richard Biener To: gcc-cvs@gcc.gnu.org Subject: [gcc(refs/users/rguenth/heads/vect-force-slp)] Do single-lane SLP discovery for reductions X-Act-Checkin: gcc X-Git-Author: Richard Biener X-Git-Refname: refs/users/rguenth/heads/vect-force-slp X-Git-Oldrev: 598e22d683e4c92eaa784238d9ef733b612b75fa X-Git-Newrev: b992691a64326ce60f1f17a0fa4c2224fb6adac9 Message-Id: <20240223120103.29BA3385829D@sourceware.org> Date: Fri, 23 Feb 2024 12:01:03 +0000 (GMT) List-Id: https://gcc.gnu.org/g:b992691a64326ce60f1f17a0fa4c2224fb6adac9 commit b992691a64326ce60f1f17a0fa4c2224fb6adac9 Author: Richard Biener Date: Fri Feb 23 11:45:50 2024 +0100 Do single-lane SLP discovery for reductions The following performs single-lane SLP discovery for reductions. This exposes a latent issue with reduction SLP in outer loop vectorization and makes gcc.dg/vect/vect-outer-4[fgkl].c FAIL execution. * tree-vect-slp.cc (vect_build_slp_tree_2): Only multi-lane discoveries are reduction chains and need special backedge treatment. (vect_analyze_slp): Fall back to single-lane SLP discovery for reductions. Diff: --- gcc/tree-vect-slp.cc | 33 +++++++++++++++++++++++++++------ 1 file changed, 27 insertions(+), 6 deletions(-) diff --git a/gcc/tree-vect-slp.cc b/gcc/tree-vect-slp.cc index 32fccba7f154..43d20da87340 100644 --- a/gcc/tree-vect-slp.cc +++ b/gcc/tree-vect-slp.cc @@ -1912,7 +1912,8 @@ vect_build_slp_tree_2 (vec_info *vinfo, slp_tree node, /* Reduction chain backedge defs are filled manually. ??? Need a better way to identify a SLP reduction chain PHI. Or a better overall way to SLP match those. */ - if (all_same && def_type == vect_reduction_def) + if (stmts.length () > 1 + && all_same && def_type == vect_reduction_def) skip_args[loop_latch_edge (loop)->dest_idx] = true; } else if (def_type != vect_internal_def) @@ -3891,11 +3892,31 @@ vect_analyze_slp (vec_info *vinfo, unsigned max_tree_size) loop_vinfo->reductions.safe_push (last); } - /* Find SLP sequences starting from groups of reductions. */ - if (loop_vinfo->reductions.length () > 1) - vect_analyze_slp_instance (vinfo, bst_map, loop_vinfo->reductions[0], - slp_inst_kind_reduc_group, max_tree_size, - &limit); + if (loop_vinfo->reductions.length () > 0 + && (loop_vinfo->reductions.length () == 1 + /* Find SLP sequences starting from groups of reductions. */ + || ! vect_analyze_slp_instance (vinfo, bst_map, + loop_vinfo->reductions[0], + slp_inst_kind_reduc_group, + max_tree_size, &limit)) + && param_vect_single_lane_slp != 0) + { + /* Do SLP discovery for single-lane reductions. */ + for (auto stmt_info : loop_vinfo->reductions) + { + vec stmts; + vec roots = vNULL; + vec remain = vNULL; + stmts.create (1); + stmts.quick_push (stmt_info); + bool res = vect_build_slp_instance (vinfo, + slp_inst_kind_reduc_group, + stmts, roots, remain, + max_tree_size, &limit, + bst_map, NULL); + gcc_assert (res); + } + } } hash_set visited_patterns;