From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by sourceware.org (Postfix) with ESMTP id 3491E3858D28 for ; Tue, 31 Jan 2023 13:06:58 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 3491E3858D28 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id A14F32F4 for ; Tue, 31 Jan 2023 05:07:39 -0800 (PST) Received: from localhost (e121540-lin.manchester.arm.com [10.32.99.50]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 3C4243F71E for ; Tue, 31 Jan 2023 05:06:57 -0800 (PST) From: Richard Sandiford To: gcc-patches@gcc.gnu.org Mail-Followup-To: gcc-patches@gcc.gnu.org, richard.sandiford@arm.com Subject: [PATCH] vect: Fix single def-use cycle for ifn reductions [PR108608] Date: Tue, 31 Jan 2023 13:06:56 +0000 Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Status: No, score=-36.3 required=5.0 tests=BAYES_00,GIT_PATCH_0,KAM_DMARC_NONE,KAM_DMARC_STATUS,KAM_LAZY_DOMAIN_SECURITY,KAM_SHORT,SPF_HELO_NONE,SPF_NONE,TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: The patch that added support for fmin/fmax reductions didn't handle single def-use cycles. In some ways, this seems like going out of our way to make things slower, but that's a discussion for another day. Tested on aarch64-linux-gnu & x86_64-linux-gnu. OK for trunk and the GCC 12 branch? Richard gcc/ PR tree-optimization/108608 * tree-vect-loop.cc (vect_transform_reduction): Handle single def-use cycles that involve function calls rather than tree codes. gcc/testsuite/ PR tree-optimization/108608 * gcc.dg/vect/pr108608.c: New test. * gcc.target/aarch64/sve/pr108608-1.c: Likewise. --- gcc/testsuite/gcc.dg/vect/pr108608.c | 24 +++++++++++++++++++ .../gcc.target/aarch64/sve/pr108608-1.c | 9 +++++++ gcc/tree-vect-loop.cc | 22 ++++++++++------- 3 files changed, 46 insertions(+), 9 deletions(-) create mode 100644 gcc/testsuite/gcc.dg/vect/pr108608.c create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/pr108608-1.c diff --git a/gcc/testsuite/gcc.dg/vect/pr108608.c b/gcc/testsuite/gcc.dg/vect/pr108608.c new file mode 100644 index 00000000000..e968141ba03 --- /dev/null +++ b/gcc/testsuite/gcc.dg/vect/pr108608.c @@ -0,0 +1,24 @@ +#include "tree-vect.h" + +double __attribute__((noipa)) +foo (double m, float *ptr) +{ + for (int i = 0; i < 256; i++) + m = __builtin_fmax (m, ptr[i]); + return m; +} + +int +main (void) +{ + check_vect (); + float ptr[256]; + for (int j = 0; j < 16; ++j) + { + for (int i = 0; i < 256; ++i) + ptr[i] = i == 128 + j ? 2 + j : i == 161 ? 1 : 0; + if (foo (0, ptr) != 2 + j) + __builtin_abort (); + } + return 0; +} diff --git a/gcc/testsuite/gcc.target/aarch64/sve/pr108608-1.c b/gcc/testsuite/gcc.target/aarch64/sve/pr108608-1.c new file mode 100644 index 00000000000..0a7d485e047 --- /dev/null +++ b/gcc/testsuite/gcc.target/aarch64/sve/pr108608-1.c @@ -0,0 +1,9 @@ +/* { dg-options "-O3" } */ + +double __attribute__((noipa)) +foo (double m, float *ptr) +{ + for (int i = 0; i < 256; i++) + m = __builtin_fmax (m, ptr[i]); + return m; +} diff --git a/gcc/tree-vect-loop.cc b/gcc/tree-vect-loop.cc index f0801c23671..f03af1efd0f 100644 --- a/gcc/tree-vect-loop.cc +++ b/gcc/tree-vect-loop.cc @@ -7755,8 +7755,6 @@ vect_transform_reduction (loop_vec_info loop_vinfo, gimple_match_op op; if (!gimple_extract_op (stmt_info->stmt, &op)) gcc_unreachable (); - gcc_assert (op.code.is_tree_code ()); - auto code = tree_code (op.code); /* All uses but the last are expected to be defined in the loop. The last use is the reduction variable. In case of nested cycle this @@ -7778,7 +7776,8 @@ vect_transform_reduction (loop_vec_info loop_vinfo, vec_num = 1; } - internal_fn cond_fn = get_conditional_internal_fn (code); + code_helper code = canonicalize_code (op.code, op.type); + internal_fn cond_fn = get_conditional_internal_fn (code, op.type); vec_loop_masks *masks = &LOOP_VINFO_MASKS (loop_vinfo); bool mask_by_cond_expr = use_mask_by_cond_expr_p (code, cond_fn, vectype_in); @@ -7802,9 +7801,10 @@ vect_transform_reduction (loop_vec_info loop_vinfo, if (reduction_type == FOLD_LEFT_REDUCTION) { internal_fn reduc_fn = STMT_VINFO_REDUC_FN (reduc_info); + gcc_assert (code.is_tree_code ()); return vectorize_fold_left_reduction - (loop_vinfo, stmt_info, gsi, vec_stmt, slp_node, reduc_def_phi, code, - reduc_fn, op.ops, vectype_in, reduc_index, masks); + (loop_vinfo, stmt_info, gsi, vec_stmt, slp_node, reduc_def_phi, + tree_code (code), reduc_fn, op.ops, vectype_in, reduc_index, masks); } bool single_defuse_cycle = STMT_VINFO_FORCE_SINGLE_CYCLE (reduc_info); @@ -7814,7 +7814,7 @@ vect_transform_reduction (loop_vec_info loop_vinfo, || code == SAD_EXPR); /* Create the destination vector */ - tree scalar_dest = gimple_assign_lhs (stmt_info->stmt); + tree scalar_dest = gimple_get_lhs (stmt_info->stmt); tree vec_dest = vect_create_destination_var (scalar_dest, vectype_out); vect_get_vec_defs (loop_vinfo, stmt_info, slp_node, ncopies, @@ -7849,7 +7849,7 @@ vect_transform_reduction (loop_vec_info loop_vinfo, /* Make sure that the reduction accumulator is vop[0]. */ if (reduc_index == 1) { - gcc_assert (commutative_tree_code (code)); + gcc_assert (commutative_binary_op_p (code, op.type)); std::swap (vop[0], vop[1]); } tree mask = vect_get_loop_mask (gsi, masks, vec_num * ncopies, @@ -7877,11 +7877,15 @@ vect_transform_reduction (loop_vec_info loop_vinfo, if (emulated_mixed_dot_prod) new_stmt = vect_emulate_mixed_dot_prod (loop_vinfo, stmt_info, gsi, vec_dest, vop); + else if (code.is_internal_fn ()) + new_stmt = gimple_build_call_internal (internal_fn (code), + op.num_ops, + vop[0], vop[1], vop[2]); else - new_stmt = gimple_build_assign (vec_dest, code, + new_stmt = gimple_build_assign (vec_dest, tree_code (op.code), vop[0], vop[1], vop[2]); new_temp = make_ssa_name (vec_dest, new_stmt); - gimple_assign_set_lhs (new_stmt, new_temp); + gimple_set_lhs (new_stmt, new_temp); vect_finish_stmt_generation (loop_vinfo, stmt_info, new_stmt, gsi); } -- 2.25.1