From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) by sourceware.org (Postfix) with ESMTPS id 55DFB3858CDB for ; Mon, 26 Feb 2024 11:34:44 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 55DFB3858CDB Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=suse.de ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 55DFB3858CDB Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=195.135.223.130 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1708947286; cv=none; b=lS/2DH9IdmHBqDY7fWMFILoy/hDveSzAhTRLhVhWiTggEtOApfi06iWUigK/LhIgu56KydRZyxMrDTHXWXhvpg6plSsZxox1YKmzzFQ+zHEAV4z8X/MKZ7x5Z9qQDo4sfW5fozv1MK+4IXRteLo1wo/IBJ2uC2InwznxXbxgDYA= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1708947286; c=relaxed/simple; bh=eTXS13LgciFyZEpgAa9Yc90xKLXZLDfmQ3QV+51Zc6Q=; h=DKIM-Signature:DKIM-Signature:DKIM-Signature:DKIM-Signature:Date: From:To:Subject:MIME-Version; b=ZH6+E+PjpcQGHnzyH1T+WiZANDGlio4BiMrKSNE5VnniZ1/quZeHNL8Sy7fQco7dBrinLXFuvKtQmGN9sNiG6jSUro1p7YVjQKlkVcSKR16ah6Uowa6zY3DreAUyS3ZGdd+M7iJgw+dNLZGYIwKG5OlpZigAbae43HV7KOiQUv8= ARC-Authentication-Results: i=1; server2.sourceware.org Received: from [10.168.4.150] (unknown [10.168.4.150]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 35E1D2243B; Mon, 26 Feb 2024 11:34:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1708947283; h=from:from:reply-to:date:date:to:to:cc:cc:mime-version:mime-version: content-type:content-type; bh=KuL/VNTCuruKlJqCmB56JpU+bJfWgpVuI74GMZMgKwg=; b=wvOJNue4e0geFPVgGM2fo7CKVmw3rgQOgOTTyh+xBCljmuOgy9m+Bt5IKIuXkof2UBXXho b3ZPU28nYuykd0fWto7LbnyTZjiOTSAYjM0Jk/59XnoyGafo8U3H6PgddeiG/SjbUoqdTN JJ1LoicmaJpVQ5eZVhXiFKT5BAYtsgc= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1708947283; h=from:from:reply-to:date:date:to:to:cc:cc:mime-version:mime-version: content-type:content-type; bh=KuL/VNTCuruKlJqCmB56JpU+bJfWgpVuI74GMZMgKwg=; b=R2CSHzBlj2l8jSS9rasxcpt7gtFIjhyNOQh4yJa2VWgKu8tIUsGBbgPo1M9Sjfi2d4S5kA n33b4I+PlGFUaUAw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1708947283; h=from:from:reply-to:date:date:to:to:cc:cc:mime-version:mime-version: content-type:content-type; bh=KuL/VNTCuruKlJqCmB56JpU+bJfWgpVuI74GMZMgKwg=; b=wvOJNue4e0geFPVgGM2fo7CKVmw3rgQOgOTTyh+xBCljmuOgy9m+Bt5IKIuXkof2UBXXho b3ZPU28nYuykd0fWto7LbnyTZjiOTSAYjM0Jk/59XnoyGafo8U3H6PgddeiG/SjbUoqdTN JJ1LoicmaJpVQ5eZVhXiFKT5BAYtsgc= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1708947283; h=from:from:reply-to:date:date:to:to:cc:cc:mime-version:mime-version: content-type:content-type; bh=KuL/VNTCuruKlJqCmB56JpU+bJfWgpVuI74GMZMgKwg=; b=R2CSHzBlj2l8jSS9rasxcpt7gtFIjhyNOQh4yJa2VWgKu8tIUsGBbgPo1M9Sjfi2d4S5kA n33b4I+PlGFUaUAw== Date: Mon, 26 Feb 2024 12:34:42 +0100 (CET) From: Richard Biener To: gcc-patches@gcc.gnu.org cc: tamar.christina@arm.com Subject: [PATCH 1/2] tree-optimization/114068 - missed virtual LC PHI after vect peeling MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Authentication-Results: smtp-out1.suse.de; none X-Spam-Score: 1.61 X-Spamd-Result: default: False [1.61 / 50.00]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; MISSING_MID(2.50)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-0.16)[-0.817]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_SPAM_LONG(2.38)[0.679]; FUZZY_BLOCKED(0.00)[rspamd.com]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; BAYES_HAM(-3.00)[100.00%] X-Spam-Status: No, score=-10.6 required=5.0 tests=BAYES_00,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,GIT_PATCH_0,MISSING_MID,SPF_HELO_NONE,SPF_PASS,TXREP,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: Message-ID: <20240226113442.aIK7oxQhkxjCxT-2UkInKT2OmsvC5XCYFr2U9PO28Vc@z> When we choose the IV exit to be one leading to no virtual use we fail to have a virtual LC PHI even though we need it for the epilog entry. The following makes sure to create it so that later updating works. Bootstrapped on x86_64-unknown-linux-gnu, testing in progress. PR tree-optimization/114068 * tree-vect-loop-manip.cc (get_live_virtual_operand_on_edge): New function. (slpeel_tree_duplicate_loop_to_edge_cfg): Add a virtual LC PHI on the main exit if needed. Remove band-aid for the case it was missing. * gcc.dg/vect/vect-early-break_118-pr114068.c: New testcase. * gcc.dg/vect/vect-early-break_119-pr114068.c: Likewise. --- .../vect/vect-early-break_118-pr114068.c | 23 ++++++++ .../vect/vect-early-break_119-pr114068.c | 25 +++++++++ gcc/tree-vect-loop-manip.cc | 52 ++++++++++++++----- 3 files changed, 87 insertions(+), 13 deletions(-) create mode 100644 gcc/testsuite/gcc.dg/vect/vect-early-break_118-pr114068.c create mode 100644 gcc/testsuite/gcc.dg/vect/vect-early-break_119-pr114068.c diff --git a/gcc/testsuite/gcc.dg/vect/vect-early-break_118-pr114068.c b/gcc/testsuite/gcc.dg/vect/vect-early-break_118-pr114068.c new file mode 100644 index 00000000000..b462a464b66 --- /dev/null +++ b/gcc/testsuite/gcc.dg/vect/vect-early-break_118-pr114068.c @@ -0,0 +1,23 @@ +/* { dg-do compile } */ +/* { dg-add-options vect_early_break } */ +/* { dg-require-effective-target vect_early_break } */ +/* { dg-require-effective-target vect_int } */ +/* { dg-additional-options "-O3" } */ + +/* { dg-final { scan-tree-dump "LOOP VECTORIZED" "vect" } } */ + +struct h { + int b; + int f; +} k; + +void n(int m) { + struct h a = k; + for (int o = m; o; ++o) { + if (a.f) + __builtin_unreachable(); + if (o > 1) + __builtin_unreachable(); + *(&k.b + o) = 1; + } +} diff --git a/gcc/testsuite/gcc.dg/vect/vect-early-break_119-pr114068.c b/gcc/testsuite/gcc.dg/vect/vect-early-break_119-pr114068.c new file mode 100644 index 00000000000..a65ef7b8c49 --- /dev/null +++ b/gcc/testsuite/gcc.dg/vect/vect-early-break_119-pr114068.c @@ -0,0 +1,25 @@ +/* { dg-do compile } */ +/* { dg-add-options vect_early_break } */ +/* { dg-require-effective-target vect_early_break } */ +/* { dg-require-effective-target vect_int } */ +/* { dg-additional-options "-O3" } */ + +/* { dg-final { scan-tree-dump "LOOP VECTORIZED" "vect" } } */ + +struct h { + int b; + int c; + int f; +} k; + +void n(int m) { + struct h a = k; + for (int o = m; o; ++o) { + if (a.f) + __builtin_unreachable(); + if (o > 1) + __builtin_unreachable(); + *(&k.b + o) = 1; + *(&k.c + o*m) = 2; + } +} diff --git a/gcc/tree-vect-loop-manip.cc b/gcc/tree-vect-loop-manip.cc index 3f974d6d839..39bac1e99ef 100644 --- a/gcc/tree-vect-loop-manip.cc +++ b/gcc/tree-vect-loop-manip.cc @@ -1429,6 +1429,32 @@ vect_set_loop_condition (class loop *loop, edge loop_e, loop_vec_info loop_vinfo (gimple *) cond_stmt); } +/* Get the virtual operand live on E. The precondition on this is valid + immediate dominators and an actual virtual definition dominating E. */ +/* ??? Costly band-aid. For the use in question we can populate a + live-on-exit/end-of-BB virtual operand when copying stmts. */ + +static tree +get_live_virtual_operand_on_edge (edge e) +{ + basic_block bb = e->src; + do + { + for (auto gsi = gsi_last_bb (bb); !gsi_end_p (gsi); gsi_prev (&gsi)) + { + gimple *stmt = gsi_stmt (gsi); + if (gimple_vdef (stmt)) + return gimple_vdef (stmt); + if (gimple_vuse (stmt)) + return gimple_vuse (stmt); + } + if (gphi *vphi = get_virtual_phi (bb)) + return gimple_phi_result (vphi); + bb = get_immediate_dominator (CDI_DOMINATORS, bb); + } + while (1); +} + /* Given LOOP this function generates a new copy of it and puts it on E which is either the entry or exit of LOOP. If SCALAR_LOOP is non-NULL, assume LOOP and SCALAR_LOOP are equivalent and copy the @@ -1595,6 +1621,18 @@ slpeel_tree_duplicate_loop_to_edge_cfg (class loop *loop, edge loop_exit, flush_pending_stmts (loop_exit); set_immediate_dominator (CDI_DOMINATORS, new_preheader, loop_exit->src); + /* If we ended up choosing an exit leading to a path not using memory + we can end up without a virtual LC PHI. Create it when it is + needed because of the epilog loop continuation. */ + if (need_virtual_phi && !get_virtual_phi (loop_exit->dest)) + { + tree header_def = gimple_phi_result (get_virtual_phi (loop->header)); + gphi *vphi = create_phi_node (copy_ssa_name (header_def), + new_preheader); + add_phi_arg (vphi, get_live_virtual_operand_on_edge (loop_exit), + loop_exit, UNKNOWN_LOCATION); + } + bool multiple_exits_p = loop_exits.length () > 1; basic_block main_loop_exit_block = new_preheader; basic_block alt_loop_exit_block = NULL; @@ -1711,19 +1749,7 @@ slpeel_tree_duplicate_loop_to_edge_cfg (class loop *loop, edge loop_exit, { /* Use the existing virtual LC SSA from exit block. */ gphi *vphi = get_virtual_phi (main_loop_exit_block); - /* ??? When the exit yields to a path without - any virtual use we can miss a LC PHI for the - live virtual operand. Simply choosing the - one live at the start of the loop header isn't - correct, but we should get here only with - early-exit vectorization which will move all - defs after the main exit, so leave a temporarily - wrong virtual operand in place. This happens - for gcc.dg/pr113659.c. */ - if (vphi) - new_arg = gimple_phi_result (vphi); - else - new_arg = gimple_phi_result (from_phi); + new_arg = gimple_phi_result (vphi); } else if ((res = new_phi_args.get (new_arg))) new_arg = *res; -- 2.35.3