From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from nikam.ms.mff.cuni.cz (nikam.ms.mff.cuni.cz [195.113.20.16]) by sourceware.org (Postfix) with ESMTPS id 399F4385DC0B for ; Fri, 21 Jul 2023 17:10:11 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 399F4385DC0B Authentication-Results: sourceware.org; dmarc=fail (p=none dis=none) header.from=ucw.cz Authentication-Results: sourceware.org; spf=none smtp.mailfrom=kam.mff.cuni.cz Received: by nikam.ms.mff.cuni.cz (Postfix, from userid 16202) id 4A9F8282628; Fri, 21 Jul 2023 19:10:09 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ucw.cz; s=gen1; t=1689959409; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=J2DSuFfxkgfc09UnBOAH/0yCRIaBksNrcehAhyI/8/0=; b=pO/qRKhboCIvJpVT3SUOgIazOK6e/X+e2CCYTJXBL1CAaSwqxQAipJfCfH2Y6+iKJgyQ8p jTBDXCphYiXD9Fxq/CV2Pi0F452S2WfMF2dqEbLWIkaBRQyaYE9q+nELtRv0P0H33XO4oC xieiCj/vwszLJiogQV91XBEih0tbPAY= Date: Fri, 21 Jul 2023 19:10:09 +0200 From: Jan Hubicka To: Tamar Christina , gcc-patches@gcc.gnu.org, nd@arm.com, Richard.Earnshaw@arm.com, Marcus.Shawcroft@arm.com, Kyrylo.Tkachov@arm.com, rguenther@suse.de, richard.sandiford@arm.com, kubanek0ondrej@gmail.com Subject: Re: [PATCH]AArch64 fix regexp for live_1.c sve test Message-ID: References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Spam-Status: No, score=-11.3 required=5.0 tests=BAYES_00,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,GIT_PATCH_0,HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE,TXREP,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: Avoid scaling flat loop profiles of vectorized loops As discussed, when vectorizing loop with static profile, it is not always good idea to divide the header frequency by vectorization factor because the profile may not realistically represent the expected number of iterations. Since in such cases we default to relatively low iteration counts (based on average for spec2k17), this will make vectorized loop body look cold. This patch makes vectorizer to look for flat profiles and only possibly reduce the profile by known upper bound on iteration counts. Bootstrapp/regtested of x86_64-linux in progress. I intend to commit this after testers pick other profile related changes from today. Tamar, Richard, it would be nice to know if it fixes the testcase you was looking at and possibly turn it into a testcase? gcc/ChangeLog: * tree-vect-loop.cc (scale_profile_for_vect_loop): Avoid scaling flat profiles by vectorization factor. (vect_transform_loop): Check for flat profiles. diff --git a/gcc/tree-vect-loop.cc b/gcc/tree-vect-loop.cc index b44fb9c7712..d036a7d4480 100644 --- a/gcc/tree-vect-loop.cc +++ b/gcc/tree-vect-loop.cc @@ -10837,11 +10837,25 @@ vect_get_loop_len (loop_vec_info loop_vinfo, gimple_stmt_iterator *gsi, } /* Scale profiling counters by estimation for LOOP which is vectorized - by factor VF. */ + by factor VF. + If FLAT is true, the loop we started with had unrealistically flat + profile. */ static void -scale_profile_for_vect_loop (class loop *loop, unsigned vf) +scale_profile_for_vect_loop (class loop *loop, unsigned vf, bool flat) { + /* For flat profiles do not scale down proportionally by VF and only + cap by known iteration count bounds. */ + if (flat) + { + if (dump_file && (dump_flags & TDF_DETAILS)) + fprintf (dump_file, + "Vectorized loop profile seems flat; not scaling iteration " + "count down by the vectorization factor %i\n", vf); + scale_loop_profile (loop, profile_probability::always (), + get_likely_max_loop_iterations_int (loop)); + return; + } /* Loop body executes VF fewer times and exit increases VF times. */ edge exit_e = single_exit (loop); profile_count entry_count = loop_preheader_edge (loop)->count (); @@ -10852,7 +10866,13 @@ scale_profile_for_vect_loop (class loop *loop, unsigned vf) while (vf > 1 && loop->header->count > entry_count && loop->header->count < entry_count * vf) - vf /= 2; + { + if (dump_file && (dump_flags & TDF_DETAILS)) + fprintf (dump_file, + "Vectorization factor %i seems too large for profile " + "prevoiusly believed to be consistent; reducing.\n", vf); + vf /= 2; + } if (entry_count.nonzero_p ()) set_edge_probability_and_rescale_others @@ -11184,6 +11204,7 @@ vect_transform_loop (loop_vec_info loop_vinfo, gimple *loop_vectorized_call) gimple *stmt; bool check_profitability = false; unsigned int th; + bool flat = maybe_flat_loop_profile (loop); DUMP_VECT_SCOPE ("vec_transform_loop"); @@ -11252,7 +11273,6 @@ vect_transform_loop (loop_vec_info loop_vinfo, gimple *loop_vectorized_call) &step_vector, &niters_vector_mult_vf, th, check_profitability, niters_no_overflow, &advance); - if (LOOP_VINFO_SCALAR_LOOP (loop_vinfo) && LOOP_VINFO_SCALAR_LOOP_SCALING (loop_vinfo).initialized_p ()) scale_loop_frequencies (LOOP_VINFO_SCALAR_LOOP (loop_vinfo), @@ -11545,7 +11565,7 @@ vect_transform_loop (loop_vec_info loop_vinfo, gimple *loop_vectorized_call) assumed_vf) - 1 : wi::udiv_floor (loop->nb_iterations_estimate + bias_for_assumed, assumed_vf) - 1); - scale_profile_for_vect_loop (loop, assumed_vf); + scale_profile_for_vect_loop (loop, assumed_vf, flat); if (dump_enabled_p ()) {