From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) by sourceware.org (Postfix) with ESMTPS id 0D7533858D28; Wed, 8 Dec 2021 06:32:58 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 0D7533858D28 Received: from pps.filterd (m0098420.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 1B85MSgH000413; Wed, 8 Dec 2021 06:32:57 GMT Received: from pps.reinject (localhost [127.0.0.1]) by mx0b-001b2d01.pphosted.com with ESMTP id 3ctpgm93qb-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 08 Dec 2021 06:32:57 +0000 Received: from m0098420.ppops.net (m0098420.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.43/8.16.0.43) with SMTP id 1B86UDJl010771; Wed, 8 Dec 2021 06:32:57 GMT Received: from ppma04ams.nl.ibm.com (63.31.33a9.ip4.static.sl-reverse.com [169.51.49.99]) by mx0b-001b2d01.pphosted.com with ESMTP id 3ctpgm93pw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 08 Dec 2021 06:32:56 +0000 Received: from pps.filterd (ppma04ams.nl.ibm.com [127.0.0.1]) by ppma04ams.nl.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 1B86DECE026975; Wed, 8 Dec 2021 06:32:55 GMT Received: from b06avi18878370.portsmouth.uk.ibm.com (b06avi18878370.portsmouth.uk.ibm.com [9.149.26.194]) by ppma04ams.nl.ibm.com with ESMTP id 3cqyyavca0-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 08 Dec 2021 06:32:54 +0000 Received: from d06av23.portsmouth.uk.ibm.com (d06av23.portsmouth.uk.ibm.com [9.149.105.59]) by b06avi18878370.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 1B86Wp7N33030438 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 8 Dec 2021 06:32:51 GMT Received: from d06av23.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 5C15DA4057; Wed, 8 Dec 2021 06:32:51 +0000 (GMT) Received: from d06av23.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 1368DA4065; Wed, 8 Dec 2021 06:32:49 +0000 (GMT) Received: from [9.200.154.17] (unknown [9.200.154.17]) by d06av23.portsmouth.uk.ibm.com (Postfix) with ESMTPS; Wed, 8 Dec 2021 06:32:48 +0000 (GMT) Message-ID: <5b652ae5-82c9-43e0-a364-e73846d0c695@linux.ibm.com> Date: Wed, 8 Dec 2021 14:32:46 +0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:91.0) Gecko/20100101 Thunderbird/91.3.2 Subject: Re: [PATCH v8 2/2] Don't move cold code out of loop by checking bb count Content-Language: en-US To: Richard Biener Cc: Segher Boessenkool , Bill Schmidt , linkw@gcc.gnu.org, GCC Patches , Jan Hubicka , David Edelsohn References: <20210802050501.159058-1-luoxhu@linux.ibm.com> <53b7c729-33c0-138f-fa06-d6efb7a43911@linux.ibm.com> <0b675ba1-cdab-652a-0bba-704b0090f1c5@linux.ibm.com> <2f0aee4a-dfea-d04d-6179-4d4623702f0a@linux.ibm.com> <409e3be5-0bfd-cdc4-2d93-9dbacdbc69cf@linux.ibm.com> <4a2eb212-6e8b-5021-3c23-ec550723f41e@linux.ibm.com> <47ec43f7-f260-fdaf-ac57-7b89fd9ceab2@linux.ibm.com> From: Xionghu Luo In-Reply-To: Content-Type: text/plain; charset=UTF-8 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: g071GQGc82-faazM-T_dILoxRmrAr87g X-Proofpoint-ORIG-GUID: fDsArViD2UGYRJEl0aKBuxUxWGuuoead Content-Transfer-Encoding: 7bit X-Proofpoint-UnRewURL: 0 URL was un-rewritten MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.790,Hydra:6.0.425,FMLib:17.11.62.513 definitions=2021-12-08_02,2021-12-06_02,2021-12-02_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 impostorscore=0 adultscore=0 suspectscore=0 lowpriorityscore=0 malwarescore=0 phishscore=0 bulkscore=0 mlxlogscore=999 priorityscore=1501 mlxscore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2110150000 definitions=main-2112080044 X-Spam-Status: No, score=-13.4 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_EF, GIT_PATCH_0, KAM_SHORT, NICE_REPLY_A, RCVD_IN_MSPIKE_H4, RCVD_IN_MSPIKE_WL, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Dec 2021 06:32:59 -0000 On 2021/12/7 20:17, Richard Biener wrote: >>> + class loop *coldest_loop = coldest_outermost_loop[loop->num]; >>> + if (loop_depth (coldest_loop) < loop_depth (outermost_loop)) >>> + { >>> + class loop *hotter_loop = hotter_than_inner_loop[loop->num]; >>> + if (!hotter_loop >>> + || loop_depth (hotter_loop) < loop_depth (outermost_loop)) >>> + return outermost_loop; >>> + >>> + /* hotter_loop is between OUTERMOST_LOOP and LOOP like: >>> + [loop tree root, ..., coldest_loop, ..., outermost_loop, ..., >>> + hotter_loop, second_coldest_loop, ..., loop] >>> + return second_coldest_loop to be the hoist target. */ >>> + class loop *aloop; >>> + for (aloop = hotter_loop->inner; aloop; aloop = aloop->next) >>> + if (flow_loop_nested_p (aloop, loop)) >> should be: >> >> if (aloop == loop || flow_loop_nested_p (aloop, loop)) > OK with that fixed. > > Are necessary prerequesites committed to avoid regressions? > I guess we need to keep a watchful eye and eventually revert > (or gate with a --param disabled by default) the new behavior if > severe regressions are discovered. > > Thanks and sorry for the repeated delays. > Richard. > Thanks for your review, I learned quite a lot and gained very useful comments & help through the period :) There are still 3 patches required to avoid regression or so, I've reorganized them and sent it out. https://gcc.gnu.org/pipermail/gcc-patches/2021-December/586371.html In addition, cooked the patch to add option for disable/enable it. Is it OK to merge it to current patch? [PATCH] Add option -fhoist-to-cold-loop gcc/ChangeLog: * common.opt: New. * loop-invariant.c (find_invariants_bb): * tree-ssa-loop-im.c (get_coldest_out_loop): (can_sm_ref_p): (loop_invariant_motion_in_fun): --- gcc/common.opt | 4 ++++ gcc/loop-invariant.c | 2 +- gcc/tree-ssa-loop-im.c | 33 ++++++++++++++++++++++----------- 3 files changed, 27 insertions(+), 12 deletions(-) diff --git a/gcc/common.opt b/gcc/common.opt index b921f5e3b25..62b82bd8b95 100644 --- a/gcc/common.opt +++ b/gcc/common.opt @@ -1171,6 +1171,10 @@ fcode-hoisting Common Var(flag_code_hoisting) Optimization Enable code hoisting. +fhoist-to-cold-loop +Common Var(flag_hoist_to_cold_loop) Init(1) Optimization +Enable hoisting code to cold loop. + fcombine-stack-adjustments Common Var(flag_combine_stack_adjustments) Optimization Looks for opportunities to reduce stack adjustments and stack references. diff --git a/gcc/loop-invariant.c b/gcc/loop-invariant.c index 5c3be7bf0eb..75b9dd47cd7 100644 --- a/gcc/loop-invariant.c +++ b/gcc/loop-invariant.c @@ -1189,7 +1189,7 @@ find_invariants_bb (class loop *loop, basic_block bb, bool always_reached, rtx_insn *insn; basic_block preheader = loop_preheader_edge (loop)->src; - if (preheader->count > bb->count) + if (flag_hoist_to_cold_loop && preheader->count > bb->count) return; FOR_BB_INSNS (bb, insn) diff --git a/gcc/tree-ssa-loop-im.c b/gcc/tree-ssa-loop-im.c index 565ee62d3f7..d745f66851b 100644 --- a/gcc/tree-ssa-loop-im.c +++ b/gcc/tree-ssa-loop-im.c @@ -450,6 +450,9 @@ static class loop * get_coldest_out_loop (class loop *outermost_loop, class loop *loop, basic_block curr_bb) { + if (!flag_hoist_to_cold_loop) + return outermost_loop; + gcc_assert (outermost_loop == loop || flow_loop_nested_p (outermost_loop, loop)); @@ -3031,8 +3034,9 @@ can_sm_ref_p (class loop *loop, im_mem_ref *ref) /* Verify whether the candidate is hot for LOOP. Only do store motion if the candidate's profile count is hot. Statement in cold BB shouldn't be moved out of it's loop_father. */ - if (!for_all_locs_in_loop (loop, ref, ref_in_loop_hot_body (loop))) - return false; + if (flag_hoist_to_cold_loop) + if (!for_all_locs_in_loop (loop, ref, ref_in_loop_hot_body (loop))) + return false; return true; } @@ -3373,8 +3377,11 @@ tree_ssa_lim_finalize (void) free (bb_loop_postorder); - coldest_outermost_loop.release (); - hotter_than_inner_loop.release (); + if (flag_hoist_to_cold_loop) + { + coldest_outermost_loop.release (); + hotter_than_inner_loop.release (); + } } /* Moves invariants from loops. Only "expensive" invariants are moved out -- @@ -3396,13 +3403,17 @@ loop_invariant_motion_in_fun (function *fun, bool store_motion) /* Pre-compute coldest outermost loop and nearest hotter loop of each loop. */ - class loop *loop; - coldest_outermost_loop.create (number_of_loops (cfun)); - coldest_outermost_loop.safe_grow_cleared (number_of_loops (cfun)); - hotter_than_inner_loop.create (number_of_loops (cfun)); - hotter_than_inner_loop.safe_grow_cleared (number_of_loops (cfun)); - for (loop = current_loops->tree_root->inner; loop != NULL; loop = loop->next) - fill_coldest_and_hotter_out_loop (loop, NULL, loop); + if (flag_hoist_to_cold_loop) + { + class loop *loop; + coldest_outermost_loop.create (number_of_loops (cfun)); + coldest_outermost_loop.safe_grow_cleared (number_of_loops (cfun)); + hotter_than_inner_loop.create (number_of_loops (cfun)); + hotter_than_inner_loop.safe_grow_cleared (number_of_loops (cfun)); + for (loop = current_loops->tree_root->inner; loop != NULL; + loop = loop->next) + fill_coldest_and_hotter_out_loop (loop, NULL, loop); + } int *rpo = XNEWVEC (int, last_basic_block_for_fn (fun)); int n = pre_and_rev_post_order_compute_fn (fun, NULL, rpo, false); -- 2.27.0.90.geebb51ba8c