From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR05-DB8-obe.outbound.protection.outlook.com (mail-db8eur05on2084.outbound.protection.outlook.com [40.107.20.84]) by sourceware.org (Postfix) with ESMTPS id 291C93858CDA for ; Wed, 11 Oct 2023 10:54:59 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 291C93858CDA Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=arm.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Zz1NLbnjCibn/agp3LRTsoVouqNUqrrq090KcyP+lQ4=; b=ncmQmXRVjRoFmASv9ujcqKxAJ0nnCeaaS8pZB8swfHM29/CZvQU3UiJ9/7vc5TRWbbTx0/+B79J8pO4jSY7hbYP4Fq+itLqjwBSkO+9OaFdwLcadbrZSnsIdWplEG06Yuv/cckwVlUpFwev5jcm+jLYOWou7y3rC5w+Q403VwCk= Received: from DBBPR09CA0035.eurprd09.prod.outlook.com (2603:10a6:10:d4::23) by GV2PR08MB7956.eurprd08.prod.outlook.com (2603:10a6:150:a9::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6863.36; Wed, 11 Oct 2023 10:54:55 +0000 Received: from DBAEUR03FT051.eop-EUR03.prod.protection.outlook.com (2603:10a6:10:d4:cafe::af) by DBBPR09CA0035.outlook.office365.com (2603:10a6:10:d4::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6886.27 via Frontend Transport; Wed, 11 Oct 2023 10:54:55 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; pr=C Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by DBAEUR03FT051.mail.protection.outlook.com (100.127.142.148) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6886.27 via Frontend Transport; Wed, 11 Oct 2023 10:54:55 +0000 Received: ("Tessian outbound 0ae75d4034ba:v211"); Wed, 11 Oct 2023 10:54:55 +0000 X-CheckRecipientChecked: true X-CR-MTA-CID: 4692f7b760145641 X-CR-MTA-TID: 64aa7808 Received: from e0b19a31a6dd.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 4A0BC226-99CE-4D7E-9650-A1A8A8AF0BB9.1; Wed, 11 Oct 2023 10:54:48 +0000 Received: from EUR02-VI1-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id e0b19a31a6dd.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Wed, 11 Oct 2023 10:54:48 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=WxZ1HfYlDILW9NV7sXfZSn5aGSVgzsI4tp6/gwVf6lrM5bj3NRwSgG9CH657GtorjfxMQ4k15wqO1l+sNS74YISwLHd5f9LYuPJwbmgjGJWGpVj9O4rRlzwoVJqzn8vOeaNVuIpYZL2VU5gahY9LSJO+Ff1+c0lGGSlKTBtqQpp8EdGIPR2JCM2oKEdt7f9nipQXGWZZQ9dIH6Z3s4A01nHIr2ftKa2MRK2EbMK+A6wYB9SFndOeFdurCvAtc97w2kq5+iAiKaXHRgQ8MTR3pJoIHPHo9k2oYxDrYLFGO6bbUfanJN60YBmWmbLq90Q0i+Hg6VurCDTasFVzvDRJVg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Zz1NLbnjCibn/agp3LRTsoVouqNUqrrq090KcyP+lQ4=; b=X5OWWEzRM0ToeL0QvVDzL4l/7J5qnOpm5sekbEwmqrIaJpnEKMkjwXuwbCqdfauWiy8grsfKSybW6Xp0wj0iuG1hAaW741ibF+vVADbfnMlYZXGdz8p9l17Rp+pfrsjfIeNTwKL9VZTLlhlzdQJ0Y2jTEQF7W65c/zmBIIaunaI2HOTETjZXQm6zFjhwNZHoX6KrPTnU2ChOdAhsKRatTe+9sW6MED4yNtn0hri7MicisLmC3XyKNoH7e4rnrvja5ohanvxRnRPq5PKLoXWXX8zL2tEjz0xYGBVoowOhilbG0t7p61tLaBYXIgT5l+qBtUPMbCzGDsDgDIJ05o2yhg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Zz1NLbnjCibn/agp3LRTsoVouqNUqrrq090KcyP+lQ4=; b=ncmQmXRVjRoFmASv9ujcqKxAJ0nnCeaaS8pZB8swfHM29/CZvQU3UiJ9/7vc5TRWbbTx0/+B79J8pO4jSY7hbYP4Fq+itLqjwBSkO+9OaFdwLcadbrZSnsIdWplEG06Yuv/cckwVlUpFwev5jcm+jLYOWou7y3rC5w+Q403VwCk= Received: from VI1PR08MB5325.eurprd08.prod.outlook.com (2603:10a6:803:13e::17) by DU0PR08MB10367.eurprd08.prod.outlook.com (2603:10a6:10:409::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6863.38; Wed, 11 Oct 2023 10:54:45 +0000 Received: from VI1PR08MB5325.eurprd08.prod.outlook.com ([fe80::bba1:2711:6992:468d]) by VI1PR08MB5325.eurprd08.prod.outlook.com ([fe80::bba1:2711:6992:468d%4]) with mapi id 15.20.6863.032; Wed, 11 Oct 2023 10:54:44 +0000 From: Tamar Christina To: Richard Biener CC: "gcc-patches@gcc.gnu.org" , nd , "jlaw@ventanamicro.com" Subject: RE: [PATCH 2/3]middle-end: updated niters analysis to handle multiple exits. Thread-Topic: [PATCH 2/3]middle-end: updated niters analysis to handle multiple exits. Thread-Index: AQHZ9QPruuL3x6j9dkWlz6DVEiSnQ7BC67IAgAGLD4A= Date: Wed, 11 Oct 2023 10:54:44 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: Authentication-Results-Original: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; x-ms-traffictypediagnostic: VI1PR08MB5325:EE_|DU0PR08MB10367:EE_|DBAEUR03FT051:EE_|GV2PR08MB7956:EE_ X-MS-Office365-Filtering-Correlation-Id: 01a130c6-9c9c-425a-7c36-08dbca488112 x-checkrecipientrouted: true nodisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: 2O2qP4oUuVhoOwWcX3VxvFyjIoyuAaBf18JKz0LbK39ALg2MjeKcHz5G9UHbJnQpfPLmkHaaGQfXV6JrUQQqZ8amAjb6YvnoCwAnmLv+KF1M7fVvckXDAdzWLyJvsjIK1EBqNjOo2Atzx/Ghy6/mQ/KjFReArXOm0OjY7bjxMRSoMbsFljkzdSc4f3VYOpTq/VRVD2JqrHviM1OdnX7cxjA3x7fMZA0lqTQYWpAKUd6fnhVjFB0IZZ5wvlrA+ebM+W8vqs4ts3ijBB63uHcg+78hB6a79IQx+qOpEU26Eo+04jRSkkD3eayVAxY6hJTodkEg8nIk0kHrldDrRa6iyZ8v59+NXw9NNEZTALfr3BLqGjQcp3g03DJYwmJU/ddD6w6vhJTFW22rBNLz1B+DDfSjty5GKfi4aBcV42anHSBk1loNz24AW+q2QHoxppipytX4T9JRW+3NR+KR4CZ7dztA9RmmwptEA0a7LTM3vjIlFKXU5KEkI5nB9rNnxfEQffSP0x69nnIFb6j9kiH9i9PqX2xRYhjoiAeFJFpOHojzL6uUvrWquhtSIPWaWFafXWbaYL3XD8Wblvl2aPw2Kf9Jyt8SBk9WQsk278R3TkrWWlsFxlg3cpf3xy4lpf9K X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR08MB5325.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(396003)(376002)(136003)(39860400002)(346002)(366004)(230922051799003)(64100799003)(1800799009)(186009)(451199024)(53546011)(9686003)(6506007)(478600001)(15650500001)(4326008)(71200400001)(66556008)(26005)(83380400001)(30864003)(7696005)(66476007)(2906002)(54906003)(5660300002)(316002)(66946007)(66446008)(64756008)(6916009)(76116006)(8936002)(41300700001)(8676002)(52536014)(38100700002)(33656002)(86362001)(38070700005)(122000001)(55016003);DIR:OUT;SFP:1101; Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU0PR08MB10367 Original-Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: DBAEUR03FT051.eop-EUR03.prod.protection.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: 58acb12e-0eac-4488-0563-08dbca487ac4 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: kTNtRMqXEDn/ZlIWimmWvSyAS/k/tVzG7UwEjSe/A70REfI66JvhcP1E09dkRiiQ2arb9Xl7eRyfg7Y04Z81H0y2VsNL9p2M91YgLrL4uGXVyGBxopHmpclLvoAjxAOWaRcwa/N9D/ubGQDLGpRLMy4iWbXY6e0o8tt9Yq6UI0Xe5ddU511Cd0IA5XcFZiBpIohp/vUJlLlU3/XMqeHZZS0QRRz2XUJcJWXrZZLuin5VtZss2A0jMh38BI3Gb6fNEuwcc00UWcdi8IsVqBwYt9LJr5ao5g/XKCZ0cn8SwGksoOVJLSNE094j19K/gULxEiCNnoBk9jEYgmfHiy7qBWcZdD4S65I8sP6rhFDaKV2EuaPIADFYq1Ry6S68WRhL5iS/debpJxJL/D0mm8J20z1YVCw7B5gb6m1qeeeGMcjEBf9LSLfTyLNBVaeLJ6YQKblOAB1dr4D8RQUul5qTOk4lsEOTt9b0o3BfHafZzre0X/SuesWlso8KE2EmVXrexm0xR4hpkreXAO5VKjrhZv6G2S89rlDRR+qSKMbP4chh2n4q7zoDgWe6LXKYhUOY7xY9Sf0lN0OyHBDL9IPHduSiFQXJ/kq+XCqG0ZB8lf9Exf1TtUcAr3tNeWXTySUrbbuSQKH2Yt7sgGAONCUAzIV5wAAO0fyeJTp0WIsm/ZIPLpwc8cpevk8ZIra66PmeqYY3J/ZfuQO5KaQyJQ+sSx68GZZXhSHwlrdu3noGGKJk22GPJ1Z2jUEcj8GphVdW X-Forefront-Antispam-Report: CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(13230031)(4636009)(376002)(396003)(39860400002)(136003)(346002)(230922051799003)(82310400011)(64100799003)(186009)(451199024)(1800799009)(40470700004)(36840700001)(46966006)(53546011)(336012)(81166007)(26005)(107886003)(7696005)(8676002)(47076005)(15650500001)(6862004)(8936002)(2906002)(478600001)(4326008)(30864003)(6506007)(70206006)(52536014)(70586007)(54906003)(5660300002)(316002)(41300700001)(55016003)(40460700003)(82740400003)(356005)(86362001)(40480700001)(33656002)(36860700001)(9686003)(83380400001);DIR:OUT;SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Oct 2023 10:54:55.4360 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 01a130c6-9c9c-425a-7c36-08dbca488112 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: DBAEUR03FT051.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: GV2PR08MB7956 X-Spam-Status: No, score=-12.0 required=5.0 tests=BAYES_00,DKIM_SIGNED,DKIM_VALID,FORGED_SPF_HELO,GIT_PATCH_0,KAM_DMARC_NONE,KAM_LOTSOFHASH,RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE,TXREP,UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: > -----Original Message----- > From: Richard Biener > Sent: Tuesday, October 10, 2023 12:14 PM > To: Tamar Christina > Cc: gcc-patches@gcc.gnu.org; nd ; jlaw@ventanamicro.com > Subject: Re: [PATCH 2/3]middle-end: updated niters analysis to handle > multiple exits. >=20 > On Mon, 2 Oct 2023, Tamar Christina wrote: >=20 > > Hi All, > > > > This second part updates niters analysis to be able to analyze any > > number of exits. If we have multiple exits we determine the main exit > > by finding the first counting IV. > > > > The change allows the vectorizer to pass analysis for multiple loops, > > but we later gracefully reject them. It does however allow us to test > > if the exit handling is using the right exit everywhere. > > > > Additionally since we analyze all exits, we now return all conditions > > for them and determine which condition belongs to the main exit. > > > > The main condition is needed because the vectorizer needs to ignore > > the main IV condition during vectorization as it will replace it during= codegen. > > > > To track versioned loops we extend the contract between ifcvt and the > > vectorizer to store the exit number in aux so that we can match it up a= gain > during peeling. > > > > Bootstrapped Regtested on aarch64-none-linux-gnu, x86_64-linux-gnu, > > and no issues. > > > > Ok for master? > > > > Thanks, > > Tamar > > > > gcc/ChangeLog: > > > > * tree-if-conv.cc (tree_if_conversion): Record exits in aux. > > * tree-vect-loop-manip.cc (slpeel_tree_duplicate_loop_to_edge_cfg): > Use > > it. > > * tree-vect-loop.cc (vect_get_loop_niters): Determine main exit. > > (vec_init_loop_exit_info): Extend analysis when multiple exits. > > (vect_analyze_loop_form): Record conds and determine main cond. > > (vect_create_loop_vinfo): Extend bookkeeping of conds. > > (vect_analyze_loop): Release conds. > > * tree-vectorizer.h (LOOP_VINFO_LOOP_CONDS, > > LOOP_VINFO_LOOP_IV_COND): New. > > (struct vect_loop_form_info): Add conds, alt_loop_conds; > > (struct loop_vec_info): Add conds, loop_iv_cond. > > > > --- inline copy of patch -- > > diff --git a/gcc/tree-if-conv.cc b/gcc/tree-if-conv.cc index > > > 799f071965e5c41eb352b5530cf1d9c7ecf7bf25..3dc2290467797ebbfcef55 > 903531 > > b22829f4fdbd 100644 > > --- a/gcc/tree-if-conv.cc > > +++ b/gcc/tree-if-conv.cc > > @@ -3795,6 +3795,13 @@ tree_if_conversion (class loop *loop, > vec *preds) > > } > > if (need_to_ifcvt) > > { > > + /* Before we rewrite edges we'll record their original position = in the > > + edge map such that we can map the edges between the ifcvt and the > > + non-ifcvt loop during peeling. */ > > + uintptr_t idx =3D 0; > > + for (edge exit : get_loop_exit_edges (loop)) > > + exit->aux =3D (void*)idx++; > > + > > /* Now all statements are if-convertible. Combine all the basic > > blocks into one huge basic block doing the if-conversion > > on-the-fly. */ > > diff --git a/gcc/tree-vect-loop-manip.cc b/gcc/tree-vect-loop-manip.cc > > index > > > e06717272aafc6d31cbdcb94840ac25de616da6d..77f8e668bcc8beca99ba4 > 052e1b1 > > 2e0d17300262 100644 > > --- a/gcc/tree-vect-loop-manip.cc > > +++ b/gcc/tree-vect-loop-manip.cc > > @@ -1470,6 +1470,18 @@ slpeel_tree_duplicate_loop_to_edge_cfg (class > loop *loop, edge loop_exit, > > scalar_loop =3D loop; > > scalar_exit =3D loop_exit; > > } > > + else if (scalar_loop =3D=3D loop) > > + scalar_exit =3D loop_exit; > > + else > > + { > > + /* Loop has been version, match exits up using the aux index. *= / > > + for (edge exit : get_loop_exit_edges (scalar_loop)) > > + if (exit->aux =3D=3D loop_exit->aux) > > + { > > + scalar_exit =3D exit; > > + break; > > + } > > + } > > > > bbs =3D XNEWVEC (basic_block, scalar_loop->num_nodes + 1); > > pbbs =3D bbs + 1; > > @@ -1501,6 +1513,8 @@ slpeel_tree_duplicate_loop_to_edge_cfg (class > loop *loop, edge loop_exit, > > exit =3D loop_exit; > > basic_block new_preheader =3D new_bbs[0]; > > > > + /* Record the new loop exit information. new_loop doesn't have SCEV > data and > > + so we must initialize the exit information. */ > > if (new_e) > > *new_e =3D new_exit; > > > > diff --git a/gcc/tree-vect-loop.cc b/gcc/tree-vect-loop.cc index > > > 6e60d84143626a8e1d801bb580f4dcebc73c7ba7..f1caa5f207d3b13da58c3 > a313b11 > > d1ef98374349 100644 > > --- a/gcc/tree-vect-loop.cc > > +++ b/gcc/tree-vect-loop.cc > > @@ -851,79 +851,106 @@ vect_fixup_scalar_cycles_with_patterns > (loop_vec_info loop_vinfo) > > in NUMBER_OF_ITERATIONSM1. Place the condition under which the > > niter information holds in ASSUMPTIONS. > > > > - Return the loop exit condition. */ > > + Return the loop exit conditions. */ > > > > > > -static gcond * > > -vect_get_loop_niters (class loop *loop, edge exit, tree *assumptions, > > +static vec > > +vect_get_loop_niters (class loop *loop, tree *assumptions, const_edge > > +main_exit, > > tree *number_of_iterations, tree > *number_of_iterationsm1) >=20 > Any reason you swap exit and main_exit? IMHO the input better pairs with > the other input 'loop'. >=20 No, I think I was just rearranging thing to fit more on a line. I'll put t= hem next to their exits. >=20 > > { > > + auto_vec exits =3D get_loop_exit_edges (loop); vec > > + conds; conds.create (exits.length ()); > > class tree_niter_desc niter_desc; > > tree niter_assumptions, niter, may_be_zero; > > - gcond *cond =3D get_loop_exit_condition (loop); > > > > *assumptions =3D boolean_true_node; > > *number_of_iterationsm1 =3D chrec_dont_know; > > *number_of_iterations =3D chrec_dont_know; > > + > > DUMP_VECT_SCOPE ("get_loop_niters"); > > > > - if (!exit) > > - return cond; > > + if (exits.is_empty ()) > > + return conds; > > + > > + if (dump_enabled_p ()) > > + dump_printf_loc (MSG_NOTE, vect_location, "Loop has %d exits.\n", > > + exits.length ()); > > + > > + edge exit; > > + unsigned int i; > > + FOR_EACH_VEC_ELT (exits, i, exit) > > + { > > + gcond *cond =3D get_loop_exit_condition (exit); > > + if (cond) > > + conds.safe_push (cond); > > + > > + if (dump_enabled_p ()) > > + dump_printf_loc (MSG_NOTE, vect_location, "Analyzing exit %d...\n", > > +i); > > > > - may_be_zero =3D NULL_TREE; > > - if (!number_of_iterations_exit_assumptions (loop, exit, &niter_desc, > NULL) > > - || chrec_contains_undetermined (niter_desc.niter)) > > - return cond; > > + may_be_zero =3D NULL_TREE; > > + if (!number_of_iterations_exit_assumptions (loop, exit, &niter_d= esc, > NULL) > > + || chrec_contains_undetermined (niter_desc.niter)) > > + continue; > > > > - niter_assumptions =3D niter_desc.assumptions; > > - may_be_zero =3D niter_desc.may_be_zero; > > - niter =3D niter_desc.niter; > > + niter_assumptions =3D niter_desc.assumptions; > > + may_be_zero =3D niter_desc.may_be_zero; > > + niter =3D niter_desc.niter; > > > > - if (may_be_zero && integer_zerop (may_be_zero)) > > - may_be_zero =3D NULL_TREE; > > + if (may_be_zero && integer_zerop (may_be_zero)) > > + may_be_zero =3D NULL_TREE; > > > > - if (may_be_zero) > > - { > > - if (COMPARISON_CLASS_P (may_be_zero)) > > + if (may_be_zero) > > { > > - /* Try to combine may_be_zero with assumptions, this can simplify > > - computation of niter expression. */ > > - if (niter_assumptions && !integer_nonzerop (niter_assumptions)) > > - niter_assumptions =3D fold_build2 (TRUTH_AND_EXPR, > boolean_type_node, > > - niter_assumptions, > > - fold_build1 (TRUTH_NOT_EXPR, > > - boolean_type_node, > > - may_be_zero)); > > + if (COMPARISON_CLASS_P (may_be_zero)) > > + { > > + /* Try to combine may_be_zero with assumptions, this can simpli= fy > > + computation of niter expression. */ > > + if (niter_assumptions && !integer_nonzerop (niter_assumptions)) > > + niter_assumptions =3D fold_build2 (TRUTH_AND_EXPR, > boolean_type_node, > > + niter_assumptions, > > + fold_build1 > (TRUTH_NOT_EXPR, > > + > boolean_type_node, > > + may_be_zero)); > > + else > > + niter =3D fold_build3 (COND_EXPR, TREE_TYPE (niter), > may_be_zero, > > + build_int_cst (TREE_TYPE (niter), 0), > > + rewrite_to_non_trapping_overflow (niter)); > > + > > + may_be_zero =3D NULL_TREE; > > + } > > + else if (integer_nonzerop (may_be_zero) && exit =3D=3D main_exit) > > + { > > + *number_of_iterationsm1 =3D build_int_cst (TREE_TYPE (niter), 0= ); > > + *number_of_iterations =3D build_int_cst (TREE_TYPE (niter), 1); > > + continue; > > + } > > else > > - niter =3D fold_build3 (COND_EXPR, TREE_TYPE (niter), may_be_zero, > > - build_int_cst (TREE_TYPE (niter), 0), > > - rewrite_to_non_trapping_overflow (niter)); > > + continue; > > + } > > > > - may_be_zero =3D NULL_TREE; > > - } > > - else if (integer_nonzerop (may_be_zero)) > > + /* Loop assumptions are based off the normal exit. */ > > + if (exit =3D=3D main_exit) >=20 > It's a bit hard to follow in patch form but I wonder why you even analyze= the > number of iterations of the non-main exits riskying possibly clobbering t= he > *number_* outputs which we later assume to be for the main exit? >=20 My original goal here was that if we can't analyze the other exits, we prob= ably can't vectorize them. So I don't really need the results but I thought it u= seful to check. I can skip them. Thanks, Tamar > > { > > - *number_of_iterationsm1 =3D build_int_cst (TREE_TYPE (niter), 0); > > - *number_of_iterations =3D build_int_cst (TREE_TYPE (niter), 1); > > - return cond; > > + *assumptions =3D niter_assumptions; > > + *number_of_iterationsm1 =3D niter; > > + > > + /* We want the number of loop header executions which is the > number > > + of latch executions plus one. > > + ??? For UINT_MAX latch executions this number overflows to zero > > + for loops like do { n++; } while (n !=3D 0); */ > > + if (niter && !chrec_contains_undetermined (niter)) > > + niter =3D fold_build2 (PLUS_EXPR, TREE_TYPE (niter), > > + unshare_expr (niter), > > + build_int_cst (TREE_TYPE (niter), 1)); > > + *number_of_iterations =3D niter; > > } > > - else > > - return cond; > > } > > > > - *assumptions =3D niter_assumptions; > > - *number_of_iterationsm1 =3D niter; > > - > > - /* We want the number of loop header executions which is the number > > - of latch executions plus one. > > - ??? For UINT_MAX latch executions this number overflows to zero > > - for loops like do { n++; } while (n !=3D 0); */ > > - if (niter && !chrec_contains_undetermined (niter)) > > - niter =3D fold_build2 (PLUS_EXPR, TREE_TYPE (niter), unshare_expr = (niter), > > - build_int_cst (TREE_TYPE (niter), 1)); > > - *number_of_iterations =3D niter; > > + if (dump_enabled_p ()) > > + dump_printf_loc (MSG_NOTE, vect_location, "All loop exits > > + successfully analyzed.\n"); > > > > - return cond; > > + return conds; > > } > > > > /* Determine the main loop exit for the vectorizer. */ @@ -936,8 > > +963,25 @@ vec_init_loop_exit_info (class loop *loop) > > auto_vec exits =3D get_loop_exit_edges (loop); > > if (exits.length () =3D=3D 1) > > return exits[0]; > > - else > > - return NULL; > > + > > + /* If we have multiple exits we only support counting IV at the mome= nt. > Analyze > > + all exits and return one */ > > + class tree_niter_desc niter_desc; > > + edge candidate =3D NULL; > > + for (edge exit : exits) > > + { > > + if (!get_loop_exit_condition (exit)) > > + continue; > > + > > + if (number_of_iterations_exit_assumptions (loop, exit, &niter_de= sc, > NULL) > > + && !chrec_contains_undetermined (niter_desc.niter)) > > + { > > + if (!niter_desc.may_be_zero || !candidate) > > + candidate =3D exit; > > + } > > + } > > + > > + return candidate; > > } > > > > /* Function bb_in_loop_p > > @@ -1788,21 +1832,31 @@ vect_analyze_loop_form (class loop *loop, > vect_loop_form_info *info) > > "not vectorized: latch block not empty.\n"); > > > > /* Make sure the exit is not abnormal. */ > > - edge e =3D single_exit (loop); > > - if (e->flags & EDGE_ABNORMAL) > > + if (exit_e->flags & EDGE_ABNORMAL) > > return opt_result::failure_at (vect_location, > > "not vectorized:" > > " abnormal loop exit edge.\n"); > > > > - info->loop_cond > > - =3D vect_get_loop_niters (loop, e, &info->assumptions, > > + info->conds > > + =3D vect_get_loop_niters (loop, &info->assumptions, exit_e, > > &info->number_of_iterations, > > &info->number_of_iterationsm1); > > - if (!info->loop_cond) > > + > > + if (info->conds.is_empty ()) > > return opt_result::failure_at > > (vect_location, > > "not vectorized: complicated exit condition.\n"); > > > > + /* Determine what the primary and alternate exit conds are. */ > > + info->alt_loop_conds.create (info->conds.length () - 1); > > + for (gcond *cond : info->conds) > > + { > > + if (exit_e->src !=3D gimple_bb (cond)) > > + info->alt_loop_conds.quick_push (cond); > > + else > > + info->loop_cond =3D cond; > > + } > > + >=20 > IMHO it would be simpler to have the primary exit condition in > info->conds[0] and the rest after that? That avoids having two > arrays and one scalar in vect_loop_form_info. >=20 > > if (integer_zerop (info->assumptions) > > || !info->number_of_iterations > > || chrec_contains_undetermined (info->number_of_iterations)) @@ > > -1847,8 +1901,13 @@ vect_create_loop_vinfo (class loop *loop, > vec_info_shared *shared, > > if (!integer_onep (info->assumptions) && !main_loop_info) > > LOOP_VINFO_NITERS_ASSUMPTIONS (loop_vinfo) =3D info->assumptions; > > > > - stmt_vec_info loop_cond_info =3D loop_vinfo->lookup_stmt > > (info->loop_cond); > > - STMT_VINFO_TYPE (loop_cond_info) =3D loop_exit_ctrl_vec_info_type; > > + for (gcond *cond : info->conds) > > + { > > + stmt_vec_info loop_cond_info =3D loop_vinfo->lookup_stmt (cond); > > + STMT_VINFO_TYPE (loop_cond_info) =3D loop_exit_ctrl_vec_info_typ= e; > > + } > > + LOOP_VINFO_LOOP_CONDS (loop_vinfo).safe_splice > > + (info->alt_loop_conds); LOOP_VINFO_LOOP_IV_COND (loop_vinfo) =3D > > + info->loop_cond; > > LOOP_VINFO_IV_EXIT (loop_vinfo) =3D info->loop_exit; > > > > @@ -3594,7 +3653,11 @@ vect_analyze_loop (class loop *loop, > vec_info_shared *shared) > > && LOOP_VINFO_PEELING_FOR_NITER > (first_loop_vinfo) > > && !loop->simduid); > > if (!vect_epilogues) > > - return first_loop_vinfo; > > + { > > + loop_form_info.conds.release (); > > + loop_form_info.alt_loop_conds.release (); > > + return first_loop_vinfo; > > + } >=20 > I think there's 'inner' where you leak these. Maybe use auto_vec<> in > vect_loop_form_info instead? >=20 > Otherwise looks OK. >=20 > Thanks, > Richard. >=20 > > /* Now analyze first_loop_vinfo for epilogue vectorization. */ > > poly_uint64 lowest_th =3D LOOP_VINFO_VERSIONING_THRESHOLD > > (first_loop_vinfo); @@ -3694,6 +3757,9 @@ vect_analyze_loop (class loop > *loop, vec_info_shared *shared) > > (first_loop_vinfo->epilogue_vinfos[0]- > >vector_mode)); > > } > > > > + loop_form_info.conds.release (); > > + loop_form_info.alt_loop_conds.release (); > > + > > return first_loop_vinfo; > > } > > > > diff --git a/gcc/tree-vectorizer.h b/gcc/tree-vectorizer.h index > > > afa7a8e30891c782a0e5e3740ecc4377f5a31e54..55b6771b271d5072fa132 > 7d595e1 > > dddb112cfdf6 100644 > > --- a/gcc/tree-vectorizer.h > > +++ b/gcc/tree-vectorizer.h > > @@ -882,6 +882,12 @@ public: > > we need to peel off iterations at the end to form an epilogue loo= p. */ > > bool peeling_for_niter; > > > > + /* List of loop additional IV conditionals found in the loop. */ > > + auto_vec conds; > > + > > + /* Main loop IV cond. */ > > + gcond* loop_iv_cond; > > + > > /* True if there are no loop carried data dependencies in the loop. > > If loop->safelen <=3D 1, then this is always true, either the loo= p > > didn't have any loop carried data dependencies, or the loop is > > being @@ -984,6 +990,8 @@ public: > > #define LOOP_VINFO_REDUCTION_CHAINS(L) (L)->reduction_chains > > #define LOOP_VINFO_PEELING_FOR_GAPS(L) (L)->peeling_for_gaps > > #define LOOP_VINFO_PEELING_FOR_NITER(L) (L)->peeling_for_niter > > +#define LOOP_VINFO_LOOP_CONDS(L) (L)->conds > > +#define LOOP_VINFO_LOOP_IV_COND(L) (L)->loop_iv_cond > > #define LOOP_VINFO_NO_DATA_DEPENDENCIES(L) (L)- > >no_data_dependencies > > #define LOOP_VINFO_SCALAR_LOOP(L) (L)->scalar_loop > > #define LOOP_VINFO_SCALAR_LOOP_SCALING(L) (L)->scalar_loop_scaling > > @@ -2373,7 +2381,9 @@ struct vect_loop_form_info > > tree number_of_iterations; > > tree number_of_iterationsm1; > > tree assumptions; > > + vec conds; > > gcond *loop_cond; > > + vec alt_loop_conds; > > gcond *inner_loop_cond; > > edge loop_exit; > > }; > > > > > > > > > > >=20 > -- > Richard Biener > SUSE Software Solutions Germany GmbH, > Frankenstrasse 146, 90461 Nuernberg, Germany; > GF: Ivo Totev, Andrew McDonald, Werner Knoblich; (HRB 36809, AG > Nuernberg)