From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR04-VI1-obe.outbound.protection.outlook.com (mail-vi1eur04on2050.outbound.protection.outlook.com [40.107.8.50]) by sourceware.org (Postfix) with ESMTPS id 8ADB73858D20 for ; Tue, 14 Nov 2023 23:59:16 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 8ADB73858D20 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=arm.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 8ADB73858D20 Authentication-Results: server2.sourceware.org; arc=pass smtp.remote-ip=40.107.8.50 ARC-Seal: i=3; a=rsa-sha256; d=sourceware.org; s=key; t=1700006358; cv=pass; b=UnZeti3UzAEsoPl+VZew6uwAm3CqItBSu4zwHefN58BXF9EMC/BRmfwaJ5PVIq27LygLMrksViSJ8RazBUZyzTw6ATtpG+TxwrSPLb9DcFkIMlVzbjVDLMDwv44wi2uhUR1Bt73/vU4OIs8MlP0nRoORiPmnOG7Fm++wHVmcDzQ= ARC-Message-Signature: i=3; a=rsa-sha256; d=sourceware.org; s=key; t=1700006358; c=relaxed/simple; bh=je71Pn7K8RIyQbpNGFXRSwPA5hiQWyF1UtNBgrvMOYk=; h=DKIM-Signature:DKIM-Signature:From:To:Subject:Date:Message-ID: MIME-Version; b=hp5C440m7J3D3qCLr75hDT6/qhty30I/mTNfxiBjqkHRiYyFEHQMZq+/GlihATUai8xiVkWemgnEtS/1yr9uZUq1M5krJpLabsrnq6kBIEvjAdB8724aTt+uAAnzupHrNUvIee/f0dwL4gV7tb0xYW6zccwjpxPtMTZs2gl6ARc= ARC-Authentication-Results: i=3; server2.sourceware.org ARC-Seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass; b=ZDYbRl+gcUMmy5UQG0GUn3Uwf+srrvhVJ5AUMcyh3htipPQOeNrpD5jzAyqmUDdJMe3RJre5F9LEPALbPmmH+6j96mFc9Fw5kKlz3nvKyZ2eBKKyBFCQ8+X2NuiU1r8jeWTmHFFcdeaIMRIINiRnksXky74d6ciL21HCG05Sj3WvzfVAUzwB3lfzQ7L/LN2UbClp7OE+LRRHhQTKad6hGtw3B1i6dHmIWVnlHnr6gvDk1wRZIaesxGYNVjMppgXca/K1KOgWa8BMVh2L1Y6lpLJEjBh78wkTZWHdGSlKXt1lbLa5ZGG+DhYAbKrdzX+BR6j3RHgFSMydTEei6GKA4g== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=v0Sr3Mx367Ndn8N9wYaHtTsfZh4csILBrwLNk5SiX4w=; b=muygNR3cgNe1Y6gyGEvg17nklGAaObyRzHOVw0Utm1t4E78HldwRAX6EJaZi8DQuRsdv+Sy/i4jczBtvafzIg+3hmOhNcmWaVkSNeB7AKsG1ezhxcalYO1xNk8U1ObsERgvLN+3enYwkNHUG11KTiTKlHPOkNxvgfW8JjLHBWEosdDSXsU4oOZB6ScKqoWy1xayELzCoLJgZ9UsUijYqBhCHuAzn5OmYYABUh86/VBbzealET6xUd0bPrB7lSD/8JuExXbtHMRgZd0jB//OdnJiFzb19uxNTOVdcVUGqdELVMNNybEZQHoQCL63A5Hwy5mBpz5YqoOSOMojmsnOQ1g== ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is 63.35.35.123) smtp.rcpttodomain=gcc.gnu.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com; arc=pass (0 oda=1 ltdi=1 spf=[1,1,smtp.mailfrom=arm.com] dkim=[1,1,header.d=arm.com] dmarc=[1,1,header.from=arm.com]) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=v0Sr3Mx367Ndn8N9wYaHtTsfZh4csILBrwLNk5SiX4w=; b=NdX8cmfGISzmhMCdt9p7Oakbqnu+10sdjVkbGP8QmpmUYxTtuSQXGqeoscwC5c60cCQ5B66HUs1oUWTsXjG3Ew8pC1AqyafaS3TnOTZUdxt5F/EdEfINFSrPuYVMTG6LmUUVPhk0YXLolBk/Ffjsfm6VLyqx45zigb9XOCrGNuI= Received: from DB9PR05CA0011.eurprd05.prod.outlook.com (2603:10a6:10:1da::16) by PAVPR08MB9016.eurprd08.prod.outlook.com (2603:10a6:102:325::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6977.29; Tue, 14 Nov 2023 23:59:13 +0000 Received: from DB1PEPF00039232.eurprd03.prod.outlook.com (2603:10a6:10:1da:cafe::81) by DB9PR05CA0011.outlook.office365.com (2603:10a6:10:1da::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6977.31 via Frontend Transport; Tue, 14 Nov 2023 23:59:13 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; pr=C Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by DB1PEPF00039232.mail.protection.outlook.com (10.167.8.105) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7002.14 via Frontend Transport; Tue, 14 Nov 2023 23:59:13 +0000 Received: ("Tessian outbound 26ee1d40577c:v228"); Tue, 14 Nov 2023 23:59:13 +0000 X-CheckRecipientChecked: true X-CR-MTA-CID: 7588485e0777656c X-CR-MTA-TID: 64aa7808 Received: from 46d9ca0217f9.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 333F447A-2D8A-4DAB-9701-C112071D899E.1; Tue, 14 Nov 2023 23:59:06 +0000 Received: from EUR03-DBA-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 46d9ca0217f9.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Tue, 14 Nov 2023 23:59:06 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=UHCDxfG6bhq1hu1JOuxymMV7UryTopzuhp9AHu/6OSvBlRp/bBZ3GThXMp7B8zrXpHMl9D9p5O101qqEbiNCginjjq0QWfgYHpwerucpWNp+FU93r92YFF7jdQjtbFWTKa+bWQGxipmVnI0ODkORx4wGdj3pleRoqcDjn7PJxXShhINQzEhe/2HBx5Wvz4qf0ny2KGVoMPuc07vQxC0lkVpLQ7/DBX0BJt56LZ0vAI+hvIuALOznzATL67oao9FEkgr9kHqwckggtMiPHHUc3qqAdPs3KKYZ2diDhd9jDxCgPJQlxlamzXLiGrHSrLswLqCmq7wWeqyUAKNq1YT4oA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=v0Sr3Mx367Ndn8N9wYaHtTsfZh4csILBrwLNk5SiX4w=; b=m+tDLUBwMBEBhXxDow24v1nCCiwIxMcUXULtW8wc6Y1SWKVuDHe+DxOH/Odnlsejq3Q261zipZm4eTwo3iqQC5jLWbwc7jgETKri0V8xjXcc94o8AfhMCFB5Z+Su9eN1XEo6lUD7YdqLMlha0gHx2Ih/x8P1f57treEgy99MIrbuKkJ2ulicYHIxEwGvgSyhyLptVo5XCabBnqjutIA1W8UGPQ8yYKJ+EVt+mzVyt7zevCMN7qIZzLeo0nkyCm2r9fbTbeI5JMHAnsNQAhcPmAOXD60NcCmcA73DwkK1O6lBHITuOuLV7j8lGFP0He+Ccls3S2PFqG0iS1r/3+5olA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=v0Sr3Mx367Ndn8N9wYaHtTsfZh4csILBrwLNk5SiX4w=; b=NdX8cmfGISzmhMCdt9p7Oakbqnu+10sdjVkbGP8QmpmUYxTtuSQXGqeoscwC5c60cCQ5B66HUs1oUWTsXjG3Ew8pC1AqyafaS3TnOTZUdxt5F/EdEfINFSrPuYVMTG6LmUUVPhk0YXLolBk/Ffjsfm6VLyqx45zigb9XOCrGNuI= Received: from VI1PR08MB5325.eurprd08.prod.outlook.com (2603:10a6:803:13e::17) by AS8PR08MB6360.eurprd08.prod.outlook.com (2603:10a6:20b:33e::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6977.31; Tue, 14 Nov 2023 23:59:04 +0000 Received: from VI1PR08MB5325.eurprd08.prod.outlook.com ([fe80::9679:2ab0:99c6:54a3]) by VI1PR08MB5325.eurprd08.prod.outlook.com ([fe80::9679:2ab0:99c6:54a3%6]) with mapi id 15.20.6977.029; Tue, 14 Nov 2023 23:59:03 +0000 From: Tamar Christina To: Richard Biener CC: "gcc-patches@gcc.gnu.org" , nd , "jlaw@ventanamicro.com" Subject: RE: [PATCH 5/21]middle-end: update vectorizer's control update to support picking an exit other than loop latch Thread-Topic: [PATCH 5/21]middle-end: update vectorizer's control update to support picking an exit other than loop latch Thread-Index: AQHaEIQ+Iau5X0e+Z0WdQvM9MBiwy7Bu9kUAgAnCFxCAAMalgIAAAgFAgAEKb6A= Date: Tue, 14 Nov 2023 23:59:03 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: yes X-MS-TNEF-Correlator: Authentication-Results-Original: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; x-ms-traffictypediagnostic: VI1PR08MB5325:EE_|AS8PR08MB6360:EE_|DB1PEPF00039232:EE_|PAVPR08MB9016:EE_ X-MS-Office365-Filtering-Correlation-Id: 209ec08b-b958-4fe5-0a03-08dbe56db3da x-checkrecipientrouted: true nodisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: 19U3Z4I+Zqk6Rco/1GxSIBIWQ1JmwTo2ULOVglGiKTRN9kmRn+sU1NS8yvlJJu3iJ+yG6YY17J/SR54aHLFA+vXOVH4ffqs/E/ZPZ6z3dQKQCDPi9na3+TTWQAE7Iw+7sx9hxlAV8w8hATT+arpw/vyi9QAeZFlg0a5q4ZUiPgHqsVk5aKK9IRH9OV5u2xA4XLRfUnr9a1u/T72q/eZBeBZ62t9wsf2WBTc6nRrtURv5mgP5gT/PgbIfvPLCD0lbbsZbW37k9zpTWBDxmjf9a/Ek+qmalEXu5BoG/yttOdw3vGm9yPFaFzt6X3PsNEOzzSF+F+Gc92uDmviA9ea+jSpgN8RfBYNcqScSCUpV2C0OuClc7tJBwiGL7fZv4uV5NYJUGv4SMhAsfEqg4Yjq1/4PH3AhZiSyoaSlW1gXDU5Z6m+Ebni77CFMGHpLVrUbd+VwhtzU7xakHUeXFcxseoLZxsbcOJOQvPsxczaUuvOJ18IpDv7K5oa2fmYSZAyJSmwsgytb3I1WUbNHq1cB0tYnqfYj35p14MeeE1n4AWPXj2Gzy3Fwm/daVZk+4bMgv7vHmZWQcvtk2mTKCHmyopOhUmuBcHWSjUcUOGFz5q2PDVRddptxWsP54MSAbm8X X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR08MB5325.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(376002)(366004)(136003)(396003)(346002)(39860400002)(230922051799003)(64100799003)(1800799009)(186009)(451199024)(2940100002)(66899024)(7696005)(6506007)(9686003)(26005)(478600001)(86362001)(52536014)(8936002)(8676002)(41300700001)(2906002)(4326008)(33656002)(5660300002)(15650500001)(38070700009)(76116006)(6916009)(66446008)(66946007)(66476007)(66556008)(54906003)(64756008)(316002)(71200400001)(38100700002)(55016003)(122000001)(99936003)(83380400001);DIR:OUT;SFP:1101; Content-Type: multipart/mixed; boundary="_002_VI1PR08MB532540B66504566059C98526FFB2AVI1PR08MB5325eurp_" MIME-Version: 1.0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB6360 Original-Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: DB1PEPF00039232.eurprd03.prod.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: 152b948d-1b9a-45df-0242-08dbe56dadf0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 3TzWZvpv2JhnuwcHRsoIYqksbc3X4zEto8MxiIEzSCEujUm/EQjJ2WIrxpKxt1DBm56eT86Kixhh/n/45rntYnkoz1VHBW4KXCGZHqo1JUen5Reo4nalehrHvD/SM5Bhibekp9gNQJsUvV3GzltXn0SBgU0t9SePtesPcCCu9YVx3tDRgCB7xotwvpot5x+EZXifCMjMAzd+CcnGDux2ZJmqdsLfZp6UBQKeQhVGmovo30fP+Y/RBmKCYXL9xzuna5L9HMSw8XskCTAdu9h1v601xZDHRwrPlb2C1H32CcKc6QqzOKLIMF86DukmwnvQ+FLGUlCTMSHnRG40scr8QvBaB+iVMK4QGOM94BC2bG40lRQUornep0bpFDYt9pQSoqeGOZzAnT1xISCVXl9dmRGkt7MH8fUu/ZsXtFz2O9R7wKxXXjjoNNZ7dQxkvsTUfhCRO9VsCBNvr9s0PwWpm3YPvM/cjYs75d43QpFI9A5wlC4OicaRer1qA05iXihJApvj9ZmYbW65gnwQrUP88/KWKcVj7tGoKH/oEk8WCtyHXjWSkbn/wgyvYXd5OxXELQCZ99Ndsd4ZgQCHWvhiCKoj0XfVettfij7dEBJPyS5uwWUAM2OEkU+4LAVSI8vY6FzvxtXLtYiMyw4AkyoYENDkmM2quEzG9ln85cL0gGFaE1bKYqhby+Gzo+DdKTheAyz3WWTPzAElKwmmXCVU8SOP4C/zjZz4Ns2Jc7v8l2AtKSfVq1eNRC2LC8CLtHx9 X-Forefront-Antispam-Report: CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(13230031)(4636009)(346002)(136003)(376002)(396003)(39860400002)(230922051799003)(451199024)(64100799003)(186009)(82310400011)(1800799009)(36840700001)(40470700004)(46966006)(41300700001)(55016003)(66899024)(40480700001)(2906002)(15650500001)(7696005)(6506007)(40460700003)(235185007)(8936002)(6862004)(8676002)(4326008)(52536014)(33656002)(70586007)(81166007)(86362001)(478600001)(316002)(107886003)(70206006)(2940100002)(54906003)(9686003)(26005)(99936003)(82740400003)(336012)(83380400001)(5660300002)(356005)(47076005)(36860700001);DIR:OUT;SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Nov 2023 23:59:13.4664 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 209ec08b-b958-4fe5-0a03-08dbe56db3da X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: DB1PEPF00039232.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAVPR08MB9016 X-Spam-Status: No, score=-12.1 required=5.0 tests=BAYES_00,DKIM_SIGNED,DKIM_VALID,FORGED_SPF_HELO,GIT_PATCH_0,KAM_DMARC_NONE,RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE,TXREP,T_SCC_BODY_TEXT_LINE,UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: --_002_VI1PR08MB532540B66504566059C98526FFB2AVI1PR08MB5325eurp_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable > > OK, but then I think the fix is to not use > > standard_iv_increment_position (it's a weird API anyway). Instead inse= rt > before the main exit condition. >=20 > I figured as much, Almost done respinning it with the vectorizer's own si= mpler > copy. > Should be out today with the rest. >=20 > > > > Btw, I assumed this order of main / early exit cannot happen. But I > > didn't re- review the main exit identification code yet. > > >=20 > It can happen because we allowed vec_init_loop_exit_info to pick the last > analyzable exit. In cases like these it happens because the final exit h= as no > Information from SCEV. It then picks the last exit it could analyze which= by > default is an early exit. >=20 > It's very tricky to deal with and have just finished cleaning up the IV u= pdate > code to make it easier to follow... but it does seem to add about 970 mor= e > vectorized cases (most of which are execution tests). >=20 Bootstrapped Regtested on aarch64-none-linux-gnu and no issues. Ok for master? Thanks, Tamar gcc/ChangeLog: * tree-vect-loop-manip.cc (vect_iv_increment_position): New. (vect_set_loop_controls_directly): Use it. (vect_set_loop_condition_partial_vectors_avx512): Likewise. (vect_set_loop_condition_normal): Likewise. --- inline copy of patch --- diff --git a/gcc/tree-vect-loop-manip.cc b/gcc/tree-vect-loop-manip.cc index fafbf924e8db18eb4eec7a4a1906d10f6ce9812f..a5a612dc6b47436730592469176= 623685a7a413f 100644 --- a/gcc/tree-vect-loop-manip.cc +++ b/gcc/tree-vect-loop-manip.cc @@ -448,6 +448,20 @@ vect_adjust_loop_lens_control (tree iv_type, gimple_se= q *seq, } } =20 +/* Stores the standard position for induction variable increment in belong= ing to + LOOP_EXIT (just before the exit condition of the given exit to BSI. + INSERT_AFTER is set to true if the increment should be inserted after + *BSI. */ + +static void +vect_iv_increment_position (edge loop_exit, gimple_stmt_iterator *bsi, + bool *insert_after) +{ + basic_block bb =3D loop_exit->src; + *bsi =3D gsi_last_bb (bb); + *insert_after =3D false; +} + /* Helper for vect_set_loop_condition_partial_vectors. Generate definitio= ns for all the rgroup controls in RGC and return a control that is nonzero when the loop needs to iterate. Add any new preheader statements to @@ -531,7 +545,8 @@ vect_set_loop_controls_directly (class loop *loop, loop= _vec_info loop_vinfo, tree index_before_incr, index_after_incr; gimple_stmt_iterator incr_gsi; bool insert_after; - standard_iv_increment_position (loop, &incr_gsi, &insert_after); + edge exit_e =3D LOOP_VINFO_IV_EXIT (loop_vinfo); + vect_iv_increment_position (exit_e, &incr_gsi, &insert_after); if (LOOP_VINFO_USING_DECREMENTING_IV_P (loop_vinfo)) { /* Create an IV that counts down from niters_total and whose step @@ -1017,7 +1032,7 @@ vect_set_loop_condition_partial_vectors_avx512 (class= loop *loop, tree index_before_incr, index_after_incr; gimple_stmt_iterator incr_gsi; bool insert_after; - standard_iv_increment_position (loop, &incr_gsi, &insert_after); + vect_iv_increment_position (exit_edge, &incr_gsi, &insert_after); create_iv (niters_adj, MINUS_EXPR, iv_step, NULL_TREE, loop, &incr_gsi, insert_after, &index_before_incr, &index_after_incr); @@ -1185,7 +1200,7 @@ vect_set_loop_condition_partial_vectors_avx512 (class= loop *loop, loop handles exactly VF scalars per iteration. */ =20 static gcond * -vect_set_loop_condition_normal (loop_vec_info /* loop_vinfo */, edge exit_= edge, +vect_set_loop_condition_normal (loop_vec_info loop_vinfo, edge exit_edge, class loop *loop, tree niters, tree step, tree final_iv, bool niters_maybe_zero, gimple_stmt_iterator loop_cond_gsi) @@ -1278,7 +1293,7 @@ vect_set_loop_condition_normal (loop_vec_info /* loop= _vinfo */, edge exit_edge, } } =20 - standard_iv_increment_position (loop, &incr_gsi, &insert_after); + vect_iv_increment_position (exit_edge, &incr_gsi, &insert_after); create_iv (init, PLUS_EXPR, step, NULL_TREE, loop, &incr_gsi, insert_after, &indx_before_incr, &indx_after_incr)= ; indx_after_incr =3D force_gimple_operand_gsi (&loop_cond_gsi, indx_after= _incr, @@ -1446,7 +1461,7 @@ slpeel_tree_duplicate_loop_for_vectorization (class l= oop *loop, edge loop_exit, redirect_edge_and_branch (exit, dest); } =20 - /* Only fush the main exit, the remaining exits we need to match the ord= er + /* Only flush the main exit, the remaining exits we need to match the or= der in the loop->header which with multiple exits may not be the same. *= / flush_pending_stmts (loop_exit); =20 @@ -1519,7 +1534,9 @@ slpeel_tree_duplicate_loop_for_vectorization (class l= oop *loop, edge loop_exit, SET_PHI_ARG_DEF (alt_lcssa_phi, main_e->dest_idx, new_arg); } new_arg =3D alt_res; /* Push it down to the new_loop header. */ - } else if (!res) { + } + else if (!res) + { /* For non-early break we need to keep the possibly live values in the exit block. For early break these are kept in the merge block in the code above. */ --_002_VI1PR08MB532540B66504566059C98526FFB2AVI1PR08MB5325eurp_ Content-Type: application/octet-stream; name="rb17965.patch" Content-Description: rb17965.patch Content-Disposition: attachment; filename="rb17965.patch"; size=3814; creation-date="Tue, 14 Nov 2023 23:58:14 GMT"; modification-date="Tue, 14 Nov 2023 23:59:03 GMT" Content-Transfer-Encoding: base64 ZGlmZiAtLWdpdCBhL2djYy90cmVlLXZlY3QtbG9vcC1tYW5pcC5jYyBiL2djYy90cmVlLXZlY3Qt bG9vcC1tYW5pcC5jYwppbmRleCBmYWZiZjkyNGU4ZGIxOGViNGVlYzdhNGExOTA2ZDEwZjZjZTk4 MTJmLi5hNWE2MTJkYzZiNDc0MzY3MzA1OTI0NjkxNzY2MjM2ODVhN2E0MTNmIDEwMDY0NAotLS0g YS9nY2MvdHJlZS12ZWN0LWxvb3AtbWFuaXAuY2MKKysrIGIvZ2NjL3RyZWUtdmVjdC1sb29wLW1h bmlwLmNjCkBAIC00NDgsNiArNDQ4LDIwIEBAIHZlY3RfYWRqdXN0X2xvb3BfbGVuc19jb250cm9s ICh0cmVlIGl2X3R5cGUsIGdpbXBsZV9zZXEgKnNlcSwKICAgICB9CiB9CiAKKy8qIFN0b3JlcyB0 aGUgc3RhbmRhcmQgcG9zaXRpb24gZm9yIGluZHVjdGlvbiB2YXJpYWJsZSBpbmNyZW1lbnQgaW4g YmVsb25naW5nIHRvCisgICBMT09QX0VYSVQgKGp1c3QgYmVmb3JlIHRoZSBleGl0IGNvbmRpdGlv biBvZiB0aGUgZ2l2ZW4gZXhpdCB0byBCU0kuCisgICBJTlNFUlRfQUZURVIgaXMgc2V0IHRvIHRy dWUgaWYgdGhlIGluY3JlbWVudCBzaG91bGQgYmUgaW5zZXJ0ZWQgYWZ0ZXIKKyAgICpCU0kuICAq LworCitzdGF0aWMgdm9pZAordmVjdF9pdl9pbmNyZW1lbnRfcG9zaXRpb24gKGVkZ2UgbG9vcF9l eGl0LCBnaW1wbGVfc3RtdF9pdGVyYXRvciAqYnNpLAorCQkJICAgIGJvb2wgKmluc2VydF9hZnRl cikKK3sKKyAgYmFzaWNfYmxvY2sgYmIgPSBsb29wX2V4aXQtPnNyYzsKKyAgKmJzaSA9IGdzaV9s YXN0X2JiIChiYik7CisgICppbnNlcnRfYWZ0ZXIgPSBmYWxzZTsKK30KKwogLyogSGVscGVyIGZv ciB2ZWN0X3NldF9sb29wX2NvbmRpdGlvbl9wYXJ0aWFsX3ZlY3RvcnMuICBHZW5lcmF0ZSBkZWZp bml0aW9ucwogICAgZm9yIGFsbCB0aGUgcmdyb3VwIGNvbnRyb2xzIGluIFJHQyBhbmQgcmV0dXJu IGEgY29udHJvbCB0aGF0IGlzIG5vbnplcm8KICAgIHdoZW4gdGhlIGxvb3AgbmVlZHMgdG8gaXRl cmF0ZS4gIEFkZCBhbnkgbmV3IHByZWhlYWRlciBzdGF0ZW1lbnRzIHRvCkBAIC01MzEsNyArNTQ1 LDggQEAgdmVjdF9zZXRfbG9vcF9jb250cm9sc19kaXJlY3RseSAoY2xhc3MgbG9vcCAqbG9vcCwg bG9vcF92ZWNfaW5mbyBsb29wX3ZpbmZvLAogICB0cmVlIGluZGV4X2JlZm9yZV9pbmNyLCBpbmRl eF9hZnRlcl9pbmNyOwogICBnaW1wbGVfc3RtdF9pdGVyYXRvciBpbmNyX2dzaTsKICAgYm9vbCBp bnNlcnRfYWZ0ZXI7Ci0gIHN0YW5kYXJkX2l2X2luY3JlbWVudF9wb3NpdGlvbiAobG9vcCwgJmlu Y3JfZ3NpLCAmaW5zZXJ0X2FmdGVyKTsKKyAgZWRnZSBleGl0X2UgPSBMT09QX1ZJTkZPX0lWX0VY SVQgKGxvb3BfdmluZm8pOworICB2ZWN0X2l2X2luY3JlbWVudF9wb3NpdGlvbiAoZXhpdF9lLCAm aW5jcl9nc2ksICZpbnNlcnRfYWZ0ZXIpOwogICBpZiAoTE9PUF9WSU5GT19VU0lOR19ERUNSRU1F TlRJTkdfSVZfUCAobG9vcF92aW5mbykpCiAgICAgewogICAgICAgLyogQ3JlYXRlIGFuIElWIHRo YXQgY291bnRzIGRvd24gZnJvbSBuaXRlcnNfdG90YWwgYW5kIHdob3NlIHN0ZXAKQEAgLTEwMTcs NyArMTAzMiw3IEBAIHZlY3Rfc2V0X2xvb3BfY29uZGl0aW9uX3BhcnRpYWxfdmVjdG9yc19hdng1 MTIgKGNsYXNzIGxvb3AgKmxvb3AsCiAgIHRyZWUgaW5kZXhfYmVmb3JlX2luY3IsIGluZGV4X2Fm dGVyX2luY3I7CiAgIGdpbXBsZV9zdG10X2l0ZXJhdG9yIGluY3JfZ3NpOwogICBib29sIGluc2Vy dF9hZnRlcjsKLSAgc3RhbmRhcmRfaXZfaW5jcmVtZW50X3Bvc2l0aW9uIChsb29wLCAmaW5jcl9n c2ksICZpbnNlcnRfYWZ0ZXIpOworICB2ZWN0X2l2X2luY3JlbWVudF9wb3NpdGlvbiAoZXhpdF9l ZGdlLCAmaW5jcl9nc2ksICZpbnNlcnRfYWZ0ZXIpOwogICBjcmVhdGVfaXYgKG5pdGVyc19hZGos IE1JTlVTX0VYUFIsIGl2X3N0ZXAsIE5VTExfVFJFRSwgbG9vcCwKIAkgICAgICZpbmNyX2dzaSwg aW5zZXJ0X2FmdGVyLCAmaW5kZXhfYmVmb3JlX2luY3IsCiAJICAgICAmaW5kZXhfYWZ0ZXJfaW5j cik7CkBAIC0xMTg1LDcgKzEyMDAsNyBAQCB2ZWN0X3NldF9sb29wX2NvbmRpdGlvbl9wYXJ0aWFs X3ZlY3RvcnNfYXZ4NTEyIChjbGFzcyBsb29wICpsb29wLAogICAgbG9vcCBoYW5kbGVzIGV4YWN0 bHkgVkYgc2NhbGFycyBwZXIgaXRlcmF0aW9uLiAgKi8KIAogc3RhdGljIGdjb25kICoKLXZlY3Rf c2V0X2xvb3BfY29uZGl0aW9uX25vcm1hbCAobG9vcF92ZWNfaW5mbyAvKiBsb29wX3ZpbmZvICov LCBlZGdlIGV4aXRfZWRnZSwKK3ZlY3Rfc2V0X2xvb3BfY29uZGl0aW9uX25vcm1hbCAobG9vcF92 ZWNfaW5mbyBsb29wX3ZpbmZvLCBlZGdlIGV4aXRfZWRnZSwKIAkJCQljbGFzcyBsb29wICpsb29w LCB0cmVlIG5pdGVycywgdHJlZSBzdGVwLAogCQkJCXRyZWUgZmluYWxfaXYsIGJvb2wgbml0ZXJz X21heWJlX3plcm8sCiAJCQkJZ2ltcGxlX3N0bXRfaXRlcmF0b3IgbG9vcF9jb25kX2dzaSkKQEAg LTEyNzgsNyArMTI5Myw3IEBAIHZlY3Rfc2V0X2xvb3BfY29uZGl0aW9uX25vcm1hbCAobG9vcF92 ZWNfaW5mbyAvKiBsb29wX3ZpbmZvICovLCBlZGdlIGV4aXRfZWRnZSwKIAl9CiAgICAgfQogCi0g IHN0YW5kYXJkX2l2X2luY3JlbWVudF9wb3NpdGlvbiAobG9vcCwgJmluY3JfZ3NpLCAmaW5zZXJ0 X2FmdGVyKTsKKyAgdmVjdF9pdl9pbmNyZW1lbnRfcG9zaXRpb24gKGV4aXRfZWRnZSwgJmluY3Jf Z3NpLCAmaW5zZXJ0X2FmdGVyKTsKICAgY3JlYXRlX2l2IChpbml0LCBQTFVTX0VYUFIsIHN0ZXAs IE5VTExfVFJFRSwgbG9vcCwKICAgICAgICAgICAgICAmaW5jcl9nc2ksIGluc2VydF9hZnRlciwg JmluZHhfYmVmb3JlX2luY3IsICZpbmR4X2FmdGVyX2luY3IpOwogICBpbmR4X2FmdGVyX2luY3Ig PSBmb3JjZV9naW1wbGVfb3BlcmFuZF9nc2kgKCZsb29wX2NvbmRfZ3NpLCBpbmR4X2FmdGVyX2lu Y3IsCkBAIC0xNDQ2LDcgKzE0NjEsNyBAQCBzbHBlZWxfdHJlZV9kdXBsaWNhdGVfbG9vcF9mb3Jf dmVjdG9yaXphdGlvbiAoY2xhc3MgbG9vcCAqbG9vcCwgZWRnZSBsb29wX2V4aXQsCiAgICAgICBy ZWRpcmVjdF9lZGdlX2FuZF9icmFuY2ggKGV4aXQsIGRlc3QpOwogICAgIH0KIAotICAvKiBPbmx5 IGZ1c2ggdGhlIG1haW4gZXhpdCwgdGhlIHJlbWFpbmluZyBleGl0cyB3ZSBuZWVkIHRvIG1hdGNo IHRoZSBvcmRlcgorICAvKiBPbmx5IGZsdXNoIHRoZSBtYWluIGV4aXQsIHRoZSByZW1haW5pbmcg ZXhpdHMgd2UgbmVlZCB0byBtYXRjaCB0aGUgb3JkZXIKICAgICAgaW4gdGhlIGxvb3AtPmhlYWRl ciB3aGljaCB3aXRoIG11bHRpcGxlIGV4aXRzIG1heSBub3QgYmUgdGhlIHNhbWUuICAqLwogICBm bHVzaF9wZW5kaW5nX3N0bXRzIChsb29wX2V4aXQpOwogCkBAIC0xNTE5LDcgKzE1MzQsOSBAQCBz bHBlZWxfdHJlZV9kdXBsaWNhdGVfbG9vcF9mb3JfdmVjdG9yaXphdGlvbiAoY2xhc3MgbG9vcCAq bG9vcCwgZWRnZSBsb29wX2V4aXQsCiAJCSAgU0VUX1BISV9BUkdfREVGIChhbHRfbGNzc2FfcGhp LCBtYWluX2UtPmRlc3RfaWR4LCBuZXdfYXJnKTsKIAkJfQogCSAgICBuZXdfYXJnID0gYWx0X3Jl czsgLyogUHVzaCBpdCBkb3duIHRvIHRoZSBuZXdfbG9vcCBoZWFkZXIuICAqLwotCSAgfSBlbHNl IGlmICghcmVzKSB7CisJICB9CisJZWxzZSBpZiAoIXJlcykKKwkgIHsKIAkgICAgLyogRm9yIG5v bi1lYXJseSBicmVhayB3ZSBuZWVkIHRvIGtlZXAgdGhlIHBvc3NpYmx5IGxpdmUgdmFsdWVzIGlu CiAJICAgICAgIHRoZSBleGl0IGJsb2NrLiAgRm9yIGVhcmx5IGJyZWFrIHRoZXNlIGFyZSBrZXB0 IGluIHRoZSBtZXJnZQogCSAgICAgICBibG9jayBpbiB0aGUgY29kZSBhYm92ZS4gICovCg== --_002_VI1PR08MB532540B66504566059C98526FFB2AVI1PR08MB5325eurp_--