From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR05-VI1-obe.outbound.protection.outlook.com (mail-vi1eur05on2076.outbound.protection.outlook.com [40.107.21.76]) by sourceware.org (Postfix) with ESMTPS id DA8E63858C98 for ; Wed, 15 Nov 2023 13:09:55 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org DA8E63858C98 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=arm.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org DA8E63858C98 Authentication-Results: server2.sourceware.org; arc=pass smtp.remote-ip=40.107.21.76 ARC-Seal: i=3; a=rsa-sha256; d=sourceware.org; s=key; t=1700053798; cv=pass; b=Dztu6jAEitnKmeLyxp+VpJsqbORmT8k6wdUY7UhhqzoZOmiS2OD5guO9He7Ere1acC2H9fSPAc2Ppen08DN9lAnRiu0yq73DI4dQoJp+CWA+52TLHtNrYNVi2YDQqenydGgLl00ZIhPpvwbargQKVhRzuwOTkBadoiHUuDCjemM= ARC-Message-Signature: i=3; a=rsa-sha256; d=sourceware.org; s=key; t=1700053798; c=relaxed/simple; bh=GGK7y+ZkOZfHshsYx/6CGyNa2glpTDEpEs2loPwzvjE=; h=DKIM-Signature:DKIM-Signature:From:To:Subject:Date:Message-ID: MIME-Version; b=WxzADnpU81Wb2WiLcUbcVF4kFMBlkeYcWf+tVES6qJpCc9qB5cth7pekDIfoAw8DimzKjukqNplDRxyWfZD8QvAPFvt475TGHgCeYXHBx4psU5RtYDaS6cbt7NzYNUTRKrzS3eOaoRizM5fOIEqipyMT2lin3DqV+tvZbo23bKc= ARC-Authentication-Results: i=3; server2.sourceware.org ARC-Seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass; b=VY14G55VKfV9EgY3ho2Lf/MF4zJBr6TTRcmGsmPusk8SfD3fWs/7gar8a2HDceHM/CsJ6WmqGCNBEWpbT4O1j5UwK8uXalCm6cNYfREPjbNnEj2DDtmjO11hPCbol0j334EPFWtkV8xSC6BSd2VwqVzVRle4vUbSfDXOfKtBF48ErzAnahRdy+GiPGOyhmsHNkvbgftWLpP8xTX/KMqgH3Mt4DFr1yJkGBBe/9c9Ropo1K1zBRAey4oUZc/+OYJZpx3+fE0JKhpS64IXJPAZIb8Jerfn7MVk5NB1jrO33QkIe/PIusSxSpLrrpgJeaiCX4tWn/KgajLXoAp2J2DV3Q== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=TpsDxWIs+D2x6dy72AKOhtST3MI+S/zHS01pQ3GPuB0=; b=JePYMV8DDMyYQTdDcur4faAeqHo50CHthnfDcQA0lHo+mCa+VEH2632o2geSkNbv7q4BVnREtvkm+SEm9HeYmibVkoAtGqX/ugavxISvDIKcz0/sJUYXyKbnQ/N/jsSzh+yHU3qcpIVo2w03J0FheJrq2hPvUZzifVAouo/zT/KBE2gcSMCU7VlMII8zl4BhER5XNA7k8rN1hLfO4JAD3ZQm0FE4/xZgYigMKVFuhNmNNDHSRL1DQpyCI9Wy8sXM/za5lSQccOfk3sZK2IoqWJ5hBRO2ycQe60TyAjmGQYOepiZlv2BTHq8fnnMbdTSBLd9FcBB4BgDcgdE6PuVyJw== ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is 63.35.35.123) smtp.rcpttodomain=gcc.gnu.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com; arc=pass (0 oda=1 ltdi=1 spf=[1,1,smtp.mailfrom=arm.com] dkim=[1,1,header.d=arm.com] dmarc=[1,1,header.from=arm.com]) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=TpsDxWIs+D2x6dy72AKOhtST3MI+S/zHS01pQ3GPuB0=; b=vN23JAXUwPHyWQhS6RZOnnN5SbFhKgPX0QmTnDvsooab5uN4PZASk75KiEIZdCdHSOBB9AfnIKnPDKi73vg5psT2EThQxlIxzEIpK4OGC0MnFXNm1s6CnanyzKgYFcOQ6TQtOy2eKsO0VBS+IipfvJlzub/9wMLijFQytrF2igQ= Received: from AM6P191CA0001.EURP191.PROD.OUTLOOK.COM (2603:10a6:209:8b::14) by DB3PR08MB9134.eurprd08.prod.outlook.com (2603:10a6:10:42b::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6977.29; Wed, 15 Nov 2023 13:09:52 +0000 Received: from AMS0EPF00000192.eurprd05.prod.outlook.com (2603:10a6:209:8b:cafe::d4) by AM6P191CA0001.outlook.office365.com (2603:10a6:209:8b::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6977.29 via Frontend Transport; Wed, 15 Nov 2023 13:09:52 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; pr=C Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by AMS0EPF00000192.mail.protection.outlook.com (10.167.16.218) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7002.20 via Frontend Transport; Wed, 15 Nov 2023 13:09:52 +0000 Received: ("Tessian outbound 26ee1d40577c:v228"); Wed, 15 Nov 2023 13:09:51 +0000 X-CheckRecipientChecked: true X-CR-MTA-CID: a948f26242cbfa86 X-CR-MTA-TID: 64aa7808 Received: from 92d79a82d33a.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id F28D150D-D562-4640-A751-B322C3B4DD3A.1; Wed, 15 Nov 2023 13:09:46 +0000 Received: from EUR04-DB3-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 92d79a82d33a.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Wed, 15 Nov 2023 13:09:46 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=dcjYaQv1++JeTmj336JaV0R5u70r8aFXU5ZUM1gD7lbsuxirrfPzmpJC9wpv4V7sfqp4ytVZcwiNY4yT4VfBcq6EvVM5wjDQ8qS+ATsezgkBEl00p6QU/TdmarsIqmMhXlPowhsy94nI/Gycy6RVwfESo5E0vhBT+t44j8siZ2eReyzBGzc/N5xrOu4/Wptm6s6kwiT8nNvHhHq408Eg+63NSq1B5jcbH2cyjFEcP9VpsWCTXg2sWzBhDVdhGjiyxyW9mh5eNiT+riibT3TtAHOW8BVY+uB8Le1L30r79x/RaGsC4iiHkhhluTV1vVLE9BN1O4BGBNqyxF4tQvLTMg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=TpsDxWIs+D2x6dy72AKOhtST3MI+S/zHS01pQ3GPuB0=; b=NI4FNucI6CnVzN8itYB54lKDH1G7jaOdT80wZC5iRe0f58JdNZxNc1TJvBxow16dZQrJmzky/mnmoLNR9b5HTnPrueu9bknNSJ4zoE7QEsHc0akn9EUyL6ZwnEUsMZN2MFbVOBJnUA5UHmwJ23GPZ9obelpzh2ZBn+lSi19GWUD02cE/CJp8KbxepqADNhvh9+/It+ae5108cUzN+mmsAuCENPJX5QwjG4SncTXvQYO2mG0hdUC8bucu2uMI+WcAtimwd9441a5VCxEZjn4Vz6yTCdtlcj3Vur0rvy5HgkwCCdxHxPUeoBhZ1qXn0ruRXGzEqgOMTvgGHAZUk1wSEQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=TpsDxWIs+D2x6dy72AKOhtST3MI+S/zHS01pQ3GPuB0=; b=vN23JAXUwPHyWQhS6RZOnnN5SbFhKgPX0QmTnDvsooab5uN4PZASk75KiEIZdCdHSOBB9AfnIKnPDKi73vg5psT2EThQxlIxzEIpK4OGC0MnFXNm1s6CnanyzKgYFcOQ6TQtOy2eKsO0VBS+IipfvJlzub/9wMLijFQytrF2igQ= Received: from VI1PR08MB5325.eurprd08.prod.outlook.com (2603:10a6:803:13e::17) by AS2PR08MB9785.eurprd08.prod.outlook.com (2603:10a6:20b:606::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7002.18; Wed, 15 Nov 2023 13:09:43 +0000 Received: from VI1PR08MB5325.eurprd08.prod.outlook.com ([fe80::9679:2ab0:99c6:54a3]) by VI1PR08MB5325.eurprd08.prod.outlook.com ([fe80::9679:2ab0:99c6:54a3%6]) with mapi id 15.20.6977.029; Wed, 15 Nov 2023 13:09:43 +0000 From: Tamar Christina To: Richard Biener CC: "gcc-patches@gcc.gnu.org" , nd , "jlaw@ventanamicro.com" Subject: RE: [PATCH 7/21]middle-end: update IV update code to support early breaks and arbitrary exits Thread-Topic: [PATCH 7/21]middle-end: update IV update code to support early breaks and arbitrary exits Thread-Index: AQHaEIRUWLSyh++pSEG+iB4lA+c+BLB6jLQwgADZ7QCAAAD4UA== Date: Wed, 15 Nov 2023 13:09:43 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: Authentication-Results-Original: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; x-ms-traffictypediagnostic: VI1PR08MB5325:EE_|AS2PR08MB9785:EE_|AMS0EPF00000192:EE_|DB3PR08MB9134:EE_ X-MS-Office365-Filtering-Correlation-Id: 9ce9c045-7e21-4ce3-5e99-08dbe5dc277c x-checkrecipientrouted: true nodisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: LbiK2vtnEELzCHUXU36rbcceYZOX5xtUT4PU81pvSkKYfftCTUBx3NTbgMac2Nu3qHYhC4FTjk4QFts6LXK0g7O3IqgZ8UkrK+A7A8bWR8Z1eJd+USE5YTAd6IWs5LJg0Wskkfx/SdFmkYD4SWPobTX1M7Pjybe9W2DzSdfzi1XefjqjRGrUD+HdsN7DAp5m0sGVHgfUzPffYAO5sIG6oE6RNO7equHhlEw7QNO3VuRirHMYTSbm6esjk/mbBCNnCKP08WOZm+Jo5S8Z7oDvihkqZl1R/yGQ0QrNm1WO6DmtFHU0u5XBiK6FLXPmhDB/WPTq6lPlI+726RSEapqJSDUbxZvMZyVg7WUZp9KTp9+0fi7y57WoeLZeMjV0u/nM9thYzM2H3A2QIdOcvqzxfuHhHhrfv2NHCxJUasIt2n+O3hLD9AoCondYbSwoetIlHYt7pBN4rrU/VZvaBng4mDim2Z+Cd7qPe1OdiyZcQczkxJF1TFVgmWaUZ+fHNSneiUSk8Kzn2g1BRgJR0Etn0g0zZyDL/gE5+g0vkhPJtwCbzoHWhyvNdsu3lPMl/SG0We24n1Pbhy3AZa+iRv11GrXvvpg2q/vs9JBoFESWCl0= X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR08MB5325.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(136003)(39860400002)(396003)(376002)(346002)(366004)(230922051799003)(186009)(451199024)(1800799009)(64100799003)(38100700002)(64756008)(122000001)(66946007)(66446008)(66476007)(54906003)(83380400001)(66556008)(6916009)(76116006)(316002)(66899024)(38070700009)(4326008)(15650500001)(8936002)(8676002)(52536014)(5660300002)(30864003)(2906002)(966005)(26005)(478600001)(7696005)(55016003)(9686003)(6506007)(53546011)(71200400001)(33656002)(86362001)(41300700001);DIR:OUT;SFP:1101; Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS2PR08MB9785 Original-Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: AMS0EPF00000192.eurprd05.prod.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: 68a41394-f643-44e7-1186-08dbe5dc227b X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: h1/YeaNCnNhNaalAs9CXlrWyT/6LsGIsKZAMekRpz/WP2gocDOFccE9VSF/5o5m8+RZOr7M6h1zNSueelAVChMz9opIY6ObDotFSBlz5/md7Xri/hVDehw1Zg7ulQ9ZzWEruUKytCOmDSLblS+40BpsC4Lc4pohU9Wue6ORbtVfWNHaIYPWSI8oMtng8Hw4B7HGPicE/s0Ehgy7YfPYF+ZCukT5OmMmqrBnd4WB0ix4iX0Yqmj+ETdIWTTbdegsXTQNoclk4YH7p/0YfOBMPSzBlnY6yPRiv7/glr0NduEWNBc34GY9Obq8plpz1HzpIZ9Rk+vocW5zhlYxCkYCx2Ejr5yiPJYggViVOuEv2ecIJ4iG69fGqIJjIk21Xd4H6jGIbS/bPUkHHwLpfTV9m3GTzLSzitVvvcm0uY3mamI8aGiE3v3c5x1D4Jo9TEoyUAK5d7OXv9Gdpf+7dEeTA1ZDNCFYSXlfIrX4h01QERhAnL+SYZ5VQ3TS7/whV3Bhww4xM4xLKhhabHE3pPj5wXJJ4qoq54TvPHeaaRJmv89Jovm+uYk1klgs+ZGH0MkSX2fa9khVxOj+HtLXB9yaSzMKK9/aBOBUE9oilYfIV8/9YPcTHMXezpG/fqwECeT1PNdD+n2nzBm2wCOJB/yegXP2pJbzNTVyCA30eg5GtWNvD+/KV/x8eLgjGxobo9YoRMBXjU/zpL1FRyhoeBuwd9w== X-Forefront-Antispam-Report: CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(13230031)(4636009)(396003)(346002)(136003)(39850400004)(376002)(230922051799003)(64100799003)(451199024)(186009)(82310400011)(1800799009)(46966006)(36840700001)(2906002)(8936002)(6862004)(52536014)(54906003)(70586007)(316002)(4326008)(8676002)(33656002)(41300700001)(30864003)(15650500001)(86362001)(5660300002)(66899024)(83380400001)(55016003)(81166007)(47076005)(356005)(82740400003)(107886003)(336012)(26005)(36860700001)(478600001)(966005)(70206006)(9686003)(40480700001)(6506007)(53546011)(7696005);DIR:OUT;SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Nov 2023 13:09:52.0460 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 9ce9c045-7e21-4ce3-5e99-08dbe5dc277c X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: AMS0EPF00000192.eurprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB3PR08MB9134 X-Spam-Status: No, score=-12.1 required=5.0 tests=BAYES_00,DKIM_SIGNED,DKIM_VALID,FORGED_SPF_HELO,GIT_PATCH_0,KAM_DMARC_NONE,RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE,TXREP,T_SCC_BODY_TEXT_LINE,UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: > -----Original Message----- > From: Richard Biener > Sent: Wednesday, November 15, 2023 1:01 PM > To: Tamar Christina > Cc: gcc-patches@gcc.gnu.org; nd ; jlaw@ventanamicro.com > Subject: RE: [PATCH 7/21]middle-end: update IV update code to support ear= ly > breaks and arbitrary exits >=20 > On Wed, 15 Nov 2023, Tamar Christina wrote: >=20 > > Patch updated to latest trunk: > > > > Hi All, > > > > This changes the PHI node updates to support early breaks. > > It has to support both the case where the loop's exit matches the > > normal loop exit and one where the early exit is "inverted", i.e. it's = an early > exit edge. > > > > In the latter case we must always restart the loop for VF iterations. > > For an early exit the reason is obvious, but there are cases where the > > "normal" exit is located before the early one. This exit then does a > > check on ivtmp resulting in us leaving the loop since it thinks we're d= one. > > > > In these case we may still have side-effects to perform so we also go > > to the scalar loop. > > > > For the "normal" exit niters has already been adjusted for peeling, > > for the early exits we must find out how many iterations we actually > > did. So we have to recalculate the new position for each exit. > > > > Thanks, > > Tamar > > > > gcc/ChangeLog: > > > > * tree-vect-loop-manip.cc (vect_set_loop_condition_normal): Hide > unused. > > (vect_update_ivs_after_vectorizer): Support early break. > > (vect_do_peeling): Use it. > > > > --- inline copy of patch --- > > > > diff --git a/gcc/tree-vect-loop-manip.cc b/gcc/tree-vect-loop-manip.cc > > index > > > d3fa8699271c4d7f404d648a38a95beabeabc99a..e1d210ab4617c894dab3 > d2654cf1 > > c842baac58f5 100644 > > --- a/gcc/tree-vect-loop-manip.cc > > +++ b/gcc/tree-vect-loop-manip.cc > > @@ -1200,7 +1200,7 @@ vect_set_loop_condition_partial_vectors_avx512 > (class loop *loop, > > loop handles exactly VF scalars per iteration. */ > > > > static gcond * > > -vect_set_loop_condition_normal (loop_vec_info loop_vinfo, edge > > exit_edge, > > +vect_set_loop_condition_normal (loop_vec_info /* loop_vinfo */, edge > > +exit_edge, > > class loop *loop, tree niters, tree step, > > tree final_iv, bool niters_maybe_zero, > > gimple_stmt_iterator loop_cond_gsi) @@ - > 1412,7 +1412,7 @@ > > vect_set_loop_condition (class loop *loop, edge loop_e, loop_vec_info > loop_vinfo > > When this happens we need to flip the understanding of main and oth= er > > exits by peeling and IV updates. */ > > > > -bool inline > > +bool > > vect_is_loop_exit_latch_pred (edge loop_exit, class loop *loop) { > > return single_pred (loop->latch) =3D=3D loop_exit->src; @@ -2142,6 > > +2142,7 @@ vect_can_advance_ivs_p (loop_vec_info loop_vinfo) > > Input: > > - LOOP - a loop that is going to be vectorized. The last few iter= ations > > of LOOP were peeled. > > + - VF - The chosen vectorization factor for LOOP. > > - NITERS - the number of iterations that LOOP executes (before it= is > > vectorized). i.e, the number of times the ivs should b= e bumped. > > - UPDATE_E - a successor edge of LOOP->exit that is on the > > (only) path >=20 > the comment on this is now a bit misleading, can you try to update it and= /or > move the comment bits to the docs on EARLY_EXIT? >=20 > > @@ -2152,6 +2153,9 @@ vect_can_advance_ivs_p (loop_vec_info > loop_vinfo) > > The phi args associated with the edge UPDATE_E in th= e bb > > UPDATE_E->dest are updated accordingly. > > > > + - restart_loop - Indicates whether the scalar loop needs to > > + restart the >=20 > params are ALL_CAPS >=20 > > + iteration count where the vector loop began. > > + > > Assumption 1: Like the rest of the vectorizer, this function assu= mes > > a single loop exit that has a single predecessor. > > > > @@ -2169,18 +2173,22 @@ vect_can_advance_ivs_p (loop_vec_info > loop_vinfo) > > */ > > > > static void > > -vect_update_ivs_after_vectorizer (loop_vec_info loop_vinfo, > > - tree niters, edge update_e) > > +vect_update_ivs_after_vectorizer (loop_vec_info loop_vinfo, > > +poly_uint64 vf, >=20 > LOOP_VINFO_VECT_FACTOR? >=20 > > + tree niters, edge update_e, bool > restart_loop) >=20 > I think 'bool early_exit' is better here? I wonder if we have an "early" > exit after the main exit we are probably sure there are no side-effects t= o re- > execute and could avoid this restarting? Side effects yes, but the actual check may not have been performed yet. If you remember https://gist.github.com/Mistuke/66f14fe5c1be32b91ce149bd9b8= bb35f There in the clz loop through the "main" exit you still have to see if that= iteration did not contain the entry. This is because the loop counter is incremented before you iterate. >=20 > > { > > gphi_iterator gsi, gsi1; > > class loop *loop =3D LOOP_VINFO_LOOP (loop_vinfo); > > basic_block update_bb =3D update_e->dest; > > - > > - basic_block exit_bb =3D LOOP_VINFO_IV_EXIT (loop_vinfo)->dest; > > - > > - /* Make sure there exists a single-predecessor exit bb: */ > > - gcc_assert (single_pred_p (exit_bb)); > > - gcc_assert (single_succ_edge (exit_bb) =3D=3D update_e); > > + bool inversed_iv > > + =3D !vect_is_loop_exit_latch_pred (LOOP_VINFO_IV_EXIT (loop_vinfo), > > + LOOP_VINFO_LOOP (loop_vinfo)); > > + bool needs_interm_block =3D LOOP_VINFO_EARLY_BREAKS (loop_vinfo) > > + && flow_bb_inside_loop_p (loop, update_e->src); > > + edge loop_e =3D LOOP_VINFO_IV_EXIT (loop_vinfo); > > + gcond *cond =3D get_loop_exit_condition (loop_e); > > + basic_block exit_bb =3D loop_e->dest; > > + basic_block iv_block =3D NULL; > > + gimple_stmt_iterator last_gsi =3D gsi_last_bb (exit_bb); > > > > for (gsi =3D gsi_start_phis (loop->header), gsi1 =3D gsi_start_phis = (update_bb); > > !gsi_end_p (gsi) && !gsi_end_p (gsi1); @@ -2190,7 +2198,6 @@ > > vect_update_ivs_after_vectorizer (loop_vec_info loop_vinfo, > > tree step_expr, off; > > tree type; > > tree var, ni, ni_name; > > - gimple_stmt_iterator last_gsi; > > > > gphi *phi =3D gsi.phi (); > > gphi *phi1 =3D gsi1.phi (); > > @@ -2222,11 +2229,52 @@ vect_update_ivs_after_vectorizer > (loop_vec_info loop_vinfo, > > enum vect_induction_op_type induction_type > > =3D STMT_VINFO_LOOP_PHI_EVOLUTION_TYPE (phi_info); > > > > - if (induction_type =3D=3D vect_step_op_add) > > + tree iv_var =3D PHI_ARG_DEF_FROM_EDGE (phi, loop_latch_edge (loo= p)); > > + /* create_iv always places it on the LHS. Alternatively we can = set a > > + property during create_iv to identify it. */ > > + bool ivtemp =3D gimple_cond_lhs (cond) =3D=3D iv_var; > > + if (restart_loop && ivtemp) > > { > > + type =3D TREE_TYPE (gimple_phi_result (phi)); > > + ni =3D build_int_cst (type, vf); > > + if (inversed_iv) > > + ni =3D fold_build2 (MINUS_EXPR, type, ni, > > + fold_convert (type, step_expr)); > > + } > > + else if (induction_type =3D=3D vect_step_op_add) > > + { > > + > > tree stype =3D TREE_TYPE (step_expr); > > - off =3D fold_build2 (MULT_EXPR, stype, > > - fold_convert (stype, niters), step_expr); > > + > > + /* Early exits always use last iter value not niters. */ > > + if (restart_loop) > > + { > > + /* Live statements in the non-main exit shouldn't be adjusted. = We > > + normally didn't have this problem with a single exit as live > > + values would be in the exit block. However when dealing with > > + multiple exits all exits are redirected to the merge block > > + and we restart the iteration. */ >=20 > Hmm, I fail to see how this works - we're either using the value to conti= nue the > induction or not, independent of STMT_VINFO_LIVE_P. That becomes clear in the patch to update live reductions. Essentially any= live Reductions inside an alternative exit will reduce to the first element rather than the last and use that as the seed for the scalar loop. It has to do this since you have to perform the side effects for the non-ma= tching elements still. Regards, Tamar >=20 > > + if (STMT_VINFO_LIVE_P (phi_info)) > > + continue; > > + > > + /* For early break the final loop IV is: > > + init + (final - init) * vf which takes into account peeling > > + values and non-single steps. The main exit can use niters > > + since if you exit from the main exit you've done all vector > > + iterations. For an early exit we don't know when we exit so > we > > + must re-calculate this on the exit. */ > > + tree start_expr =3D gimple_phi_result (phi); > > + off =3D fold_build2 (MINUS_EXPR, stype, > > + fold_convert (stype, start_expr), > > + fold_convert (stype, init_expr)); > > + /* Now adjust for VF to get the final iteration value. */ > > + off =3D fold_build2 (MULT_EXPR, stype, off, > > + build_int_cst (stype, vf)); > > + } > > + else > > + off =3D fold_build2 (MULT_EXPR, stype, > > + fold_convert (stype, niters), step_expr); > > + > > if (POINTER_TYPE_P (type)) > > ni =3D fold_build_pointer_plus (init_expr, off); > > else > > @@ -2238,6 +2286,8 @@ vect_update_ivs_after_vectorizer (loop_vec_info > loop_vinfo, > > /* Don't bother call vect_peel_nonlinear_iv_init. */ > > else if (induction_type =3D=3D vect_step_op_neg) > > ni =3D init_expr; > > + else if (restart_loop) > > + continue; >=20 > This looks all a bit complicated - why wouldn't we simply always use the = PHI > result when 'restart_loop'? Isn't that the correct old start value in al= l cases? >=20 > > else > > ni =3D vect_peel_nonlinear_iv_init (&stmts, init_expr, > > niters, step_expr, > > @@ -2245,9 +2295,20 @@ vect_update_ivs_after_vectorizer > (loop_vec_info > > loop_vinfo, > > > > var =3D create_tmp_var (type, "tmp"); > > > > - last_gsi =3D gsi_last_bb (exit_bb); > > gimple_seq new_stmts =3D NULL; > > ni_name =3D force_gimple_operand (ni, &new_stmts, false, var); > > + > > + /* For non-main exit create an intermediat edge to get any updat= ed iv > > + calculations. */ > > + if (needs_interm_block > > + && !iv_block > > + && (!gimple_seq_empty_p (stmts) || !gimple_seq_empty_p > (new_stmts))) > > + { > > + iv_block =3D split_edge (update_e); > > + update_e =3D single_succ_edge (update_e->dest); > > + last_gsi =3D gsi_last_bb (iv_block); > > + } > > + > > /* Exit_bb shouldn't be empty. */ > > if (!gsi_end_p (last_gsi)) > > { > > @@ -3342,8 +3403,26 @@ vect_do_peeling (loop_vec_info loop_vinfo, tree > niters, tree nitersm1, > > niters_vector_mult_vf steps. */ > > gcc_checking_assert (vect_can_advance_ivs_p (loop_vinfo)); > > update_e =3D skip_vector ? e : loop_preheader_edge (epilog); > > - vect_update_ivs_after_vectorizer (loop_vinfo, niters_vector_mult= _vf, > > - update_e); > > + if (LOOP_VINFO_EARLY_BREAKS (loop_vinfo)) > > + update_e =3D single_succ_edge (e->dest); > > + bool inversed_iv > > + =3D !vect_is_loop_exit_latch_pred (LOOP_VINFO_IV_EXIT (loop_vinfo), > > + LOOP_VINFO_LOOP (loop_vinfo)); >=20 > You are computing this here and in vect_update_ivs_after_vectorizer? >=20 > > + > > + /* Update the main exit first. */ > > + vect_update_ivs_after_vectorizer (loop_vinfo, vf, niters_vector_= mult_vf, > > + update_e, inversed_iv); > > + > > + /* And then update the early exits. */ > > + for (auto exit : get_loop_exit_edges (loop)) > > + { > > + if (exit =3D=3D LOOP_VINFO_IV_EXIT (loop_vinfo)) > > + continue; > > + > > + vect_update_ivs_after_vectorizer (loop_vinfo, vf, > > + niters_vector_mult_vf, > > + exit, true); >=20 > ... why does the same not work here? Wouldn't the proper condition be > !dominated_by_p (CDI_DOMINATORS, exit->src, LOOP_VINFO_IV_EXIT > (loop_vinfo)->src) or similar? That is, whether the exit is at or after = the main IV > exit? (consider having two) >=20 > > + } > > > > if (skip_epilog) > > { > >