From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR01-HE1-obe.outbound.protection.outlook.com (mail-he1eur01on2058.outbound.protection.outlook.com [40.107.13.58]) by sourceware.org (Postfix) with ESMTPS id 5CCE03858D38 for ; Wed, 6 Dec 2023 09:10:30 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 5CCE03858D38 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=arm.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 5CCE03858D38 Authentication-Results: server2.sourceware.org; arc=pass smtp.remote-ip=40.107.13.58 ARC-Seal: i=3; a=rsa-sha256; d=sourceware.org; s=key; t=1701853839; cv=pass; b=yARilo1r7FxMJ059JImjwuabpOmPUGGiMywCjriwDXKYBJ38sZTGIr5OSPmAyNuAvI8wPHda9buFHZhnH3AuW5BUFAOXYn0Ltf1PF6PMQ9OcKaJKv+WvsY+xp+pzW6JBr7Vjl3GAfGKcnSEjhbfKENGHUaUkCyn7Q13VQYFS2qs= ARC-Message-Signature: i=3; a=rsa-sha256; d=sourceware.org; s=key; t=1701853839; c=relaxed/simple; bh=ZGIcs7Q1RlLwEG+y3V/+L/DEMSBT+awgsqdWjbbM9EU=; h=DKIM-Signature:DKIM-Signature:From:To:Subject:Date:Message-ID: MIME-Version; b=lqOa4tftILCTMyhd4smMlRLObLK+WfPTbIodOTWJzSPcQnNA+db1cW8kJ66+dezAvXISfa4C1FLVUuWF0TEi9TbAVxosTQpVwsBgkKnfVvvSQUnSyT5j2+e+mvd8K8mUQlg4H4s9VrlqHu0Vz5VaFL55ON3nfWUgTv5Ye3noxds= ARC-Authentication-Results: i=3; server2.sourceware.org ARC-Seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass; b=Zsm0T9WEKLxxbdRxqAInA6qNKDe3I3t9Y28et/yRFBIObH/OdU12Iuomt3rRYXOKXjTp/AsNc4JrM2lfqmCr/v3D6HoE2/VKPyUk/qzOq8b1++Lqyf+V6xfl0zonUPb+pGulVm6CUxM9BkswuJtLFd66f1UjLv29TylyMmFcChF2G5oynpeEyUMq1uXAPxKIFDcJYDkplQQD0AKbhJWS9GKU/VbK6TNBVUQJE32jgwPnBMfy37wk+PPTYd3U+6uBbh08mJjTEZB/Bv8wfp3Yi/L4hh19StB7N+/owVBh7IbKUnY587z+PVOXvDSHTe/TzoK5WY5ay9WLu4t6awwQow== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Fv4Y/TcGDd8VT9yO03LUMoiXji909aYCRXKu9+ufeBQ=; b=WWvILfG0OLK4yiPPILZeiG7G2RmV/rLDxORXgeoWBDrD6hr/I4P28gCu+jSAsm1xKHXOgdZErwCG5ZH2ynwUquLl28H2tWsGi203K3gjnFA3fN4qpFcZiNarfCVT/Qov3WrwBksi3EZlNQ1GzZH+4IF7POFwLEj4UhcItoUTHsXCKfVhkYtXFccK9fcIZtxhAQWOFfBp6KPrs+KHDVWA/B+Gusn4MqziCcrH9VdPGRIJykXG/nPlf6ww3lI/4/D/jsGx/QWi8JbsZTo9qjaZcmuULa9PxQBZ65tI+erweHyd5tSwJd4fWUybXCN+xhDtsNk/3LaxYfez0FqsG9GC6Q== ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is 63.35.35.123) smtp.rcpttodomain=gcc.gnu.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com; arc=pass (0 oda=1 ltdi=1 spf=[1,1,smtp.mailfrom=arm.com] dkim=[1,1,header.d=arm.com] dmarc=[1,1,header.from=arm.com]) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Fv4Y/TcGDd8VT9yO03LUMoiXji909aYCRXKu9+ufeBQ=; b=8Z/FjgBJlIVmuJ1DqaUUgV79znqQRafiIkV5d8DGWpkrTh38qP6lNOxsFIXNH3rfpzdNnb9QxY2EFPCkhGfS7cugcFuzgfwo5lb0XEnmcQnyxy2/z13+YMIO+RgskCHsNbuajD5KI/4QSuR8n+ECRpQ2clxE15KvGjdl1464jso= Received: from DUZPR01CA0032.eurprd01.prod.exchangelabs.com (2603:10a6:10:468::11) by PR3PR08MB5834.eurprd08.prod.outlook.com (2603:10a6:102:90::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7046.34; Wed, 6 Dec 2023 09:10:25 +0000 Received: from DU6PEPF0000B622.eurprd02.prod.outlook.com (2603:10a6:10:468:cafe::4a) by DUZPR01CA0032.outlook.office365.com (2603:10a6:10:468::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7046.34 via Frontend Transport; Wed, 6 Dec 2023 09:10:25 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; pr=C Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by DU6PEPF0000B622.mail.protection.outlook.com (10.167.8.139) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7068.20 via Frontend Transport; Wed, 6 Dec 2023 09:10:25 +0000 Received: ("Tessian outbound 5d213238733f:v228"); Wed, 06 Dec 2023 09:10:25 +0000 X-CheckRecipientChecked: true X-CR-MTA-CID: 2a869c29c33531c8 X-CR-MTA-TID: 64aa7808 Received: from 3da88a30e36d.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 71DA0138-8C5F-4D69-B44C-3E42C7EF827C.1; Wed, 06 Dec 2023 09:10:19 +0000 Received: from EUR01-HE1-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 3da88a30e36d.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Wed, 06 Dec 2023 09:10:19 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=PjhSDvSah9JUZrMJO2vcrFeURaWbZ7dS+nKKDnlUa3nQQgrfvMYBJLpvGCpRFydo5sm+F3yNKrf6b+1gOq8/utJ6ZN9URifmfgFkHXMCS+3MifyxzMp3RqKlVjO6+OqZE4mknmxrRIsJPLVlk+DqChWH39Odja2DWSTKRghj/hv33XlbDvKbRaIfx5/ANuYxSF49eiWy/BznGSPleFpzIhvBT6wnk+xOm6pz9AdOW3SNGmb6VBE2BxHFKckb1pWEK1+BFziKWybtUYmb3Y75X0/8vbtcOTK2Gbo/nEn2JYgIbqksyRsnv8pt/xcZtmfVbTQSQz+4mmkCMUGW5Ivw1Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Fv4Y/TcGDd8VT9yO03LUMoiXji909aYCRXKu9+ufeBQ=; b=JL+FF1pMo3duip7TjQWVtLEw9RDZTEcO75zs1JTMKgP8YdYHcxOpQJoGowiADbMiPa/JRooKwr7DMrxP7MXocm/lDYb7F65cnIlqMeLgnDZOQw1Ovg1BajpvRmb5vWWbOF4XVHBHMX3A/2HW7tWYmZ7uCwkfy5L3/SkGSXMWAkEGtsoZ7FFC3lQwVlw5L8oqP3dM/XVSARtNhrRF7iSYBx08Nv3FG88CYk/wk1o/PIqb2ZKoWDSuASbqKfLlxRJCqHrI+VJJAHemguxetkhJk+PqfAUv6C+J9Z+nX6U8Y4M1rpnVW8SC0qMtOpxfbTpqvTxgPrIdqQC/IUAii8XMyQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Fv4Y/TcGDd8VT9yO03LUMoiXji909aYCRXKu9+ufeBQ=; b=8Z/FjgBJlIVmuJ1DqaUUgV79znqQRafiIkV5d8DGWpkrTh38qP6lNOxsFIXNH3rfpzdNnb9QxY2EFPCkhGfS7cugcFuzgfwo5lb0XEnmcQnyxy2/z13+YMIO+RgskCHsNbuajD5KI/4QSuR8n+ECRpQ2clxE15KvGjdl1464jso= Received: from VI1PR08MB5325.eurprd08.prod.outlook.com (2603:10a6:803:13e::17) by PAWPR08MB9853.eurprd08.prod.outlook.com (2603:10a6:102:2e6::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7046.34; Wed, 6 Dec 2023 09:10:13 +0000 Received: from VI1PR08MB5325.eurprd08.prod.outlook.com ([fe80::9679:2ab0:99c6:54a3]) by VI1PR08MB5325.eurprd08.prod.outlook.com ([fe80::9679:2ab0:99c6:54a3%6]) with mapi id 15.20.7046.034; Wed, 6 Dec 2023 09:10:13 +0000 From: Tamar Christina To: Richard Biener CC: "gcc-patches@gcc.gnu.org" , nd , "jlaw@ventanamicro.com" Subject: RE: [PATCH 12/21]middle-end: Add remaining changes to peeling and vectorizer to support early breaks Thread-Topic: [PATCH 12/21]middle-end: Add remaining changes to peeling and vectorizer to support early breaks Thread-Index: AQHaEISHXPfUARGhJEW/vHP8j+hCHrCcHESAgAAJeOA= Date: Wed, 6 Dec 2023 09:10:13 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: Authentication-Results-Original: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; x-ms-traffictypediagnostic: VI1PR08MB5325:EE_|PAWPR08MB9853:EE_|DU6PEPF0000B622:EE_|PR3PR08MB5834:EE_ X-MS-Office365-Filtering-Correlation-Id: dfde4f91-f3af-46bd-9d7e-08dbf63b2ecd x-checkrecipientrouted: true nodisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: 8RnlQQlW8X+eH6D9kyli2w/auhd0di1iqlnE8Ur1rpGach2w+ZauzoGLXNST0tRlLruqzB5UUaJ6ZTr9RSQZ74iGpcmtj9nOTD9rlDfSljYS6zAJ89F/FMNRHiJaHvpCegjVV3ri84iEGg/gaGC9uk4qMgi8EH5FykfBqMUjcVlYjDlfwAZhMFBwfDst4Ds/HeHe8gzpLoHeuduPn50bG9XRj0IIP1oK4Ibpr2kwhrLZ+2ZARCNO1As1sqI3RLWRus0g1xzFh79m8mMycgWvuSNrZM88KV8eNs8M6T+n3HX/PFon9jsX1cO/Z4FxJprbBSwCpMDPYr38rCvy70ov9eDbNIYuBuO42BRDCf20hns3SqV7W3M7OmFbsL82eL70TOlsPTMshPgfr4xCumW1LLJnv2n3wx/7iHDkL+U07LRNQLGEapqG6q0hzjLjFgHHzlsGXYVT4dfSXBsE3A5SCOIXRunVLqO/HI/LxVDLxwzQktCE+NmacBlA5HwQ0Qx7ejTaf8C79nmT0ZDjJ9+9BVCqn/JOhQbK5g7i2VQd789OVYnWzCERH2kEUETgj3gfnH5AboPCL35e7Rl43PWMbwVmTBk9dzTnTfmOyC2//szzeMePgkjbH6roOhyL6nn4 X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR08MB5325.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(376002)(366004)(136003)(39860400002)(346002)(396003)(230922051799003)(451199024)(1800799012)(186009)(64100799003)(83380400001)(26005)(38100700002)(122000001)(2906002)(5660300002)(41300700001)(71200400001)(478600001)(9686003)(53546011)(6506007)(7696005)(8936002)(4326008)(8676002)(52536014)(54906003)(316002)(64756008)(66476007)(66556008)(76116006)(66946007)(6916009)(66446008)(55016003)(33656002)(38070700009)(86362001);DIR:OUT;SFP:1101; Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAWPR08MB9853 Original-Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: DU6PEPF0000B622.eurprd02.prod.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: 847e115d-2f72-46ad-7767-08dbf63b2804 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: WGvGDA3QiWK4vU5ze/YwhTmhgE3AiMPIXjr5Jjrc9kqOUWyWznIxKmaXtGdxq5cvIZvHtypB8T4ukxGNbjTcjMGJQ9GeU5tiAkxWsWT9zPNbmr/4/a+DQCzCSP3j4tzPQKsmiR+sDc8OT6lcH9JnPnluK9mvoICkl6snk40MayRRUVd89SXpsNxD3Szk0sieD0cqGX/L/1riwFhN0QgQGCorsXwy2jMs8rCm0TfOXU2U/jI54qXG27QFqStDDGlvbjOyNnbe3POfbf+f2Ji+lSQbBU4DJvvlt30vyt4mjADJJuGsfLq1GrWHH8h7lgFTa6fw7/+1juaTG1eR4IbHCUUByC2qMR7c2MpH6/3JceDLvEk4iVdLRbHUDsQVXIYElPSmrO76jAFiYS9y/r3D9aJz7liBkcigSlL7v35r49+16mAi9JblgZ7PWzmLCstsT7WIIih7gKdQMppS2MxH0WVAH3gvRiT/oSI/EBA6fQcs6bgIH+dbmHSicXcf9Pz+u5fzOGDwGgpxmn+FIuBu1sFuqkK4dHLN92W11GPuacRJSOdZWztnaoV0jibqRdI4Faf+NDym9Kw2ZFr1ySBrsAlbmzeNdvDsUheoXRdxaWAskB4ouZB92OoP6qr8U8WVgsNHkxcEuTUbHvgbejIEl/aTloqw8yt5odnsRxKgPTqprmo6FNe7QjBQ541boUIwMYFPIZA1nAOb2Gwh7OO24/WUxJTGb/+FWfJaj9hlKt8X76TXhdD/9GzuOnW4tU0v X-Forefront-Antispam-Report: CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(13230031)(4636009)(346002)(396003)(39860400002)(136003)(376002)(230922051799003)(82310400011)(64100799003)(451199024)(1800799012)(186009)(40470700004)(36840700001)(46966006)(40460700003)(316002)(81166007)(82740400003)(5660300002)(2906002)(86362001)(41300700001)(36860700001)(33656002)(356005)(336012)(47076005)(6506007)(53546011)(107886003)(26005)(6862004)(478600001)(7696005)(8676002)(52536014)(70586007)(70206006)(9686003)(4326008)(54906003)(8936002)(83380400001)(40480700001)(55016003);DIR:OUT;SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Dec 2023 09:10:25.1787 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: dfde4f91-f3af-46bd-9d7e-08dbf63b2ecd X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: DU6PEPF0000B622.eurprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PR3PR08MB5834 X-Spam-Status: No, score=-12.1 required=5.0 tests=BAYES_00,DKIM_SIGNED,DKIM_VALID,FORGED_SPF_HELO,GIT_PATCH_0,KAM_DMARC_NONE,RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE,TXREP,T_SCC_BODY_TEXT_LINE,UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: > -----Original Message----- > From: Richard Biener > Sent: Wednesday, December 6, 2023 8:32 AM > To: Tamar Christina > Cc: gcc-patches@gcc.gnu.org; nd ; jlaw@ventanamicro.com > Subject: Re: [PATCH 12/21]middle-end: Add remaining changes to peeling an= d > vectorizer to support early breaks >=20 > On Mon, 6 Nov 2023, Tamar Christina wrote: >=20 > > Hi All, > > > > This finishes wiring that didn't fit in any of the other patches. > > Essentially just adding related changes so peeling for early break work= s. > > > > Bootstrapped Regtested on aarch64-none-linux-gnu and no issues. > > > > Ok for master? > > > > Thanks, > > Tamar > > > > gcc/ChangeLog: > > > > * tree-vect-loop-manip.cc (vect_set_loop_condition_normal, > > vect_do_peeling): Support early breaks. > > * tree-vect-loop.cc (vect_need_peeling_or_partial_vectors_p): Likewise= . > > * tree-vectorizer.cc (pass_vectorize::execute): Check all exits. > > > > --- inline copy of patch -- > > diff --git a/gcc/tree-vect-loop-manip.cc b/gcc/tree-vect-loop-manip.cc > > index > eef2bb50c1505f5cf802d5d80300affc2cbe69f6..9c1405d79fd8fe8689007df3b7 > 605b7a3d3ecdd7 100644 > > --- a/gcc/tree-vect-loop-manip.cc > > +++ b/gcc/tree-vect-loop-manip.cc > > @@ -1187,7 +1187,7 @@ vect_set_loop_condition_partial_vectors_avx512 > (class loop *loop, > > loop handles exactly VF scalars per iteration. */ > > > > static gcond * > > -vect_set_loop_condition_normal (loop_vec_info /* loop_vinfo */, edge > exit_edge, > > +vect_set_loop_condition_normal (loop_vec_info loop_vinfo, edge exit_ed= ge, > > class loop *loop, tree niters, tree step, > > tree final_iv, bool niters_maybe_zero, > > gimple_stmt_iterator loop_cond_gsi) > > @@ -1296,7 +1296,8 @@ vect_set_loop_condition_normal (loop_vec_info /* > loop_vinfo */, edge exit_edge, > > gsi_insert_before (&loop_cond_gsi, cond_stmt, GSI_SAME_STMT); > > > > /* Record the number of latch iterations. */ > > - if (limit =3D=3D niters) > > + if (limit =3D=3D niters > > + || LOOP_VINFO_EARLY_BREAKS (loop_vinfo)) > > /* Case A: the loop iterates NITERS times. Subtract one to get th= e > > latch count. */ > > loop->nb_iterations =3D fold_build2 (MINUS_EXPR, niters_type, nite= rs, > > @@ -3242,6 +3243,16 @@ vect_do_peeling (loop_vec_info loop_vinfo, tree > niters, tree nitersm1, > > bound_epilog +=3D vf - 1; > > if (LOOP_VINFO_PEELING_FOR_GAPS (loop_vinfo)) > > bound_epilog +=3D 1; > > + > > + /* For early breaks the scalar loop needs to execute at most VF time= s > > + to find the element that caused the break. */ > > + if (LOOP_VINFO_EARLY_BREAKS (loop_vinfo)) > > + { > > + bound_epilog =3D vf; > > + /* Force a scalar epilogue as we can't vectorize the index findi= ng. */ > > + vect_epilogues =3D false; >=20 > This is originally initialized with >=20 > bool vect_epilogues =3D loop_vinfo->epilogue_vinfos.length () > 0; >=20 > so I think we should avoid filling that with LOOP_VINFO_EARLY_BREAKS > rather than fixing up after the fact? That is in vect_analyze_loop > adjust >=20 > /* Only vectorize epilogues if PARAM_VECT_EPILOGUES_NOMASK is > enabled, SIMDUID is not set, it is the innermost loop and we have > either already found the loop's SIMDLEN or there was no SIMDLEN to > begin with. > TODO: Enable epilogue vectorization for loops with SIMDUID set. */ > bool vect_epilogues =3D (!simdlen > && loop->inner =3D=3D NULL > && param_vect_epilogues_nomask > && LOOP_VINFO_PEELING_FOR_NITER > (first_loop_vinfo) > && !loop->simduid); >=20 > and add !LOOP_VINFO_EARLY_BREAKS? >=20 > > + } > > + > > bool epilog_peeling =3D maybe_ne (bound_epilog, 0U); > > poly_uint64 bound_scalar =3D bound_epilog; > > > > @@ -3376,14 +3387,23 @@ vect_do_peeling (loop_vec_info loop_vinfo, tree > niters, tree nitersm1, > > bound_prolog + bound_epilog) > > : (!LOOP_REQUIRES_VERSIONING (loop_vinfo) > > || vect_epilogues)); > > + > > + /* We only support early break vectorization on known bounds at this= time. > > + This means that if the vector loop can't be entered then we won't= generate > > + it at all. So for now force skip_vector off because the addition= al control > > + flow messes with the BB exits and we've already analyzed them. *= / > > + skip_vector =3D skip_vector && !LOOP_VINFO_EARLY_BREAKS (loop_vinfo); > > + >=20 > bool skip_vector =3D (LOOP_VINFO_NITERS_KNOWN_P (loop_vinfo) > ? maybe_lt (LOOP_VINFO_INT_NITERS (loop_vinfo), > bound_prolog + bound_epilog) > : (!LOOP_REQUIRES_VERSIONING (loop_vinfo) > || vect_epilogues)); >=20 > to me that looks like >=20 > gcc_assert (!skip_vector || !LOOP_VINFO_EARLY_BREAKS (loop_vinfo)); >=20 > should work? You are basically relying on cost modeling rejecting > vectorization that doesn't enter the vector loop. >=20 > > /* Epilog loop must be executed if the number of iterations for epil= og > > loop is known at compile time, otherwise we need to add a check a= t > > the end of vector loop and skip to the end of epilog loop. */ > > bool skip_epilog =3D (prolog_peeling < 0 > > || !LOOP_VINFO_NITERS_KNOWN_P (loop_vinfo) > > || !vf.is_constant ()); > > - /* PEELING_FOR_GAPS is special because epilog loop must be executed.= */ > > - if (LOOP_VINFO_PEELING_FOR_GAPS (loop_vinfo)) > > + /* PEELING_FOR_GAPS and peeling for early breaks are special because= epilog > > + loop must be executed. */ > > + if (LOOP_VINFO_PEELING_FOR_GAPS (loop_vinfo) > > + || LOOP_VINFO_EARLY_BREAKS (loop_vinfo)) > > skip_epilog =3D false; > > > > class loop *scalar_loop =3D LOOP_VINFO_SCALAR_LOOP (loop_vinfo); > > diff --git a/gcc/tree-vect-loop.cc b/gcc/tree-vect-loop.cc > > index > 55d6aee3d29151e6b528f6fdde15c693e5bdd847..51a054c5b035ac80dfbbf3b5 > ba2f6da82fda91f6 100644 > > --- a/gcc/tree-vect-loop.cc > > +++ b/gcc/tree-vect-loop.cc > > @@ -1236,6 +1236,14 @@ vect_need_peeling_or_partial_vectors_p > (loop_vec_info loop_vinfo) > > th =3D LOOP_VINFO_COST_MODEL_THRESHOLD > (LOOP_VINFO_ORIG_LOOP_INFO > > (loop_vinfo)); > > > > + /* When we have multiple exits and VF is unknown, we must require pa= rtial > > + vectors because the loop bounds is not a minimum but a maximum. = That is > to > > + say we cannot unpredicate the main loop unless we peel or use par= tial > > + vectors in the epilogue. */ > > + if (LOOP_VINFO_EARLY_BREAKS (loop_vinfo) > > + && !LOOP_VINFO_VECT_FACTOR (loop_vinfo).is_constant ()) > > + return true; > > + >=20 > I don't quite understand the !.is_constant (), early breaks always > require peeling? It's mostly to force the use of an unpredicated main loop, since this forces LOOP_VINFO_PEELING_FOR_NITER. I can alternatively not set it and then mask the individual comparisons in vectorizable_early_break. Both work equally, I guess that may be better since the assumption is that the break is not taken. Any preference? Thanks, Tamar >=20 > > if (LOOP_VINFO_NITERS_KNOWN_P (loop_vinfo) > > && LOOP_VINFO_PEELING_FOR_ALIGNMENT (loop_vinfo) >=3D 0) > > { > > @@ -3149,7 +3157,8 @@ start_over: > > > > /* If an epilogue loop is required make sure we can create one. */ > > if (LOOP_VINFO_PEELING_FOR_GAPS (loop_vinfo) > > - || LOOP_VINFO_PEELING_FOR_NITER (loop_vinfo)) > > + || LOOP_VINFO_PEELING_FOR_NITER (loop_vinfo) > > + || LOOP_VINFO_EARLY_BREAKS (loop_vinfo)) > > { > > if (dump_enabled_p ()) > > dump_printf_loc (MSG_NOTE, vect_location, "epilog loop require= d\n"); > > diff --git a/gcc/tree-vectorizer.cc b/gcc/tree-vectorizer.cc > > index > d97e2b54c25ac60378935392aa7b73476efed74b..8b495fc7ca137109fc0097c7 > 7837233c660e032a 100644 > > --- a/gcc/tree-vectorizer.cc > > +++ b/gcc/tree-vectorizer.cc > > @@ -1381,7 +1381,9 @@ pass_vectorize::execute (function *fun) > > predicates that need to be shared for optimal predicate usage. > > However reassoc will re-order them and prevent CSE from working > > as it should. CSE only the loop body, not the entry. */ > > - bitmap_set_bit (exit_bbs, single_exit (loop)->dest->index); > > + auto_vec exits =3D get_loop_exit_edges (loop); > > + for (edge exit : exits) > > + bitmap_set_bit (exit_bbs, exit->dest->index); > > > > edge entry =3D EDGE_PRED (loop_preheader_edge (loop)->src, 0); > > do_rpo_vn (fun, entry, exit_bbs); >=20 > Otherwise OK. >=20 > Richard.