From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR04-VI1-obe.outbound.protection.outlook.com (mail-vi1eur04on2055.outbound.protection.outlook.com [40.107.8.55]) by sourceware.org (Postfix) with ESMTPS id B42723861937 for ; Mon, 6 Nov 2023 07:41:05 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org B42723861937 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=arm.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org B42723861937 Authentication-Results: server2.sourceware.org; arc=pass smtp.remote-ip=40.107.8.55 ARC-Seal: i=3; a=rsa-sha256; d=sourceware.org; s=key; t=1699256467; cv=pass; b=xBrEvNTjbfr14NR/Lf7mhnRAHgTmOdVUjyYj1CXasVSGNU4bxJBvrPSdc0xHbKSW+Nk2uJi2XysrBBM0oUQvEH6L9FA9spa0qh8MabAPcDLDKsQQfo3aIh3QHNSfBooIPJA4KWpbYZtKNS22oO6L5OrzBMmFftt55G2KdeK/Yjs= ARC-Message-Signature: i=3; a=rsa-sha256; d=sourceware.org; s=key; t=1699256467; c=relaxed/simple; bh=QAygj23G0ORueYNoNH9wyw/kTSYmGNgRW9b6eE/nSv8=; h=DKIM-Signature:DKIM-Signature:Date:From:To:Subject:Message-ID: MIME-Version; b=YbbX7EQRcOt3CRBzUTpc3qxUzNLbC5l0fFE3VjbDn1I4B/RZcHvBt8d5B10MxvtxlNXra68LTpGP/JtYkplFcDwnaVt65hM7D/8pzCWhOzERW45Qg0Uq8eAts+fHsStIt3hOo6qfgTJRgXLcIuYM8amZNaRtz9ck3oi+LXOH6nA= ARC-Authentication-Results: i=3; server2.sourceware.org ARC-Seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass; b=bEWy8Zu3MRMIF27MMtCKaHoKR4BLYhJNQ29U8omNxwPoSqTHfncxZjRAe8dtlNmN0Hgxg748N+OgSyXwmN0osRBHaFFQo+TWTu/eUj3RWRaogB2z+d2lEPhJ4i9beTl+gvF2MT+oLvI5VKYunteKDNhcIcUAmuMTlqjk6hSBd4qStUMwDZReZ4eSNr+pwW6OoxETSogBL883PbUHm84wE2VP3wooETDDw9xCIsN0oFso4kOV5SRNjsYloQW2GVS67BeEqEe7CcPLYNrZ9g0DnYeE7zJ9sMBJTf0zAIoLLtIZOzBbYeflNTetn85l1Rd7mM9LaR9bGq9pi/7/gwPumw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=JRvqajftYmCK0LywoTwWitEVFyn1vZW/XBfjdaX61D0=; b=DpsCQ+lXF4RUG11a/cpFsyzQUhHB2hbkz7X3qctLHOGTznYUA/UKJ7mnveBIS58xbGSny5tn43FXkDsMqodIo5QrFo8zdFnOklysDrzDXGQmxoiz3M4jCvR2Za2rF5n8lNZ84HLHr1ZcssjxbcqS4qBUardfJijLvzBsanblYaaHth5Orv6twVZUl4mXPLmNM6Qyt4bJjM9VOWepgR7LiLsc2l8qa6STKQq6H6Lj1KRGwQVkOlLVM3QyvaNqJZPnC5PBfPN+p/giu4Fasp7SRXAdrlhpa3En5LcLlfyDwdNR4licNZL+B6ER9ZTfeHNPKCHOvGare7hAxLEaN2Q/1Q== ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is 63.35.35.123) smtp.rcpttodomain=gcc.gnu.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com; arc=pass (0 oda=1 ltdi=1 spf=[1,1,smtp.mailfrom=arm.com] dkim=[1,1,header.d=arm.com] dmarc=[1,1,header.from=arm.com]) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=JRvqajftYmCK0LywoTwWitEVFyn1vZW/XBfjdaX61D0=; b=dQGnfk6BgU5G8iidKViCf/OBVekJ7PxJCBGbo1tsUyqWnmlg3V4reJ/YGqK2AAQHhQSeczzgX3Ab1kFY8kbXFvsyP5xU8WvaWVIW2j+q20j/a2IwhEvlydetMSOvL5ZjnrJQxJPckqItZNDcko79YtlyNAENGSffh7DL5AIL8KY= Received: from DUZPR01CA0081.eurprd01.prod.exchangelabs.com (2603:10a6:10:46a::6) by PAXPR08MB6623.eurprd08.prod.outlook.com (2603:10a6:102:155::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6954.28; Mon, 6 Nov 2023 07:41:01 +0000 Received: from DB5PEPF00014B9D.eurprd02.prod.outlook.com (2603:10a6:10:46a:cafe::39) by DUZPR01CA0081.outlook.office365.com (2603:10a6:10:46a::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6954.28 via Frontend Transport; Mon, 6 Nov 2023 07:41:01 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; pr=C Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by DB5PEPF00014B9D.mail.protection.outlook.com (10.167.8.164) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6977.16 via Frontend Transport; Mon, 6 Nov 2023 07:41:01 +0000 Received: ("Tessian outbound 20615a7e7970:v228"); Mon, 06 Nov 2023 07:41:01 +0000 X-CheckRecipientChecked: true X-CR-MTA-CID: e64a56eba983efb4 X-CR-MTA-TID: 64aa7808 Received: from a362611094f2.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 15982DA6-742F-461C-AB56-C22DE8490DBB.1; Mon, 06 Nov 2023 07:40:54 +0000 Received: from EUR05-AM6-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id a362611094f2.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Mon, 06 Nov 2023 07:40:54 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Fg2Le6sMRKf9nQ+gHkcZ+5kALVZQAkPv4GyCUXsDjYWhRWOakrKiV4F+rKVql/eWzQIju1n0m9HIMcUvx+dygxldmC7CpmlzTTD9E6Yd40JjSocDzwJPfAICAioldhVH+q192tT19bEZ1WGrMvojjzrWBL69SykskUJPYvFG5vYwjuwlRPBjcpEijRnUPQXnf8Jon8C5NnoOOtF0iclWp4Fao63JDH2m89p1iIb/Pup9qAhJvJGr2SXLlZaEAtQvQe6InrWRv8DLJ8bvJpPspImZ3XMS2bB/zAm8QWhP7lKMXrX9adInyL+nHJ6zjCpRHekGjgapWyrcO5n/orj7+w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=JRvqajftYmCK0LywoTwWitEVFyn1vZW/XBfjdaX61D0=; b=UtUQddhjaV6+vb0SkQK7XBDKEkJ5lhf7LL1WQz4evGaJgHN9ZkfndgaXbfw6VBP+3JHqD2xcMdfGDHAJuRa29KJDOK2e+EU/trzsfGpoBfGhFMhEJQw3uIAETEMbV2xq6guM1Gb2BmQwqA/Bbv70mAdTjoRNsQ8r4VQuBleU4qf29Q0uoC0/WJSsYTZk9G6McEeo9vPaWdBpOnvNBrhaxSZr5ikpPr1FN1T2t2DaVuN9inZcDQUTWYzObb0IafjGHyacB7+rBplYLV2kp1y+7qEOM3xXSyUUDf81/Hh+2jjPYIRtRrDf/7zPwCoDwfO/36IQNEZN35OMhY1FTkKIPQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=JRvqajftYmCK0LywoTwWitEVFyn1vZW/XBfjdaX61D0=; b=dQGnfk6BgU5G8iidKViCf/OBVekJ7PxJCBGbo1tsUyqWnmlg3V4reJ/YGqK2AAQHhQSeczzgX3Ab1kFY8kbXFvsyP5xU8WvaWVIW2j+q20j/a2IwhEvlydetMSOvL5ZjnrJQxJPckqItZNDcko79YtlyNAENGSffh7DL5AIL8KY= Authentication-Results-Original: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; Received: from VI1PR08MB5325.eurprd08.prod.outlook.com (2603:10a6:803:13e::17) by DU0PR08MB9581.eurprd08.prod.outlook.com (2603:10a6:10:44b::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6954.28; Mon, 6 Nov 2023 07:40:52 +0000 Received: from VI1PR08MB5325.eurprd08.prod.outlook.com ([fe80::26aa:efdd:a74a:27d0]) by VI1PR08MB5325.eurprd08.prod.outlook.com ([fe80::26aa:efdd:a74a:27d0%5]) with mapi id 15.20.6954.028; Mon, 6 Nov 2023 07:40:51 +0000 Date: Mon, 6 Nov 2023 07:40:48 +0000 From: Tamar Christina To: gcc-patches@gcc.gnu.org Cc: nd@arm.com, rguenther@suse.de, jlaw@ventanamicro.com Subject: [PATCH 13/21]middle-end: Update loop form analysis to support early break Message-ID: Content-Type: multipart/mixed; boundary="wq4B12A2A70CvW/4" Content-Disposition: inline In-Reply-To: X-ClientProxiedBy: LO4P265CA0209.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:33a::16) To VI1PR08MB5325.eurprd08.prod.outlook.com (2603:10a6:803:13e::17) MIME-Version: 1.0 X-MS-TrafficTypeDiagnostic: VI1PR08MB5325:EE_|DU0PR08MB9581:EE_|DB5PEPF00014B9D:EE_|PAXPR08MB6623:EE_ X-MS-Office365-Filtering-Correlation-Id: 4ad21660-47bd-41db-e940-08dbde9bb93f x-checkrecipientrouted: true NoDisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: VFqIpNwvwqxMaYLbixCbq6XsEvlQ8DmZNbJWiTMIL8SBBtDHM0yoBgT5MI8xt+tAg2+gwoHlooCpwWTPRC7ZcGIIXiDIMdNy2iM13UwvKLi5ceVnV9K0UcRBPwVQv7i0g5KEK/6UhBz0zxYXf5FYasFAF3VJdKXc6iCU8U01TV4pHb/oO1XGbUhql9eoELjrspvBLTixps5y4jbbW6ZxtGx59zZiLAo2Ssn4bQMqOPjd+UpWuojA8U2nRke60NkzHFNX+d1t2Sra3VsfuWaptfEUi8LoLIeQG+2SKtpP/5FXun+hwCDYk+hhL+2Qeokzti6aqXPl4nV6AT1jgfQhFlaJ8LUgYhqUcx4L9+TUcGapvP7MoRp9qJhofIoAspMX3DTLN+bFiXVSAO53l0rbl0PSukPUHGQPfL0qWPMBTKZ8IAUUilHeQzc1D5RlB3crkdiNzQOJIZiXKyzojXh+2lc+qoCFsbiXzhQUGtMYCoMMzRTMv6yIwxmcMYZT9W9zbFzJG2j6QkeQBENfKRV2lxGQgahj8ZPTw0eBs2zLQpzNvOyUn2At/sZyR7ilsM7SBuYd/rB71LrhEZV437VmExn0FO9v15pEqHFYnbmDvnvW5vYO4pJIj6B1Eyisru3+ X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR08MB5325.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(396003)(376002)(136003)(346002)(39860400002)(366004)(230922051799003)(64100799003)(451199024)(186009)(1800799009)(6506007)(478600001)(2616005)(44144004)(6512007)(6666004)(6486002)(33964004)(36756003)(38100700002)(86362001)(66946007)(5660300002)(235185007)(44832011)(316002)(66476007)(66556008)(41300700001)(4743002)(83380400001)(2906002)(15650500001)(26005)(8676002)(4326008)(6916009)(8936002)(2700100001);DIR:OUT;SFP:1101; X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU0PR08MB9581 Original-Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: DB5PEPF00014B9D.eurprd02.prod.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: f1b3a5f9-b395-4571-113b-08dbde9bb3a2 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: oMajB6LCU50wjN+YjaTEGYNV2ExnCOmybi1CO1GnWP4uF/SX69weNBmH70NpCiYGOBlYIeq7bS3+G4tU0+/T1H6Qy7gGYXx0TOzsDZjfKkmO9b/xo+Jg4dZI5fXyRXJaJh0vK5SIQ8ceyXAEZpRAGURFEPDm688Ny26xiz6OY+EIQ94N7ogXVNSPYvEpVKiYbU6H2ktmOV0MVL8gNyUSKgITLNJ3gDsTwJJq3noLMm9oztDJ8fa4ulOwleObLzc+/zVQX66rWKvmpO98XgjSw8RFgbX/7SVXvTvE2D+D70iG53i4tfGZsURPVX4AYOONyXgcdOKeQ+4J/Z4TGVKBSrvFgldkpaImdVlo/vhgko+uH6Azdo5GTlJbn+gT4OqW/WfpJziSgKBGh55hH4m6frwaZlt2LY1mHRnMOgJlCUjTojlEY7FzC2Gf8FQc7BUJ4GUUXaVoHV3E2sWHRUSDC5/ooi6eiL71yj+xKgHuYyl5IHf9DGUrxWb1P9xpovIG9Uma+YTlwqy5d3Fi2BY3KBBeL1K83F66U8vWfHfLblSzH+2fEkOVDpM/7agQUDf4XiN38fg0o0iWLDwTFmIwkM0kdy35ZOzONPNiJu22vdOLbGuSumf4nIxj8SOTDGUTMQuYTSFCPKjDoT7Pu05kv3Y2mkirEy+iZqYBSWhUkoATElgrgib12ELQ2BJrsupm+qmD0WYIctVPgj26xCGS1qCmBvTVYeb8eqdGejWx0y81/BeyTUfaLEGiUmuluStjpJdQLQ64BaWcnRUlHEQv3w== X-Forefront-Antispam-Report: CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(13230031)(4636009)(396003)(346002)(39860400002)(136003)(376002)(230922051799003)(451199024)(64100799003)(82310400011)(1800799009)(186009)(36840700001)(46966006)(40470700004)(40460700003)(40480700001)(6666004)(2616005)(6512007)(44144004)(6506007)(478600001)(6486002)(33964004)(15650500001)(36860700001)(36756003)(86362001)(356005)(82740400003)(81166007)(235185007)(41300700001)(5660300002)(70586007)(70206006)(44832011)(47076005)(83380400001)(4743002)(2906002)(107886003)(26005)(336012)(6916009)(316002)(8676002)(4326008)(8936002)(2700100001);DIR:OUT;SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Nov 2023 07:41:01.2327 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 4ad21660-47bd-41db-e940-08dbde9bb93f X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: DB5PEPF00014B9D.eurprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR08MB6623 X-Spam-Status: No, score=-12.1 required=5.0 tests=BAYES_00,DKIM_SIGNED,DKIM_VALID,FORGED_SPF_HELO,GIT_PATCH_0,KAM_DMARC_NONE,RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE,TXREP,T_SCC_BODY_TEXT_LINE,UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: --wq4B12A2A70CvW/4 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Hi All, This sets LOOP_VINFO_EARLY_BREAKS and does some misc changes so the other patches are self contained. Bootstrapped Regtested on aarch64-none-linux-gnu and no issues. Ok for master? Thanks, Tamar gcc/ChangeLog: * tree-vect-loop.cc (vect_analyze_loop_form): Analyse all exits. (vect_create_loop_vinfo): Set LOOP_VINFO_EARLY_BREAKS. (vect_transform_loop): Use it. --- inline copy of patch -- diff --git a/gcc/tree-vect-loop.cc b/gcc/tree-vect-loop.cc index 51a054c5b035ac80dfbbf3b5ba2f6da82fda91f6..f9483eff6e9606e835906fb991f07cd6052491d0 100644 --- a/gcc/tree-vect-loop.cc +++ b/gcc/tree-vect-loop.cc @@ -1700,12 +1700,12 @@ vect_compute_single_scalar_iteration_cost (loop_vec_info loop_vinfo) loop_vinfo->scalar_costs->finish_cost (nullptr); } - /* Function vect_analyze_loop_form. Verify that certain CFG restrictions hold, including: - the loop has a pre-header - - the loop has a single entry and exit + - the loop has a single entry + - nested loops can have only a single exit. - the loop exit condition is simple enough - the number of iterations can be analyzed, i.e, a countable loop. The niter could be analyzed under some assumptions. */ @@ -1841,10 +1841,14 @@ vect_analyze_loop_form (class loop *loop, vect_loop_form_info *info) "not vectorized: latch block not empty.\n"); /* Make sure the exit is not abnormal. */ - if (exit_e->flags & EDGE_ABNORMAL) - return opt_result::failure_at (vect_location, - "not vectorized:" - " abnormal loop exit edge.\n"); + auto_vec exits = get_loop_exit_edges (loop); + for (edge e : exits) + { + if (e->flags & EDGE_ABNORMAL) + return opt_result::failure_at (vect_location, + "not vectorized:" + " abnormal loop exit edge.\n"); + } info->conds = vect_get_loop_niters (loop, exit_e, &info->assumptions, @@ -1920,6 +1924,10 @@ vect_create_loop_vinfo (class loop *loop, vec_info_shared *shared, LOOP_VINFO_IV_EXIT (loop_vinfo) = info->loop_exit; + /* Check to see if we're vectorizing multiple exits. */ + LOOP_VINFO_EARLY_BREAKS (loop_vinfo) + = !LOOP_VINFO_LOOP_CONDS (loop_vinfo).is_empty (); + if (info->inner_loop_cond) { stmt_vec_info inner_loop_cond_info @@ -11577,7 +11585,7 @@ vect_transform_loop (loop_vec_info loop_vinfo, gimple *loop_vectorized_call) /* Make sure there exists a single-predecessor exit bb. Do this before versioning. */ edge e = LOOP_VINFO_IV_EXIT (loop_vinfo); - if (! single_pred_p (e->dest)) + if (! single_pred_p (e->dest) && !LOOP_VINFO_EARLY_BREAKS (loop_vinfo)) { split_loop_exit_edge (e, true); if (dump_enabled_p ()) -- --wq4B12A2A70CvW/4 Content-Type: text/plain; charset=utf-8 Content-Disposition: attachment; filename="rb17973.patch" diff --git a/gcc/tree-vect-loop.cc b/gcc/tree-vect-loop.cc index 51a054c5b035ac80dfbbf3b5ba2f6da82fda91f6..f9483eff6e9606e835906fb991f07cd6052491d0 100644 --- a/gcc/tree-vect-loop.cc +++ b/gcc/tree-vect-loop.cc @@ -1700,12 +1700,12 @@ vect_compute_single_scalar_iteration_cost (loop_vec_info loop_vinfo) loop_vinfo->scalar_costs->finish_cost (nullptr); } - /* Function vect_analyze_loop_form. Verify that certain CFG restrictions hold, including: - the loop has a pre-header - - the loop has a single entry and exit + - the loop has a single entry + - nested loops can have only a single exit. - the loop exit condition is simple enough - the number of iterations can be analyzed, i.e, a countable loop. The niter could be analyzed under some assumptions. */ @@ -1841,10 +1841,14 @@ vect_analyze_loop_form (class loop *loop, vect_loop_form_info *info) "not vectorized: latch block not empty.\n"); /* Make sure the exit is not abnormal. */ - if (exit_e->flags & EDGE_ABNORMAL) - return opt_result::failure_at (vect_location, - "not vectorized:" - " abnormal loop exit edge.\n"); + auto_vec exits = get_loop_exit_edges (loop); + for (edge e : exits) + { + if (e->flags & EDGE_ABNORMAL) + return opt_result::failure_at (vect_location, + "not vectorized:" + " abnormal loop exit edge.\n"); + } info->conds = vect_get_loop_niters (loop, exit_e, &info->assumptions, @@ -1920,6 +1924,10 @@ vect_create_loop_vinfo (class loop *loop, vec_info_shared *shared, LOOP_VINFO_IV_EXIT (loop_vinfo) = info->loop_exit; + /* Check to see if we're vectorizing multiple exits. */ + LOOP_VINFO_EARLY_BREAKS (loop_vinfo) + = !LOOP_VINFO_LOOP_CONDS (loop_vinfo).is_empty (); + if (info->inner_loop_cond) { stmt_vec_info inner_loop_cond_info @@ -11577,7 +11585,7 @@ vect_transform_loop (loop_vec_info loop_vinfo, gimple *loop_vectorized_call) /* Make sure there exists a single-predecessor exit bb. Do this before versioning. */ edge e = LOOP_VINFO_IV_EXIT (loop_vinfo); - if (! single_pred_p (e->dest)) + if (! single_pred_p (e->dest) && !LOOP_VINFO_EARLY_BREAKS (loop_vinfo)) { split_loop_exit_edge (e, true); if (dump_enabled_p ()) --wq4B12A2A70CvW/4--