From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR04-HE1-obe.outbound.protection.outlook.com (mail-he1eur04on2042.outbound.protection.outlook.com [40.107.7.42]) by sourceware.org (Postfix) with ESMTPS id A67393858414 for ; Sun, 25 Feb 2024 15:36:21 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org A67393858414 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=arm.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org A67393858414 Authentication-Results: server2.sourceware.org; arc=pass smtp.remote-ip=40.107.7.42 ARC-Seal: i=3; a=rsa-sha256; d=sourceware.org; s=key; t=1708875384; cv=pass; b=DObOtH4Im+OTw7nPmKXb+51aW3XkQLxw7PLGg12E8eAVvL35N6uFsmDLQkaHUO3CtqFlXvFXuzWA67CVS5Yxu+ocF9/C7TtB09tkkrQkEAO3nyswMB36z5iKMzkGXSMHJKahshZqT06vDb0vfR/wf/fhdrazCThnHYtlNP+ydDM= ARC-Message-Signature: i=3; a=rsa-sha256; d=sourceware.org; s=key; t=1708875384; c=relaxed/simple; bh=nKXeEfl/vFqA+KZC4mLSIGCXSdVMUHPiIh/6nPLEtMI=; h=DKIM-Signature:DKIM-Signature:Date:From:To:Subject:Message-ID: MIME-Version; b=B7wSLBIhJJoUpMvjyWDRcbieKkGsi+EysAqOQLsVLVo1WuBULt7fPcUEeK8tmCHxDTD79KfOg/GQniDQt9ax/rzatIMx8eV0is7QkPcoyVbZRRHOcV8+VCHLm4IuQX87cpM5XiSWxMKgDsa47kgkwJjnr3mAaIvGhwebs7nTpLE= ARC-Authentication-Results: i=3; server2.sourceware.org ARC-Seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass; b=kBZJRjOE4qyH4D6WQbPG+wTLkLwWfcd5goOR27/pA312UF3WpN8yykt0wNoteENt1yAnfJpFUPgsX1KrVICiS+rbu7x6aW2ptEc4KNZqsQuHoFqRJDHlr0FapdiNCBds0tOu5bxVSe3KA38S1mYqufmea4R60rL1bH6y6MNljPvQKwNqp4DKMKT3kVEYXZelPgMACHw2pqqYuV31ee7vpkzY2Yx7wdsdllnB8yneZ1kOmQ50tunv9aIT7OJupzwIIf19ll9X16toCGqoGLeD7XlQP0PrxIqJDgQtCOcs8+Xrh+nHfwMbrl9HbTMqpBP4vpDlRSaoBNa8wrINpgDybA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=dfMNf4YQpFnh+a4ikDe1funkWGv62ExclzMV6i1Wjic=; b=WT83/Oaatah2YqCgtMbSNWG6b2x5bNpNm4IlEmdSsK2tuWoaqdcQzyZ1P1F51g5GaN/FizPPMGDXlwWjiZVx7Q6mC0sIaWQNmtTX93IEPmTLIrxiMKJWzx5ozu9u1JogH4lGQ9bIYceJxuMQyP+gFvuPGIN0xhttvceVzAniq2KZFHIgSFebE9KYwHE0/1L4UbvL460aLdeloRwzg/7ntBQfK+F6Jm0AVoeui9kkmdjq8JYRcmvRrHDXsS4cXzeKo2EyezB3qC/yy7ZRv9chzPn050O2ViYuKi9DLW5LiKtAMkjC7GPBsClsd2XuawBMVLAwGPiuo3jjh3HeCLX0ew== ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is 63.35.35.123) smtp.rcpttodomain=gcc.gnu.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com; arc=pass (0 oda=1 ltdi=1 spf=[1,1,smtp.mailfrom=arm.com] dkim=[1,1,header.d=arm.com] dmarc=[1,1,header.from=arm.com]) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=dfMNf4YQpFnh+a4ikDe1funkWGv62ExclzMV6i1Wjic=; b=dwkaou4UPNRgnbW21On+CjlOLkOIKY9xfVnf13gOaFKrCi5zyszot/FcWWsGa6VOqix0O4yyl9Ww6IujeYBfX1puNe2DJaPMhx0NIqsp6cFfg+p1x48ck3o8ejdackJcOV96oplejRgIQWqMadPhBZ1rHM6+ulSZ10KmDZauH7g= Received: from AS9PR06CA0017.eurprd06.prod.outlook.com (2603:10a6:20b:462::13) by AS8PR08MB6279.eurprd08.prod.outlook.com (2603:10a6:20b:294::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7316.33; Sun, 25 Feb 2024 15:36:16 +0000 Received: from AM4PEPF00027A66.eurprd04.prod.outlook.com (2603:10a6:20b:462:cafe::67) by AS9PR06CA0017.outlook.office365.com (2603:10a6:20b:462::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.49 via Frontend Transport; Sun, 25 Feb 2024 15:36:16 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; pr=C Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by AM4PEPF00027A66.mail.protection.outlook.com (10.167.16.91) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.25 via Frontend Transport; Sun, 25 Feb 2024 15:36:15 +0000 Received: ("Tessian outbound 778f5fac34db:v228"); Sun, 25 Feb 2024 15:36:15 +0000 X-CheckRecipientChecked: true X-CR-MTA-CID: b928ee9afccb0740 X-CR-MTA-TID: 64aa7808 Received: from 828a15340cd7.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 2DE349A9-AFD1-4437-A896-3725F73A34B2.1; Sun, 25 Feb 2024 15:36:08 +0000 Received: from EUR05-DB8-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 828a15340cd7.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Sun, 25 Feb 2024 15:36:08 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=MzaZQ5zRpACIIvvIc6aWPX8c2t+RYqoXEVjOuj6rB3v4HceGWYo62zOBx6g7KThNqSE91p2lXFzQSM2A5Hviby0o9h4CPLJLUrzmSad1wzGbEzsC8K/L04kypPapZ+rV7jyol++jBXIFNPmMlAXz0dXXXDqs1SZCWw99vwFSktykJe6ZysIfh0wFkvNEbehQ3jHsD5H+LeUKA/TvM96+z3T4Crf5K8Nww3K0KAKHSRqVgoE9RuD24CS57BQnXOEGyrPv5FQoVdn923TI7qqJI8vtXmZZN5xAcup4GAlTxe08niueDgzMbu28lzi2Eb8+HTFecbMPKpSb6nKMTuOLAw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=dfMNf4YQpFnh+a4ikDe1funkWGv62ExclzMV6i1Wjic=; b=mqAXmkkjPcvaXlr4xtqAfds6mEJs6xBhorYoEzh5SZfxKzUFiFo2D4Rft2fcl6tpc0rN7Z1bPal6FnFPOczr+gRykLS0478A3egG0gsLLOlgWBmxq7+0h+B3/HQSELkf+21PIAxvbOuIbVyypDsEIKlbZVQ64m2d/ibFTTFsqW758LSr/4q1DuCGM/y/Bvwv4eN9uaOiXFuE0QtpfVJmRH3cdjEZaRXq0G2yBTS7D2jP1YGh9/RszRTOMHEXXWUrQ6OsrKwARQjlk/vhhHSZrUN8E3ggBP2klGQV4tk/+6cBBfSFpsO5WPrNoY1fZtLhwfGe57lqNaNzcaXg2Ffbng== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=dfMNf4YQpFnh+a4ikDe1funkWGv62ExclzMV6i1Wjic=; b=dwkaou4UPNRgnbW21On+CjlOLkOIKY9xfVnf13gOaFKrCi5zyszot/FcWWsGa6VOqix0O4yyl9Ww6IujeYBfX1puNe2DJaPMhx0NIqsp6cFfg+p1x48ck3o8ejdackJcOV96oplejRgIQWqMadPhBZ1rHM6+ulSZ10KmDZauH7g= Authentication-Results-Original: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; Received: from VI1PR08MB5325.eurprd08.prod.outlook.com (2603:10a6:803:13e::17) by DU0PR08MB8561.eurprd08.prod.outlook.com (2603:10a6:10:406::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7316.33; Sun, 25 Feb 2024 15:36:05 +0000 Received: from VI1PR08MB5325.eurprd08.prod.outlook.com ([fe80::810c:8495:3f0a:ef8]) by VI1PR08MB5325.eurprd08.prod.outlook.com ([fe80::810c:8495:3f0a:ef8%7]) with mapi id 15.20.7316.032; Sun, 25 Feb 2024 15:36:05 +0000 Date: Sun, 25 Feb 2024 15:36:03 +0000 From: Tamar Christina To: gcc-patches@gcc.gnu.org Cc: nd@arm.com, rguenther@suse.de, jlaw@ventanamicro.com Subject: [PATCH]middle-end: delay updating of dominators until later during vectorization. [PR114081] Message-ID: Content-Type: multipart/mixed; boundary="4dXfLpipNSx4/eZz" Content-Disposition: inline X-ClientProxiedBy: LO2P123CA0100.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:139::15) To VI1PR08MB5325.eurprd08.prod.outlook.com (2603:10a6:803:13e::17) MIME-Version: 1.0 X-MS-TrafficTypeDiagnostic: VI1PR08MB5325:EE_|DU0PR08MB8561:EE_|AM4PEPF00027A66:EE_|AS8PR08MB6279:EE_ X-MS-Office365-Filtering-Correlation-Id: 84113302-aad7-4c78-75dc-08dc36178122 x-checkrecipientrouted: true NoDisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: o0QT6gYIAEbm2ql28chRfofoTKZ1QBB79xA4M8WNZA1jKaWp5TYEunHGToJkm1opoOnixC2edhEbLj8FektMSHr6r8O88wpxVZ8iTUWD3gPZ+ETeQJuQFzTCI9eqPMliS0yFNv4BxlT3PBN9P3dP0ICNmyGpVjHmUiNWWlq2YdsGyN5EqzrWkKWXL6zRW4eXTzpmKfipIc9/pBKKsF7vHmQIvJtSfYjxwUcHkMEU7mEwbJwOzace2Cg9G5SjXZ6U6V+4h4/+hmO8xJtuclOe71A/vgxcYSkLYVLm4KBmo9kJlx83vHsDgqGkobkLQ25f5quG1qtAOUVwmJU8KF3xUMnSY6xcpKfnX2d3YYrGG57gXpZuXvf21Sx2ugf8ydzouDC+t5UeLZy4O5NSqEyk1rgi0IRz/ZY8REUw2gkM46aOSr+NbD+J4uUwf+3Yy65o8TWhV00OoQl8DI4DQd2SttS1U8J18AKn77Lel6a3udHKPraKPcH0k9gGQyEoZSLctuKlu3bz+RNA1U6BBGgtBQ== X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR08MB5325.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031);DIR:OUT;SFP:1101; X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU0PR08MB8561 Original-Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: AM4PEPF00027A66.eurprd04.prod.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: 2b4dbfbc-96da-4c3e-6744-08dc36177b05 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: ZSdA8GV1n8lJrTx1wam9aOsr1hpDVaIXkPEACLGstSPhVRLBGr2CxqaH2pzCC1xVRcL6CIg4JwBM/5QjkjyG/x/kqEStxVp0rQc1C1ChOmZWK2pCsIEpsQ6BCuBs/J0aWEc/D2SeAjQuKrHYy4FLw5JEwi3BzcIjUusjOFX1VmVGG4i92tXTVrwcFW1rPRYPh60nvjn1LxiTmTY78TinW4w5FPDI0rDn1C1UNe7c4DKfliRtJGP1QhiOUaCTSmtYaO3Dh2BVcTwg7Acn7UsqGd4DfnLfLp01H9YHul06U4eGdstVlu/cBOv3kesKz3tmb588hAH3jJZ54fFd1NU/BHWh+dia6qsbFr72FbwTlxb5YBuPhvJ39yAw3V+J/DS/NUhX1s2wINBmyBF6wHSFhiEJ48hMjoaNNC5Bo2muOUfcm53eOzBfJn9aPHDjeDt5RUDKAH9kB3iaYsxtRl07RNgFePL21Y4WgKaDIzw0Ai3xtvC3qxJl6+XTWANE7m7y/fBaJm8Yogh4yNea3Yt+bj/qgFkB9qzfgnsPtp/+wBYOa1+lhc0gBCgs0+UlcTl9H2ywC52IkTwNX2yAPvE/Ug9Y0t+gwynkcmWDdy819jF3eDohm2u4kmkcvJjcERTlKKRrOK+1FgoBGaHaptp8Dg== X-Forefront-Antispam-Report: CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(13230031)(36860700004)(46966006);DIR:OUT;SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Feb 2024 15:36:15.7653 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 84113302-aad7-4c78-75dc-08dc36178122 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: AM4PEPF00027A66.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB6279 X-Spam-Status: No, score=-11.2 required=5.0 tests=BAYES_00,DKIM_SIGNED,DKIM_VALID,FORGED_SPF_HELO,GIT_PATCH_0,KAM_DMARC_NONE,KAM_LOTSOFHASH,RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE,TXREP,T_SCC_BODY_TEXT_LINE,UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: --4dXfLpipNSx4/eZz Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Hi All, The testcase shows an interesting case where we have multiple loops sharing a live value and have an early exit that go to the same location. The additional complication is that on x86_64 with -mavx we seem to also do prologue peeling on the loops. We correctly identify which BB we need their dominators updated for, but we do so too early. Instead of adding more dominator update we can solve this by for the cases with multiple exits not to verify dominators at the end of peeling if peeling for vectorization. We can then perform the final dominator updates just before vectorization when all loop transformations are done. This also means we reduce the number of dominator updates needed by at least 50% and fixes the ICE. Bootstrapped Regtested on aarch64-none-linux-gnu and x86_64-pc-linux-gnu no issues. Ok for master? Thanks, Tamar gcc/ChangeLog: PR tree-optimization/114081 PR tree-optimization/113290 * tree-vect-loop-manip.cc (slpeel_tree_duplicate_loop_to_edge_cfg): Skip dominator update when multiple exit. (vect_do_peeling): Remove multiple exit dominator update. * tree-vect-loop.cc (vect_transform_loop): Update dominators when multiple exits. * tree-vectorizer.h (LOOP_VINFO_DOMS_NEED_UPDATE, dominators_needing_update): New. gcc/testsuite/ChangeLog: PR tree-optimization/114081 PR tree-optimization/113290 * gcc.dg/vect/vect-early-break_120-pr114081.c: New test. * gcc.dg/vect/vect-early-break_121-pr114081.c: New test. --- inline copy of patch -- diff --git a/gcc/testsuite/gcc.dg/vect/vect-early-break_120-pr114081.c b/gcc/testsuite/gcc.dg/vect/vect-early-break_120-pr114081.c new file mode 100644 index 0000000000000000000000000000000000000000..2cd4ce1e4ac573ba6e41730fd2216f0ec8061376 --- /dev/null +++ b/gcc/testsuite/gcc.dg/vect/vect-early-break_120-pr114081.c @@ -0,0 +1,38 @@ +/* { dg-do compile } */ +/* { dg-add-options vect_early_break } */ +/* { dg-require-effective-target vect_early_break } */ +/* { dg-require-effective-target vect_int } */ +/* { dg-additional-options "-O3" } */ + +/* { dg-final { scan-tree-dump "LOOP VECTORIZED" "vect" } } */ + +typedef struct filter_list_entry { + const char *name; + int id; + void (*function)(); +} filter_list_entry; + +static const filter_list_entry filter_list[9] = {0}; + +void php_zval_filter(int filter, int id1) { + filter_list_entry filter_func; + + int size = 9; + for (int i = 0; i < size; ++i) { + if (filter_list[i].id == filter) { + filter_func = filter_list[i]; + goto done; + } + } + +#pragma GCC novector + for (int i = 0; i < size; ++i) { + if (filter_list[i].id == 0x0204) { + filter_func = filter_list[i]; + goto done; + } + } +done: + if (!filter_func.id) + filter_func.function(); +} diff --git a/gcc/testsuite/gcc.dg/vect/vect-early-break_121-pr114081.c b/gcc/testsuite/gcc.dg/vect/vect-early-break_121-pr114081.c new file mode 100644 index 0000000000000000000000000000000000000000..feebdb7a6c9b8981d7be31dd1c741f9e36738515 --- /dev/null +++ b/gcc/testsuite/gcc.dg/vect/vect-early-break_121-pr114081.c @@ -0,0 +1,37 @@ +/* { dg-do compile } */ +/* { dg-add-options vect_early_break } */ +/* { dg-require-effective-target vect_early_break } */ +/* { dg-require-effective-target vect_int } */ +/* { dg-additional-options "-O3" } */ + +/* { dg-final { scan-tree-dump "LOOP VECTORIZED" "vect" } } */ + +typedef struct filter_list_entry { + const char *name; + int id; + void (*function)(); +} filter_list_entry; + +static const filter_list_entry filter_list[9] = {0}; + +void php_zval_filter(int filter, int id1) { + filter_list_entry filter_func; + + int size = 9; + for (int i = 0; i < size; ++i) { + if (filter_list[i].id == filter) { + filter_func = filter_list[i]; + goto done; + } + } + + for (int i = 0; i < size; ++i) { + if (filter_list[i].id == 0x0204) { + filter_func = filter_list[i]; + goto done; + } + } +done: + if (!filter_func.id) + filter_func.function(); +} diff --git a/gcc/tree-vect-loop-manip.cc b/gcc/tree-vect-loop-manip.cc index 3f974d6d839e32516ae316f28ca25316e43d7d86..b5e158bc5cfb5107d5ff461e489d306f81e090d0 100644 --- a/gcc/tree-vect-loop-manip.cc +++ b/gcc/tree-vect-loop-manip.cc @@ -1917,7 +1917,6 @@ slpeel_tree_duplicate_loop_to_edge_cfg (class loop *loop, edge loop_exit, doms.safe_push (e->dest); } - iterate_fix_dominators (CDI_DOMINATORS, doms, false); if (updated_doms) updated_doms->safe_splice (doms); } @@ -1925,7 +1924,9 @@ slpeel_tree_duplicate_loop_to_edge_cfg (class loop *loop, edge loop_exit, free (new_bbs); free (bbs); - checking_verify_dominators (CDI_DOMINATORS); + /* If we're peeling for vectorization then delay verifying dominators. */ + if (!flow_loops || !multiple_exits_p) + checking_verify_dominators (CDI_DOMINATORS); return new_loop; } @@ -3404,7 +3405,7 @@ vect_do_peeling (loop_vec_info loop_vinfo, tree niters, tree nitersm1, epilog = vect_epilogues ? get_loop_copy (loop) : scalar_loop; edge epilog_e = vect_epilogues ? e : scalar_e; edge new_epilog_e = NULL; - auto_vec doms; + auto_vec& doms = LOOP_VINFO_DOMS_NEED_UPDATE (loop_vinfo); epilog = slpeel_tree_duplicate_loop_to_edge_cfg (loop, e, epilog, epilog_e, e, &new_epilog_e, true, &doms); @@ -3554,10 +3555,6 @@ vect_do_peeling (loop_vec_info loop_vinfo, tree niters, tree nitersm1, scale_loop_profile (epilog, prob_epilog, -1); } - /* Recalculate the dominators after adding the guard edge. */ - if (LOOP_VINFO_EARLY_BREAKS (loop_vinfo)) - iterate_fix_dominators (CDI_DOMINATORS, doms, false); - /* When we do not have a loop-around edge to the epilog we know the vector loop covered at least VF scalar iterations unless we have early breaks. diff --git a/gcc/tree-vect-loop.cc b/gcc/tree-vect-loop.cc index 35f1f8c7d4245135ace7ffff40ff9be548919587..ab19ad6a6be516e3ee1f0fbeaaeeffeae1fb900f 100644 --- a/gcc/tree-vect-loop.cc +++ b/gcc/tree-vect-loop.cc @@ -11987,7 +11987,12 @@ vect_transform_loop (loop_vec_info loop_vinfo, gimple *loop_vectorized_call) /* Handle any code motion that we need to for early-break vectorization after we've done peeling but just before we start vectorizing. */ if (LOOP_VINFO_EARLY_BREAKS (loop_vinfo)) - move_early_exit_stmts (loop_vinfo); + { + /* Recalculate the dominators. */ + iterate_fix_dominators (CDI_DOMINATORS, + LOOP_VINFO_DOMS_NEED_UPDATE (loop_vinfo), false); + move_early_exit_stmts (loop_vinfo); + } /* Schedule the SLP instances first, then handle loop vectorization below. */ diff --git a/gcc/tree-vectorizer.h b/gcc/tree-vectorizer.h index db44d730b7026492c4dd8c394ab505f227edfc8e..6bb5b6d69a8340c0b68cbcbbb110c0380b7d73aa 100644 --- a/gcc/tree-vectorizer.h +++ b/gcc/tree-vectorizer.h @@ -961,6 +961,10 @@ public: /* Statements whose VUSES need updating if early break vectorization is to happen. */ auto_vec early_break_vuses; + + /* Dominators that need to be recalculated that have been deferred until after + all peeling. */ + auto_vec dominators_needing_update; } *loop_vec_info; /* Access Functions. */ @@ -1021,6 +1025,7 @@ public: (single_pred ((L)->loop->latch) != (L)->vec_loop_iv_exit->src) #define LOOP_VINFO_EARLY_BRK_DEST_BB(L) (L)->early_break_dest_bb #define LOOP_VINFO_EARLY_BRK_VUSES(L) (L)->early_break_vuses +#define LOOP_VINFO_DOMS_NEED_UPDATE(L) (L)->dominators_needing_update #define LOOP_VINFO_LOOP_CONDS(L) (L)->conds #define LOOP_VINFO_LOOP_IV_COND(L) (L)->loop_iv_cond #define LOOP_VINFO_NO_DATA_DEPENDENCIES(L) (L)->no_data_dependencies -- --4dXfLpipNSx4/eZz Content-Type: text/plain; charset=utf-8 Content-Disposition: attachment; filename="rb18326.patch" diff --git a/gcc/testsuite/gcc.dg/vect/vect-early-break_120-pr114081.c b/gcc/testsuite/gcc.dg/vect/vect-early-break_120-pr114081.c new file mode 100644 index 0000000000000000000000000000000000000000..2cd4ce1e4ac573ba6e41730fd2216f0ec8061376 --- /dev/null +++ b/gcc/testsuite/gcc.dg/vect/vect-early-break_120-pr114081.c @@ -0,0 +1,38 @@ +/* { dg-do compile } */ +/* { dg-add-options vect_early_break } */ +/* { dg-require-effective-target vect_early_break } */ +/* { dg-require-effective-target vect_int } */ +/* { dg-additional-options "-O3" } */ + +/* { dg-final { scan-tree-dump "LOOP VECTORIZED" "vect" } } */ + +typedef struct filter_list_entry { + const char *name; + int id; + void (*function)(); +} filter_list_entry; + +static const filter_list_entry filter_list[9] = {0}; + +void php_zval_filter(int filter, int id1) { + filter_list_entry filter_func; + + int size = 9; + for (int i = 0; i < size; ++i) { + if (filter_list[i].id == filter) { + filter_func = filter_list[i]; + goto done; + } + } + +#pragma GCC novector + for (int i = 0; i < size; ++i) { + if (filter_list[i].id == 0x0204) { + filter_func = filter_list[i]; + goto done; + } + } +done: + if (!filter_func.id) + filter_func.function(); +} diff --git a/gcc/testsuite/gcc.dg/vect/vect-early-break_121-pr114081.c b/gcc/testsuite/gcc.dg/vect/vect-early-break_121-pr114081.c new file mode 100644 index 0000000000000000000000000000000000000000..feebdb7a6c9b8981d7be31dd1c741f9e36738515 --- /dev/null +++ b/gcc/testsuite/gcc.dg/vect/vect-early-break_121-pr114081.c @@ -0,0 +1,37 @@ +/* { dg-do compile } */ +/* { dg-add-options vect_early_break } */ +/* { dg-require-effective-target vect_early_break } */ +/* { dg-require-effective-target vect_int } */ +/* { dg-additional-options "-O3" } */ + +/* { dg-final { scan-tree-dump "LOOP VECTORIZED" "vect" } } */ + +typedef struct filter_list_entry { + const char *name; + int id; + void (*function)(); +} filter_list_entry; + +static const filter_list_entry filter_list[9] = {0}; + +void php_zval_filter(int filter, int id1) { + filter_list_entry filter_func; + + int size = 9; + for (int i = 0; i < size; ++i) { + if (filter_list[i].id == filter) { + filter_func = filter_list[i]; + goto done; + } + } + + for (int i = 0; i < size; ++i) { + if (filter_list[i].id == 0x0204) { + filter_func = filter_list[i]; + goto done; + } + } +done: + if (!filter_func.id) + filter_func.function(); +} diff --git a/gcc/tree-vect-loop-manip.cc b/gcc/tree-vect-loop-manip.cc index 3f974d6d839e32516ae316f28ca25316e43d7d86..b5e158bc5cfb5107d5ff461e489d306f81e090d0 100644 --- a/gcc/tree-vect-loop-manip.cc +++ b/gcc/tree-vect-loop-manip.cc @@ -1917,7 +1917,6 @@ slpeel_tree_duplicate_loop_to_edge_cfg (class loop *loop, edge loop_exit, doms.safe_push (e->dest); } - iterate_fix_dominators (CDI_DOMINATORS, doms, false); if (updated_doms) updated_doms->safe_splice (doms); } @@ -1925,7 +1924,9 @@ slpeel_tree_duplicate_loop_to_edge_cfg (class loop *loop, edge loop_exit, free (new_bbs); free (bbs); - checking_verify_dominators (CDI_DOMINATORS); + /* If we're peeling for vectorization then delay verifying dominators. */ + if (!flow_loops || !multiple_exits_p) + checking_verify_dominators (CDI_DOMINATORS); return new_loop; } @@ -3404,7 +3405,7 @@ vect_do_peeling (loop_vec_info loop_vinfo, tree niters, tree nitersm1, epilog = vect_epilogues ? get_loop_copy (loop) : scalar_loop; edge epilog_e = vect_epilogues ? e : scalar_e; edge new_epilog_e = NULL; - auto_vec doms; + auto_vec& doms = LOOP_VINFO_DOMS_NEED_UPDATE (loop_vinfo); epilog = slpeel_tree_duplicate_loop_to_edge_cfg (loop, e, epilog, epilog_e, e, &new_epilog_e, true, &doms); @@ -3554,10 +3555,6 @@ vect_do_peeling (loop_vec_info loop_vinfo, tree niters, tree nitersm1, scale_loop_profile (epilog, prob_epilog, -1); } - /* Recalculate the dominators after adding the guard edge. */ - if (LOOP_VINFO_EARLY_BREAKS (loop_vinfo)) - iterate_fix_dominators (CDI_DOMINATORS, doms, false); - /* When we do not have a loop-around edge to the epilog we know the vector loop covered at least VF scalar iterations unless we have early breaks. diff --git a/gcc/tree-vect-loop.cc b/gcc/tree-vect-loop.cc index 35f1f8c7d4245135ace7ffff40ff9be548919587..ab19ad6a6be516e3ee1f0fbeaaeeffeae1fb900f 100644 --- a/gcc/tree-vect-loop.cc +++ b/gcc/tree-vect-loop.cc @@ -11987,7 +11987,12 @@ vect_transform_loop (loop_vec_info loop_vinfo, gimple *loop_vectorized_call) /* Handle any code motion that we need to for early-break vectorization after we've done peeling but just before we start vectorizing. */ if (LOOP_VINFO_EARLY_BREAKS (loop_vinfo)) - move_early_exit_stmts (loop_vinfo); + { + /* Recalculate the dominators. */ + iterate_fix_dominators (CDI_DOMINATORS, + LOOP_VINFO_DOMS_NEED_UPDATE (loop_vinfo), false); + move_early_exit_stmts (loop_vinfo); + } /* Schedule the SLP instances first, then handle loop vectorization below. */ diff --git a/gcc/tree-vectorizer.h b/gcc/tree-vectorizer.h index db44d730b7026492c4dd8c394ab505f227edfc8e..6bb5b6d69a8340c0b68cbcbbb110c0380b7d73aa 100644 --- a/gcc/tree-vectorizer.h +++ b/gcc/tree-vectorizer.h @@ -961,6 +961,10 @@ public: /* Statements whose VUSES need updating if early break vectorization is to happen. */ auto_vec early_break_vuses; + + /* Dominators that need to be recalculated that have been deferred until after + all peeling. */ + auto_vec dominators_needing_update; } *loop_vec_info; /* Access Functions. */ @@ -1021,6 +1025,7 @@ public: (single_pred ((L)->loop->latch) != (L)->vec_loop_iv_exit->src) #define LOOP_VINFO_EARLY_BRK_DEST_BB(L) (L)->early_break_dest_bb #define LOOP_VINFO_EARLY_BRK_VUSES(L) (L)->early_break_vuses +#define LOOP_VINFO_DOMS_NEED_UPDATE(L) (L)->dominators_needing_update #define LOOP_VINFO_LOOP_CONDS(L) (L)->conds #define LOOP_VINFO_LOOP_IV_COND(L) (L)->loop_iv_cond #define LOOP_VINFO_NO_DATA_DEPENDENCIES(L) (L)->no_data_dependencies --4dXfLpipNSx4/eZz--