From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR03-DBA-obe.outbound.protection.outlook.com (mail-dbaeur03on2086.outbound.protection.outlook.com [40.107.104.86]) by sourceware.org (Postfix) with ESMTPS id 90EAE3858C31 for ; Wed, 28 Jun 2023 13:41:39 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 90EAE3858C31 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=arm.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=GFudQkdviZu5/tTMp8cLL7Vch/f/MQd4AIY9fvo9rMA=; b=XN8tX0TKvQPfHX38EcBUzeI1oOCHjWMlVv7TzQkISwISlGW8S/ghf6xHoCaywW4C+RQnnjyhCTf6NeYS2A3+HyzhDeBz6lhbdiO775xD6F5VW7//KiGwwa5P5C7w1YGgN/9LMJXp3JkY8h2E6LREkCSYMqc35eHjFTjD8V8YwyE= Received: from AM6PR04CA0035.eurprd04.prod.outlook.com (2603:10a6:20b:92::48) by AS8PR08MB6342.eurprd08.prod.outlook.com (2603:10a6:20b:31a::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6521.26; Wed, 28 Jun 2023 13:41:34 +0000 Received: from AM7EUR03FT029.eop-EUR03.prod.protection.outlook.com (2603:10a6:20b:92:cafe::18) by AM6PR04CA0035.outlook.office365.com (2603:10a6:20b:92::48) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6544.19 via Frontend Transport; Wed, 28 Jun 2023 13:41:34 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; pr=C Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by AM7EUR03FT029.mail.protection.outlook.com (100.127.140.143) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6544.18 via Frontend Transport; Wed, 28 Jun 2023 13:41:34 +0000 Received: ("Tessian outbound 52217515e112:v142"); Wed, 28 Jun 2023 13:41:34 +0000 X-CheckRecipientChecked: true X-CR-MTA-CID: 9531fbda2c421124 X-CR-MTA-TID: 64aa7808 Received: from 2e3920b91d91.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 728A8054-8687-4062-9206-931E3E3A8952.1; Wed, 28 Jun 2023 13:41:27 +0000 Received: from EUR04-VI1-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 2e3920b91d91.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Wed, 28 Jun 2023 13:41:27 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=mt/bmDD28g6aPAWXOgaA5XnvoM8DhMTSUXWImT9vy8MXx7j+aPoxyz3HYi203JkFE0qhmGc97ZpxB7OamEXUnv/OSSt/Oy6/GPO4G7r+TznSs8mIhF4lokJB9iUtSnq4pUgK6WuGCRelbYatN8mlcx+Tog7FnJ7mZ+Mf4oLS/PFfSzBJXvXyJyoFQpiBV8BA9ZNTetjMBAqp8e5PSrS6inyNsCofIXVzR2yhsYEA6EuMyEkw0cXQ0aCkYFUvtlORhig8WKwGPHCvPTF/XyYWuxyO77cV571+ZAWtnUwe+M5v8YLv9AUcNX8wnu+8tLeVMl4dpCqTAqH2ejF1CCIpEw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=GFudQkdviZu5/tTMp8cLL7Vch/f/MQd4AIY9fvo9rMA=; b=JI5S0TziupFQrHFkBgDkukDT+EJalzgQUebkqXh1a3VmDabZwHvNEFVHyqfP0fkmmGGv4O8vAr5azIxK0WXh1wvpBuK2dEZ4zir7mZmC8J82EbOJCktjIzMwyQ7iJYcAPLYSInFo5kXd0ph8+zPALMILXQvtnbrXvpMzKnUPbL+lYyf4nXAkT7bNpudViaeIFvV91TlFCmESjXDbHqMhH0teLK7em0BaeyPMyDk1VTCAIEADXmx6Af2s8lcKH9/LoqP9yzPQTR46yA3Uof+b/7E3+Jx75gzl4ZWCEF9V9gaKn3mkkKWbmv+mUOBr/hfrs2TLYUMDoXhj/D5tKl7ZhQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=GFudQkdviZu5/tTMp8cLL7Vch/f/MQd4AIY9fvo9rMA=; b=XN8tX0TKvQPfHX38EcBUzeI1oOCHjWMlVv7TzQkISwISlGW8S/ghf6xHoCaywW4C+RQnnjyhCTf6NeYS2A3+HyzhDeBz6lhbdiO775xD6F5VW7//KiGwwa5P5C7w1YGgN/9LMJXp3JkY8h2E6LREkCSYMqc35eHjFTjD8V8YwyE= Authentication-Results-Original: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; Received: from VI1PR08MB5325.eurprd08.prod.outlook.com (2603:10a6:803:13e::17) by DBAPR08MB5576.eurprd08.prod.outlook.com (2603:10a6:10:1ae::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6521.26; Wed, 28 Jun 2023 13:41:25 +0000 Received: from VI1PR08MB5325.eurprd08.prod.outlook.com ([fe80::2301:1cde:cfe7:eaf0]) by VI1PR08MB5325.eurprd08.prod.outlook.com ([fe80::2301:1cde:cfe7:eaf0%6]) with mapi id 15.20.6521.026; Wed, 28 Jun 2023 13:41:25 +0000 Date: Wed, 28 Jun 2023 14:41:21 +0100 From: Tamar Christina To: gcc-patches@gcc.gnu.org Cc: nd@arm.com, rguenther@suse.de, jlaw@ventanamicro.com Subject: [PATCH 1/19]middle-end ifcvt: Support bitfield lowering of multiple-exit loops Message-ID: Content-Type: multipart/mixed; boundary="VezUAqWp1VxrrBSk" Content-Disposition: inline In-Reply-To: X-ClientProxiedBy: LO2P265CA0122.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:9f::14) To VI1PR08MB5325.eurprd08.prod.outlook.com (2603:10a6:803:13e::17) MIME-Version: 1.0 X-MS-TrafficTypeDiagnostic: VI1PR08MB5325:EE_|DBAPR08MB5576:EE_|AM7EUR03FT029:EE_|AS8PR08MB6342:EE_ X-MS-Office365-Filtering-Correlation-Id: 1fb90388-551b-4921-0a3c-08db77dd6398 x-checkrecipientrouted: true NoDisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: LumKpFuc4tFihRH1CEdkjX1C3GqBs/vBD4Q3++eNHnTb35KwErcKQdjPs0K/Q3D5EMLDfmPikWQuCJRtdmDMDOAz/L6zCHmeYmgBYo9M+K3U+lHSEjOqpBdJBusI6REf8sD34c0GPlE8/NCZzfghG06dp1C2esoUxvC0rev+MzYjlfdQVsN0acPWgoRYmJ1AqD+CWRG+di0mlfaAAZsJPzCnjuPWJXXm9LsXNDSFpUdBQSDS/j5CnoNSVIj8p+SpNRyLwRXxZ0wgh7m+uDr/7jLR9ROElZM6oRrEEHbM2Ylf0g3hbPTm0DFKgsnaxMuOmyWqOH+hllyq0NlqacVGY+pH+VXty6d1oHRKVgHrKWiaGSo8oO6PmJp3tM72QICZkoe1pG8yHkFFMgrHSp8222/a9WPADrxs4Yhum3uU7tIVCe04EyqjSVCRGAKgkhvPSSVq8hDm6qw6s44+8MuMxanRanDlBIW2nbtx4C6M0XSTSRZg0tM+Ltc4R7RDVvnsW1zsEwdjtzNqIcNM0QorJ+7jFUf5gAn8mUN97OVOKfxwkq7bkrcs9xf5fYMobI5TltxMVzlH/67wOHwG75nHupO6kyNeP5DGdkChRwFUkpp5aFO9yTwzGObByvAxRUuNNS/4G6omOD8jFotnOv9m8w== X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR08MB5325.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230028)(4636009)(136003)(346002)(39860400002)(396003)(376002)(366004)(451199021)(6486002)(2616005)(83380400001)(33964004)(6666004)(6512007)(6506007)(26005)(44144004)(2906002)(186003)(478600001)(235185007)(30864003)(44832011)(6916009)(5660300002)(36756003)(4326008)(66946007)(316002)(38100700002)(8936002)(41300700001)(66476007)(8676002)(86362001)(84970400001)(66556008)(2700100001);DIR:OUT;SFP:1101; X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBAPR08MB5576 Original-Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: AM7EUR03FT029.eop-EUR03.prod.protection.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: c02c737b-3ab7-41cf-270c-08db77dd5df4 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: ZBRiesHwYpWDg48CgaHkMfnmw/srXqa7cHXBgK6XNDwwg7wq8z2iWPcM5ogOelDrymf/fz7s47fuFUZmxrTAs/cARDZHB2TbeXUiUaczcld5jAHz9K1tG6ExUuUGWgdk0MJ5sjP1WFXMYBvx7Ah9IqSt9JYbTVm9cDW5K0u1q5Qmp2/7APs0So7toLJDvqjVS3qjeJAYto1go43pT0vFmfKnVB9nu+M/5EuNeZQHaQD2c0/q6FJbGJM2Y3XbGYNXGi25uXRnlpi97AoCODOU8cCsjNV0MqXDh+kQqAaBFHahsBDKf0VwxiDrN6x6xrpo8Vs2MGsJ0CKSRxCBS+B1UkSTWsER4WCP4Ttd6p2uIOeDI2GIztNdDA/26da0l4ILvMIufor8CU2KCKRWvGnPOAmuCTwUX47kG+V9zsuRvrF/sSAoBUACIztMjXVW50ohIxUitjYTIRU2v1NGb94D01PqNs3NFW0TrINgA+KnzRPxny12XjohyFEs21AydSYIa+fjjm0GlPpxpPO59U7GBdIDuF3iHDhSFWgBM3DC/0zPewF0Zw7DahRWTm6PQSUnQpf9dEu6pAcfz+YdvUrWNNELQfjMeX1vIBztBJJTBxtK61O1wxNPOpchAhMaqb1YCCdmv+7eQ0MjzPaVM53zlury9WgWq2BwxDRzSFqi7FLrVTYWZ6CEUL0NogLPGGjXY6aAsv162EvEd0m+VxmhmRx795+DceHR1pu9o8MA794aGJOaMjEglnEf/vZnRek8a6UqhhqJucuhNvVILlC90FGl7t08nKC9zML4B1Xalxc= X-Forefront-Antispam-Report: CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(13230028)(4636009)(136003)(39860400002)(396003)(346002)(376002)(451199021)(46966006)(36840700001)(40470700004)(6512007)(84970400001)(82310400005)(36860700001)(26005)(36756003)(356005)(30864003)(82740400003)(235185007)(44832011)(70206006)(86362001)(41300700001)(6916009)(40460700003)(40480700001)(4326008)(81166007)(8936002)(70586007)(316002)(8676002)(107886003)(47076005)(5660300002)(6506007)(6486002)(186003)(83380400001)(44144004)(33964004)(6666004)(2906002)(336012)(2616005)(478600001)(2700100001);DIR:OUT;SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Jun 2023 13:41:34.4662 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 1fb90388-551b-4921-0a3c-08db77dd6398 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: AM7EUR03FT029.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB6342 X-Spam-Status: No, score=-12.6 required=5.0 tests=BAYES_00,DKIM_SIGNED,DKIM_VALID,GIT_PATCH_0,KAM_DMARC_NONE,KAM_LOTSOFHASH,RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE,TXREP,T_SCC_BODY_TEXT_LINE,UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: --VezUAqWp1VxrrBSk Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Hi, With the patch enabling the vectorization of early-breaks, we'd like to allow bitfield lowering in such loops, which requires the relaxation of allowing multiple exits when doing so. In order to avoid a similar issue to PR107275, the code that rejects loops with certain types of gimple_stmts was hoisted from 'if_convertible_loop_p_1' to 'get_loop_body_in_if_conv_order', to avoid trying to lower bitfields in loops we are not going to vectorize anyway. This also ensures 'ifcvt_local_dec' doesn't accidentally remove statements it shouldn't as it will never come across them. I made sure to add a comment to make clear that there is a direct connection between the two and if we were to enable vectorization of any other gimple statement we should make sure both handle it. NOTE: This patch accepted before but never committed because it is a no-op without the early break patch. This is a respun version of Andre's patch and rebased to changes in ifcvt and updated to handle multiple exits. Bootstrappend and regression tested on aarch64-none-linux-gnu and x86_64-pc-linux-gnu and no issues. gcc/ChangeLog: * tree-if-conv.cc (if_convertible_loop_p_1): Move check from here ... (get_loop_body_if_conv_order): ... to here. (if_convertible_loop_p): Remove single_exit check. (tree_if_conversion): Move single_exit check to if-conversion part and support multiple exits. gcc/testsuite/ChangeLog: * gcc.dg/vect/vect-bitfield-read-1-not.c: New test. * gcc.dg/vect/vect-bitfield-read-2-not.c: New test. * gcc.dg/vect/vect-bitfield-read-8.c: New test. * gcc.dg/vect/vect-bitfield-read-9.c: New test. Co-Authored-By: Andre Vieira --- inline copy of patch -- diff --git a/gcc/testsuite/gcc.dg/vect/vect-bitfield-read-1-not.c b/gcc/testsuite/gcc.dg/vect/vect-bitfield-read-1-not.c new file mode 100644 index 0000000000000000000000000000000000000000..0d91067ebb27b1db2b2352975c43bce8b4171e3f --- /dev/null +++ b/gcc/testsuite/gcc.dg/vect/vect-bitfield-read-1-not.c @@ -0,0 +1,60 @@ +/* { dg-require-effective-target vect_shift } */ +/* { dg-require-effective-target vect_long_long } */ +/* { dg-additional-options { "-fdump-tree-ifcvt-all" } } */ + +#include +#include "tree-vect.h" + +extern void abort(void); + +struct s { + char a : 4; +}; + +#define N 32 +#define ELT0 {0} +#define ELT1 {1} +#define ELT2 {2} +#define ELT3 {3} +#define RES 56 +struct s A[N] + = { ELT0, ELT1, ELT2, ELT3, ELT0, ELT1, ELT2, ELT3, + ELT0, ELT1, ELT2, ELT3, ELT0, ELT1, ELT2, ELT3, + ELT0, ELT1, ELT2, ELT3, ELT0, ELT1, ELT2, ELT3, + ELT0, ELT1, ELT2, ELT3, ELT0, ELT1, ELT2, ELT3}; + +int __attribute__ ((noipa)) +f(struct s *ptr, unsigned n) { + int res = 0; + for (int i = 0; i < n; ++i) + { + switch (ptr[i].a) + { + case 0: + res += ptr[i].a + 1; + break; + case 1: + case 2: + case 3: + res += ptr[i].a; + break; + default: + return 0; + } + } + return res; +} + +int main (void) +{ + check_vect (); + + if (f(&A[0], N) != RES) + abort (); + + return 0; +} + +/* { dg-final { scan-tree-dump-not "Bitfield OK to lower." "ifcvt" } } */ + + diff --git a/gcc/testsuite/gcc.dg/vect/vect-bitfield-read-2-not.c b/gcc/testsuite/gcc.dg/vect/vect-bitfield-read-2-not.c new file mode 100644 index 0000000000000000000000000000000000000000..4ac7b3fc0dfd1c9d0b5e94a2ba6a745545577ec1 --- /dev/null +++ b/gcc/testsuite/gcc.dg/vect/vect-bitfield-read-2-not.c @@ -0,0 +1,49 @@ +/* { dg-require-effective-target vect_shift } */ +/* { dg-require-effective-target vect_long_long } */ +/* { dg-additional-options { "-fdump-tree-ifcvt-all" } } */ + +#include +#include "tree-vect.h" + +extern void abort(void); + +struct s { + char a : 4; +}; + +#define N 32 +#define ELT0 {0} +#define ELT1 {1} +#define ELT2 {2} +#define ELT3 {3} +#define RES 48 +struct s A[N] + = { ELT0, ELT1, ELT2, ELT3, ELT0, ELT1, ELT2, ELT3, + ELT0, ELT1, ELT2, ELT3, ELT0, ELT1, ELT2, ELT3, + ELT0, ELT1, ELT2, ELT3, ELT0, ELT1, ELT2, ELT3, + ELT0, ELT1, ELT2, ELT3, ELT0, ELT1, ELT2, ELT3}; + +int __attribute__ ((noipa)) +f(struct s *ptr, unsigned n) { + int res = 0; + for (int i = 0; i < n; ++i) + { + asm volatile ("" ::: "memory"); + res += ptr[i].a; + } + return res; +} + +int main (void) +{ + check_vect (); + + if (f(&A[0], N) != RES) + abort (); + + return 0; +} + +/* { dg-final { scan-tree-dump-not "Bitfield OK to lower." "ifcvt" } } */ + + diff --git a/gcc/testsuite/gcc.dg/vect/vect-bitfield-read-8.c b/gcc/testsuite/gcc.dg/vect/vect-bitfield-read-8.c new file mode 100644 index 0000000000000000000000000000000000000000..52cfd33d937ae90f3fe9556716c90e098b768ac8 --- /dev/null +++ b/gcc/testsuite/gcc.dg/vect/vect-bitfield-read-8.c @@ -0,0 +1,49 @@ +/* { dg-require-effective-target vect_int } */ +/* { dg-require-effective-target vect_shift } */ +/* { dg-additional-options { "-fdump-tree-ifcvt-all" } } */ + +#include +#include "tree-vect.h" + +extern void abort(void); + +struct s { int i : 31; }; + +#define ELT0 {0} +#define ELT1 {1} +#define ELT2 {2} +#define ELT3 {3} +#define ELT4 {4} +#define N 32 +#define RES 25 +struct s A[N] + = { ELT0, ELT1, ELT2, ELT3, ELT0, ELT1, ELT2, ELT3, + ELT0, ELT1, ELT2, ELT3, ELT0, ELT1, ELT2, ELT3, + ELT0, ELT1, ELT4, ELT3, ELT0, ELT1, ELT2, ELT3, + ELT0, ELT1, ELT2, ELT3, ELT0, ELT1, ELT2, ELT3}; + +int __attribute__ ((noipa)) +f(struct s *ptr, unsigned n) { + int res = 0; + for (int i = 0; i < n; ++i) + { + if (ptr[i].i == 4) + return res; + res += ptr[i].i; + } + + return res; +} + +int main (void) +{ + check_vect (); + + if (f(&A[0], N) != RES) + abort (); + + return 0; +} + +/* { dg-final { scan-tree-dump "Bitfield OK to lower." "ifcvt" } } */ + diff --git a/gcc/testsuite/gcc.dg/vect/vect-bitfield-read-9.c b/gcc/testsuite/gcc.dg/vect/vect-bitfield-read-9.c new file mode 100644 index 0000000000000000000000000000000000000000..ab814698131a5905def181eeed85d8a3c62b924b --- /dev/null +++ b/gcc/testsuite/gcc.dg/vect/vect-bitfield-read-9.c @@ -0,0 +1,51 @@ +/* { dg-require-effective-target vect_shift } */ +/* { dg-require-effective-target vect_long_long } */ +/* { dg-additional-options { "-fdump-tree-ifcvt-all" } } */ + +#include +#include "tree-vect.h" + +extern void abort(void); + +struct s { + unsigned i : 31; + char a : 4; +}; + +#define N 32 +#define ELT0 {0x7FFFFFFFUL, 0} +#define ELT1 {0x7FFFFFFFUL, 1} +#define ELT2 {0x7FFFFFFFUL, 2} +#define ELT3 {0x7FFFFFFFUL, 3} +#define ELT4 {0x7FFFFFFFUL, 4} +#define RES 9 +struct s A[N] + = { ELT0, ELT4, ELT2, ELT3, ELT0, ELT1, ELT2, ELT3, + ELT0, ELT1, ELT2, ELT3, ELT0, ELT1, ELT2, ELT3, + ELT0, ELT1, ELT2, ELT3, ELT0, ELT1, ELT2, ELT3, + ELT0, ELT1, ELT2, ELT3, ELT0, ELT1, ELT2, ELT3}; + +int __attribute__ ((noipa)) +f(struct s *ptr, unsigned n) { + int res = 0; + for (int i = 0; i < n; ++i) + { + if (ptr[i].a) + return 9; + res += ptr[i].a; + } + return res; +} + +int main (void) +{ + check_vect (); + + if (f(&A[0], N) != RES) + abort (); + + return 0; +} + +/* { dg-final { scan-tree-dump "Bitfield OK to lower." "ifcvt" } } */ + diff --git a/gcc/tree-if-conv.cc b/gcc/tree-if-conv.cc index e342532a343a3c066142adeec5fdfaf736a653e5..cdb0fe4c29dfa531e3277925022d127b13ffcc16 100644 --- a/gcc/tree-if-conv.cc +++ b/gcc/tree-if-conv.cc @@ -586,7 +586,7 @@ add_to_dst_predicate_list (class loop *loop, edge e, /* Return true if one of the successor edges of BB exits LOOP. */ static bool -bb_with_exit_edge_p (class loop *loop, basic_block bb) +bb_with_exit_edge_p (const class loop *loop, basic_block bb) { edge e; edge_iterator ei; @@ -1268,6 +1268,44 @@ get_loop_body_in_if_conv_order (const class loop *loop) } free (blocks_in_bfs_order); BITMAP_FREE (visited); + + /* Go through loop and reject if-conversion or lowering of bitfields if we + encounter statements we do not believe the vectorizer will be able to + handle. If adding a new type of statement here, make sure + 'ifcvt_local_dce' is also able to handle it propertly. */ + for (index = 0; index < loop->num_nodes; index++) + { + basic_block bb = blocks[index]; + gimple_stmt_iterator gsi; + + bool may_have_nonlocal_labels + = bb_with_exit_edge_p (loop, bb) || bb == loop->latch; + for (gsi = gsi_start_bb (bb); !gsi_end_p (gsi); gsi_next (&gsi)) + switch (gimple_code (gsi_stmt (gsi))) + { + case GIMPLE_LABEL: + if (!may_have_nonlocal_labels) + { + tree label + = gimple_label_label (as_a (gsi_stmt (gsi))); + if (DECL_NONLOCAL (label) || FORCED_LABEL (label)) + { + free (blocks); + return NULL; + } + } + /* Fallthru. */ + case GIMPLE_ASSIGN: + case GIMPLE_CALL: + case GIMPLE_DEBUG: + case GIMPLE_COND: + gimple_set_uid (gsi_stmt (gsi), 0); + break; + default: + free (blocks); + return NULL; + } + } return blocks; } @@ -1438,36 +1476,6 @@ if_convertible_loop_p_1 (class loop *loop, vec *refs) exit_bb = bb; } - for (i = 0; i < loop->num_nodes; i++) - { - basic_block bb = ifc_bbs[i]; - gimple_stmt_iterator gsi; - - bool may_have_nonlocal_labels - = bb_with_exit_edge_p (loop, bb) || bb == loop->latch; - for (gsi = gsi_start_bb (bb); !gsi_end_p (gsi); gsi_next (&gsi)) - switch (gimple_code (gsi_stmt (gsi))) - { - case GIMPLE_LABEL: - if (!may_have_nonlocal_labels) - { - tree label - = gimple_label_label (as_a (gsi_stmt (gsi))); - if (DECL_NONLOCAL (label) || FORCED_LABEL (label)) - return false; - } - /* Fallthru. */ - case GIMPLE_ASSIGN: - case GIMPLE_CALL: - case GIMPLE_DEBUG: - case GIMPLE_COND: - gimple_set_uid (gsi_stmt (gsi), 0); - break; - default: - return false; - } - } - data_reference_p dr; innermost_DR_map @@ -1579,14 +1587,6 @@ if_convertible_loop_p (class loop *loop, vec *refs) return false; } - /* More than one loop exit is too much to handle. */ - if (!single_exit (loop)) - { - if (dump_file && (dump_flags & TDF_DETAILS)) - fprintf (dump_file, "multiple exits\n"); - return false; - } - /* If one of the loop header's edge is an exit edge then do not apply if-conversion. */ FOR_EACH_EDGE (e, ei, loop->header->succs) @@ -3566,9 +3566,6 @@ tree_if_conversion (class loop *loop, vec *preds) aggressive_if_conv = true; } - if (!single_exit (loop)) - goto cleanup; - /* If there are more than two BBs in the loop then there is at least one if to convert. */ if (loop->num_nodes > 2 @@ -3588,15 +3585,25 @@ tree_if_conversion (class loop *loop, vec *preds) if (loop->num_nodes > 2) { - need_to_ifcvt = true; + /* More than one loop exit is too much to handle. */ + if (!single_exit (loop)) + { + if (dump_file && (dump_flags & TDF_DETAILS)) + fprintf (dump_file, "Can not ifcvt due to multiple exits\n"); + } + else + { + need_to_ifcvt = true; - if (!if_convertible_loop_p (loop, &refs) || !dbg_cnt (if_conversion_tree)) - goto cleanup; + if (!if_convertible_loop_p (loop, &refs) + || !dbg_cnt (if_conversion_tree)) + goto cleanup; - if ((need_to_predicate || any_complicated_phi) - && ((!flag_tree_loop_vectorize && !loop->force_vectorize) - || loop->dont_vectorize)) - goto cleanup; + if ((need_to_predicate || any_complicated_phi) + && ((!flag_tree_loop_vectorize && !loop->force_vectorize) + || loop->dont_vectorize)) + goto cleanup; + } } if ((flag_tree_loop_vectorize || loop->force_vectorize) @@ -3687,7 +3694,8 @@ tree_if_conversion (class loop *loop, vec *preds) PHIs, those are to be kept in sync with the non-if-converted copy. ??? We'll still keep dead stores though. */ exit_bbs = BITMAP_ALLOC (NULL); - bitmap_set_bit (exit_bbs, single_exit (loop)->dest->index); + for (edge exit : get_loop_exit_edges (loop)) + bitmap_set_bit (exit_bbs, exit->dest->index); bitmap_set_bit (exit_bbs, loop->latch->index); std::pair *name_pair; -- --VezUAqWp1VxrrBSk Content-Type: text/plain; charset=utf-8 Content-Disposition: attachment; filename="rb17495.patch" diff --git a/gcc/testsuite/gcc.dg/vect/vect-bitfield-read-1-not.c b/gcc/testsuite/gcc.dg/vect/vect-bitfield-read-1-not.c new file mode 100644 index 0000000000000000000000000000000000000000..0d91067ebb27b1db2b2352975c43bce8b4171e3f --- /dev/null +++ b/gcc/testsuite/gcc.dg/vect/vect-bitfield-read-1-not.c @@ -0,0 +1,60 @@ +/* { dg-require-effective-target vect_shift } */ +/* { dg-require-effective-target vect_long_long } */ +/* { dg-additional-options { "-fdump-tree-ifcvt-all" } } */ + +#include +#include "tree-vect.h" + +extern void abort(void); + +struct s { + char a : 4; +}; + +#define N 32 +#define ELT0 {0} +#define ELT1 {1} +#define ELT2 {2} +#define ELT3 {3} +#define RES 56 +struct s A[N] + = { ELT0, ELT1, ELT2, ELT3, ELT0, ELT1, ELT2, ELT3, + ELT0, ELT1, ELT2, ELT3, ELT0, ELT1, ELT2, ELT3, + ELT0, ELT1, ELT2, ELT3, ELT0, ELT1, ELT2, ELT3, + ELT0, ELT1, ELT2, ELT3, ELT0, ELT1, ELT2, ELT3}; + +int __attribute__ ((noipa)) +f(struct s *ptr, unsigned n) { + int res = 0; + for (int i = 0; i < n; ++i) + { + switch (ptr[i].a) + { + case 0: + res += ptr[i].a + 1; + break; + case 1: + case 2: + case 3: + res += ptr[i].a; + break; + default: + return 0; + } + } + return res; +} + +int main (void) +{ + check_vect (); + + if (f(&A[0], N) != RES) + abort (); + + return 0; +} + +/* { dg-final { scan-tree-dump-not "Bitfield OK to lower." "ifcvt" } } */ + + diff --git a/gcc/testsuite/gcc.dg/vect/vect-bitfield-read-2-not.c b/gcc/testsuite/gcc.dg/vect/vect-bitfield-read-2-not.c new file mode 100644 index 0000000000000000000000000000000000000000..4ac7b3fc0dfd1c9d0b5e94a2ba6a745545577ec1 --- /dev/null +++ b/gcc/testsuite/gcc.dg/vect/vect-bitfield-read-2-not.c @@ -0,0 +1,49 @@ +/* { dg-require-effective-target vect_shift } */ +/* { dg-require-effective-target vect_long_long } */ +/* { dg-additional-options { "-fdump-tree-ifcvt-all" } } */ + +#include +#include "tree-vect.h" + +extern void abort(void); + +struct s { + char a : 4; +}; + +#define N 32 +#define ELT0 {0} +#define ELT1 {1} +#define ELT2 {2} +#define ELT3 {3} +#define RES 48 +struct s A[N] + = { ELT0, ELT1, ELT2, ELT3, ELT0, ELT1, ELT2, ELT3, + ELT0, ELT1, ELT2, ELT3, ELT0, ELT1, ELT2, ELT3, + ELT0, ELT1, ELT2, ELT3, ELT0, ELT1, ELT2, ELT3, + ELT0, ELT1, ELT2, ELT3, ELT0, ELT1, ELT2, ELT3}; + +int __attribute__ ((noipa)) +f(struct s *ptr, unsigned n) { + int res = 0; + for (int i = 0; i < n; ++i) + { + asm volatile ("" ::: "memory"); + res += ptr[i].a; + } + return res; +} + +int main (void) +{ + check_vect (); + + if (f(&A[0], N) != RES) + abort (); + + return 0; +} + +/* { dg-final { scan-tree-dump-not "Bitfield OK to lower." "ifcvt" } } */ + + diff --git a/gcc/testsuite/gcc.dg/vect/vect-bitfield-read-8.c b/gcc/testsuite/gcc.dg/vect/vect-bitfield-read-8.c new file mode 100644 index 0000000000000000000000000000000000000000..52cfd33d937ae90f3fe9556716c90e098b768ac8 --- /dev/null +++ b/gcc/testsuite/gcc.dg/vect/vect-bitfield-read-8.c @@ -0,0 +1,49 @@ +/* { dg-require-effective-target vect_int } */ +/* { dg-require-effective-target vect_shift } */ +/* { dg-additional-options { "-fdump-tree-ifcvt-all" } } */ + +#include +#include "tree-vect.h" + +extern void abort(void); + +struct s { int i : 31; }; + +#define ELT0 {0} +#define ELT1 {1} +#define ELT2 {2} +#define ELT3 {3} +#define ELT4 {4} +#define N 32 +#define RES 25 +struct s A[N] + = { ELT0, ELT1, ELT2, ELT3, ELT0, ELT1, ELT2, ELT3, + ELT0, ELT1, ELT2, ELT3, ELT0, ELT1, ELT2, ELT3, + ELT0, ELT1, ELT4, ELT3, ELT0, ELT1, ELT2, ELT3, + ELT0, ELT1, ELT2, ELT3, ELT0, ELT1, ELT2, ELT3}; + +int __attribute__ ((noipa)) +f(struct s *ptr, unsigned n) { + int res = 0; + for (int i = 0; i < n; ++i) + { + if (ptr[i].i == 4) + return res; + res += ptr[i].i; + } + + return res; +} + +int main (void) +{ + check_vect (); + + if (f(&A[0], N) != RES) + abort (); + + return 0; +} + +/* { dg-final { scan-tree-dump "Bitfield OK to lower." "ifcvt" } } */ + diff --git a/gcc/testsuite/gcc.dg/vect/vect-bitfield-read-9.c b/gcc/testsuite/gcc.dg/vect/vect-bitfield-read-9.c new file mode 100644 index 0000000000000000000000000000000000000000..ab814698131a5905def181eeed85d8a3c62b924b --- /dev/null +++ b/gcc/testsuite/gcc.dg/vect/vect-bitfield-read-9.c @@ -0,0 +1,51 @@ +/* { dg-require-effective-target vect_shift } */ +/* { dg-require-effective-target vect_long_long } */ +/* { dg-additional-options { "-fdump-tree-ifcvt-all" } } */ + +#include +#include "tree-vect.h" + +extern void abort(void); + +struct s { + unsigned i : 31; + char a : 4; +}; + +#define N 32 +#define ELT0 {0x7FFFFFFFUL, 0} +#define ELT1 {0x7FFFFFFFUL, 1} +#define ELT2 {0x7FFFFFFFUL, 2} +#define ELT3 {0x7FFFFFFFUL, 3} +#define ELT4 {0x7FFFFFFFUL, 4} +#define RES 9 +struct s A[N] + = { ELT0, ELT4, ELT2, ELT3, ELT0, ELT1, ELT2, ELT3, + ELT0, ELT1, ELT2, ELT3, ELT0, ELT1, ELT2, ELT3, + ELT0, ELT1, ELT2, ELT3, ELT0, ELT1, ELT2, ELT3, + ELT0, ELT1, ELT2, ELT3, ELT0, ELT1, ELT2, ELT3}; + +int __attribute__ ((noipa)) +f(struct s *ptr, unsigned n) { + int res = 0; + for (int i = 0; i < n; ++i) + { + if (ptr[i].a) + return 9; + res += ptr[i].a; + } + return res; +} + +int main (void) +{ + check_vect (); + + if (f(&A[0], N) != RES) + abort (); + + return 0; +} + +/* { dg-final { scan-tree-dump "Bitfield OK to lower." "ifcvt" } } */ + diff --git a/gcc/tree-if-conv.cc b/gcc/tree-if-conv.cc index e342532a343a3c066142adeec5fdfaf736a653e5..cdb0fe4c29dfa531e3277925022d127b13ffcc16 100644 --- a/gcc/tree-if-conv.cc +++ b/gcc/tree-if-conv.cc @@ -586,7 +586,7 @@ add_to_dst_predicate_list (class loop *loop, edge e, /* Return true if one of the successor edges of BB exits LOOP. */ static bool -bb_with_exit_edge_p (class loop *loop, basic_block bb) +bb_with_exit_edge_p (const class loop *loop, basic_block bb) { edge e; edge_iterator ei; @@ -1268,6 +1268,44 @@ get_loop_body_in_if_conv_order (const class loop *loop) } free (blocks_in_bfs_order); BITMAP_FREE (visited); + + /* Go through loop and reject if-conversion or lowering of bitfields if we + encounter statements we do not believe the vectorizer will be able to + handle. If adding a new type of statement here, make sure + 'ifcvt_local_dce' is also able to handle it propertly. */ + for (index = 0; index < loop->num_nodes; index++) + { + basic_block bb = blocks[index]; + gimple_stmt_iterator gsi; + + bool may_have_nonlocal_labels + = bb_with_exit_edge_p (loop, bb) || bb == loop->latch; + for (gsi = gsi_start_bb (bb); !gsi_end_p (gsi); gsi_next (&gsi)) + switch (gimple_code (gsi_stmt (gsi))) + { + case GIMPLE_LABEL: + if (!may_have_nonlocal_labels) + { + tree label + = gimple_label_label (as_a (gsi_stmt (gsi))); + if (DECL_NONLOCAL (label) || FORCED_LABEL (label)) + { + free (blocks); + return NULL; + } + } + /* Fallthru. */ + case GIMPLE_ASSIGN: + case GIMPLE_CALL: + case GIMPLE_DEBUG: + case GIMPLE_COND: + gimple_set_uid (gsi_stmt (gsi), 0); + break; + default: + free (blocks); + return NULL; + } + } return blocks; } @@ -1438,36 +1476,6 @@ if_convertible_loop_p_1 (class loop *loop, vec *refs) exit_bb = bb; } - for (i = 0; i < loop->num_nodes; i++) - { - basic_block bb = ifc_bbs[i]; - gimple_stmt_iterator gsi; - - bool may_have_nonlocal_labels - = bb_with_exit_edge_p (loop, bb) || bb == loop->latch; - for (gsi = gsi_start_bb (bb); !gsi_end_p (gsi); gsi_next (&gsi)) - switch (gimple_code (gsi_stmt (gsi))) - { - case GIMPLE_LABEL: - if (!may_have_nonlocal_labels) - { - tree label - = gimple_label_label (as_a (gsi_stmt (gsi))); - if (DECL_NONLOCAL (label) || FORCED_LABEL (label)) - return false; - } - /* Fallthru. */ - case GIMPLE_ASSIGN: - case GIMPLE_CALL: - case GIMPLE_DEBUG: - case GIMPLE_COND: - gimple_set_uid (gsi_stmt (gsi), 0); - break; - default: - return false; - } - } - data_reference_p dr; innermost_DR_map @@ -1579,14 +1587,6 @@ if_convertible_loop_p (class loop *loop, vec *refs) return false; } - /* More than one loop exit is too much to handle. */ - if (!single_exit (loop)) - { - if (dump_file && (dump_flags & TDF_DETAILS)) - fprintf (dump_file, "multiple exits\n"); - return false; - } - /* If one of the loop header's edge is an exit edge then do not apply if-conversion. */ FOR_EACH_EDGE (e, ei, loop->header->succs) @@ -3566,9 +3566,6 @@ tree_if_conversion (class loop *loop, vec *preds) aggressive_if_conv = true; } - if (!single_exit (loop)) - goto cleanup; - /* If there are more than two BBs in the loop then there is at least one if to convert. */ if (loop->num_nodes > 2 @@ -3588,15 +3585,25 @@ tree_if_conversion (class loop *loop, vec *preds) if (loop->num_nodes > 2) { - need_to_ifcvt = true; + /* More than one loop exit is too much to handle. */ + if (!single_exit (loop)) + { + if (dump_file && (dump_flags & TDF_DETAILS)) + fprintf (dump_file, "Can not ifcvt due to multiple exits\n"); + } + else + { + need_to_ifcvt = true; - if (!if_convertible_loop_p (loop, &refs) || !dbg_cnt (if_conversion_tree)) - goto cleanup; + if (!if_convertible_loop_p (loop, &refs) + || !dbg_cnt (if_conversion_tree)) + goto cleanup; - if ((need_to_predicate || any_complicated_phi) - && ((!flag_tree_loop_vectorize && !loop->force_vectorize) - || loop->dont_vectorize)) - goto cleanup; + if ((need_to_predicate || any_complicated_phi) + && ((!flag_tree_loop_vectorize && !loop->force_vectorize) + || loop->dont_vectorize)) + goto cleanup; + } } if ((flag_tree_loop_vectorize || loop->force_vectorize) @@ -3687,7 +3694,8 @@ tree_if_conversion (class loop *loop, vec *preds) PHIs, those are to be kept in sync with the non-if-converted copy. ??? We'll still keep dead stores though. */ exit_bbs = BITMAP_ALLOC (NULL); - bitmap_set_bit (exit_bbs, single_exit (loop)->dest->index); + for (edge exit : get_loop_exit_edges (loop)) + bitmap_set_bit (exit_bbs, exit->dest->index); bitmap_set_bit (exit_bbs, loop->latch->index); std::pair *name_pair; --VezUAqWp1VxrrBSk--