From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR03-DB5-obe.outbound.protection.outlook.com (mail-eopbgr40066.outbound.protection.outlook.com [40.107.4.66]) by sourceware.org (Postfix) with ESMTPS id DA0D8398B400 for ; Fri, 25 Sep 2020 14:28:38 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org DA0D8398B400 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=Tamar.Christina@arm.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=XvUJYK3D9BrLpGFRZ9fgUfu/dJo3vZ61Fskb03aOBVs=; b=z4Tla2oqrmhY/7hzDHVFa5IY2p1t6B8hDN7JYAGzkmVnk0Ek/Fai7NZusK2AlR6IoNEevFe2nFpXoeTAPygapJezSOl0u5Q1QLY1Va7bciCkKRkpCU8RMGemd8UGfxn7Yfg07Ywlf/eY54jRZNLgP8K36ySQhxQe9aqmOsdatuw= Received: from DB6PR0501CA0045.eurprd05.prod.outlook.com (2603:10a6:4:67::31) by HE1PR0802MB2298.eurprd08.prod.outlook.com (2603:10a6:3:c4::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3391.19; Fri, 25 Sep 2020 14:28:36 +0000 Received: from DB5EUR03FT037.eop-EUR03.prod.protection.outlook.com (2603:10a6:4:67:cafe::5f) by DB6PR0501CA0045.outlook.office365.com (2603:10a6:4:67::31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3412.22 via Frontend Transport; Fri, 25 Sep 2020 14:28:36 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; gcc.gnu.org; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;gcc.gnu.org; dmarc=bestguesspass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by DB5EUR03FT037.mail.protection.outlook.com (10.152.20.215) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3412.21 via Frontend Transport; Fri, 25 Sep 2020 14:28:36 +0000 Received: ("Tessian outbound 7a6fb63c1e64:v64"); Fri, 25 Sep 2020 14:28:36 +0000 X-CheckRecipientChecked: true X-CR-MTA-CID: 413c71d75b32518b X-CR-MTA-TID: 64aa7808 Received: from 7259db9ac837.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 916CB34C-9F37-4A08-BD1B-C6F493BDE58A.1; Fri, 25 Sep 2020 14:28:29 +0000 Received: from EUR03-DB5-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 7259db9ac837.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Fri, 25 Sep 2020 14:28:29 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=KAZ2Pr8TAoI2B8Ga1JzFqQwANMyPdnnSkwq83krRXGnLzsA3xBcghtf6Ewf/fZU0Le0futYNJXknS4h1ZE1fAhW9YjmzRTig5xnhtdJt/++jDM4WUdcCTJH6utJOvMC1Z9lWUSf2cLPKfvKVUyLjdaVZFmHoH/UcjO/ntjsOwxdtgHevFPCPXWS+A71R9SAcNttdaCah0Ni6TRhcqUL2i8EIj5qSfCWLPbgiB8rEfdjxEX1GgCC4VjeBODdaKhCpyLLr9IX2L77qfxl0QtaoARseRl23pyV8+cjyf+y+bSWfULTRZv29bPk8mfD1LBKKuFdCtEtgdIANWL6JJYI7Ow== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=XvUJYK3D9BrLpGFRZ9fgUfu/dJo3vZ61Fskb03aOBVs=; b=BkBym8GA+6iH/kkrKmRN1HJz0vS+mUVZTOXE17Ql0MvdmzCnvzZps6E44dcoZ/yK9cYov45aa/zmeguCC0RdSZ5jMcp5Lh3GFuSRYGejuM2KiZpI5hFRCzvF5C/1+nPDunQpbYLdUzDkMneDd6zf11l0RvjkiV9MYe5p0wsVi3223B8uGGorOwFq36jnvE8rEM+rU95tqe/8+wGK5O4+yNPykP8Aubq+1LcdCghshXvgWW2Nu1HfrBqJWYyGTlA6VnQziyBt2uUh3Fv4gL1zlascGEoc8mnGqVluF9EvDdRKwUXQTXnFAVzVDjIHjneml08FkKtSblA2kz21wpdyJA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=XvUJYK3D9BrLpGFRZ9fgUfu/dJo3vZ61Fskb03aOBVs=; b=z4Tla2oqrmhY/7hzDHVFa5IY2p1t6B8hDN7JYAGzkmVnk0Ek/Fai7NZusK2AlR6IoNEevFe2nFpXoeTAPygapJezSOl0u5Q1QLY1Va7bciCkKRkpCU8RMGemd8UGfxn7Yfg07Ywlf/eY54jRZNLgP8K36ySQhxQe9aqmOsdatuw= Authentication-Results-Original: gcc.gnu.org; dkim=none (message not signed) header.d=none;gcc.gnu.org; dmarc=none action=none header.from=arm.com; Received: from VI1PR08MB5325.eurprd08.prod.outlook.com (2603:10a6:803:13e::17) by VE1PR08MB5678.eurprd08.prod.outlook.com (2603:10a6:800:1a0::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3391.11; Fri, 25 Sep 2020 14:28:29 +0000 Received: from VI1PR08MB5325.eurprd08.prod.outlook.com ([fe80::d0e7:49cd:4dae:a2a2]) by VI1PR08MB5325.eurprd08.prod.outlook.com ([fe80::d0e7:49cd:4dae:a2a2%7]) with mapi id 15.20.3412.024; Fri, 25 Sep 2020 14:28:28 +0000 Date: Fri, 25 Sep 2020 15:28:20 +0100 From: Tamar Christina To: gcc-patches@gcc.gnu.org Cc: nd@arm.com, rguenther@suse.de, ook@ucw.cz Subject: [PATCH v2 4/16]middle-end: Add dissolve code for when SLP fails and non-SLP loop vectorization is to be tried. Message-ID: <20200925142816.GA14929@arm.com> Content-Type: multipart/mixed; boundary="G4iJoqBmSsgzjUCe" Content-Disposition: inline User-Agent: Mutt/1.9.4 (2018-02-28) X-ClientProxiedBy: SA9PR10CA0028.namprd10.prod.outlook.com (2603:10b6:806:a7::33) To VI1PR08MB5325.eurprd08.prod.outlook.com (2603:10a6:803:13e::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from arm.com (217.140.106.53) by SA9PR10CA0028.namprd10.prod.outlook.com (2603:10b6:806:a7::33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3412.20 via Frontend Transport; Fri, 25 Sep 2020 14:28:27 +0000 X-Originating-IP: [217.140.106.53] X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: 5160e319-d7aa-46f2-bc12-08d8615f49e1 X-MS-TrafficTypeDiagnostic: VE1PR08MB5678:|HE1PR0802MB2298: X-Microsoft-Antispam-PRVS: x-checkrecipientrouted: true NoDisclaimer: true X-MS-Oob-TLC-OOBClassifiers: OLM:10000;OLM:10000; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: kGnRN+GmN1GR9fHo2Lbz1Pw1Isopy/4cL0o6SOqDbgxlSxXpkpnHvDb4FbhR5QSmrjdbItb0AiLxDu3TkwC1ra27pwTMrjvNfeiPdx2bsnJ144rjv++9XI26HepnyZoks7ptY13rwrcIlFom2pdlwVgU1vgauLkllCDVidKlj9hDZwb++qgRJTijHJ7SqaFAs8ZEnmJeLI9wgvaQ+cStm6UTFVOYqakJxFogOCLjNAvihaAeLIYJPz9FklradG+Agka8JnlnVD1ItqcyelIG+WmxccVOMlsS2m8oOW9Y70ROWhpu1G6dXeKszn7DhNlS4RbWuoXF9AqHnOS9lU5Schg+xuNJRX+aRqJQtqlk48FGPMH45YUunE/qozkv02lKcBgouHH/cUcDZY8dNPBmtKrF3izujj0O8uhqW78nXKc= X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:VI1PR08MB5325.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(396003)(366004)(136003)(39860400002)(346002)(376002)(8936002)(1076003)(55016002)(33656002)(8886007)(66946007)(66556008)(86362001)(66616009)(36756003)(66476007)(478600001)(6666004)(6916009)(235185007)(44144004)(5660300002)(16526019)(186003)(4743002)(4326008)(33964004)(52116002)(44832011)(7696005)(316002)(2616005)(26005)(2906002)(8676002)(956004)(2700100001); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: YldIEJaJT0ao71dVhfrzQdYVj97ZuL+xHC5r9VaY+pmvVDuOjdzHN14W8WRRl7f8UfRYHTDLmwv3afolBXxOhpwoALcnn1tZiknEJl4ig1+pkzmkIZrdsyGrwT5Ps8xEfqekVfI+JdhsKSmIlnef0ovtfh0Qp1XARfJPQb8seRVauhejhkW2HLHwoHb7FJIkr9Ylap6cQbdBeqWMzxy8DRO0QU+dizxXO7/za+zbqit25+yrZF9oA1M9jKp5nNK6b3qL1CefVtVUrX0Z5ADtRnxzP39A808uWryDt0NLBNXiSvnsL+zg0q7hXu5rHa6/z1C9YNATAZnTtuwlzgICfj5fsxQ666ssbiPKQRkgS1e+pey1J8SyOcYeHQD/eI8RCmj2XoRxSPd6UfBw5CRTcp0PAC5zGjbh5Zlj8Lw/jGZ77TJ3nmVuzF2Y5p3ewUtxf2id9gnp+EzmIf4FXzK1CStUV8n+eA74UUKnlqGOeffTph0SL9KYwUd4vTICc8Sr2IcCTJHesXxD/Kh+BpoZK2dUHKJb7Pnoj6b6Sy9kSkU1qAN8iTO3R/vWsHJR7+G0gMMc3BnXBOlrAHZGy4VJamz/TF7AHgGtSCI+uwHu4TTcdwwsoiglG81Xxua/rhzIqDkrhme0x2DTrcemKnB7Rg== X-MS-Exchange-Transport-Forked: True X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR08MB5678 Original-Authentication-Results: gcc.gnu.org; dkim=none (message not signed) header.d=none;gcc.gnu.org; dmarc=none action=none header.from=arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: DB5EUR03FT037.eop-EUR03.prod.protection.outlook.com X-MS-Office365-Filtering-Correlation-Id-Prvs: da7958b5-1028-4952-0686-08d8615f454c X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 0Q4HKVJDOjIeT4QVveJU9x7iOCkTU6Mn5vhUyFcTi3E1gOESN4KagIeVagNGH0QBxBuCS/esHiRqVWO8jp1q3HTGQrgOSILV6tzicqw8ZfEJxu8R5IwUFFFK2uXvOOpP1EMQ7LJgXWzGs7xEAW3PVx/lIn9kqWDm+TnAISURKE6y1ar+5HfnCGRjYhowPa6PsuzcJbY4YRdgrxV4RIZOb3Sr8s6sy380X43+N8C7Yqg2PKWG/VlO9WBZDwEQysiVUm2AzYaojedtx0SleWl9AXFk7mm2LUOeRK4clgvRUiphllTqrhdcF3e3srTwUmQrEjiWb26dbqj2A0Il32LUImwLsIY8b/z/K6uZwv9RjFdKTtOVECHNGjDXRpgLV3OgcdXlqyIocbgrM672fE0ik00tpWL4bEx8QDq0kKlHWos= X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFS:(4636009)(39860400002)(396003)(376002)(346002)(136003)(46966005)(44144004)(4743002)(70586007)(356005)(235185007)(70206006)(1076003)(66616009)(5660300002)(44832011)(8936002)(336012)(16526019)(82740400003)(2616005)(7696005)(186003)(47076004)(36756003)(86362001)(82310400003)(2906002)(33964004)(55016002)(6916009)(8886007)(6666004)(478600001)(316002)(33656002)(26005)(8676002)(4326008)(81166007)(956004)(2700100001); DIR:OUT; SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Sep 2020 14:28:36.2173 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 5160e319-d7aa-46f2-bc12-08d8615f49e1 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: DB5EUR03FT037.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: HE1PR0802MB2298 X-Spam-Status: No, score=-14.1 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, GIT_PATCH_0, MSGID_FROM_MTA_HEADER, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_PASS, SPF_PASS, TXREP, UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Sep 2020 14:28:40 -0000 --G4iJoqBmSsgzjUCe Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Hi All, This adds the dissolve code to undo the patterns created by the pattern matcher in case SLP is to be aborted. As mentioned in the cover letter this has one issue in that the number of copies can needed can change depending on whether TWO_OPERATORS is needed or not. Because of this I don't analyze the original statement when it's replaced by a pattern and attempt to correct it here by analyzing it after dissolve. This however seems too late and I would need to change the unroll factor, which seems a bit odd. Any advice would be appreciated. Bootstrapped Regtested on aarch64-none-linux-gnu and no issues. Thanks, Tamar gcc/ChangeLog: * tree-vect-loop.c (vect_dissolve_slp_only_patterns): New (vect_dissolve_slp_only_groups): Call vect_dissolve_slp_only_patterns. -- --G4iJoqBmSsgzjUCe Content-Type: text/x-diff; charset=utf-8 Content-Disposition: attachment; filename="rb13508.patch" diff --git a/gcc/tree-vect-loop.c b/gcc/tree-vect-loop.c index b1a6e1508c7f00f5f369ec873f927f30d673059e..8231ad6452af6ff111911a7bfb6aab2257df9fc0 100644 --- a/gcc/tree-vect-loop.c +++ b/gcc/tree-vect-loop.c @@ -1956,6 +1956,92 @@ vect_get_datarefs_in_loop (loop_p loop, basic_block *bbs, return opt_result::success (); } +/* For every SLP only pattern created by the pattern matched rooted in ROOT + restore the relevancy of the original statements over those of the pattern + and destroy the pattern relationship. This restores the SLP tree to a state + where it can be used when SLP build is cancelled or re-tried. */ + +static opt_result +vect_dissolve_slp_only_patterns (loop_vec_info loop_vinfo, + hash_set *visited, slp_tree root) +{ + if (!root || visited->contains (root)) + return opt_result::success (); + + unsigned int i; + slp_tree node; + opt_result res = opt_result::success (); + stmt_vec_info stmt_info; + stmt_vec_info related_stmt_info; + bool need_to_vectorize = false; + auto_vec cost_vec; + + visited->add (root); + + FOR_EACH_VEC_ELT (SLP_TREE_SCALAR_STMTS (root), i, stmt_info) + if (STMT_VINFO_SLP_VECT_ONLY (stmt_info) + && (related_stmt_info = STMT_VINFO_RELATED_STMT (stmt_info)) != NULL) + { + if (dump_enabled_p ()) + dump_printf_loc (MSG_NOTE, vect_location, + "dissolving relevancy of %G over %G", + STMT_VINFO_STMT (stmt_info), + STMT_VINFO_STMT (related_stmt_info)); + STMT_VINFO_RELEVANT (stmt_info) = vect_unused_in_scope; + STMT_VINFO_RELEVANT (related_stmt_info) = vect_used_in_scope; + STMT_VINFO_IN_PATTERN_P (related_stmt_info) = false; + STMT_SLP_TYPE (related_stmt_info) = hybrid; + /* Now we have to re-analyze the statement since we skipped it in the + the initial analysis due to the differences in copies. */ + res = vect_analyze_stmt (loop_vinfo, related_stmt_info, + &need_to_vectorize, NULL, NULL, &cost_vec); + + if (!res) + return res; + } + + FOR_EACH_VEC_ELT (SLP_TREE_CHILDREN (root), i, node) + { + res = vect_dissolve_slp_only_patterns (loop_vinfo, visited, node); + if (!res) + return res; + } + + return res; +} + +/* Lookup any SLP Only Pattern statements created by the SLP pattern matcher in + all slp_instances in LOOP_VINFO and undo the relevancy of statements such + that the original SLP tree before the pattern matching is used. */ + +static opt_result +vect_dissolve_slp_only_patterns (loop_vec_info loop_vinfo) +{ + + unsigned int i; + opt_result res = opt_result::success (); + hash_set *visited = new hash_set (); + + DUMP_VECT_SCOPE ("vect_dissolve_slp_only_patterns"); + + /* Unmark any SLP only patterns as relevant and restore the STMT_INFO of the + related instruction. */ + slp_instance instance; + FOR_EACH_VEC_ELT (LOOP_VINFO_SLP_INSTANCES (loop_vinfo), i, instance) + { + res = vect_dissolve_slp_only_patterns (loop_vinfo, visited, + SLP_INSTANCE_TREE (instance)); + if (!res) + { + delete visited; + return res; + } + } + + delete visited; + return res; +} + /* Look for SLP-only access groups and turn each individual access into its own group. */ static void @@ -2427,6 +2513,11 @@ again: /* Ensure that "ok" is false (with an opt_problem if dumping is enabled). */ gcc_assert (!ok); + /* Dissolve any SLP patterns created by the SLP pattern matcher. */ + opt_result dissolved = vect_dissolve_slp_only_patterns (loop_vinfo); + if (!dissolved) + return dissolved; + /* Try again with SLP forced off but if we didn't do any SLP there is no point in re-trying. */ if (!slp) --G4iJoqBmSsgzjUCe--