From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) by sourceware.org (Postfix) with ESMTPS id 1E111385842A for ; Thu, 14 Sep 2023 03:12:18 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 1E111385842A Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=linux.ibm.com Received: from pps.filterd (m0360083.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 38E39boq021124; Thu, 14 Sep 2023 03:12:14 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=6l68TugUHT9n0p00kuhrHFYdDmKgcJC1VWtgmGj7kuc=; b=oEbGWQbrJ/XnzHhIGclcHirJl7qPu9Dnx+UkUjdpgTDnjf5qDysK2d8Q1VHcqd0PiCjK j+YPzAhiTLPo3IjY76XQWoU9bYCoibkhTYYS5CEVdlOp/nlkbguWckpVzsadmBkYkBFL p2jvocFa6YeHhJ0HJ9eskzQYg82EPhKBohY7Jo+vedFwOY6QnGRGXVr2AGTwnoeTQ66B Lqmv5gL1uvbwnacOPHVIgSq7x7DTWs2ZH5HsQQukOffn/7rJuuqFwYBW/wI8X6GzBqXn B4Eqea0dwVfidcFyip1i/YUsTjh5jhsZ9iQ3ZGVP56+uAmrPKC+xEGVMAmXf15w10mu+ yg== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3t3s628wmc-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 14 Sep 2023 03:12:14 +0000 Received: from m0360083.ppops.net (m0360083.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 38E3AIjV023165; Thu, 14 Sep 2023 03:12:14 GMT Received: from ppma13.dal12v.mail.ibm.com (dd.9e.1632.ip4.static.sl-reverse.com [50.22.158.221]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3t3s628wm4-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 14 Sep 2023 03:12:13 +0000 Received: from pps.filterd (ppma13.dal12v.mail.ibm.com [127.0.0.1]) by ppma13.dal12v.mail.ibm.com (8.17.1.19/8.17.1.19) with ESMTP id 38E0S9kT002352; Thu, 14 Sep 2023 03:12:13 GMT Received: from smtprelay06.fra02v.mail.ibm.com ([9.218.2.230]) by ppma13.dal12v.mail.ibm.com (PPS) with ESMTPS id 3t158kffj1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 14 Sep 2023 03:12:12 +0000 Received: from smtpav06.fra02v.mail.ibm.com (smtpav06.fra02v.mail.ibm.com [10.20.54.105]) by smtprelay06.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 38E3CBx044827092 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Sep 2023 03:12:11 GMT Received: from smtpav06.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 392CA20040; Thu, 14 Sep 2023 03:12:11 +0000 (GMT) Received: from smtpav06.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 6EFCB2004B; Thu, 14 Sep 2023 03:12:10 +0000 (GMT) Received: from trout.aus.stglabs.ibm.com (unknown [9.40.194.100]) by smtpav06.fra02v.mail.ibm.com (Postfix) with ESMTP; Thu, 14 Sep 2023 03:12:10 +0000 (GMT) From: Kewen Lin To: gcc-patches@gcc.gnu.org Cc: richard.guenther@gmail.com, richard.sandiford@arm.com Subject: [PATCH 09/10] vect: Get rid of vect_model_store_cost Date: Wed, 13 Sep 2023 22:11:58 -0500 Message-Id: X-Mailer: git-send-email 2.31.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-TM-AS-GCONF: 00 X-Proofpoint-GUID: _ljovXMqTxoxOOpH2eryTEMea9vxCyYr X-Proofpoint-ORIG-GUID: W3DzvZd6zhSwjPuWNtOU-mxbGdJGKxcq X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.267,Aquarius:18.0.980,Hydra:6.0.601,FMLib:17.11.176.26 definitions=2023-09-13_19,2023-09-13_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxlogscore=999 spamscore=0 mlxscore=0 bulkscore=0 suspectscore=0 lowpriorityscore=0 malwarescore=0 priorityscore=1501 impostorscore=0 clxscore=1015 phishscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2308100000 definitions=main-2309140025 X-Spam-Status: No, score=-12.3 required=5.0 tests=BAYES_00,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,GIT_PATCH_0,RCVD_IN_MSPIKE_H4,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_PASS,TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: This patch is to eventually get rid of vect_model_store_cost, it adjusts the costing for the remaining memory access types VMAT_CONTIGUOUS{, _DOWN, _REVERSE} by moving costing close to the transform code. Note that in vect_model_store_cost, there is one special handling for vectorizing a store into the function result, since it's extra penalty and the transform part doesn't have it, this patch keep it alone. gcc/ChangeLog: * tree-vect-stmts.cc (vect_model_store_cost): Remove. (vectorizable_store): Adjust the costing for the remaining memory access types VMAT_CONTIGUOUS{, _DOWN, _REVERSE}. --- gcc/tree-vect-stmts.cc | 137 +++++++++++++---------------------------- 1 file changed, 44 insertions(+), 93 deletions(-) diff --git a/gcc/tree-vect-stmts.cc b/gcc/tree-vect-stmts.cc index e3ba8077091..3d451c80bca 100644 --- a/gcc/tree-vect-stmts.cc +++ b/gcc/tree-vect-stmts.cc @@ -951,81 +951,6 @@ cfun_returns (tree decl) return false; } -/* Function vect_model_store_cost - - Models cost for stores. In the case of grouped accesses, one access - has the overhead of the grouped access attributed to it. */ - -static void -vect_model_store_cost (vec_info *vinfo, stmt_vec_info stmt_info, int ncopies, - vect_memory_access_type memory_access_type, - dr_alignment_support alignment_support_scheme, - int misalignment, - vec_load_store_type vls_type, slp_tree slp_node, - stmt_vector_for_cost *cost_vec) -{ - gcc_assert (memory_access_type != VMAT_GATHER_SCATTER - && memory_access_type != VMAT_ELEMENTWISE - && memory_access_type != VMAT_STRIDED_SLP - && memory_access_type != VMAT_LOAD_STORE_LANES - && memory_access_type != VMAT_CONTIGUOUS_PERMUTE); - - unsigned int inside_cost = 0, prologue_cost = 0; - - /* ??? Somehow we need to fix this at the callers. */ - if (slp_node) - ncopies = SLP_TREE_NUMBER_OF_VEC_STMTS (slp_node); - - if (vls_type == VLS_STORE_INVARIANT) - { - if (!slp_node) - prologue_cost += record_stmt_cost (cost_vec, 1, scalar_to_vec, - stmt_info, 0, vect_prologue); - } - - - /* Costs of the stores. */ - vect_get_store_cost (vinfo, stmt_info, ncopies, alignment_support_scheme, - misalignment, &inside_cost, cost_vec); - - /* When vectorizing a store into the function result assign - a penalty if the function returns in a multi-register location. - In this case we assume we'll end up with having to spill the - vector result and do piecewise loads as a conservative estimate. */ - tree base = get_base_address (STMT_VINFO_DATA_REF (stmt_info)->ref); - if (base - && (TREE_CODE (base) == RESULT_DECL - || (DECL_P (base) && cfun_returns (base))) - && !aggregate_value_p (base, cfun->decl)) - { - rtx reg = hard_function_value (TREE_TYPE (base), cfun->decl, 0, 1); - /* ??? Handle PARALLEL in some way. */ - if (REG_P (reg)) - { - int nregs = hard_regno_nregs (REGNO (reg), GET_MODE (reg)); - /* Assume that a single reg-reg move is possible and cheap, - do not account for vector to gp register move cost. */ - if (nregs > 1) - { - /* Spill. */ - prologue_cost += record_stmt_cost (cost_vec, ncopies, - vector_store, - stmt_info, 0, vect_epilogue); - /* Loads. */ - prologue_cost += record_stmt_cost (cost_vec, ncopies * nregs, - scalar_load, - stmt_info, 0, vect_epilogue); - } - } - } - - if (dump_enabled_p ()) - dump_printf_loc (MSG_NOTE, vect_location, - "vect_model_store_cost: inside_cost = %d, " - "prologue_cost = %d .\n", inside_cost, prologue_cost); -} - - /* Calculate cost of DR's memory access. */ void vect_get_store_cost (vec_info *, stmt_vec_info stmt_info, int ncopies, @@ -9223,6 +9148,11 @@ vectorizable_store (vec_info *vinfo, return true; } + gcc_assert (memory_access_type == VMAT_CONTIGUOUS + || memory_access_type == VMAT_CONTIGUOUS_DOWN + || memory_access_type == VMAT_CONTIGUOUS_PERMUTE + || memory_access_type == VMAT_CONTIGUOUS_REVERSE); + unsigned inside_cost = 0, prologue_cost = 0; auto_vec result_chain (group_size); auto_vec vec_oprnds; @@ -9257,10 +9187,9 @@ vectorizable_store (vec_info *vinfo, that there is no interleaving, DR_GROUP_SIZE is 1, and only one iteration of the loop will be executed. */ op = vect_get_store_rhs (next_stmt_info); - if (costing_p - && memory_access_type == VMAT_CONTIGUOUS_PERMUTE) + if (costing_p) update_prologue_cost (&prologue_cost, op); - else if (!costing_p) + else { vect_get_vec_defs_for_operand (vinfo, next_stmt_info, ncopies, op, @@ -9352,10 +9281,9 @@ vectorizable_store (vec_info *vinfo, { if (costing_p) { - if (memory_access_type == VMAT_CONTIGUOUS_PERMUTE) - vect_get_store_cost (vinfo, stmt_info, 1, - alignment_support_scheme, misalignment, - &inside_cost, cost_vec); + vect_get_store_cost (vinfo, stmt_info, 1, + alignment_support_scheme, misalignment, + &inside_cost, cost_vec); if (!slp) { @@ -9550,18 +9478,41 @@ vectorizable_store (vec_info *vinfo, if (costing_p) { - if (memory_access_type == VMAT_CONTIGUOUS_PERMUTE) - { - if (dump_enabled_p ()) - dump_printf_loc (MSG_NOTE, vect_location, - "vect_model_store_cost: inside_cost = %d, " - "prologue_cost = %d .\n", - inside_cost, prologue_cost); + /* When vectorizing a store into the function result assign + a penalty if the function returns in a multi-register location. + In this case we assume we'll end up with having to spill the + vector result and do piecewise loads as a conservative estimate. */ + tree base = get_base_address (STMT_VINFO_DATA_REF (stmt_info)->ref); + if (base + && (TREE_CODE (base) == RESULT_DECL + || (DECL_P (base) && cfun_returns (base))) + && !aggregate_value_p (base, cfun->decl)) + { + rtx reg = hard_function_value (TREE_TYPE (base), cfun->decl, 0, 1); + /* ??? Handle PARALLEL in some way. */ + if (REG_P (reg)) + { + int nregs = hard_regno_nregs (REGNO (reg), GET_MODE (reg)); + /* Assume that a single reg-reg move is possible and cheap, + do not account for vector to gp register move cost. */ + if (nregs > 1) + { + /* Spill. */ + prologue_cost + += record_stmt_cost (cost_vec, ncopies, vector_store, + stmt_info, 0, vect_epilogue); + /* Loads. */ + prologue_cost + += record_stmt_cost (cost_vec, ncopies * nregs, scalar_load, + stmt_info, 0, vect_epilogue); + } + } } - else - vect_model_store_cost (vinfo, stmt_info, ncopies, memory_access_type, - alignment_support_scheme, misalignment, vls_type, - slp_node, cost_vec); + if (dump_enabled_p ()) + dump_printf_loc (MSG_NOTE, vect_location, + "vect_model_store_cost: inside_cost = %d, " + "prologue_cost = %d .\n", + inside_cost, prologue_cost); } return true; -- 2.31.1