From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 93595 invoked by alias); 23 May 2017 15:58:35 -0000 Mailing-List: contact gcc-patches-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Archive: List-Post: List-Help: Sender: gcc-patches-owner@gcc.gnu.org Received: (qmail 93378 invoked by uid 89); 23 May 2017 15:58:35 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-25.1 required=5.0 tests=AWL,BAYES_00,GIT_PATCH_0,GIT_PATCH_1,GIT_PATCH_2,GIT_PATCH_3,KAM_LAZY_DOMAIN_SECURITY,RCVD_IN_DNSWL_LOW autolearn=ham version=3.3.2 spammy=dr's, DRs, DR's, drs X-HELO: mx0a-001b2d01.pphosted.com Received: from mx0b-001b2d01.pphosted.com (HELO mx0a-001b2d01.pphosted.com) (148.163.158.5) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Tue, 23 May 2017 15:58:31 +0000 Received: from pps.filterd (m0098416.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.20/8.16.0.20) with SMTP id v4NFcrXo144571 for ; Tue, 23 May 2017 11:58:33 -0400 Received: from e06smtp14.uk.ibm.com (e06smtp14.uk.ibm.com [195.75.94.110]) by mx0b-001b2d01.pphosted.com with ESMTP id 2ampbsyj3h-1 (version=TLSv1.2 cipher=AES256-SHA bits=256 verify=NOT) for ; Tue, 23 May 2017 11:58:33 -0400 Received: from localhost by e06smtp14.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Tue, 23 May 2017 16:58:31 +0100 Received: from b06cxnps4075.portsmouth.uk.ibm.com (9.149.109.197) by e06smtp14.uk.ibm.com (192.168.101.144) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Tue, 23 May 2017 16:58:30 +0100 Received: from d06av26.portsmouth.uk.ibm.com (d06av26.portsmouth.uk.ibm.com [9.149.105.62]) by b06cxnps4075.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id v4NFwTlW34537698; Tue, 23 May 2017 15:58:29 GMT Received: from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 8DAE7AE051; Tue, 23 May 2017 16:56:28 +0100 (BST) Received: from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 5F22EAE055; Tue, 23 May 2017 16:56:28 +0100 (BST) Received: from oc6142347168.ibm.com (unknown [9.152.212.171]) by d06av26.portsmouth.uk.ibm.com (Postfix) with ESMTP; Tue, 23 May 2017 16:56:28 +0100 (BST) Subject: [PATCH 2/5 v3] Vect peeling cost model To: Richard Biener References: <0296a54f-cb8d-d9b8-380a-9cc553dbb6da@linux.vnet.ibm.com> <2804E9EF-67D1-4EFD-AF29-65C634EBE24F@gmail.com> <6f1194a0-9e57-0028-faf4-6190beec2009@linux.vnet.ibm.com> <3e575f6d-874a-b260-1fc2-f4db1250c32b@linux.vnet.ibm.com> <15b3df5a-f6a1-23f0-57ff-1f065420df19@linux.vnet.ibm.com> <765ae66b-6294-6c39-a101-8c54a6be42ae@linux.vnet.ibm.com> Cc: GCC Patches , "Bin.Cheng" , Andreas Krebbel From: Robin Dapp Date: Tue, 23 May 2017 15:58:00 -0000 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/mixed; boundary="------------05427F0382E0F31993C30D22" X-TM-AS-GCONF: 00 x-cbid: 17052315-0016-0000-0000-000004A83336 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 17052315-0017-0000-0000-000027C92428 Message-Id: <1c1dfe6c-9e0f-2243-bf79-ff1a8bb0bb02@linux.vnet.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,, definitions=2017-05-23_04:,, signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 suspectscore=0 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1703280000 definitions=main-1705230081 X-SW-Source: 2017-05/txt/msg01773.txt.bz2 This is a multi-part message in MIME format. --------------05427F0382E0F31993C30D22 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Content-length: 382 gcc/ChangeLog: 2017-05-23 Robin Dapp * tree-vect-data-refs.c (vect_update_misalignment_for_peel): Rename. (vect_get_peeling_costs_all_drs): Create function. (vect_peeling_hash_get_lowest_cost): Use vect_get_peeling_costs_all_drs. (vect_peeling_supportable): Create function. (vect_enhance_data_refs_alignment): Use vect_peeling_supportable. --------------05427F0382E0F31993C30D22 Content-Type: text/x-patch; name="gcc-peeling-p2.diff" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="gcc-peeling-p2.diff" Content-length: 8234 diff --git a/gcc/tree-vect-data-refs.c b/gcc/tree-vect-data-refs.c index 874fdb5..fe398ea 100644 --- a/gcc/tree-vect-data-refs.c +++ b/gcc/tree-vect-data-refs.c @@ -903,7 +903,11 @@ vect_compute_data_ref_alignment (struct data_reference *dr) } -/* Function vect_update_misalignment_for_peel +/* Function vect_update_misalignment_for_peel. + Sets DR's misalignment + - to 0 if it has the same alignment as DR_PEEL, + - to the misalignment computed using NPEEL if DR's salignment is known, + - to -1 (unknown) otherwise. DR - the data reference whose misalignment is to be adjusted. DR_PEEL - the data reference whose misalignment is being made @@ -916,7 +920,7 @@ vect_update_misalignment_for_peel (struct data_reference *dr, struct data_reference *dr_peel, int npeel) { unsigned int i; - vec same_align_drs; + vec same_aligned_drs; struct data_reference *current_dr; int dr_size = GET_MODE_SIZE (TYPE_MODE (TREE_TYPE (DR_REF (dr)))); int dr_peel_size = GET_MODE_SIZE (TYPE_MODE (TREE_TYPE (DR_REF (dr_peel)))); @@ -932,9 +936,9 @@ vect_update_misalignment_for_peel (struct data_reference *dr, /* It can be assumed that the data refs with the same alignment as dr_peel are aligned in the vector loop. */ - same_align_drs + same_aligned_drs = STMT_VINFO_SAME_ALIGN_REFS (vinfo_for_stmt (DR_STMT (dr_peel))); - FOR_EACH_VEC_ELT (same_align_drs, i, current_dr) + FOR_EACH_VEC_ELT (same_aligned_drs, i, current_dr) { if (current_dr != dr) continue; @@ -1234,27 +1238,23 @@ vect_peeling_hash_get_most_frequent (_vect_peel_info **slot, return 1; } +/* Get the costs of peeling NPEEL iterations checking data access costs + for all data refs. */ -/* Traverse peeling hash table and calculate cost for each peeling option. - Find the one with the lowest cost. */ - -int -vect_peeling_hash_get_lowest_cost (_vect_peel_info **slot, - _vect_peel_extended_info *min) +static void +vect_get_peeling_costs_all_drs (struct data_reference *dr0, + unsigned int *inside_cost, + unsigned int *outside_cost, + stmt_vector_for_cost *body_cost_vec, + unsigned int npeel, unsigned int vf) { - vect_peel_info elem = *slot; - int save_misalignment, dummy; - unsigned int inside_cost = 0, outside_cost = 0, i; - gimple *stmt = DR_STMT (elem->dr); + gimple *stmt = DR_STMT (dr0); stmt_vec_info stmt_info = vinfo_for_stmt (stmt); loop_vec_info loop_vinfo = STMT_VINFO_LOOP_VINFO (stmt_info); vec datarefs = LOOP_VINFO_DATAREFS (loop_vinfo); - struct data_reference *dr; - stmt_vector_for_cost prologue_cost_vec, body_cost_vec, epilogue_cost_vec; - prologue_cost_vec.create (2); - body_cost_vec.create (2); - epilogue_cost_vec.create (2); + unsigned i; + data_reference *dr; FOR_EACH_VEC_ELT (datarefs, i, dr) { @@ -1272,12 +1272,40 @@ vect_peeling_hash_get_lowest_cost (_vect_peel_info **slot, && !STMT_VINFO_GROUPED_ACCESS (stmt_info)) continue; + int save_misalignment; save_misalignment = DR_MISALIGNMENT (dr); - vect_update_misalignment_for_peel (dr, elem->dr, elem->npeel); - vect_get_data_access_cost (dr, &inside_cost, &outside_cost, - &body_cost_vec); + if (dr == dr0 && npeel == vf / 2) + SET_DR_MISALIGNMENT (dr, 0); + else + vect_update_misalignment_for_peel (dr, dr0, npeel); + vect_get_data_access_cost (dr, inside_cost, outside_cost, + body_cost_vec); SET_DR_MISALIGNMENT (dr, save_misalignment); } +} + +/* Traverse peeling hash table and calculate cost for each peeling option. + Find the one with the lowest cost. */ + +int +vect_peeling_hash_get_lowest_cost (_vect_peel_info **slot, + _vect_peel_extended_info *min) +{ + vect_peel_info elem = *slot; + int dummy; + unsigned int inside_cost = 0, outside_cost = 0; + gimple *stmt = DR_STMT (elem->dr); + stmt_vec_info stmt_info = vinfo_for_stmt (stmt); + loop_vec_info loop_vinfo = STMT_VINFO_LOOP_VINFO (stmt_info); + stmt_vector_for_cost prologue_cost_vec, body_cost_vec, + epilogue_cost_vec; + + prologue_cost_vec.create (2); + body_cost_vec.create (2); + epilogue_cost_vec.create (2); + + vect_get_peeling_costs_all_drs (elem->dr, &inside_cost, &outside_cost, + &body_cost_vec, elem->npeel, 0); outside_cost += vect_get_known_peeling_cost (loop_vinfo, elem->npeel, &dummy, @@ -1292,7 +1320,8 @@ vect_peeling_hash_get_lowest_cost (_vect_peel_info **slot, epilogue_cost_vec.release (); if (inside_cost < min->inside_cost - || (inside_cost == min->inside_cost && outside_cost < min->outside_cost)) + || (inside_cost == min->inside_cost + && outside_cost < min->outside_cost)) { min->inside_cost = inside_cost; min->outside_cost = outside_cost; @@ -1300,6 +1329,7 @@ vect_peeling_hash_get_lowest_cost (_vect_peel_info **slot, min->body_cost_vec = body_cost_vec; min->peel_info.dr = elem->dr; min->peel_info.npeel = elem->npeel; + min->peel_info.count = elem->count; } else body_cost_vec.release (); @@ -1342,6 +1372,52 @@ vect_peeling_hash_choose_best_peeling (hash_table *peeling_hta return res.peel_info.dr; } +/* Return true if the new peeling NPEEL is supported. */ + +static bool +vect_peeling_supportable (loop_vec_info loop_vinfo, struct data_reference *dr0, + unsigned npeel) +{ + unsigned i; + struct data_reference *dr = NULL; + vec datarefs = LOOP_VINFO_DATAREFS (loop_vinfo); + gimple *stmt; + stmt_vec_info stmt_info; + enum dr_alignment_support supportable_dr_alignment; + + /* Ensure that all data refs can be vectorized after the peel. */ + FOR_EACH_VEC_ELT (datarefs, i, dr) + { + int save_misalignment; + + if (dr == dr0) + continue; + + stmt = DR_STMT (dr); + stmt_info = vinfo_for_stmt (stmt); + /* For interleaving, only the alignment of the first access + matters. */ + if (STMT_VINFO_GROUPED_ACCESS (stmt_info) + && GROUP_FIRST_ELEMENT (stmt_info) != stmt) + continue; + + /* Strided accesses perform only component accesses, alignment is + irrelevant for them. */ + if (STMT_VINFO_STRIDED_P (stmt_info) + && !STMT_VINFO_GROUPED_ACCESS (stmt_info)) + continue; + + save_misalignment = DR_MISALIGNMENT (dr); + vect_update_misalignment_for_peel (dr, dr0, npeel); + supportable_dr_alignment = vect_supportable_dr_alignment (dr, false); + SET_DR_MISALIGNMENT (dr, save_misalignment); + + if (!supportable_dr_alignment) + return false; + } + + return true; +} /* Function vect_enhance_data_refs_alignment @@ -1780,40 +1856,11 @@ vect_enhance_data_refs_alignment (loop_vec_info loop_vinfo) "Try peeling by %d\n", npeel); } - /* Ensure that all data refs can be vectorized after the peel. */ - FOR_EACH_VEC_ELT (datarefs, i, dr) - { - int save_misalignment; - - if (dr == dr0) - continue; - - stmt = DR_STMT (dr); - stmt_info = vinfo_for_stmt (stmt); - /* For interleaving, only the alignment of the first access - matters. */ - if (STMT_VINFO_GROUPED_ACCESS (stmt_info) - && GROUP_FIRST_ELEMENT (stmt_info) != stmt) - continue; - - /* Strided accesses perform only component accesses, alignment is - irrelevant for them. */ - if (STMT_VINFO_STRIDED_P (stmt_info) - && !STMT_VINFO_GROUPED_ACCESS (stmt_info)) - continue; - - save_misalignment = DR_MISALIGNMENT (dr); - vect_update_misalignment_for_peel (dr, dr0, npeel); - supportable_dr_alignment = vect_supportable_dr_alignment (dr, false); - SET_DR_MISALIGNMENT (dr, save_misalignment); - - if (!supportable_dr_alignment) - { - do_peeling = false; - break; - } - } + /* Ensure that all datarefs can be vectorized after the peel. */ + if (!vect_peeling_supportable (loop_vinfo, dr0, npeel)) + do_peeling = false; + /* Check if all datarefs are supportable and log. */ if (do_peeling && known_alignment_for_access_p (dr0) && npeel == 0) { stat = vect_verify_datarefs_alignment (loop_vinfo); --------------05427F0382E0F31993C30D22--