From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 80174 invoked by alias); 30 Sep 2019 08:53:59 -0000 Mailing-List: contact gcc-patches-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Archive: List-Post: List-Help: Sender: gcc-patches-owner@gcc.gnu.org Received: (qmail 80159 invoked by uid 89); 30 Sep 2019 08:53:58 -0000 Authentication-Results: sourceware.org; auth=none X-Spam-SWARE-Status: No, score=-23.7 required=5.0 tests=AWL,BAYES_00,GIT_PATCH_0,GIT_PATCH_1,GIT_PATCH_2,GIT_PATCH_3,RCVD_IN_DNSWL_NONE,SPF_HELO_PASS,SPF_PASS autolearn=ham version=3.3.1 spammy=gathered, released, nb X-HELO: NAM02-BL2-obe.outbound.protection.outlook.com Received: from mail-eopbgr750108.outbound.protection.outlook.com (HELO NAM02-BL2-obe.outbound.protection.outlook.com) (40.107.75.108) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Mon, 30 Sep 2019 08:53:48 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=JOE4Sjq6dTF1yXhXb/XEtHsaKoEH1+6SoZCMCMQHcSRi+LK0+sRa4p9TaDHz5duitquVwToLpy5BHW0Ea3LnUKTZCKeG6BKuwdn4Pe4thTaecp3FPPzaqnXaORdR5PsixYeuyzi7T1FJBhXNa8IxWJgwVIM7N48aaMObYAkYcd+06p7FibeHZwLpvpK7rhjlZGi5gQcmbLqP74nd8H9tXho0OHtNhVKYXGRYzpa50j7LDZeJ/Yl/AcefVbHX7mUdmPzNDnrvQdM9oVyQ6acfrLzJ9VfahDsW3/ypfL9Pea6APw4TdnpOaRC4Pr++z69mDnOhBcZzs72FaiKR6GEjbQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=9yTqXtSHoeTmZDpxPu8tz1q3Yb5FdCSYnDX5/K3CAro=; b=MbXnZxTlV8YZHh675NiofX0c6BdJ9BtoQ3Sc4Y95NbrmbRMKTDE4dyassvFTG0gxXymCBXWbG2oDtsQtKDYfWOXxAvevlVBpIG3O9n9YOdsZjqgvQU/Y9IHqv1pU4mWKMgNVvnB1Cbt81LJyH5X741QAI84F/FnHG2drW4N3pGX0KzYYRxMnTaXfBlKj0yOxFF9h7BM7PIzuAflvHYSARoIKa5ha5dtVK2QmdbHMMMjEdJUTrJWhi9frZF5huNR6Qyc4KT5hMbJUgeuj5+jJwvWXRaoIjvLW8pwlD5EeAHK3K72FW0mVdXVG9Odfm/RJJWD77s0Eh/RekkjWeo6E1w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=os.amperecomputing.com; dmarc=pass action=none header.from=os.amperecomputing.com; dkim=pass header.d=os.amperecomputing.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=os.amperecomputing.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=9yTqXtSHoeTmZDpxPu8tz1q3Yb5FdCSYnDX5/K3CAro=; b=NoEinjjfGxx24OjBOluDMqL0yo93Mg40Vbk67anxj5oLjNgZj2X5PMrPtRBMUK9ChSqoDng9s4Fbq4G0pQdsV4kG9Kh+qck6L3qoVlivEaTOjEyf1S4f9x0p0aUtXjbfhPHyiyFszc6xmuGl4DGOnh+wVetCjAsvE1PNMSA/e8g= Received: from BYAPR01MB4869.prod.exchangelabs.com (20.177.228.18) by BYAPR01MB3814.prod.exchangelabs.com (52.135.193.142) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2305.20; Mon, 30 Sep 2019 08:53:43 +0000 Received: from BYAPR01MB4869.prod.exchangelabs.com ([fe80::a016:e802:d3d0:f1c7]) by BYAPR01MB4869.prod.exchangelabs.com ([fe80::a016:e802:d3d0:f1c7%3]) with mapi id 15.20.2305.017; Mon, 30 Sep 2019 08:53:37 +0000 From: Feng Xue OS To: Martin Jambor , Jan Hubicka , "gcc-patches@gcc.gnu.org" Subject: Ping: [PATCH V4] Extend IPA-CP to support arithmetically-computed value-passing on by-ref argument (PR ipa/91682) Date: Mon, 30 Sep 2019 08:53:00 -0000 Message-ID: References: ,,, In-Reply-To: authentication-results: spf=none (sender IP is ) smtp.mailfrom=fxue@os.amperecomputing.com; x-ms-oob-tlc-oobclassifiers: OLM:9508; received-spf: None (protection.outlook.com: os.amperecomputing.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-ms-exchange-transport-forked: True Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: cvefoB7XAxq6SYe3zlUNPl4ieNQidWMww9FjCbxWpsqgJp/tq2dVGYqVidCtxc9If6gcipnbN3WxmNMjy1YVVqd/V+CxwNS66NfJFm4rVv8= X-IsSubscribed: yes X-SW-Source: 2019-09/txt/msg01735.txt.bz2 Hi Honza & Martin, And also hope your comments on this patch. Thanks. Feng ________________________________________ From: Feng Xue OS Sent: Thursday, September 19, 2019 10:30 PM To: Martin Jambor; Jan Hubicka; gcc-patches@gcc.gnu.org Subject: [PATCH V4] Extend IPA-CP to support arithmetically-computed value-= passing on by-ref argument (PR ipa/91682) Fix a bug on unary/binary operation check. Feng --- diff --git a/gcc/ipa-cp.c b/gcc/ipa-cp.c index 33d52fe5537..f218f1093b8 100644 --- a/gcc/ipa-cp.c +++ b/gcc/ipa-cp.c @@ -1244,23 +1244,23 @@ initialize_node_lattices (struct cgraph_node *node) } } -/* Return the result of a (possibly arithmetic) pass through jump function - JFUNC on the constant value INPUT. RES_TYPE is the type of the paramet= er - to which the result is passed. Return NULL_TREE if that cannot be - determined or be considered an interprocedural invariant. */ +/* Return the result of a (possibly arithmetic) operation on the constant + value INPUT. OPERAND is 2nd operand for binary operation. RES_TYPE is + the type of the parameter to which the result is passed. Return + NULL_TREE if that cannot be determined or be considered an + interprocedural invariant. */ static tree -ipa_get_jf_pass_through_result (struct ipa_jump_func *jfunc, tree input, - tree res_type) +ipa_get_jf_arith_result (enum tree_code opcode, tree input, tree operand, + tree res_type) { tree res; - if (ipa_get_jf_pass_through_operation (jfunc) =3D=3D NOP_EXPR) + if (opcode =3D=3D NOP_EXPR) return input; if (!is_gimple_ip_invariant (input)) return NULL_TREE; - tree_code opcode =3D ipa_get_jf_pass_through_operation (jfunc); if (!res_type) { if (TREE_CODE_CLASS (opcode) =3D=3D tcc_comparison) @@ -1274,8 +1274,7 @@ ipa_get_jf_pass_through_result (struct ipa_jump_func = *jfunc, tree input, if (TREE_CODE_CLASS (opcode) =3D=3D tcc_unary) res =3D fold_unary (opcode, res_type, input); else - res =3D fold_binary (opcode, res_type, input, - ipa_get_jf_pass_through_operand (jfunc)); + res =3D fold_binary (opcode, res_type, input, operand); if (res && !is_gimple_ip_invariant (res)) return NULL_TREE; @@ -1283,6 +1282,21 @@ ipa_get_jf_pass_through_result (struct ipa_jump_func= *jfunc, tree input, return res; } +/* Return the result of a (possibly arithmetic) pass through jump function + JFUNC on the constant value INPUT. RES_TYPE is the type of the paramet= er + to which the result is passed. Return NULL_TREE if that cannot be + determined or be considered an interprocedural invariant. */ + +static tree +ipa_get_jf_pass_through_result (struct ipa_jump_func *jfunc, tree input, + tree res_type) +{ + return ipa_get_jf_arith_result (ipa_get_jf_pass_through_operation (jfunc= ), + input, + ipa_get_jf_pass_through_operand (jfunc), + res_type); +} + /* Return the result of an ancestor jump function JFUNC on the constant va= lue INPUT. Return NULL_TREE if that cannot be determined. */ @@ -1416,6 +1430,146 @@ ipa_context_from_jfunc (ipa_node_params *info, cgra= ph_edge *cs, int csidx, return ctx; } +/* See if NODE is a clone with a known aggregate value at a given OFFSET o= f a + parameter with the given INDEX. */ + +static tree +get_clone_agg_value (struct cgraph_node *node, HOST_WIDE_INT offset, + int index) +{ + struct ipa_agg_replacement_value *aggval; + + aggval =3D ipa_get_agg_replacements_for_node (node); + while (aggval) + { + if (aggval->offset =3D=3D offset + && aggval->index =3D=3D index) + return aggval->value; + aggval =3D aggval->next; + } + return NULL_TREE; +} + +/* Determine whether ITEM, jump function for an aggregate part, evaluates = to a + single known constant value and if so, return it. Otherwise return NUL= L. + NODE and INFO describes the caller node or the one it is inlined to, and + its related info. */ + +static tree +ipa_agg_value_from_node (class ipa_node_params *info, + struct cgraph_node *node, + struct ipa_agg_jf_item *item) +{ + tree value =3D NULL_TREE; + int src_idx; + + if (item->offset < 0 || item->jftype =3D=3D IPA_JF_UNKNOWN) + return NULL_TREE; + + if (item->jftype =3D=3D IPA_JF_CONST) + return item->value.constant; + + gcc_checking_assert (item->jftype =3D=3D IPA_JF_PASS_THROUGH + || item->jftype =3D=3D IPA_JF_LOAD_AGG); + + src_idx =3D item->value.pass_through.formal_id; + + if (info->ipcp_orig_node) + { + if (item->jftype =3D=3D IPA_JF_PASS_THROUGH) + value =3D info->known_csts[src_idx]; + else + value =3D get_clone_agg_value (node, item->value.load_agg.offset, + src_idx); + } + else if (info->lattices) + { + class ipcp_param_lattices *src_plats + =3D ipa_get_parm_lattices (info, src_idx); + + if (item->jftype =3D=3D IPA_JF_PASS_THROUGH) + { + struct ipcp_lattice *lat =3D &src_plats->itself; + + if (!lat->is_single_const ()) + return NULL_TREE; + + value =3D lat->values->value; + } + else if (src_plats->aggs + && !src_plats->aggs_bottom + && !src_plats->aggs_contain_variable + && src_plats->aggs_by_ref =3D=3D item->value.load_agg.by_ref) + { + struct ipcp_agg_lattice *aglat; + + for (aglat =3D src_plats->aggs; aglat; aglat =3D aglat->next) + { + if (aglat->offset > item->value.load_agg.offset) + break; + + if (aglat->offset =3D=3D item->value.load_agg.offset) + { + if (aglat->is_single_const ()) + value =3D aglat->values->value; + break; + } + } + } + } + + if (!value) + return NULL_TREE; + + if (item->jftype =3D=3D IPA_JF_LOAD_AGG) + { + tree load_type =3D item->value.load_agg.type; + tree value_type =3D TREE_TYPE (value); + + /* Ensure value type is compatible with load type. */ + if (!useless_type_conversion_p (load_type, value_type)) + return NULL_TREE; + } + + return ipa_get_jf_arith_result (item->value.pass_through.operation, + value, + item->value.pass_through.operand, + item->type); +} + +/* Determine whether AGG_JFUNC evaluates to a set of known constant value = for + an aggregate and if so, return it. Otherwise return an empty set. NODE + and INFO describes the caller node or the one it is inlined to, and its + related info. */ + +struct ipa_agg_value_set +ipa_agg_value_set_from_jfunc (class ipa_node_params *info, cgraph_node *no= de, + struct ipa_agg_jump_function *agg_jfunc) +{ + struct ipa_agg_value_set agg; + struct ipa_agg_jf_item *item; + int i; + + agg.items =3D vNULL; + agg.by_ref =3D agg_jfunc->by_ref; + + FOR_EACH_VEC_SAFE_ELT (agg_jfunc->items, i, item) + { + tree value =3D ipa_agg_value_from_node (info, node, item); + + if (value) + { + struct ipa_agg_value value_item; + + value_item.offset =3D item->offset; + value_item.value =3D value; + + agg.items.safe_push (value_item); + } + } + return agg; +} + /* If checking is enabled, verify that no lattice is in the TOP state, i.e= . not bottom, not containing a variable component and without any known value= at the same time. */ @@ -1592,16 +1746,25 @@ ipcp_lattice::add_value (valtype newval, c= graph_edge *cs, return true; } -/* Propagate values through a pass-through jump function JFUNC associated = with - edge CS, taking values from SRC_LAT and putting them into DEST_LAT. SR= C_IDX - is the index of the source parameter. PARM_TYPE is the type of the - parameter to which the result is passed. */ +/* Propagate values through an arithmetic transformation described by a ju= mp + function associated with edge CS, taking values from SRC_LAT and putting + them into DEST_LAT. OPND1_TYPE is expected type for the values in SRC_= LAT. + OPND2 is a constant value if transformation is a binary operation. + SRC_OFFSET specifies offset in an aggregate if SRC_LAT describes lattic= e of + a part of the aggregate. SRC_IDX is the index of the source parameter. + RES_TYPE is the value type of result being propagated into. Return tru= e if + DEST_LAT changed. */ static bool -propagate_vals_across_pass_through (cgraph_edge *cs, ipa_jump_func *jfunc, - ipcp_lattice *src_lat, - ipcp_lattice *dest_lat, int src_i= dx, - tree parm_type) +propagate_vals_across_arith_jfunc (cgraph_edge *cs, + enum tree_code opcode, + tree opnd1_type, + tree opnd2, + ipcp_lattice *src_lat, + ipcp_lattice *dest_lat, + HOST_WIDE_INT src_offset, + int src_idx, + tree res_type) { ipcp_value *src_val; bool ret =3D false; @@ -1611,17 +1774,22 @@ propagate_vals_across_pass_through (cgraph_edge *cs= , ipa_jump_func *jfunc, number of them and we would just make lattices bottom. If this condi= tion is ever relaxed we have to detect self-feeding recursive calls in cgraph_edge_brings_value_p in a smarter way. */ - if ((ipa_get_jf_pass_through_operation (jfunc) !=3D NOP_EXPR) - && ipa_edge_within_scc (cs)) + if (opcode !=3D NOP_EXPR && ipa_edge_within_scc (cs)) ret =3D dest_lat->set_contains_variable (); else for (src_val =3D src_lat->values; src_val; src_val =3D src_val->next) { - tree cstval =3D ipa_get_jf_pass_through_result (jfunc, src_val->val= ue, - parm_type); + tree opnd1 =3D src_val->value; + tree cstval =3D NULL_TREE; + + /* Skip source values that is incompatible with specified type. */ + if (!opnd1_type + || useless_type_conversion_p (opnd1_type, TREE_TYPE (opnd1))) + cstval =3D ipa_get_jf_arith_result (opcode, opnd1, opnd2, res_typ= e); if (cstval) - ret |=3D dest_lat->add_value (cstval, cs, src_val, src_idx); + ret |=3D dest_lat->add_value (cstval, cs, src_val, src_idx, + src_offset); else ret |=3D dest_lat->set_contains_variable (); } @@ -1629,6 +1797,24 @@ propagate_vals_across_pass_through (cgraph_edge *cs,= ipa_jump_func *jfunc, return ret; } +/* Propagate values through a pass-through jump function JFUNC associated = with + edge CS, taking values from SRC_LAT and putting them into DEST_LAT. SR= C_IDX + is the index of the source parameter. PARM_TYPE is the type of the + parameter to which the result is passed. */ + +static bool +propagate_vals_across_pass_through (cgraph_edge *cs, ipa_jump_func *jfunc, + ipcp_lattice *src_lat, + ipcp_lattice *dest_lat, int src_i= dx, + tree parm_type) +{ + return propagate_vals_across_arith_jfunc (cs, + ipa_get_jf_pass_through_operation (jfunc), + NULL_TREE, + ipa_get_jf_pass_through_operand (jfunc), + src_lat, dest_lat, -1, src_idx, parm_type); +} + /* Propagate values through an ancestor jump function JFUNC associated with edge CS, taking values from SRC_LAT and putting them into DEST_LAT. SR= C_IDX is the index of the source parameter. */ @@ -1789,7 +1975,6 @@ propagate_context_across_jump_function (cgraph_edge *= cs, added_sth =3D true; } } - } prop_fail: @@ -2145,6 +2330,85 @@ agg_pass_through_permissible_p (class ipcp_param_lat= tices *src_plats, || ipa_get_jf_pass_through_agg_preserved (jfunc)); } +/* Propagate values through ITEM, jump function for a part of an aggregate, + into corresponding aggregate lattice AGLAT. CS is the call graph edge + associated with the jump function. Return true if AGLAT changed in any + way. */ + +static bool +propagate_aggregate_lattice (struct cgraph_edge *cs, + struct ipa_agg_jf_item *item, + struct ipcp_agg_lattice *aglat) +{ + class ipa_node_params *caller_info; + class ipcp_param_lattices *src_plats; + struct ipcp_lattice *src_lat; + HOST_WIDE_INT src_offset; + int src_idx; + tree load_type; + bool ret; + + if (item->jftype =3D=3D IPA_JF_CONST) + { + tree value =3D item->value.constant; + + gcc_checking_assert (is_gimple_ip_invariant (value)); + return aglat->add_value (value, cs, NULL, 0); + } + + gcc_checking_assert (item->jftype =3D=3D IPA_JF_PASS_THROUGH + || item->jftype =3D=3D IPA_JF_LOAD_AGG); + + caller_info =3D IPA_NODE_REF (cs->caller); + src_idx =3D item->value.pass_through.formal_id; + src_plats =3D ipa_get_parm_lattices (caller_info, src_idx); + + if (item->jftype =3D=3D IPA_JF_PASS_THROUGH) + { + load_type =3D NULL_TREE; + src_lat =3D &src_plats->itself; + src_offset =3D -1; + } + else + { + HOST_WIDE_INT load_offset =3D item->value.load_agg.offset; + struct ipcp_agg_lattice *src_aglat; + + for (src_aglat =3D src_plats->aggs; src_aglat; src_aglat =3D src_agl= at->next) + if (src_aglat->offset >=3D load_offset) + break; + + load_type =3D item->value.load_agg.type; + if (!src_aglat + || src_aglat->offset > load_offset + || src_aglat->size !=3D tree_to_shwi (TYPE_SIZE (load_type)) + || src_plats->aggs_by_ref !=3D item->value.load_agg.by_ref) + return aglat->set_contains_variable (); + + src_lat =3D src_aglat; + src_offset =3D load_offset; + } + + if (src_lat->bottom + || (!ipcp_versionable_function_p (cs->caller) + && !src_lat->is_single_const ())) + return aglat->set_contains_variable (); + + ret =3D propagate_vals_across_arith_jfunc (cs, + item->value.pass_through.operati= on, + load_type, + item->value.pass_through.operand, + src_lat, aglat, + src_offset, + src_idx, + item->type); + + if (src_lat->contains_variable) + ret |=3D aglat->set_contains_variable (); + + return ret; +} + /* Propagate scalar values across jump function JFUNC that is associated w= ith edge CS and put the values into DEST_LAT. */ @@ -2212,15 +2476,14 @@ propagate_aggs_across_jump_function (struct cgraph_= edge *cs, { HOST_WIDE_INT val_size; - if (item->offset < 0) + if (item->offset < 0 || item->jftype =3D=3D IPA_JF_UNKNOWN) continue; - gcc_checking_assert (is_gimple_ip_invariant (item->value)); - val_size =3D tree_to_uhwi (TYPE_SIZE (TREE_TYPE (item->value))); + val_size =3D tree_to_shwi (TYPE_SIZE (item->type)); if (merge_agg_lats_step (dest_plats, item->offset, val_size, &aglat, pre_existing, &ret)) { - ret |=3D (*aglat)->add_value (item->value, cs, NULL, 0, 0); + ret |=3D propagate_aggregate_lattice (cs, item, *aglat); aglat =3D &(*aglat)->next; } else if (dest_plats->aggs_bottom) @@ -2326,7 +2589,7 @@ static tree ipa_get_indirect_edge_target_1 (struct cgraph_edge *ie, vec known_csts, vec known_con= texts, - vec known_aggs, + vec known_aggs, struct ipa_agg_replacement_value *agg_reps, bool *speculative) { @@ -2364,9 +2627,9 @@ ipa_get_indirect_edge_target_1 (struct cgraph_edge *i= e, } if (!t) { - struct ipa_agg_jump_function *agg; + struct ipa_agg_value_set *agg; if (known_aggs.length () > (unsigned int) param_index) - agg =3D known_aggs[param_index]; + agg =3D &known_aggs[param_index]; else agg =3D NULL; bool from_global_constant; @@ -2420,8 +2683,7 @@ ipa_get_indirect_edge_target_1 (struct cgraph_edge *i= e, if (!t && known_aggs.length () > (unsigned int) param_index && !ie->indirect_info->by_ref) { - struct ipa_agg_jump_function *agg; - agg =3D known_aggs[param_index]; + struct ipa_agg_value_set *agg =3D &known_aggs[param_index]; t =3D ipa_find_agg_cst_for_param (agg, known_csts[param_index], ie->indirect_info->offset, true); } @@ -2543,7 +2805,7 @@ tree ipa_get_indirect_edge_target (struct cgraph_edge *ie, vec known_csts, vec known_conte= xts, - vec known_aggs, + vec known_aggs, bool *speculative) { return ipa_get_indirect_edge_target_1 (ie, known_csts, known_contexts, @@ -2557,7 +2819,7 @@ static int devirtualization_time_bonus (struct cgraph_node *node, vec known_csts, vec known_contex= ts, - vec known_aggs) + vec known_aggs) { struct cgraph_edge *ie; int res =3D 0; @@ -2691,25 +2953,25 @@ good_cloning_opportunity_p (struct cgraph_node *nod= e, int time_benefit, /* Return all context independent values from aggregate lattices in PLATS = in a vector. Return NULL if there are none. */ -static vec * +static vec context_independent_aggregate_values (class ipcp_param_lattices *plats) { - vec *res =3D NULL; + vec res =3D vNULL; if (plats->aggs_bottom || plats->aggs_contain_variable || plats->aggs_count =3D=3D 0) - return NULL; + return vNULL; for (struct ipcp_agg_lattice *aglat =3D plats->aggs; aglat; aglat =3D aglat->next) if (aglat->is_single_const ()) { - struct ipa_agg_jf_item item; + struct ipa_agg_value item; item.offset =3D aglat->offset; item.value =3D aglat->values->value; - vec_safe_push (res, item); + res.safe_push (item); } return res; } @@ -2725,7 +2987,7 @@ gather_context_independent_values (class ipa_node_par= ams *info, vec *known_csts, vec *known_contexts, - vec *known_aggs, + vec *known_aggs, int *removable_params_cost) { int i, count =3D ipa_get_param_count (info); @@ -2775,40 +3037,20 @@ gather_context_independent_values (class ipa_node_p= arams *info, if (known_aggs) { - vec *agg_items; - struct ipa_agg_jump_function *ajf; + vec agg_items; + struct ipa_agg_value_set *agg; agg_items =3D context_independent_aggregate_values (plats); - ajf =3D &(*known_aggs)[i]; - ajf->items =3D agg_items; - ajf->by_ref =3D plats->aggs_by_ref; - ret |=3D agg_items !=3D NULL; + agg =3D &(*known_aggs)[i]; + agg->items =3D agg_items; + agg->by_ref =3D plats->aggs_by_ref; + ret |=3D !agg_items.is_empty (); } } return ret; } -/* The current interface in ipa-inline-analysis requires a pointer vector. - Create it. - - FIXME: That interface should be re-worked, this is slightly silly. Sti= ll, - I'd like to discuss how to change it first and this demonstrates the - issue. */ - -static vec -agg_jmp_p_vec_for_t_vec (vec known_aggs) -{ - vec ret; - struct ipa_agg_jump_function *ajf; - int i; - - ret.create (known_aggs.length ()); - FOR_EACH_VEC_ELT (known_aggs, i, ajf) - ret.quick_push (ajf); - return ret; -} - /* Perform time and size measurement of NODE with the context given in KNOWN_CSTS, KNOWN_CONTEXTS and KNOWN_AGGS, calculate the benefit and co= st given BASE_TIME of the node without specialization, REMOVABLE_PARAMS_CO= ST of @@ -2818,7 +3060,7 @@ agg_jmp_p_vec_for_t_vec (vec k= nown_aggs) static void perform_estimation_of_a_value (cgraph_node *node, vec known_csts, vec known_cont= exts, - vec known_aggs_ptrs, + vec known_aggs, int removable_params_cost, int est_move_cost, ipcp_value_base *val) { @@ -2827,7 +3069,7 @@ perform_estimation_of_a_value (cgraph_node *node, vec= known_csts, ipa_hints hints; estimate_ipcp_clone_size_and_time (node, known_csts, known_contexts, - known_aggs_ptrs, &size, &time, + known_aggs, &size, &time, &base_time, &hints); base_time -=3D time; if (base_time > 65535) @@ -2841,7 +3083,7 @@ perform_estimation_of_a_value (cgraph_node *node, vec= known_csts, else time_benefit =3D base_time.to_int () + devirtualization_time_bonus (node, known_csts, known_contexts, - known_aggs_ptrs) + known_aggs) + hint_time_bonus (hints) + removable_params_cost + est_move_cost; @@ -2867,8 +3109,7 @@ estimate_local_effects (struct cgraph_node *node) int i, count =3D ipa_get_param_count (info); vec known_csts; vec known_contexts; - vec known_aggs; - vec known_aggs_ptrs; + vec known_aggs; bool always_const; int removable_params_cost; @@ -2881,9 +3122,8 @@ estimate_local_effects (struct cgraph_node *node) always_const =3D gather_context_independent_values (info, &known_csts, &known_contexts, &known= _aggs, &removable_params_cost); - known_aggs_ptrs =3D agg_jmp_p_vec_for_t_vec (known_aggs); int devirt_bonus =3D devirtualization_time_bonus (node, known_csts, - known_contexts, known_aggs_ptrs); + known_contexts, known_aggs); if (always_const || devirt_bonus || (removable_params_cost && node->local.can_change_signature)) { @@ -2896,7 +3136,7 @@ estimate_local_effects (struct cgraph_node *node) node->call_for_symbol_thunks_and_aliases (gather_caller_stats, &stat= s, false); estimate_ipcp_clone_size_and_time (node, known_csts, known_contexts, - known_aggs_ptrs, &size, &time, + known_aggs, &size, &time, &base_time, &hints); time -=3D devirt_bonus; time -=3D hint_time_bonus (hints); @@ -2959,7 +3199,7 @@ estimate_local_effects (struct cgraph_node *node) int emc =3D estimate_move_cost (TREE_TYPE (val->value), true); perform_estimation_of_a_value (node, known_csts, known_contexts, - known_aggs_ptrs, + known_aggs, removable_params_cost, emc, val); if (dump_file && (dump_flags & TDF_DETAILS)) @@ -2994,7 +3234,7 @@ estimate_local_effects (struct cgraph_node *node) { known_contexts[i] =3D val->value; perform_estimation_of_a_value (node, known_csts, known_contexts, - known_aggs_ptrs, + known_aggs, removable_params_cost, 0, val); if (dump_file && (dump_flags & TDF_DETAILS)) @@ -3013,13 +3253,13 @@ estimate_local_effects (struct cgraph_node *node) for (i =3D 0; i < count; i++) { class ipcp_param_lattices *plats =3D ipa_get_parm_lattices (info, i); - struct ipa_agg_jump_function *ajf; + struct ipa_agg_value_set *agg; struct ipcp_agg_lattice *aglat; if (plats->aggs_bottom || !plats->aggs) continue; - ajf =3D &known_aggs[i]; + agg =3D &known_aggs[i]; for (aglat =3D plats->aggs; aglat; aglat =3D aglat->next) { ipcp_value *val; @@ -3031,14 +3271,14 @@ estimate_local_effects (struct cgraph_node *node) for (val =3D aglat->values; val; val =3D val->next) { - struct ipa_agg_jf_item item; + struct ipa_agg_value item; item.offset =3D aglat->offset; item.value =3D val->value; - vec_safe_push (ajf->items, item); + agg->items.safe_push (item); perform_estimation_of_a_value (node, known_csts, known_contex= ts, - known_aggs_ptrs, + known_aggs, removable_params_cost, 0, val); if (dump_file && (dump_flags & TDF_DETAILS)) @@ -3054,18 +3294,14 @@ estimate_local_effects (struct cgraph_node *node) val->local_time_benefit, val->local_size_cost); } - ajf->items->pop (); + agg->items.pop (); } } } - for (i =3D 0; i < count; i++) - vec_free (known_aggs[i].items); - known_csts.release (); known_contexts.release (); - known_aggs.release (); - known_aggs_ptrs.release (); + ipa_release_agg_values (known_aggs); } @@ -3433,26 +3669,6 @@ edge_clone_summary_t::duplicate (cgraph_edge *src_ed= ge, cgraph_edge *dst_edge, src_data->next_clone =3D dst_edge; } -/* See if NODE is a clone with a known aggregate value at a given OFFSET o= f a - parameter with the given INDEX. */ - -static tree -get_clone_agg_value (struct cgraph_node *node, HOST_WIDE_INT offset, - int index) -{ - struct ipa_agg_replacement_value *aggval; - - aggval =3D ipa_get_agg_replacements_for_node (node); - while (aggval) - { - if (aggval->offset =3D=3D offset - && aggval->index =3D=3D index) - return aggval->value; - aggval =3D aggval->next; - } - return NULL_TREE; -} - /* Return true is NODE is DEST or its clone for all contexts. */ static bool @@ -4074,10 +4290,10 @@ find_more_contexts_for_caller_subset (cgraph_node *= node, /* Go through PLATS and create a vector of values consisting of values and offsets (minus OFFSET) of lattices that contain only a single value. */ -static vec +static vec copy_plats_to_inter (class ipcp_param_lattices *plats, HOST_WIDE_INT offse= t) { - vec res =3D vNULL; + vec res =3D vNULL; if (!plats->aggs || plats->aggs_contain_variable || plats->aggs_bottom) return vNULL; @@ -4085,7 +4301,7 @@ copy_plats_to_inter (class ipcp_param_lattices *plats= , HOST_WIDE_INT offset) for (struct ipcp_agg_lattice *aglat =3D plats->aggs; aglat; aglat =3D ag= lat->next) if (aglat->is_single_const ()) { - struct ipa_agg_jf_item ti; + struct ipa_agg_value ti; ti.offset =3D aglat->offset - offset; ti.value =3D aglat->values->value; res.safe_push (ti); @@ -4098,11 +4314,11 @@ copy_plats_to_inter (class ipcp_param_lattices *pla= ts, HOST_WIDE_INT offset) static void intersect_with_plats (class ipcp_param_lattices *plats, - vec *inter, + vec *inter, HOST_WIDE_INT offset) { struct ipcp_agg_lattice *aglat; - struct ipa_agg_jf_item *item; + struct ipa_agg_value *item; int k; if (!plats->aggs || plats->aggs_contain_variable || plats->aggs_bottom) @@ -4140,18 +4356,18 @@ intersect_with_plats (class ipcp_param_lattices *pl= ats, /* Copy aggregate replacement values of NODE (which is an IPA-CP clone) to= the vector result while subtracting OFFSET from the individual value offset= s. */ -static vec +static vec agg_replacements_to_vector (struct cgraph_node *node, int index, HOST_WIDE_INT offset) { struct ipa_agg_replacement_value *av; - vec res =3D vNULL; + vec res =3D vNULL; for (av =3D ipa_get_agg_replacements_for_node (node); av; av =3D av->nex= t) if (av->index =3D=3D index && (av->offset - offset) >=3D 0) { - struct ipa_agg_jf_item item; + struct ipa_agg_value item; gcc_checking_assert (av->value); item.offset =3D av->offset - offset; item.value =3D av->value; @@ -4167,11 +4383,11 @@ agg_replacements_to_vector (struct cgraph_node *nod= e, int index, static void intersect_with_agg_replacements (struct cgraph_node *node, int index, - vec *inter, + vec *inter, HOST_WIDE_INT offset) { struct ipa_agg_replacement_value *srcvals; - struct ipa_agg_jf_item *item; + struct ipa_agg_value *item; int i; srcvals =3D ipa_get_agg_replacements_for_node (node); @@ -4208,9 +4424,9 @@ intersect_with_agg_replacements (struct cgraph_node *= node, int index, copy all incoming values to it. If we determine we ended up with no va= lues whatsoever, return a released vector. */ -static vec +static vec intersect_aggregates_with_edge (struct cgraph_edge *cs, int index, - vec inter) + vec inter) { struct ipa_jump_func *jfunc; jfunc =3D ipa_get_ith_jump_func (IPA_EDGE_REF (cs), index); @@ -4291,12 +4507,26 @@ intersect_aggregates_with_edge (struct cgraph_edge = *cs, int index, } else if (jfunc->agg.items) { - struct ipa_agg_jf_item *item; + class ipa_node_params *caller_info =3D IPA_NODE_REF (cs->caller); + struct ipa_agg_value *item; int k; if (!inter.exists ()) for (unsigned i =3D 0; i < jfunc->agg.items->length (); i++) - inter.safe_push ((*jfunc->agg.items)[i]); + { + struct ipa_agg_jf_item *agg_item =3D &(*jfunc->agg.items)[i]; + tree value =3D ipa_agg_value_from_node (caller_info, cs->caller, + agg_item); + if (value) + { + struct ipa_agg_value agg_value; + + agg_value.offset =3D agg_item->offset; + agg_value.value =3D value; + + inter.safe_push (agg_value); + } + } else FOR_EACH_VEC_ELT (inter, k, item) { @@ -4314,9 +4544,10 @@ intersect_aggregates_with_edge (struct cgraph_edge *= cs, int index, break; if (ti->offset =3D=3D item->offset) { - gcc_checking_assert (ti->value); - if (values_equal_for_ipcp_p (item->value, - ti->value)) + tree value =3D ipa_agg_value_from_node (caller_info, + cs->caller, ti); + if (value + && values_equal_for_ipcp_p (item->value, value)) found =3D true; break; } @@ -4329,7 +4560,7 @@ intersect_aggregates_with_edge (struct cgraph_edge *c= s, int index, else { inter.release (); - return vec(); + return vNULL; } return inter; } @@ -4357,8 +4588,8 @@ find_aggregate_values_for_callers_subset (struct cgra= ph_node *node, for (i =3D 0; i < count; i++) { struct cgraph_edge *cs; - vec inter =3D vNULL; - struct ipa_agg_jf_item *item; + vec inter =3D vNULL; + struct ipa_agg_value *item; class ipcp_param_lattices *plats =3D ipa_get_parm_lattices (dest_inf= o, i); int j; @@ -4465,7 +4696,7 @@ cgraph_edge_brings_all_agg_vals_for_node (struct cgra= ph_edge *cs, for (i =3D 0; i < count; i++) { - static vec values =3D vec(); + static vec values =3D vNULL; class ipcp_param_lattices *plats; bool interesting =3D false; for (struct ipa_agg_replacement_value *av =3D aggval; av; av =3D av-= >next) @@ -4488,7 +4719,7 @@ cgraph_edge_brings_all_agg_vals_for_node (struct cgra= ph_edge *cs, for (struct ipa_agg_replacement_value *av =3D aggval; av; av =3D av-= >next) if (aggval->index =3D=3D i) { - struct ipa_agg_jf_item *item; + struct ipa_agg_value *item; int j; bool found =3D false; FOR_EACH_VEC_ELT (values, j, item) @@ -4726,7 +4957,6 @@ decide_whether_version_node (struct cgraph_node *node) int i, count =3D ipa_get_param_count (info); vec known_csts; vec known_contexts; - vec known_aggs =3D vNULL; bool ret =3D false; if (count =3D=3D 0) @@ -4737,8 +4967,7 @@ decide_whether_version_node (struct cgraph_node *node) node->dump_name ()); gather_context_independent_values (info, &known_csts, &known_contexts, - info->do_clone_for_all_contexts ? &known_= aggs - : NULL, NULL); + NULL, NULL); for (i =3D 0; i < count;i++) { @@ -4807,9 +5036,6 @@ decide_whether_version_node (struct cgraph_node *node) info =3D IPA_NODE_REF (node); info->do_clone_for_all_contexts =3D false; IPA_NODE_REF (clone)->is_all_contexts_clone =3D true; - for (i =3D 0; i < count; i++) - vec_free (known_aggs[i].items); - known_aggs.release (); ret =3D true; } else diff --git a/gcc/ipa-fnsummary.c b/gcc/ipa-fnsummary.c index 6de060aa3fc..f8725d8dbfe 100644 --- a/gcc/ipa-fnsummary.c +++ b/gcc/ipa-fnsummary.c @@ -306,9 +306,9 @@ set_hint_predicate (predicate **p, predicate new_predic= ate) the fact that parameter is indeed a constant. KNOWN_VALS is partial mapping of parameters of NODE to constant values. - KNOWN_AGGS is a vector of aggreggate jump functions for each parameter. - Return clause of possible truths. When INLINE_P is true, assume that we= are - inlining. + KNOWN_AGGS is a vector of aggreggate known offset/value set for each + parameter. Return clause of possible truths. When INLINE_P is true, a= ssume + that we are inlining. ERROR_MARK means compile time invariant. */ @@ -316,8 +316,7 @@ static void evaluate_conditions_for_known_args (struct cgraph_node *node, bool inline_p, vec known_vals, - vec - known_aggs, + vec known_aggs, clause_t *ret_clause, clause_t *ret_nonspec_clause) { @@ -347,7 +346,7 @@ evaluate_conditions_for_known_args (struct cgraph_node = *node, if (c->agg_contents) { - struct ipa_agg_jump_function *agg; + struct ipa_agg_value_set *agg; if (c->code =3D=3D predicate::changed && !c->by_ref @@ -356,7 +355,7 @@ evaluate_conditions_for_known_args (struct cgraph_node = *node, if (known_aggs.exists ()) { - agg =3D known_aggs[c->operand_num]; + agg =3D &known_aggs[c->operand_num]; val =3D ipa_find_agg_cst_for_param (agg, known_vals[c->operan= d_num], c->offset, c->by_ref); } @@ -420,12 +419,12 @@ evaluate_properties_for_edge (struct cgraph_edge *e, = bool inline_p, vec *known_vals_ptr, vec *known_contexts_ptr, - vec *known_aggs_ptr) + vec *known_aggs_ptr) { struct cgraph_node *callee =3D e->callee->ultimate_alias_target (); class ipa_fn_summary *info =3D ipa_fn_summaries->get (callee); vec known_vals =3D vNULL; - vec known_aggs =3D vNULL; + vec known_aggs =3D vNULL; if (clause_ptr) *clause_ptr =3D inline_p ? 0 : 1 << predicate::not_inlined_condition; @@ -438,15 +437,17 @@ evaluate_properties_for_edge (struct cgraph_edge *e, = bool inline_p, && !e->call_stmt_cannot_inline_p && ((clause_ptr && info->conds) || known_vals_ptr || known_contexts_= ptr)) { + struct cgraph_node *caller; class ipa_node_params *caller_parms_info, *callee_pi; class ipa_edge_args *args =3D IPA_EDGE_REF (e); class ipa_call_summary *es =3D ipa_call_summaries->get (e); int i, count =3D ipa_get_cs_argument_count (args); if (e->caller->global.inlined_to) - caller_parms_info =3D IPA_NODE_REF (e->caller->global.inlined_to); + caller =3D e->caller->global.inlined_to; else - caller_parms_info =3D IPA_NODE_REF (e->caller); + caller =3D e->caller; + caller_parms_info =3D IPA_NODE_REF (caller); callee_pi =3D IPA_NODE_REF (e->callee); if (count && (info->conds || known_vals_ptr)) @@ -481,10 +482,9 @@ evaluate_properties_for_edge (struct cgraph_edge *e, b= ool inline_p, if (known_contexts_ptr) (*known_contexts_ptr)[i] =3D ipa_context_from_jfunc (caller_parms_info, e, i, jf); - /* TODO: When IPA-CP starts propagating and merging aggregate jump - functions, use its knowledge of the caller too, just like the - scalar case above. */ - known_aggs[i] =3D &jf->agg; + + known_aggs[i] =3D ipa_agg_value_set_from_jfunc (caller_parms_info, + caller, &jf->agg); } } else if (e->call_stmt && !e->call_stmt_cannot_inline_p @@ -516,7 +516,7 @@ evaluate_properties_for_edge (struct cgraph_edge *e, bo= ol inline_p, if (known_aggs_ptr) *known_aggs_ptr =3D known_aggs; else - known_aggs.release (); + ipa_release_agg_values (known_aggs); } @@ -2662,7 +2662,7 @@ estimate_edge_devirt_benefit (struct cgraph_edge *ie, int *size, int *time, vec known_vals, vec known_conte= xts, - vec known_aggs) + vec known_aggs) { tree target; struct cgraph_node *callee; @@ -2711,7 +2711,7 @@ estimate_edge_size_and_time (struct cgraph_edge *e, i= nt *size, int *min_size, int prob, vec known_vals, vec known_contex= ts, - vec known_aggs, + vec known_aggs, ipa_hints *hints) { class ipa_call_summary *es =3D ipa_call_summaries->get (e); @@ -2746,7 +2746,7 @@ estimate_calls_size_and_time (struct cgraph_node *nod= e, int *size, clause_t possible_truths, vec known_vals, vec known_conte= xts, - vec known_aggs) + vec known_aggs) { struct cgraph_edge *e; for (e =3D node->callees; e; e =3D e->next_callee) @@ -2809,7 +2809,7 @@ estimate_node_size_and_time (struct cgraph_node *node, clause_t nonspec_possible_truths, vec known_vals, vec known_contex= ts, - vec known_aggs, + vec known_aggs, int *ret_size, int *ret_min_size, sreal *ret_time, sreal *ret_nonspecialized_time, @@ -2945,7 +2945,7 @@ estimate_ipcp_clone_size_and_time (struct cgraph_node= *node, vec known_vals, vec known_contexts, - vec known_aggs, + vec known_aggs, int *ret_size, sreal *ret_time, sreal *ret_nonspec_time, ipa_hints *hints) diff --git a/gcc/ipa-fnsummary.h b/gcc/ipa-fnsummary.h index 173d3f2a652..7e561dab400 100644 --- a/gcc/ipa-fnsummary.h +++ b/gcc/ipa-fnsummary.h @@ -260,7 +260,7 @@ void inline_analyze_function (struct cgraph_node *node); void estimate_ipcp_clone_size_and_time (struct cgraph_node *, vec, vec, - vec, + vec, int *, sreal *, sreal *, ipa_hints *); void ipa_merge_fn_summary_after_inlining (struct cgraph_edge *edge); @@ -274,13 +274,13 @@ void evaluate_properties_for_edge (struct cgraph_edge= *e, bool inline_p, vec *known_vals_ptr, vec *known_contexts_ptr, - vec *); + vec *); void estimate_node_size_and_time (struct cgraph_node *node, clause_t possible_truths, clause_t nonspec_possible_truths, vec known_vals, vec, - vec known_aggs, + vec known_aggs, int *ret_size, int *ret_min_size, sreal *ret_time, sreal *ret_nonspecialized_time, diff --git a/gcc/ipa-inline-analysis.c b/gcc/ipa-inline-analysis.c index a66af277d03..bf4e6ea3392 100644 --- a/gcc/ipa-inline-analysis.c +++ b/gcc/ipa-inline-analysis.c @@ -127,7 +127,7 @@ do_estimate_edge_time (struct cgraph_edge *edge) clause_t clause, nonspec_clause; vec known_vals; vec known_contexts; - vec known_aggs; + vec known_aggs; class ipa_call_summary *es =3D ipa_call_summaries->get (edge); int min_size; @@ -154,7 +154,7 @@ do_estimate_edge_time (struct cgraph_edge *edge) known_vals.release (); known_contexts.release (); - known_aggs.release (); + ipa_release_agg_values (known_aggs); gcc_checking_assert (size >=3D 0); gcc_checking_assert (time >=3D 0); @@ -186,7 +186,7 @@ do_estimate_edge_size (struct cgraph_edge *edge) clause_t clause, nonspec_clause; vec known_vals; vec known_contexts; - vec known_aggs; + vec known_aggs; /* When we do caching, use do_estimate_edge_time to populate the entry. = */ @@ -211,7 +211,7 @@ do_estimate_edge_size (struct cgraph_edge *edge) NULL, NULL, vNULL); known_vals.release (); known_contexts.release (); - known_aggs.release (); + ipa_release_agg_values (known_aggs); return size; } @@ -227,7 +227,7 @@ do_estimate_edge_hints (struct cgraph_edge *edge) clause_t clause, nonspec_clause; vec known_vals; vec known_contexts; - vec known_aggs; + vec known_aggs; /* When we do caching, use do_estimate_edge_time to populate the entry. = */ @@ -252,7 +252,7 @@ do_estimate_edge_hints (struct cgraph_edge *edge) NULL, NULL, &hints, vNULL); known_vals.release (); known_contexts.release (); - known_aggs.release (); + ipa_release_agg_values (known_aggs); hints |=3D simple_edge_hints (edge); return hints; } diff --git a/gcc/ipa-prop.c b/gcc/ipa-prop.c index a23aa2590a0..be281293eb7 100644 --- a/gcc/ipa-prop.c +++ b/gcc/ipa-prop.c @@ -359,18 +359,45 @@ ipa_print_node_jump_functions_for_edge (FILE *f, stru= ct cgraph_edge *cs) fprintf (f, " Aggregate passed by %s:\n", jump_func->agg.by_ref ? "reference" : "value"); - FOR_EACH_VEC_SAFE_ELT (jump_func->agg.items, j, item) + FOR_EACH_VEC_ELT (*jump_func->agg.items, j, item) { fprintf (f, " offset: " HOST_WIDE_INT_PRINT_DEC ", = ", item->offset); - if (TYPE_P (item->value)) - fprintf (f, "clobber of " HOST_WIDE_INT_PRINT_DEC " bits", - tree_to_uhwi (TYPE_SIZE (item->value))); - else + fprintf (f, "type: "); + print_generic_expr (f, item->type); + fprintf (f, ", "); + if (item->jftype =3D=3D IPA_JF_PASS_THROUGH) + fprintf (f, "PASS THROUGH: %d,", + item->value.pass_through.formal_id); + else if (item->jftype =3D=3D IPA_JF_LOAD_AGG) + { + fprintf (f, "LOAD AGG: %d", + item->value.pass_through.formal_id); + fprintf (f, " [offset: " HOST_WIDE_INT_PRINT_DEC ", by %s= ],", + item->value.load_agg.offset, + item->value.load_agg.by_ref ? "reference" + : "value"); + } + + if (item->jftype =3D=3D IPA_JF_PASS_THROUGH + || item->jftype =3D=3D IPA_JF_LOAD_AGG) + { + fprintf (f, " op %s", + get_tree_code_name (item->value.pass_through.operation= )); + if (item->value.pass_through.operation !=3D NOP_EXPR) + { + fprintf (f, " "); + print_generic_expr (f, item->value.pass_through.opera= nd); + } + } + else if (item->jftype =3D=3D IPA_JF_CONST) { - fprintf (f, "cst: "); - print_generic_expr (f, item->value); + fprintf (f, "CONST: "); + print_generic_expr (f, item->value.constant); } + else if (item->jftype =3D=3D IPA_JF_UNKNOWN) + fprintf (f, "UNKNOWN: " HOST_WIDE_INT_PRINT_DEC " bits", + tree_to_uhwi (TYPE_SIZE (item->type))); fprintf (f, "\n"); } } @@ -1135,6 +1162,67 @@ ipa_load_from_parm_agg (struct ipa_func_body_info *f= bi, return false; } +/* If STMT is an assignment that loads a value from a parameter declaratio= n, + or from an aggregate passed as the parameter either by value or referen= ce, + return the index of the parameter in ipa_node_params. Otherwise return= -1. + + FBI holds gathered information about the function. INFO describes + parameters of the function, STMT is the assignment statement. If it is= a + memory load from an aggregate, *OFFSET_P is filled with offset within t= he + aggregate, and *BY_REF_P specifies whether the aggregate is passed by + reference. */ + +static int +load_from_unmodified_param_or_agg (struct ipa_func_body_info *fbi, + class ipa_node_params *info, + gimple *stmt, + HOST_WIDE_INT *offset_p, + bool *by_ref_p) +{ + int index =3D load_from_unmodified_param (fbi, info->descriptors, stmt); + poly_int64 size; + + /* Load value from a parameter declaration. */ + if (index >=3D 0) + { + *offset_p =3D -1; + return index; + } + + if (!gimple_assign_load_p (stmt)) + return -1; + + tree rhs =3D gimple_assign_rhs1 (stmt); + + /* Skip memory reference containing VIEW_CONVERT_EXPR. */ + for (tree t =3D rhs; handled_component_p (t); t =3D TREE_OPERAND (t, 0)) + if (TREE_CODE (t) =3D=3D VIEW_CONVERT_EXPR) + return -1; + + /* Skip memory reference containing bit-field. */ + if (TREE_CODE (rhs) =3D=3D BIT_FIELD_REF + || contains_bitfld_component_ref_p (rhs)) + return -1; + + if (!ipa_load_from_parm_agg (fbi, info->descriptors, stmt, rhs, &index, + offset_p, &size, by_ref_p)) + return -1; + + gcc_assert (!maybe_ne (tree_to_poly_int64 (TYPE_SIZE (TREE_TYPE (rhs))), + size)); + if (!*by_ref_p) + { + tree param_type =3D ipa_get_type (info, index); + + if (!param_type || !AGGREGATE_TYPE_P (param_type)) + return -1; + } + else if (TREE_THIS_VOLATILE (rhs)) + return -1; + + return index; +} + /* Given that an actual argument is an SSA_NAME (given in NAME) and is a r= esult of an assignment statement STMT, try to determine whether we are actual= ly handling any of the following cases and construct an appropriate jump @@ -1438,11 +1526,11 @@ type_like_member_ptr_p (tree type, tree *method_ptr= , tree *delta) } /* If RHS is an SSA_NAME and it is defined by a simple copy assign stateme= nt, - return the rhs of its defining statement. Otherwise return RHS as it - is. */ + return the rhs of its defining statement, and this statement is stored = in + *RHS_STMT. Otherwise return RHS as it is. */ static inline tree -get_ssa_def_if_simple_copy (tree rhs) +get_ssa_def_if_simple_copy (tree rhs, gimple **rhs_stmt) { while (TREE_CODE (rhs) =3D=3D SSA_NAME && !SSA_NAME_IS_DEFAULT_DEF (rhs)) { @@ -1452,25 +1540,31 @@ get_ssa_def_if_simple_copy (tree rhs) rhs =3D gimple_assign_rhs1 (def_stmt); else break; + *rhs_stmt =3D def_stmt; } return rhs; } -/* Simple linked list, describing known contents of an aggregate before - call. */ +/* Simple linked list, describing contents of an aggregate before call. */ struct ipa_known_agg_contents_list { /* Offset and size of the described part of the aggregate. */ HOST_WIDE_INT offset, size; - /* Known constant value or NULL if the contents is known to be unknown. = */ - tree constant; + + /* Type of the described part of the aggregate. */ + tree type; + + /* Known constant value or jump function data describing contents. */ + struct ipa_load_agg_data value; + /* Pointer to the next structure in the list. */ struct ipa_known_agg_contents_list *next; }; -/* Add a known content item into a linked list of ipa_known_agg_contents_l= ist - structure, in which all elements are sorted ascendingly by offset. */ +/* Add an aggregate content item into a linked list of + ipa_known_agg_contents_list structure, in which all elements + are sorted ascendingly by offset. */ static inline void add_to_agg_contents_list (struct ipa_known_agg_contents_list **plist, @@ -1490,7 +1584,7 @@ add_to_agg_contents_list (struct ipa_known_agg_conten= ts_list **plist, *plist =3D item; } -/* Check whether a given known content is clobbered by certain element in +/* Check whether a given aggregate content is clobbered by certain element= in a linked list of ipa_known_agg_contents_list. */ static inline bool @@ -1510,27 +1604,189 @@ clobber_by_agg_contents_list_p (struct ipa_known_a= gg_contents_list *list, } /* Build aggregate jump function from LIST, assuming there are exactly - CONST_COUNT constant entries there and that offset of the passed argume= nt + VALUE_COUNT entries there and that offset of the passed argument is ARG_OFFSET and store it into JFUNC. */ static void build_agg_jump_func_from_list (struct ipa_known_agg_contents_list *list, - int const_count, HOST_WIDE_INT arg_offset, + int value_count, HOST_WIDE_INT arg_offset, struct ipa_jump_func *jfunc) { - vec_alloc (jfunc->agg.items, const_count); - while (list) + vec_alloc (jfunc->agg.items, value_count); + for (; list; list =3D list->next) + { + struct ipa_agg_jf_item item; + tree operand =3D list->value.pass_through.operand; + + if (list->value.pass_through.formal_id >=3D 0) + { + /* Content value is derived from some formal paramerter. */ + if (list->value.offset >=3D 0) + item.jftype =3D IPA_JF_LOAD_AGG; + else + item.jftype =3D IPA_JF_PASS_THROUGH; + + item.value.load_agg =3D list->value; + if (operand) + item.value.pass_through.operand + =3D unshare_expr_without_location (operand); + } + else if (operand) + { + /* Content value is known constant. */ + item.jftype =3D IPA_JF_CONST; + item.value.constant =3D unshare_expr_without_location (operand); + } + else + continue; + + item.type =3D list->type; + gcc_assert (tree_to_shwi (TYPE_SIZE (list->type)) =3D=3D list->size); + + item.offset =3D list->offset - arg_offset; + gcc_assert ((item.offset % BITS_PER_UNIT) =3D=3D 0); + + jfunc->agg.items->quick_push (item); + } +} + +/* Given an assignment statement STMT, try to collect information into + AGG_VALUE that will be used to construct jump function for RHS of the + assignment, from which content value of an aggregate part comes. + + Besides constant and simple pass-through jump functions, also try to + identify whether it matches the following pattern that can be described= by + a load-value-from-aggregate jump function, which is a derivative of sim= ple + pass-through jump function. + + foo (int *p) + { + ... + + *(q_5 + 4) =3D *(p_3(D) + 28) op 1; + bar (q_5); + } + + Since load-value-from-aggregate jump function data structure is informa= tive + enough to describe constant and simple pass-through jump function, here= we + do not need a jump function type, merely use FORMAL_ID and OPERAND in + IPA_LOAD_AGG_DATA to disginguish different jump functions. */ + +static void +compute_assign_agg_jump_func (struct ipa_func_body_info *fbi, + struct ipa_load_agg_data *agg_value, + gimple *stmt) +{ + tree lhs =3D gimple_assign_lhs (stmt); + tree rhs1 =3D gimple_assign_rhs1 (stmt); + enum tree_code code; + int index =3D -1; + + /* Initialize jump function data for the aggregate part. */ + memset (agg_value, 0, sizeof (*agg_value)); + agg_value->pass_through.operation =3D NOP_EXPR; + agg_value->pass_through.formal_id =3D -1; + agg_value->offset =3D -1; + + if (AGGREGATE_TYPE_P (TREE_TYPE (lhs)) /* TODO: Support aggregate type.= */ + || TREE_THIS_VOLATILE (lhs) + || TREE_CODE (lhs) =3D=3D BIT_FIELD_REF + || contains_bitfld_component_ref_p (lhs)) + return; + + /* Skip SSA copies. */ + while (gimple_assign_rhs_class (stmt) =3D=3D GIMPLE_SINGLE_RHS) + { + if (TREE_CODE (rhs1) !=3D SSA_NAME || SSA_NAME_IS_DEFAULT_DEF (rhs1)) + break; + + if (!is_gimple_assign (stmt =3D SSA_NAME_DEF_STMT (rhs1))) + return; + + rhs1 =3D gimple_assign_rhs1 (stmt); + } + + code =3D gimple_assign_rhs_code (stmt); + switch (gimple_assign_rhs_class (stmt)) { - if (list->constant) + case GIMPLE_SINGLE_RHS: + if (is_gimple_ip_invariant (rhs1)) { - struct ipa_agg_jf_item item; - item.offset =3D list->offset - arg_offset; - gcc_assert ((item.offset % BITS_PER_UNIT) =3D=3D 0); - item.value =3D unshare_expr_without_location (list->constant); - jfunc->agg.items->quick_push (item); + agg_value->pass_through.operand =3D rhs1; + return; } - list =3D list->next; + code =3D NOP_EXPR; + break; + + case GIMPLE_UNARY_RHS: + /* NOTE: A GIMPLE_UNARY_RHS operation might not be tcc_unary + (truth_not_expr is example), GIMPLE_BINARY_RHS does not imply + tcc_binary, this subtleness is somewhat misleading. + + Since tcc_unary is widely used in IPA-CP code to check an operation + with one operand, here we only allow tc_unary operation to avoid + possible problem. Then we can use (opclass =3D=3D tc_unary) or no= t to + distinguish unary and binary. */ + if (TREE_CODE_CLASS (code) !=3D tcc_unary || CONVERT_EXPR_CODE_P (co= de)) + return; + + rhs1 =3D get_ssa_def_if_simple_copy (rhs1, &stmt); + break; + + case GIMPLE_BINARY_RHS: + { + gimple *rhs1_stmt =3D stmt; + gimple *rhs2_stmt =3D stmt; + tree rhs2 =3D gimple_assign_rhs2 (stmt); + + rhs1 =3D get_ssa_def_if_simple_copy (rhs1, &rhs1_stmt); + rhs2 =3D get_ssa_def_if_simple_copy (rhs2, &rhs2_stmt); + + if (is_gimple_ip_invariant (rhs2)) + { + agg_value->pass_through.operand =3D rhs2; + stmt =3D rhs1_stmt; + } + else if (is_gimple_ip_invariant (rhs1)) + { + if (TREE_CODE_CLASS (code) =3D=3D tcc_comparison) + code =3D swap_tree_comparison (code); + else if (!commutative_tree_code (code)) + return; + + agg_value->pass_through.operand =3D rhs1; + stmt =3D rhs2_stmt; + rhs1 =3D rhs2; + } + else + return; + + if (TREE_CODE_CLASS (code) !=3D tcc_comparison + && !useless_type_conversion_p (TREE_TYPE (lhs), TREE_TYPE (rhs1= ))) + return; + } + break; + + default: + return; + } + + if (TREE_CODE (rhs1) !=3D SSA_NAME) + index =3D load_from_unmodified_param_or_agg (fbi, fbi->info, stmt, + &agg_value->offset, + &agg_value->by_ref); + else if (SSA_NAME_IS_DEFAULT_DEF (rhs1)) + index =3D ipa_get_param_decl_index (fbi->info, SSA_NAME_VAR (rhs1)); + + if (index >=3D 0) + { + if (agg_value->offset >=3D 0) + agg_value->type =3D TREE_TYPE (rhs1); + agg_value->pass_through.formal_id =3D index; + agg_value->pass_through.operation =3D code; } + else + agg_value->pass_through.operand =3D NULL_TREE; } /* If STMT is a memory store to the object whose address is BASE, extract @@ -1540,26 +1796,19 @@ build_agg_jump_func_from_list (struct ipa_known_agg= _contents_list *list, is expected to be in form of MEM_REF expression. */ static bool -extract_mem_content (gimple *stmt, tree base, bool check_ref, +extract_mem_content (struct ipa_func_body_info *fbi, + gimple *stmt, tree base, bool check_ref, struct ipa_known_agg_contents_list *content) { HOST_WIDE_INT lhs_offset, lhs_size; - tree lhs, rhs, lhs_base; bool reverse; - if (!gimple_assign_single_p (stmt)) - return false; - - lhs =3D gimple_assign_lhs (stmt); - rhs =3D gimple_assign_rhs1 (stmt); - - if (!is_gimple_reg_type (TREE_TYPE (rhs)) - || TREE_CODE (lhs) =3D=3D BIT_FIELD_REF - || contains_bitfld_component_ref_p (lhs)) + if (!is_gimple_assign (stmt)) return false; - lhs_base =3D get_ref_base_and_extent_hwi (lhs, &lhs_offset, - &lhs_size, &reverse); + tree lhs =3D gimple_assign_lhs (stmt); + tree lhs_base =3D get_ref_base_and_extent_hwi (lhs, &lhs_offset, &lhs_si= ze, + &reverse); if (!lhs_base) return false; @@ -1573,32 +1822,31 @@ extract_mem_content (gimple *stmt, tree base, bool = check_ref, else if (lhs_base !=3D base) return false; - rhs =3D get_ssa_def_if_simple_copy (rhs); - - content->size =3D lhs_size; content->offset =3D lhs_offset; - content->constant =3D is_gimple_ip_invariant (rhs) ? rhs : NULL_TREE; + content->size =3D lhs_size; + content->type =3D TREE_TYPE (lhs); content->next =3D NULL; + compute_assign_agg_jump_func (fbi, &content->value, stmt); return true; } /* Traverse statements from CALL backwards, scanning whether an aggregate = given - in ARG is filled in with constant values. ARG can either be an aggrega= te - expression or a pointer to an aggregate. ARG_TYPE is the type of the - aggregate. JFUNC is the jump function into which the constants are - subsequently stored. AA_WALK_BUDGET_P points to limit on number of - statements we allow get_continuation_for_phi to examine. */ + in ARG is filled in constant or value that is derived from caller's for= mal + parameter in the way described by some kind of jump function. FBI is t= he + context of the caller function for interprocedural analysis. ARG can e= ither + be an aggregate expression or a pointer to an aggregate. ARG_TYPE is t= he + type of the aggregate. JFUNC is the jump function for the aggregate. = */ static void -determine_known_aggregate_parts (gcall *call, tree arg, +determine_known_aggregate_parts (struct ipa_func_body_info *fbi, + gcall *call, tree arg, tree arg_type, - struct ipa_jump_func *jfunc, - unsigned *aa_walk_budget_p) + struct ipa_jump_func *jfunc) { struct ipa_known_agg_contents_list *list =3D NULL, *all_list =3D NULL; bitmap visited =3D NULL; - int item_count =3D 0, const_count =3D 0; + int item_count =3D 0, value_count =3D 0; int ipa_max_agg_items =3D PARAM_VALUE (PARAM_IPA_MAX_AGG_ITEMS); HOST_WIDE_INT arg_offset, arg_size; tree arg_base; @@ -1677,7 +1925,7 @@ determine_known_aggregate_parts (gcall *call, tree ar= g, if (gimple_code (stmt) =3D=3D GIMPLE_PHI) { dom_vuse =3D get_continuation_for_phi (stmt, &r, true, - *aa_walk_budget_p, + fbi->aa_walk_budget, &visited, false, NULL, NULL); continue; } @@ -1687,12 +1935,13 @@ determine_known_aggregate_parts (gcall *call, tree = arg, struct ipa_known_agg_contents_list *content =3D XALLOCA (struct ipa_known_agg_contents_list); - if (!extract_mem_content (stmt, arg_base, check_ref, content)) + if (!extract_mem_content (fbi, stmt, arg_base, check_ref, content= )) break; /* Now we get a dominating virtual operand, and need to check whether its value is clobbered any other dominating one. */ - if (content->constant + if ((content->value.pass_through.formal_id >=3D 0 + || content->value.pass_through.operand) && !clobber_by_agg_contents_list_p (all_list, content)) { struct ipa_known_agg_contents_list *copy @@ -1702,7 +1951,7 @@ determine_known_aggregate_parts (gcall *call, tree ar= g, operands, whose definitions can finally reach the call. */ add_to_agg_contents_list (&list, (*copy =3D *content, copy)); - if (++const_count =3D=3D ipa_max_agg_items) + if (++value_count =3D=3D ipa_max_agg_items) break; } @@ -1720,12 +1969,12 @@ determine_known_aggregate_parts (gcall *call, tree = arg, /* Third stage just goes over the list and creates an appropriate vector= of ipa_agg_jf_item structures out of it, of course only if there are - any known constants to begin with. */ + any meaningful items to begin with. */ - if (const_count) + if (value_count) { jfunc->agg.by_ref =3D by_ref; - build_agg_jump_func_from_list (list, const_count, arg_offset, jfunc); + build_agg_jump_func_from_list (list, value_count, arg_offset, jfunc); } } @@ -2017,8 +2266,7 @@ ipa_compute_jump_functions_for_edge (struct ipa_func_= body_info *fbi, || !ipa_get_jf_ancestor_agg_preserved (jfunc)) && (AGGREGATE_TYPE_P (TREE_TYPE (arg)) || POINTER_TYPE_P (param_type))) - determine_known_aggregate_parts (call, arg, param_type, jfunc, - &fbi->aa_walk_budget); + determine_known_aggregate_parts (fbi, call, arg, param_type, jfunc); } if (!useful_context) vec_free (args->polymorphic_call_contexts); @@ -2661,6 +2909,72 @@ update_jump_functions_after_inlining (struct cgraph_= edge *cs, class ipa_polymorphic_call_context *dst_ctx =3D ipa_get_ith_polymorhic_call_context (args, i); + if (dst->agg.items) + { + struct ipa_agg_jf_item *item; + int j; + + FOR_EACH_VEC_ELT (*dst->agg.items, j, item) + { + int dst_fid; + struct ipa_jump_func *src; + + if (item->jftype !=3D IPA_JF_PASS_THROUGH + && item->jftype !=3D IPA_JF_LOAD_AGG) + continue; + + dst_fid =3D item->value.pass_through.formal_id; + if (dst_fid >=3D ipa_get_cs_argument_count (top)) + { + item->jftype =3D IPA_JF_UNKNOWN; + continue; + } + + item->value.pass_through.formal_id =3D -1; + src =3D ipa_get_ith_jump_func (top, dst_fid); + if (src->type =3D=3D IPA_JF_CONST) + { + if (item->jftype =3D=3D IPA_JF_PASS_THROUGH + && item->value.pass_through.operation =3D=3D NOP_EXPR) + { + item->jftype =3D IPA_JF_CONST; + item->value.constant =3D src->value.constant.value; + continue; + } + } + else if (src->type =3D=3D IPA_JF_PASS_THROUGH + && src->value.pass_through.operation =3D=3D NOP_EXPR) + { + if (item->jftype =3D=3D IPA_JF_PASS_THROUGH + || !item->value.load_agg.by_ref + || src->value.pass_through.agg_preserved) + item->value.pass_through.formal_id + =3D src->value.pass_through.formal_id; + } + else if (src->type =3D=3D IPA_JF_ANCESTOR) + { + if (item->jftype =3D=3D IPA_JF_PASS_THROUGH) + { + if (!src->value.ancestor.offset) + item->value.pass_through.formal_id + =3D src->value.ancestor.formal_id; + } + else if (src->value.ancestor.agg_preserved) + { + gcc_checking_assert (item->value.load_agg.by_ref); + + item->value.pass_through.formal_id + =3D src->value.ancestor.formal_id; + item->value.load_agg.offset + +=3D src->value.ancestor.offset; + } + } + + if (item->value.pass_through.formal_id < 0) + item->jftype =3D IPA_JF_UNKNOWN; + } + } + if (dst->type =3D=3D IPA_JF_ANCESTOR) { struct ipa_jump_func *src; @@ -2700,8 +3014,11 @@ update_jump_functions_after_inlining (struct cgraph_= edge *cs, } } - if (src->agg.items - && (dst->value.ancestor.agg_preserved || !src->agg.by_ref)) + /* Parameter and argument in ancestor jump function must be point= er + type, which means access to aggregate must be by-reference. */ + gcc_checking_assert (!src->agg.items || src->agg.by_ref); + + if (src->agg.items && dst->value.ancestor.agg_preserved) { struct ipa_agg_jf_item *item; int j; @@ -3093,18 +3410,19 @@ ipa_find_agg_cst_from_init (tree scalar, HOST_WIDE_= INT offset, bool by_ref) return find_constructor_constant_at_offset (DECL_INITIAL (scalar), offse= t); } -/* Retrieve value from aggregate jump function AGG or static initializer of - SCALAR (which can be NULL) for the given OFFSET or return NULL if there= is - none. BY_REF specifies whether the value has to be passed by reference= or - by value. If FROM_GLOBAL_CONSTANT is non-NULL, then the boolean it poi= nts - to is set to true if the value comes from an initializer of a constant.= */ +/* Retrieve value from AGG, a set of known offset/value for an aggregate or + static initializer of SCALAR (which can be NULL) for the given OFFSET or + return NULL if there is none. BY_REF specifies whether the value has t= o be + passed by reference or by value. If FROM_GLOBAL_CONSTANT is non-NULL, = then + the boolean it points to is set to true if the value comes from an + initializer of a constant. */ tree -ipa_find_agg_cst_for_param (struct ipa_agg_jump_function *agg, tree scalar, +ipa_find_agg_cst_for_param (struct ipa_agg_value_set *agg, tree scalar, HOST_WIDE_INT offset, bool by_ref, bool *from_global_constant) { - struct ipa_agg_jf_item *item; + struct ipa_agg_value *item; int i; if (scalar) @@ -3122,7 +3440,7 @@ ipa_find_agg_cst_for_param (struct ipa_agg_jump_funct= ion *agg, tree scalar, || by_ref !=3D agg->by_ref) return NULL; - FOR_EACH_VEC_SAFE_ELT (agg->items, i, item) + FOR_EACH_VEC_ELT (agg->items, i, item) if (item->offset =3D=3D offset) { /* Currently we do not have clobber values, return NULL for them on= ce @@ -3218,11 +3536,13 @@ try_decrement_rdesc_refcount (struct ipa_jump_func = *jfunc) pointer formal parameter described by jump function JFUNC. TARGET_TYPE= is the type of the parameter to which the result of JFUNC is passed. If i= t can be determined, return the newly direct edge, otherwise return NULL. - NEW_ROOT_INFO is the node info that JFUNC lattices are relative to. */ + NEW_ROOT and NEW_ROOT_INFO is the node and its info that JFUNC lattices= are + relative to. */ static struct cgraph_edge * try_make_edge_direct_simple_call (struct cgraph_edge *ie, struct ipa_jump_func *jfunc, tree target_= type, + struct cgraph_node *new_root, class ipa_node_params *new_root_info) { struct cgraph_edge *cs; @@ -3232,10 +3552,14 @@ try_make_edge_direct_simple_call (struct cgraph_edg= e *ie, if (agg_contents) { bool from_global_constant; - target =3D ipa_find_agg_cst_for_param (&jfunc->agg, scalar, + ipa_agg_value_set agg =3D ipa_agg_value_set_from_jfunc (new_root_inf= o, + new_root, + &jfunc->agg); + target =3D ipa_find_agg_cst_for_param (&agg, scalar, ie->indirect_info->offset, ie->indirect_info->by_ref, &from_global_constant); + agg.release (); if (target && !from_global_constant && !ie->indirect_info->guaranteed_unmodified) @@ -3289,12 +3613,16 @@ ipa_impossible_devirt_target (struct cgraph_edge *i= e, tree target) call based on a formal parameter which is described by jump function JF= UNC and if it can be determined, make it direct and return the direct edge. Otherwise, return NULL. CTX describes the polymorphic context that the - parameter the call is based on brings along with it. */ + parameter the call is based on brings along with it. NEW_ROOT and + NEW_ROOT_INFO is the node and its info that JFUNC lattices are relative + to. */ static struct cgraph_edge * try_make_edge_direct_virtual_call (struct cgraph_edge *ie, struct ipa_jump_func *jfunc, - class ipa_polymorphic_call_context ctx) + class ipa_polymorphic_call_context ctx, + struct cgraph_node *new_root, + class ipa_node_params *new_root_info) { tree target =3D NULL; bool speculative =3D false; @@ -3312,9 +3640,13 @@ try_make_edge_direct_virtual_call (struct cgraph_edg= e *ie, unsigned HOST_WIDE_INT offset; tree scalar =3D (jfunc->type =3D=3D IPA_JF_CONST) ? ipa_get_jf_const= ant (jfunc) : NULL; - tree t =3D ipa_find_agg_cst_for_param (&jfunc->agg, scalar, + ipa_agg_value_set agg =3D ipa_agg_value_set_from_jfunc (new_root_inf= o, + new_root, + &jfunc->agg); + tree t =3D ipa_find_agg_cst_for_param (&agg, scalar, ie->indirect_info->offset, true); + agg.release (); if (t && vtable_pointer_value_to_vtable (t, &vtable, &offset)) { bool can_refer; @@ -3405,14 +3737,15 @@ update_indirect_edges_after_inlining (struct cgraph= _edge *cs, { class ipa_edge_args *top; struct cgraph_edge *ie, *next_ie, *new_direct_edge; + struct cgraph_node *new_root; class ipa_node_params *new_root_info, *inlined_node_info; bool res =3D false; ipa_check_create_edge_args (); top =3D IPA_EDGE_REF (cs); - new_root_info =3D IPA_NODE_REF (cs->caller->global.inlined_to - ? cs->caller->global.inlined_to - : cs->caller); + new_root =3D cs->caller->global.inlined_to + ? cs->caller->global.inlined_to : cs->caller; + new_root_info =3D IPA_NODE_REF (new_root); inlined_node_info =3D IPA_NODE_REF (cs->callee->function_symbol ()); for (ie =3D node->indirect_calls; ie; ie =3D next_ie) @@ -3451,13 +3784,16 @@ update_indirect_edges_after_inlining (struct cgraph= _edge *cs, { ipa_polymorphic_call_context ctx; ctx =3D ipa_context_from_jfunc (new_root_info, cs, param_index, j= func); - new_direct_edge =3D try_make_edge_direct_virtual_call (ie, jfunc,= ctx); + new_direct_edge =3D try_make_edge_direct_virtual_call (ie, jfunc,= ctx, + new_root, + new_root_inf= o); } else { tree target_type =3D ipa_get_type (inlined_node_info, param_inde= x); new_direct_edge =3D try_make_edge_direct_simple_call (ie, jfunc, target_type, + new_root, new_root_info= ); } @@ -4125,6 +4461,8 @@ ipa_write_jump_function (struct output_block *ob, bp_pack_value (&bp, jump_func->value.ancestor.agg_preserved, 1); streamer_write_bitpack (&bp); break; + default: + fatal_error (UNKNOWN_LOCATION, "invalid jump function in LTO stream"= ); } count =3D vec_safe_length (jump_func->agg.items); @@ -4138,8 +4476,36 @@ ipa_write_jump_function (struct output_block *ob, FOR_EACH_VEC_SAFE_ELT (jump_func->agg.items, i, item) { + stream_write_tree (ob, item->type, true); streamer_write_uhwi (ob, item->offset); - stream_write_tree (ob, item->value, true); + streamer_write_uhwi (ob, item->jftype); + switch (item->jftype) + { + case IPA_JF_UNKNOWN: + break; + case IPA_JF_CONST: + stream_write_tree (ob, item->value.constant, true); + break; + case IPA_JF_PASS_THROUGH: + case IPA_JF_LOAD_AGG: + streamer_write_uhwi (ob, item->value.pass_through.operation); + streamer_write_uhwi (ob, item->value.pass_through.formal_id); + if (TREE_CODE_CLASS (item->value.pass_through.operation) + !=3D tcc_unary) + stream_write_tree (ob, item->value.pass_through.operand, true); + if (item->jftype =3D=3D IPA_JF_LOAD_AGG) + { + stream_write_tree (ob, item->value.load_agg.type, true); + streamer_write_uhwi (ob, item->value.load_agg.offset); + bp =3D bitpack_create (ob->main_stream); + bp_pack_value (&bp, item->value.load_agg.by_ref, 1); + streamer_write_bitpack (&bp); + } + break; + default: + fatal_error (UNKNOWN_LOCATION, + "invalid jump function in LTO stream"); + } } bp =3D bitpack_create (ob->main_stream); @@ -4236,8 +4602,39 @@ ipa_read_jump_function (class lto_input_block *ib, for (i =3D 0; i < count; i++) { struct ipa_agg_jf_item item; + item.type =3D stream_read_tree (ib, data_in); item.offset =3D streamer_read_uhwi (ib); - item.value =3D stream_read_tree (ib, data_in); + item.jftype =3D (enum jump_func_type) streamer_read_uhwi (ib); + + switch (item.jftype) + { + case IPA_JF_UNKNOWN: + break; + case IPA_JF_CONST: + item.value.constant =3D stream_read_tree (ib, data_in); + break; + case IPA_JF_PASS_THROUGH: + case IPA_JF_LOAD_AGG: + operation =3D (enum tree_code) streamer_read_uhwi (ib); + item.value.pass_through.operation =3D operation; + item.value.pass_through.formal_id =3D streamer_read_uhwi (ib); + if (TREE_CODE_CLASS (operation) =3D=3D tcc_unary) + item.value.pass_through.operand =3D NULL_TREE; + else + item.value.pass_through.operand =3D stream_read_tree (ib, data_= in); + if (item.jftype =3D=3D IPA_JF_LOAD_AGG) + { + struct bitpack_d bp; + item.value.load_agg.type =3D stream_read_tree (ib, data_in); + item.value.load_agg.offset =3D streamer_read_uhwi (ib); + bp =3D streamer_read_bitpack (ib); + item.value.load_agg.by_ref =3D bp_unpack_value (&bp, 1); + } + break; + default: + fatal_error (UNKNOWN_LOCATION, + "invalid jump function in LTO stream"); + } if (prevails) jump_func->agg.items->quick_push (item); } diff --git a/gcc/ipa-prop.h b/gcc/ipa-prop.h index 30948fb8854..bcdcc4b7f02 100644 --- a/gcc/ipa-prop.h +++ b/gcc/ipa-prop.h @@ -39,6 +39,15 @@ along with GCC; see the file COPYING3. If not see argument. Unknown - neither of the above. + IPA_JF_LOAD_AGG is a compound pass-through jump function, in which prim= ary + operation on formal parameter is memory dereference that loads a value = from + a part of an aggregate, which is represented or pointed to by the formal + parameter. Moreover, an additional unary/binary operation can be appli= ed on + the loaded value, and final result is passed as actual argument of call= ee + (e.g. *(param_1(D) + 4) op 24 ). It is meant to describe usage of aggr= egate + parameter or by-reference parameter referenced in argument passing, com= monly + found in C++ and Fortran. + IPA_JF_ANCESTOR is a special pass-through jump function, which means th= at the result is an address of a part of the object pointed to by the form= al parameter to which the function refers. It is mainly intended to repre= sent @@ -60,6 +69,7 @@ enum jump_func_type IPA_JF_UNKNOWN =3D 0, /* newly allocated and zeroed jump functions defa= ult */ IPA_JF_CONST, /* represented by field costant */ IPA_JF_PASS_THROUGH, /* represented by field pass_through */ + IPA_JF_LOAD_AGG, /* represented by field load_agg */ IPA_JF_ANCESTOR /* represented by field ancestor */ }; @@ -97,6 +107,26 @@ struct GTY(()) ipa_pass_through_data unsigned agg_preserved : 1; }; +/* Structure holding data required to describe a load-value-from-aggregate + jump function. */ + +struct GTY(()) ipa_load_agg_data +{ + /* Inherit from pass through jump function, describing unary/binary + operation on the value loaded from aggregate that is represented or + pointed to by the formal parameter, specified by formal_id in this + pass_through jump function data structure. */ + struct ipa_pass_through_data pass_through; + /* Type of the value loaded from the aggregate. */ + tree type; + /* Offset at which the value is located within the aggregate. */ + HOST_WIDE_INT offset; + /* True if loaded by reference (the aggregate is pointed to by the formal + parameter) or false if loaded by value (the aggregate is represented + by the formal parameter). */ + bool by_ref; +}; + /* Structure holding data required to describe an ancestor pass-through jump function. */ @@ -110,38 +140,86 @@ struct GTY(()) ipa_ancestor_jf_data unsigned agg_preserved : 1; }; -/* An element in an aggegate part of a jump function describing a known va= lue - at a given offset. When it is part of a pass-through jump function with - agg_preserved set or an ancestor jump function with agg_preserved set, = all - unlisted positions are assumed to be preserved but the value can be a t= ype - node, which means that the particular piece (starting at offset and hav= ing - the size of the type) is clobbered with an unknown value. When - agg_preserved is false or the type of the containing jump function is - different, all unlisted parts are assumed to be unknown and all values = must - fulfill is_gimple_ip_invariant. */ +/* A jump function for an aggregate part at a given offset, which describe= s how + it content value is generated. All unlisted positions are assumed to h= ave a + value defined in an unknown way. */ struct GTY(()) ipa_agg_jf_item { - /* The offset at which the known value is located within the aggregate. = */ + /* The offset for the aggregate part. */ HOST_WIDE_INT offset; - /* The known constant or type if this is a clobber. */ - tree value; -}; + /* Data type of the aggregate part. */ + tree type; + /* Jump function type. */ + enum jump_func_type jftype; -/* Aggregate jump function - i.e. description of contents of aggregates pa= ssed - either by reference or value. */ + /* Represents a value of jump function. constant represents the actual c= onstant + in constant jump function content. pass_through is used only in simp= le pass + through jump function context. load_agg is for load-value-from-aggre= gate + jump function context. */ + union jump_func_agg_value + { + tree GTY ((tag ("IPA_JF_CONST"))) constant; + struct ipa_pass_through_data GTY ((tag ("IPA_JF_PASS_THROUGH"))) pass_= through; + struct ipa_load_agg_data GTY ((tag ("IPA_JF_LOAD_AGG"))) load_agg; + } GTY ((desc ("%1.jftype"))) value; +}; + +/* Jump functions describing a set of aggregate contents. */ struct GTY(()) ipa_agg_jump_function { - /* Description of the individual items. */ + /* Description of the individual jump function item. */ vec *items; - /* True if the data was passed by reference (as opposed to by value). */ + /* True if the data was passed by reference (as opposed to by value). */ + bool by_ref; +}; + +/* An element in an aggregate part describing a known value at a given off= set. + All unlisted positions are assumed to be unknown and all listed values = must + fulfill is_gimple_ip_invariant. */ + +struct GTY(()) ipa_agg_value +{ + /* The offset at which the known value is located within the aggregate. = */ + HOST_WIDE_INT offset; + + /* The known constant. */ + tree value; +}; + +/* Structure describing a set of known offset/value for aggregate. */ + +struct GTY(()) ipa_agg_value_set +{ + /* Description of the individual item. */ + vec items; + /* True if the data was passed by reference (as opposed to by value). */ bool by_ref; + + void release () + { + items.release (); + } }; -typedef struct ipa_agg_jump_function *ipa_agg_jump_function_p; +/* For vec, DO NOT call release(), use below function + instead. Because ipa_agg_value_set contains a field of vector type, we + should release this child vector in each element before reclaiming the + whole vector. */ + +static inline void +ipa_release_agg_values (vec &aggs) +{ + ipa_agg_value_set *agg; + int i; + + FOR_EACH_VEC_ELT (aggs, i, agg) + agg->release (); + aggs.release (); +} /* Information about zero/non-zero bits. */ class GTY(()) ipa_bits @@ -172,8 +250,8 @@ public: types of jump functions supported. */ struct GTY (()) ipa_jump_func { - /* Aggregate contants description. See struct ipa_agg_jump_function and= its - description. */ + /* Aggregate jump function description. See struct ipa_agg_jump_function + and its description. */ struct ipa_agg_jump_function agg; /* Information about zero/non-zero bits. The pointed to structure is sh= ared @@ -742,9 +820,9 @@ bool ipa_propagate_indirect_call_infos (struct cgraph_e= dge *cs, /* Indirect edge and binfo processing. */ tree ipa_get_indirect_edge_target (struct cgraph_edge *ie, - vec , + vec, vec, - vec, + vec, bool *); struct cgraph_edge *ipa_make_edge_direct_to_target (struct cgraph_edge *, = tree, bool speculative =3D fa= lse); @@ -757,7 +835,7 @@ ipa_bits *ipa_get_ipa_bits_for_value (const widest_int = &value, void ipa_analyze_node (struct cgraph_node *); /* Aggregate jump function related functions. */ -tree ipa_find_agg_cst_for_param (struct ipa_agg_jump_function *agg, tree s= calar, +tree ipa_find_agg_cst_for_param (struct ipa_agg_value_set *agg, tree scala= r, HOST_WIDE_INT offset, bool by_ref, bool *from_global_constant =3D NULL); bool ipa_load_from_parm_agg (struct ipa_func_body_info *fbi, @@ -803,6 +881,9 @@ ipa_polymorphic_call_context ipa_context_from_jfunc (ip= a_node_params *, cgraph_edge *, int, ipa_jump_func *); +ipa_agg_value_set ipa_agg_value_set_from_jfunc (ipa_node_params *, + cgraph_node *, + ipa_agg_jump_function *); void ipa_dump_param (FILE *, class ipa_node_params *info, int i); void ipa_release_body_info (struct ipa_func_body_info *); tree ipa_get_callee_param_type (struct cgraph_edge *e, int i); diff --git a/gcc/testsuite/gcc.dg/ipa/ipcp-agg-10.c b/gcc/testsuite/gcc.dg/= ipa/ipcp-agg-10.c index 16d62e72c9a..c61e96a842b 100644 --- a/gcc/testsuite/gcc.dg/ipa/ipcp-agg-10.c +++ b/gcc/testsuite/gcc.dg/ipa/ipcp-agg-10.c @@ -72,7 +72,7 @@ int caller2(void) return sum; } -/* { dg-final { scan-ipa-dump-times "offset: 0, cst: 1" 1 "cp" } } */ -/* { dg-final { scan-ipa-dump-times "offset: 0, cst: 2" 1 "cp" } } */ -/* { dg-final { scan-ipa-dump-times "offset: 0, cst: 3" 1 "cp" } } */ -/* { dg-final { scan-ipa-dump-times "offset: 64, cst: 4" 1 "cp" } } */ +/* { dg-final { scan-ipa-dump-times "offset: 0, type: int, CONST: 1" 1 "cp= " } } */ +/* { dg-final { scan-ipa-dump-times "offset: 0, type: int, CONST: 2" 1 "cp= " } } */ +/* { dg-final { scan-ipa-dump-times "offset: 0, type: int, CONST: 3" 1 "cp= " } } */ +/* { dg-final { scan-ipa-dump-times "offset: 64, type: int, CONST: 4" 1 "c= p" } } */ diff --git a/gcc/testsuite/gcc.dg/ipa/ipcp-agg-11.c b/gcc/testsuite/gcc.dg/= ipa/ipcp-agg-11.c new file mode 100644 index 00000000000..3c496eeef39 --- /dev/null +++ b/gcc/testsuite/gcc.dg/ipa/ipcp-agg-11.c @@ -0,0 +1,77 @@ +/* { dg-do compile } */ +/* { dg-options "-O3 -fno-ipa-sra -fdump-ipa-cp-details -fno-early-inlinin= g" } */ +/* { dg-add-options bind_pic_locally } */ + +struct S +{ + int a, b, c; +}; + +void *blah(int, void *); + +#define foo_body(p)\ +{ \ + int i, c =3D (p)->c; \ + int b =3D (p)->b; \ + void *v =3D (void *) (p); \ + \ + for (i=3D 0; i< c; i++) \ + v =3D blah(b + i, v); \ +} + +static void __attribute__ ((noinline)) +foo_v (struct S s) +{ + foo_body (&s); +} + +static void __attribute__ ((noinline)) +foo_r (struct S *p) +{ + foo_body (p); +} + +static void +goo_v (int a, int *p) +{ + struct S s; + s.a =3D 101; + s.b =3D a % 7; + s.c =3D *p + 6; + foo_v (s); +} + +static void +goo_r (int a, struct S n) +{ + struct S s; + s.a =3D 1; + s.b =3D a + 5; + s.c =3D -n.b; + foo_r (&s); +} + +void +entry () +{ + int a; + int v; + struct S s; + + a =3D 9; + v =3D 3; + goo_v (a, &v); + + a =3D 100; + s.b =3D 18; + goo_r (a, s); +} + +/* { dg-final { scan-ipa-dump "offset: 0, type: int, CONST: 1" "cp" } } */ +/* { dg-final { scan-ipa-dump "offset: 32, type: int, PASS THROUGH: 0, op = plus_expr 5" "cp" } } */ +/* { dg-final { scan-ipa-dump "offset: 64, type: int, LOAD AGG: 1 \\\[offs= et: 32, by value], op negate_expr" "cp" } } */ +/* { dg-final { scan-ipa-dump "offset: 0, type: int, CONST: 101" "cp" } } = */ +/* { dg-final { scan-ipa-dump "offset: 32, type: int, PASS THROUGH: 0, op = trunc_mod_expr 7" "cp" } } */ +/* { dg-final { scan-ipa-dump "offset: 64, type: int, LOAD AGG: 1 \\\[offs= et: 0, by reference], op plus_expr 6" "cp" } } */ +/* { dg-final { scan-ipa-dump "Aggregate replacements: 0\\\[0]=3D1, 0\\\[3= 2]=3D105, 0\\\[64]=3D-18" "cp" } } */ +/* { dg-final { scan-ipa-dump "Aggregate replacements: 0\\\[0]=3D101, 0\\\= [32]=3D2, 0\\\[64]=3D9" "cp" } } */ -- 2.17.1