From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by sourceware.org (Postfix) with ESMTPS id 675603959C5D for ; Tue, 30 Aug 2022 17:06:08 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 675603959C5D Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=suse.cz Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=suse.cz Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 1E1E21FA05; Tue, 30 Aug 2022 17:06:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1661879167; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type; bh=rRYcxAMTJRCbSAFPT2Fgwx7VgsEy+/iboFe7mXI12nw=; b=mfZvWc5PYOFxn0iRvtGZ8t5kvoFjULkpMEoYj4XcmiqZ5VD7ddYSqgL+54hPO0aYfLxqtf +LGTI4WJkZlGPIEKJcs7hRvJckRHM7Cn706xUMkjbkGIFN0/N4GeE7K3QLg68IiyPSpPqJ tfOT8FjxOktOixyRPIswt3O3MzWseOk= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1661879167; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type; bh=rRYcxAMTJRCbSAFPT2Fgwx7VgsEy+/iboFe7mXI12nw=; b=YHL5B52l3UkfvTHbpwX1yAbVdd1FvOgxxJj2eQMcxXMSOHBqVruZSTqguuFC2h5wab+hOd F95DPMNODxwW7FBQ== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 0B05A13B0C; Tue, 30 Aug 2022 17:06:07 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id ZDiuAn9DDmPbWQAAMHmgww (envelope-from ); Tue, 30 Aug 2022 17:06:07 +0000 From: Martin Jambor To: GCC Patches Cc: Jan Hubicka , Jan Hubicka Subject: [PATCH 2/2] ipa-cp: Better representation of aggregate values in call contexts User-Agent: Notmuch/0.35 (https://notmuchmail.org) Emacs/28.1 (x86_64-suse-linux-gnu) Date: Tue, 30 Aug 2022 19:06:06 +0200 Message-ID: MIME-Version: 1.0 Content-Type: text/plain X-Spam-Status: No, score=-11.7 required=5.0 tests=BAYES_00,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,GIT_PATCH_0,SPF_HELO_NONE,SPF_PASS,TXREP,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: Hi this patch extends the previous one by using the same data structure to represent aggregate values in classes ipa_auto_call_arg_values and ipa_call_arg_values. This usually simplifies handling and makes allocations of memory much cheaper because only a single vectore is needed, as opposed to vectors with each element pointing at other vecs. The only functions which unfortunately are a bit more complec are estimate_local_effects in ipa-cp.cc and ipa_call_context::equal_to but I hope not too much - the latter could probably be shorteneed at the expense of readability. The patch removes types ipa_agg_value ipa_agg_value_set which is no longer used with it. This means that we could replace the "_argagg_" part of the types introduced by the previous patches with more reasonable "_agg_" - possibly as a follow-up patch. Bootstrapped, LTO-bootstrapped and tested on x86_64-linux (and a slightly older version also on aarch64-linux). LTO-profiledbootstrap is currently underway. Given the size of the patch I assume there will be concerns/questions but I'm looking for an approval to commit a version of this. Thanks, Martin gcc/ChangeLog: 2022-08-26 Martin Jambor * ipa-prop.h (ipa_agg_value): Remove type. (ipa_agg_value_set): Likewise. (ipa_copy_agg_values): Remove function. (ipa_release_agg_values): Likewise. (ipa_auto_call_arg_values) Add a forward declaration. (ipa_call_arg_values): Likewise. (class ipa_argagg_value_list): New constructors, added member function value_for_index_p. (class ipa_auto_call_arg_values): Removed the destructor and member function safe_aggval_at. Use ipa_argagg_values for m_known_aggs. (class ipa_call_arg_values): Removed member function safe_aggval_at. Use ipa_argagg_values for m_known_aggs. (ipa_get_indirect_edge_target): Removed declaration. (ipa_find_agg_cst_for_param): Likewise. (ipa_find_agg_cst_from_init): New declaration. (ipa_agg_value_from_jfunc): Likewise. (ipa_agg_value_set_from_jfunc): Removed declaration. (ipa_push_agg_values_from_jfunc): New declaration. * ipa-cp.cc (ipa_agg_value_from_node): Renamed to ipa_agg_value_from_jfunc, made public. (ipa_agg_value_set_from_jfunc): Removed. (ipa_push_agg_values_from_jfunc): New function. (ipa_get_indirect_edge_target_1): Removed known_aggs parameter, use avs for this purpose too. (ipa_get_indirect_edge_target): Removed the overload working on ipa_auto_call_arg_values, use ipa_argagg_value_list in the remaining one. (devirtualization_time_bonus): Use ipa_argagg_value_list and ipa_get_indirect_edge_target_1 instead of ipa_get_indirect_edge_target. (context_independent_aggregate_values): Removed function. (gather_context_independent_values): Work on ipa_argagg_value_list. (estimate_local_effects): Likewise, define some iterator variables only in the construct where necessary. (ipcp_discover_new_direct_edges): Adjust the call to ipa_get_indirect_edge_target_1. (push_agg_values_for_index_from_edge): Adjust the call ipa_agg_value_from_node which has been renamed to ipa_agg_value_from_jfunc. * ipa-fnsummary.cc (evaluate_conditions_for_known_args): Work on ipa_argagg_value_list. (evaluate_properties_for_edge): Replace manual filling in aggregate values with call to ipa_push_agg_values_from_jfunc. (estimate_calls_size_and_time): Work on ipa_argagg_value_list. (ipa_cached_call_context::duplicate_from): Likewise. (ipa_cached_call_context::release): Likewise. (ipa_call_context::equal_to): Likewise. * ipa-prop.cc (ipa_find_agg_cst_from_init): Make public. (ipa_find_agg_cst_for_param): Removed function. (ipa_find_agg_cst_from_jfunc_items): New function. (try_make_edge_direct_simple_call): Replace calls to ipa_agg_value_set_from_jfunc and ipa_find_agg_cst_for_param with ipa_find_agg_cst_from_init and ipa_find_agg_cst_from_jfunc_items. (try_make_edge_direct_virtual_call): Replace calls to ipa_agg_value_set_from_jfunc and ipa_find_agg_cst_for_param with simple query of constant jump function and a call to ipa_find_agg_cst_from_jfunc_items. (ipa_auto_call_arg_values::~ipa_auto_call_arg_values): Removed. --- gcc/ipa-cp.cc | 234 +++++++++++++++++-------------------------- gcc/ipa-fnsummary.cc | 105 ++++++++++--------- gcc/ipa-prop.cc | 110 ++++++-------------- gcc/ipa-prop.h | 172 +++++++------------------------ 4 files changed, 218 insertions(+), 403 deletions(-) diff --git a/gcc/ipa-cp.cc b/gcc/ipa-cp.cc index 024f8c06b5d..098392d9b90 100644 --- a/gcc/ipa-cp.cc +++ b/gcc/ipa-cp.cc @@ -1973,10 +1973,9 @@ ipa_value_range_from_jfunc (ipa_node_params *info, cgraph_edge *cs, NODE and INFO describes the caller node or the one it is inlined to, and its related info. */ -static tree -ipa_agg_value_from_node (class ipa_node_params *info, - struct cgraph_node *node, - const ipa_agg_jf_item *item) +tree +ipa_agg_value_from_jfunc (ipa_node_params *info, cgraph_node *node, + const ipa_agg_jf_item *item) { tree value = NULL_TREE; int src_idx; @@ -2059,37 +2058,38 @@ ipa_agg_value_from_node (class ipa_node_params *info, item->type); } -/* Determine whether AGG_JFUNC evaluates to a set of known constant value for - an aggregate and if so, return it. Otherwise return an empty set. NODE - and INFO describes the caller node or the one it is inlined to, and its - related info. */ +/* Process all items in AGG_JFUNC relative to caller (or the node the original + caller is inlined to) NODE which described by INFO and push the results to + RES as describing values passed in parameter DST_INDEX. */ -struct ipa_agg_value_set -ipa_agg_value_set_from_jfunc (class ipa_node_params *info, cgraph_node *node, - struct ipa_agg_jump_function *agg_jfunc) +void +ipa_push_agg_values_from_jfunc (ipa_node_params *info, cgraph_node *node, + ipa_agg_jump_function *agg_jfunc, + unsigned dst_index, + vec *res) { - struct ipa_agg_value_set agg; - struct ipa_agg_jf_item *item; - int i; + unsigned prev_unit_offset = 0; + bool first = true; - agg.items = vNULL; - agg.by_ref = agg_jfunc->by_ref; - - FOR_EACH_VEC_SAFE_ELT (agg_jfunc->items, i, item) + for (const ipa_agg_jf_item &item : agg_jfunc->items) { - tree value = ipa_agg_value_from_node (info, node, item); + tree value = ipa_agg_value_from_jfunc (info, node, &item); + if (!value) + continue; - if (value) - { - struct ipa_agg_value value_item; + ipa_argagg_value iav; + iav.value = value; + iav.unit_offset = item.offset / BITS_PER_UNIT; + iav.index = dst_index; + iav.by_ref = agg_jfunc->by_ref; - value_item.offset = item->offset; - value_item.value = value; + gcc_assert (first + || iav.unit_offset > prev_unit_offset); + prev_unit_offset = iav.unit_offset; + first = false; - agg.items.safe_push (value_item); - } + res->safe_push (iav); } - return agg; } /* If checking is enabled, verify that no lattice is in the TOP state, i.e. not @@ -3238,8 +3238,7 @@ static tree ipa_get_indirect_edge_target_1 (struct cgraph_edge *ie, const vec &known_csts, const vec &known_contexts, - const vec &known_aggs, - const ipa_argagg_value_list *avs, + const ipa_argagg_value_list &avs, bool *speculative) { int param_index = ie->indirect_info->param_index; @@ -3259,31 +3258,16 @@ ipa_get_indirect_edge_target_1 (struct cgraph_edge *ie, if (ie->indirect_info->agg_contents) { t = NULL; - if (avs && ie->indirect_info->guaranteed_unmodified) - t = avs->get_value (param_index, - ie->indirect_info->offset / BITS_PER_UNIT, - ie->indirect_info->by_ref); - if (!t) - { - const ipa_agg_value_set *agg; - if (known_aggs.length () > (unsigned int) param_index) - agg = &known_aggs[param_index]; - else - agg = NULL; - bool from_global_constant; - t = ipa_find_agg_cst_for_param (agg, - (unsigned) param_index - < known_csts.length () - ? known_csts[param_index] - : NULL, - ie->indirect_info->offset, - ie->indirect_info->by_ref, - &from_global_constant); - if (t - && !from_global_constant - && !ie->indirect_info->guaranteed_unmodified) - t = NULL_TREE; - } + if ((unsigned) param_index < known_csts.length () + && known_csts[param_index]) + t = ipa_find_agg_cst_from_init (known_csts[param_index], + ie->indirect_info->offset, + ie->indirect_info->by_ref); + + if (!t && ie->indirect_info->guaranteed_unmodified) + t = avs.get_value (param_index, + ie->indirect_info->offset / BITS_PER_UNIT, + ie->indirect_info->by_ref); } else if ((unsigned) param_index < known_csts.length ()) t = known_csts[param_index]; @@ -3300,28 +3284,22 @@ ipa_get_indirect_edge_target_1 (struct cgraph_edge *ie, return NULL_TREE; gcc_assert (!ie->indirect_info->agg_contents); + gcc_assert (!ie->indirect_info->by_ref); anc_offset = ie->indirect_info->offset; t = NULL; - /* Try to work out value of virtual table pointer value in replacements. */ - if (!t && avs && !ie->indirect_info->by_ref) - t = avs->get_value (param_index, - ie->indirect_info->offset / BITS_PER_UNIT, - true); + if ((unsigned) param_index < known_csts.length () + && known_csts[param_index]) + t = ipa_find_agg_cst_from_init (known_csts[param_index], + ie->indirect_info->offset, true); - /* Try to work out value of virtual table pointer value in known - aggregate values. */ - if (!t && known_aggs.length () > (unsigned int) param_index - && !ie->indirect_info->by_ref) - { - const ipa_agg_value_set *agg = &known_aggs[param_index]; - t = ipa_find_agg_cst_for_param (agg, - (unsigned) param_index - < known_csts.length () - ? known_csts[param_index] : NULL, - ie->indirect_info->offset, true); - } + /* Try to work out value of virtual table pointer value in replacements. */ + /* or known aggregate values. */ + if (!t) + t = avs.get_value (param_index, + ie->indirect_info->offset / BITS_PER_UNIT, + true); /* If we found the virtual table pointer, lookup the target. */ if (t) @@ -3440,23 +3418,10 @@ ipa_get_indirect_edge_target (struct cgraph_edge *ie, ipa_call_arg_values *avals, bool *speculative) { + ipa_argagg_value_list avl (avals); return ipa_get_indirect_edge_target_1 (ie, avals->m_known_vals, avals->m_known_contexts, - avals->m_known_aggs, - NULL, speculative); -} - -/* The same functionality as above overloaded for ipa_auto_call_arg_values. */ - -tree -ipa_get_indirect_edge_target (struct cgraph_edge *ie, - ipa_auto_call_arg_values *avals, - bool *speculative) -{ - return ipa_get_indirect_edge_target_1 (ie, avals->m_known_vals, - avals->m_known_contexts, - avals->m_known_aggs, - NULL, speculative); + avl, speculative); } /* Calculate devirtualization time bonus for NODE, assuming we know information @@ -3477,7 +3442,10 @@ devirtualization_time_bonus (struct cgraph_node *node, tree target; bool speculative; - target = ipa_get_indirect_edge_target (ie, avals, &speculative); + ipa_argagg_value_list avl (avals); + target = ipa_get_indirect_edge_target_1 (ie, avals->m_known_vals, + avals->m_known_contexts, + avl, &speculative); if (!target) continue; @@ -3613,32 +3581,6 @@ good_cloning_opportunity_p (struct cgraph_node *node, sreal time_benefit, } } -/* Return all context independent values from aggregate lattices in PLATS in a - vector. Return NULL if there are none. */ - -static vec -context_independent_aggregate_values (class ipcp_param_lattices *plats) -{ - vec res = vNULL; - - if (plats->aggs_bottom - || plats->aggs_contain_variable - || plats->aggs_count == 0) - return vNULL; - - for (struct ipcp_agg_lattice *aglat = plats->aggs; - aglat; - aglat = aglat->next) - if (aglat->is_single_const ()) - { - struct ipa_agg_value item; - item.offset = aglat->offset; - item.value = aglat->values->value; - res.safe_push (item); - } - return res; -} - /* Grow vectors in AVALS and fill them with information about values of parameters that are known to be independent of the context. Only calculate m_known_aggs if CALCULATE_AGGS is true. INFO describes the function. If @@ -3658,8 +3600,6 @@ gather_context_independent_values (class ipa_node_params *info, avals->m_known_vals.safe_grow_cleared (count, true); avals->m_known_contexts.safe_grow_cleared (count, true); - if (calculate_aggs) - avals->m_known_aggs.safe_grow_cleared (count, true); if (removable_params_cost) *removable_params_cost = 0; @@ -3694,16 +3634,7 @@ gather_context_independent_values (class ipa_node_params *info, avals->m_known_contexts[i] = ctxlat->values->value; if (calculate_aggs) - { - vec agg_items; - struct ipa_agg_value_set *agg; - - agg_items = context_independent_aggregate_values (plats); - agg = &avals->m_known_aggs[i]; - agg->items = agg_items; - agg->by_ref = plats->aggs_by_ref; - ret |= !agg_items.is_empty (); - } + ret |= push_agg_values_from_plats (plats, i, 0, &avals->m_known_aggs); } return ret; @@ -3774,7 +3705,7 @@ static void estimate_local_effects (struct cgraph_node *node) { ipa_node_params *info = ipa_node_params_sum->get (node); - int i, count = ipa_get_param_count (info); + int count = ipa_get_param_count (info); bool always_const; int removable_params_cost; @@ -3840,7 +3771,7 @@ estimate_local_effects (struct cgraph_node *node) } - for (i = 0; i < count; i++) + for (int i = 0; i < count; i++) { class ipcp_param_lattices *plats = ipa_get_parm_lattices (info, i); ipcp_lattice *lat = &plats->itself; @@ -3874,7 +3805,7 @@ estimate_local_effects (struct cgraph_node *node) avals.m_known_vals[i] = NULL_TREE; } - for (i = 0; i < count; i++) + for (int i = 0; i < count; i++) { class ipcp_param_lattices *plats = ipa_get_parm_lattices (info, i); @@ -3909,30 +3840,49 @@ estimate_local_effects (struct cgraph_node *node) avals.m_known_contexts[i] = ipa_polymorphic_call_context (); } - for (i = 0; i < count; i++) + unsigned all_ctx_len = avals.m_known_aggs.length (); + auto_vec all_ctx; + all_ctx.reserve_exact (all_ctx_len); + all_ctx.splice (avals.m_known_aggs); + avals.m_known_aggs.safe_grow_cleared (all_ctx_len + 1); + + unsigned j = 0; + for (int index = 0; index < count; index++) { - class ipcp_param_lattices *plats = ipa_get_parm_lattices (info, i); + class ipcp_param_lattices *plats = ipa_get_parm_lattices (info, index); if (plats->aggs_bottom || !plats->aggs) continue; - ipa_agg_value_set *agg = &avals.m_known_aggs[i]; for (ipcp_agg_lattice *aglat = plats->aggs; aglat; aglat = aglat->next) { ipcp_value *val; if (aglat->bottom || !aglat->values - /* If the following is true, the one value is in known_aggs. */ + /* If the following is true, the one value is already part of all + context estimations. */ || (!plats->aggs_contain_variable && aglat->is_single_const ())) continue; + unsigned unit_offset = aglat->offset / BITS_PER_UNIT; + while (j < all_ctx_len + && (all_ctx[j].index < index + || (all_ctx[j].index == index + && all_ctx[j].unit_offset < unit_offset))) + { + avals.m_known_aggs[j] = all_ctx[j]; + j++; + } + + for (unsigned k = j; k < all_ctx_len; k++) + avals.m_known_aggs[k+1] = all_ctx[k]; + for (val = aglat->values; val; val = val->next) { - struct ipa_agg_value item; - - item.offset = aglat->offset; - item.value = val->value; - agg->items.safe_push (item); + avals.m_known_aggs[j].value = val->value; + avals.m_known_aggs[j].unit_offset = unit_offset; + avals.m_known_aggs[j].index = index; + avals.m_known_aggs[j].by_ref = plats->aggs_by_ref; perform_estimation_of_a_value (node, &avals, removable_params_cost, 0, val); @@ -3942,7 +3892,7 @@ estimate_local_effects (struct cgraph_node *node) fprintf (dump_file, " - estimates for value "); print_ipcp_constant_value (dump_file, val->value); fprintf (dump_file, " for "); - ipa_dump_param (dump_file, info, i); + ipa_dump_param (dump_file, info, index); fprintf (dump_file, "[%soffset: " HOST_WIDE_INT_PRINT_DEC "]: time_benefit: %g, size: %i\n", plats->aggs_by_ref ? "ref " : "", @@ -3950,8 +3900,6 @@ estimate_local_effects (struct cgraph_node *node) val->local_time_benefit.to_double (), val->local_size_cost); } - - agg->items.pop (); } } } @@ -4348,7 +4296,7 @@ ipcp_discover_new_direct_edges (struct cgraph_node *node, next_ie = ie->next_callee; ipa_argagg_value_list avs (aggvals); target = ipa_get_indirect_edge_target_1 (ie, known_csts, known_contexts, - vNULL, &avs, &speculative); + avs, &speculative); if (target) { bool agg_contents = ie->indirect_info->agg_contents; @@ -5777,8 +5725,8 @@ push_agg_values_for_index_from_edge (struct cgraph_edge *cs, int index, agg_jf.value.pass_through.operand, agg_jf.type); else - value = ipa_agg_value_from_node (caller_info, cs->caller, - &agg_jf); + value = ipa_agg_value_from_jfunc (caller_info, cs->caller, + &agg_jf); if (value) { struct ipa_argagg_value iav; diff --git a/gcc/ipa-fnsummary.cc b/gcc/ipa-fnsummary.cc index e2a86680a21..fd3d7d6c5e8 100644 --- a/gcc/ipa-fnsummary.cc +++ b/gcc/ipa-fnsummary.cc @@ -386,15 +386,6 @@ evaluate_conditions_for_known_args (struct cgraph_node *node, int j; struct expr_eval_op *op; - /* We allow call stmt to have fewer arguments than the callee function - (especially for K&R style programs). So bound check here (we assume - m_known_aggs vector is either empty or has the same length as - m_known_vals). */ - gcc_checking_assert (!avals->m_known_aggs.length () - || !avals->m_known_vals.length () - || (avals->m_known_vals.length () - == avals->m_known_aggs.length ())); - if (c->agg_contents) { if (c->code == ipa_predicate::changed @@ -402,14 +393,14 @@ evaluate_conditions_for_known_args (struct cgraph_node *node, && (avals->safe_sval_at(c->operand_num) == error_mark_node)) continue; - if (ipa_agg_value_set *agg = avals->safe_aggval_at (c->operand_num)) + if (tree sval = avals->safe_sval_at (c->operand_num)) + val = ipa_find_agg_cst_from_init (sval, c->offset, c->by_ref); + if (!val) { - tree sval = avals->safe_sval_at (c->operand_num); - val = ipa_find_agg_cst_for_param (agg, sval, c->offset, - c->by_ref); + ipa_argagg_value_list avs (avals); + val = avs.get_value (c->operand_num, c->offset / BITS_PER_UNIT, + c->by_ref); } - else - val = NULL_TREE; } else { @@ -674,17 +665,9 @@ evaluate_properties_for_edge (struct cgraph_edge *e, bool inline_p, /* Determine known aggregate values. */ if (fre_will_run_p (caller)) - { - ipa_agg_value_set agg - = ipa_agg_value_set_from_jfunc (caller_parms_info, - caller, &jf->agg); - if (agg.items.length ()) - { - if (!avals->m_known_aggs.length ()) - avals->m_known_aggs.safe_grow_cleared (count, true); - avals->m_known_aggs[i] = agg; - } - } + ipa_push_agg_values_from_jfunc (caller_parms_info, + caller, &jf->agg, i, + &avals->m_known_aggs); } /* For calls used in polymorphic calls we further determine @@ -3446,8 +3429,7 @@ estimate_calls_size_and_time (struct cgraph_node *node, int *size, { if (ipa_is_param_used_by_indirect_call (params_summary, i) && (avals->safe_sval_at (i) - || (avals->m_known_aggs.length () > i - && avals->m_known_aggs[i].items.length ()))) + || (ipa_argagg_value_list (avals).value_for_index_p (i)))) use_table = false; else if (ipa_is_param_used_by_polymorphic_call (params_summary, i) && (avals->m_known_contexts.length () > i @@ -3583,14 +3565,12 @@ ipa_cached_call_context::duplicate_from (const ipa_call_context &ctx) m_avals.m_known_aggs = vNULL; if (ctx.m_avals.m_known_aggs.exists ()) { - unsigned int n = MIN (ctx.m_avals.m_known_aggs.length (), nargs); - - for (unsigned int i = 0; i < n; i++) + const ipa_argagg_value_list avl (&ctx.m_avals); + for (unsigned int i = 0; i < nargs; i++) if (ipa_is_param_used_by_indirect_call (params_summary, i) - && !ctx.m_avals.m_known_aggs[i].is_empty ()) + && avl.value_for_index_p (i)) { - m_avals.m_known_aggs - = ipa_copy_agg_values (ctx.m_avals.m_known_aggs); + m_avals.m_known_aggs = ctx.m_avals.m_known_aggs.copy (); break; } } @@ -3607,7 +3587,7 @@ ipa_cached_call_context::release () /* See if context is initialized at first place. */ if (!m_node) return; - ipa_release_agg_values (m_avals.m_known_aggs, true); + m_avals.m_known_aggs.release (); m_avals.m_known_vals.release (); m_avals.m_known_contexts.release (); m_inline_param_summary.release (); @@ -3708,28 +3688,59 @@ ipa_call_context::equal_to (const ipa_call_context &ctx) } if (m_avals.m_known_aggs.exists () || ctx.m_avals.m_known_aggs.exists ()) { - for (unsigned int i = 0; i < nargs; i++) + unsigned i = 0, j = 0; + while (i < m_avals.m_known_aggs.length () + || j < ctx.m_avals.m_known_aggs.length ()) { - if (!ipa_is_param_used_by_indirect_call (params_summary, i)) - continue; - if (i >= m_avals.m_known_aggs.length () - || m_avals.m_known_aggs[i].is_empty ()) + if (i >= m_avals.m_known_aggs.length ()) { - if (i < ctx.m_avals.m_known_aggs.length () - && !ctx.m_avals.m_known_aggs[i].is_empty ()) + int idx2 = ctx.m_avals.m_known_aggs[j].index; + if (ipa_is_param_used_by_indirect_call (params_summary, idx2)) return false; + j++; continue; } - if (i >= ctx.m_avals.m_known_aggs.length () - || ctx.m_avals.m_known_aggs[i].is_empty ()) + if (j >= ctx.m_avals.m_known_aggs.length ()) { - if (i < m_avals.m_known_aggs.length () - && !m_avals.m_known_aggs[i].is_empty ()) + int idx1 = m_avals.m_known_aggs[i].index; + if (ipa_is_param_used_by_indirect_call (params_summary, idx1)) return false; + i++; continue; } - if (!m_avals.m_known_aggs[i].equal_to (ctx.m_avals.m_known_aggs[i])) + + int idx1 = m_avals.m_known_aggs[i].index; + int idx2 = ctx.m_avals.m_known_aggs[j].index; + if (idx1 < idx2) + { + if (ipa_is_param_used_by_indirect_call (params_summary, idx1)) + return false; + i++; + continue; + } + if (idx1 > idx2) + { + if (ipa_is_param_used_by_indirect_call (params_summary, idx2)) + return false; + j++; + continue; + } + if (!ipa_is_param_used_by_indirect_call (params_summary, idx1)) + { + i++; + j++; + continue; + } + + if ((m_avals.m_known_aggs[i].unit_offset + != ctx.m_avals.m_known_aggs[j].unit_offset) + || (m_avals.m_known_aggs[i].by_ref + != ctx.m_avals.m_known_aggs[j].by_ref) + || !operand_equal_p (m_avals.m_known_aggs[i].value, + ctx.m_avals.m_known_aggs[j].value)) return false; + i++; + j++; } } return true; diff --git a/gcc/ipa-prop.cc b/gcc/ipa-prop.cc index 6196d7e6bdb..ee1acf85a95 100644 --- a/gcc/ipa-prop.cc +++ b/gcc/ipa-prop.cc @@ -3608,7 +3608,7 @@ find_constructor_constant_at_offset (tree constructor, HOST_WIDE_INT req_offset) invariant from a static constructor and if so, return it. Otherwise return NULL. */ -static tree +tree ipa_find_agg_cst_from_init (tree scalar, HOST_WIDE_INT offset, bool by_ref) { if (by_ref) @@ -3628,47 +3628,24 @@ ipa_find_agg_cst_from_init (tree scalar, HOST_WIDE_INT offset, bool by_ref) return find_constructor_constant_at_offset (DECL_INITIAL (scalar), offset); } -/* Retrieve value from AGG, a set of known offset/value for an aggregate or - static initializer of SCALAR (which can be NULL) for the given OFFSET or - return NULL if there is none. BY_REF specifies whether the value has to be - passed by reference or by value. If FROM_GLOBAL_CONSTANT is non-NULL, then - the boolean it points to is set to true if the value comes from an - initializer of a constant. */ +/* Retrieve value from AGG_JFUNC for the given OFFSET or return NULL if there + is none. BY_REF specifies whether the value has to be passed by reference + or by value. */ -tree -ipa_find_agg_cst_for_param (const ipa_agg_value_set *agg, tree scalar, - HOST_WIDE_INT offset, bool by_ref, - bool *from_global_constant) +static tree +ipa_find_agg_cst_from_jfunc_items (struct ipa_agg_jump_function *agg_jfunc, + ipa_node_params *src_info, + cgraph_node *src_node, + HOST_WIDE_INT offset, bool by_ref) { - struct ipa_agg_value *item; - int i; + if (by_ref != agg_jfunc->by_ref) + return NULL_TREE; - if (scalar) - { - tree res = ipa_find_agg_cst_from_init (scalar, offset, by_ref); - if (res) - { - if (from_global_constant) - *from_global_constant = true; - return res; - } - } + for (const ipa_agg_jf_item &item : agg_jfunc->items) + if (item.offset == offset) + return ipa_agg_value_from_jfunc (src_info, src_node, &item); - if (!agg - || by_ref != agg->by_ref) - return NULL; - - FOR_EACH_VEC_ELT (agg->items, i, item) - if (item->offset == offset) - { - /* Currently we do not have clobber values, return NULL for them once - we do. */ - gcc_checking_assert (is_gimple_ip_invariant (item->value)); - if (from_global_constant) - *from_global_constant = false; - return item->value; - } - return NULL; + return NULL_TREE; } /* Remove a reference to SYMBOL from the list of references of a node given by @@ -3765,24 +3742,19 @@ try_make_edge_direct_simple_call (struct cgraph_edge *ie, class ipa_node_params *new_root_info) { struct cgraph_edge *cs; - tree target; + tree target = NULL_TREE; bool agg_contents = ie->indirect_info->agg_contents; tree scalar = ipa_value_from_jfunc (new_root_info, jfunc, target_type); if (agg_contents) { - bool from_global_constant; - ipa_agg_value_set agg = ipa_agg_value_set_from_jfunc (new_root_info, - new_root, - &jfunc->agg); - target = ipa_find_agg_cst_for_param (&agg, scalar, - ie->indirect_info->offset, - ie->indirect_info->by_ref, - &from_global_constant); - agg.release (); - if (target - && !from_global_constant - && !ie->indirect_info->guaranteed_unmodified) - return NULL; + if (scalar) + target = ipa_find_agg_cst_from_init (scalar, ie->indirect_info->offset, + ie->indirect_info->by_ref); + if (!target && ie->indirect_info->guaranteed_unmodified) + target = ipa_find_agg_cst_from_jfunc_items (&jfunc->agg, new_root_info, + new_root, + ie->indirect_info->offset, + ie->indirect_info->by_ref); } else target = scalar; @@ -3857,15 +3829,14 @@ try_make_edge_direct_virtual_call (struct cgraph_edge *ie, { tree vtable; unsigned HOST_WIDE_INT offset; - tree scalar = (jfunc->type == IPA_JF_CONST) ? ipa_get_jf_constant (jfunc) - : NULL; - ipa_agg_value_set agg = ipa_agg_value_set_from_jfunc (new_root_info, - new_root, - &jfunc->agg); - tree t = ipa_find_agg_cst_for_param (&agg, scalar, - ie->indirect_info->offset, - true); - agg.release (); + tree t = NULL_TREE; + if (jfunc->type == IPA_JF_CONST) + t = ipa_find_agg_cst_from_init (ipa_get_jf_constant (jfunc), + ie->indirect_info->offset, true); + if (!t) + t = ipa_find_agg_cst_from_jfunc_items (&jfunc->agg, new_root_info, + new_root, + ie->indirect_info->offset, true); if (t && vtable_pointer_value_to_vtable (t, &vtable, &offset)) { bool can_refer; @@ -6060,21 +6031,4 @@ ipcp_transform_function (struct cgraph_node *node) } -/* Return true if OTHER describes same agg value. */ -bool -ipa_agg_value::equal_to (const ipa_agg_value &other) -{ - return offset == other.offset - && operand_equal_p (value, other.value, 0); -} - -/* Destructor also removing individual aggregate values. */ - -ipa_auto_call_arg_values::~ipa_auto_call_arg_values () -{ - ipa_release_agg_values (m_known_aggs, false); -} - - - #include "gt-ipa-prop.h" diff --git a/gcc/ipa-prop.h b/gcc/ipa-prop.h index b04c1d1e8f9..1ff3cd48a6a 100644 --- a/gcc/ipa-prop.h +++ b/gcc/ipa-prop.h @@ -190,6 +190,8 @@ struct GTY(()) ipa_agg_jump_function }; class ipcp_transformation; +class ipa_auto_call_arg_values; +class ipa_call_arg_values; /* Element of a vector describing aggregate values for a number of arguments in a particular context, be it a call or the aggregate constants that a node is @@ -224,6 +226,8 @@ public: ipa_argagg_value_list (const vec *values) : m_elts (*values) {} + ipa_argagg_value_list (const ipa_auto_call_arg_values *aavals); + ipa_argagg_value_list (const ipa_call_arg_values *gavals); ipa_argagg_value_list (const ipcp_transformation *tinfo); /* Return the aggregate constant stored for INDEX at UNIT_OFFSET, if it is @@ -243,12 +247,22 @@ public: const ipa_argagg_value *get_elt (int index, unsigned unit_offset) const; + /* Return the first item describing a constant stored for parameter with INDEX, regardless of offset or reference, or NULL if there is no such constant. */ const ipa_argagg_value *get_elt_for_index (int index) const; + /* Return true if there is an aggregate constant referring to a value passed + in or by parameter with INDEX (at any offset, whether by reference or + not). */ + + bool value_for_index_p (int index) const + { + return !!get_elt_for_index (index); + } + /* Return true if all elements present in OTHER are also present in this class. */ @@ -275,105 +289,6 @@ public: array_slice m_elts; }; -/* An element in an aggregate part describing a known value at a given offset. - All unlisted positions are assumed to be unknown and all listed values must - fulfill is_gimple_ip_invariant. */ - -struct ipa_agg_value -{ - /* The offset at which the known value is located within the aggregate. */ - HOST_WIDE_INT offset; - - /* The known constant. */ - tree value; - - /* Return true if OTHER describes same agg value. */ - bool equal_to (const ipa_agg_value &other); -}; - -/* Structure describing a set of known offset/value for aggregate. */ - -struct ipa_agg_value_set -{ - /* Description of the individual item. */ - vec items; - /* True if the data was passed by reference (as opposed to by value). */ - bool by_ref; - - /* Return true if OTHER describes same agg values. */ - bool equal_to (const ipa_agg_value_set &other) - { - if (by_ref != other.by_ref) - return false; - if (items.length () != other.items.length ()) - return false; - for (unsigned int i = 0; i < items.length (); i++) - if (!items[i].equal_to (other.items[i])) - return false; - return true; - } - - /* Return true if there is any value for aggregate. */ - bool is_empty () const - { - return items.is_empty (); - } - - ipa_agg_value_set copy () const - { - ipa_agg_value_set new_copy; - - new_copy.items = items.copy (); - new_copy.by_ref = by_ref; - - return new_copy; - } - - void release () - { - items.release (); - } -}; - -/* Return copy of a vec. */ - -static inline vec -ipa_copy_agg_values (const vec &aggs) -{ - vec aggs_copy = vNULL; - - if (!aggs.is_empty ()) - { - ipa_agg_value_set *agg; - int i; - - aggs_copy.reserve_exact (aggs.length ()); - - FOR_EACH_VEC_ELT (aggs, i, agg) - aggs_copy.quick_push (agg->copy ()); - } - - return aggs_copy; -} - -/* For vec, DO NOT call release(), use below function - instead. Because ipa_agg_value_set contains a field of vector type, we - should release this child vector in each element before reclaiming the - whole vector. */ - -static inline void -ipa_release_agg_values (vec &aggs, - bool release_vector = true) -{ - ipa_agg_value_set *agg; - int i; - - FOR_EACH_VEC_ELT (aggs, i, agg) - agg->release (); - if (release_vector) - aggs.release (); -} - /* Information about zero/non-zero bits. */ class GTY(()) ipa_bits { @@ -551,28 +466,15 @@ ipa_get_jf_ancestor_keep_null (struct ipa_jump_func *jfunc) class ipa_auto_call_arg_values { public: - ~ipa_auto_call_arg_values (); - /* If m_known_vals (vector of known "scalar" values) is sufficiantly long, return its element at INDEX, otherwise return NULL. */ tree safe_sval_at (int index) { - /* TODO: Assert non-negative index here and test. */ if ((unsigned) index < m_known_vals.length ()) return m_known_vals[index]; return NULL; } - /* If m_known_aggs is sufficiantly long, return the pointer rto its element - at INDEX, otherwise return NULL. */ - ipa_agg_value_set *safe_aggval_at (int index) - { - /* TODO: Assert non-negative index here and test. */ - if ((unsigned) index < m_known_aggs.length ()) - return &m_known_aggs[index]; - return NULL; - } - /* Vector describing known values of parameters. */ auto_vec m_known_vals; @@ -580,15 +482,22 @@ public: auto_vec m_known_contexts; /* Vector describing known aggregate values. */ - auto_vec m_known_aggs; + auto_vec m_known_aggs; /* Vector describing known value ranges of arguments. */ auto_vec m_known_value_ranges; }; +inline +ipa_argagg_value_list +::ipa_argagg_value_list (const ipa_auto_call_arg_values *aavals) + : m_elts (aavals->m_known_aggs) +{} + /* Class bundling the various potentially known properties about actual arguments of a particular call. This variant does not deallocate the - bundled data in any way. */ + bundled data in any way as the vectors can either be pointing to vectors in + ipa_auto_call_arg_values or be allocated independently. */ class ipa_call_arg_values { @@ -613,22 +522,11 @@ public: return its element at INDEX, otherwise return NULL. */ tree safe_sval_at (int index) { - /* TODO: Assert non-negative index here and test. */ if ((unsigned) index < m_known_vals.length ()) return m_known_vals[index]; return NULL; } - /* If m_known_aggs is sufficiantly long, return the pointer rto its element - at INDEX, otherwise return NULL. */ - ipa_agg_value_set *safe_aggval_at (int index) - { - /* TODO: Assert non-negative index here and test. */ - if ((unsigned) index < m_known_aggs.length ()) - return &m_known_aggs[index]; - return NULL; - } - /* Vector describing known values of parameters. */ vec m_known_vals = vNULL; @@ -636,12 +534,17 @@ public: vec m_known_contexts = vNULL; /* Vector describing known aggregate values. */ - vec m_known_aggs = vNULL; + vec m_known_aggs = vNULL; /* Vector describing known value ranges of arguments. */ vec m_known_value_ranges = vNULL; }; +inline +ipa_argagg_value_list +::ipa_argagg_value_list (const ipa_call_arg_values *gavals) + : m_elts (gavals->m_known_aggs) +{} /* Summary describing a single formal parameter. */ @@ -1190,9 +1093,6 @@ bool ipa_propagate_indirect_call_infos (struct cgraph_edge *cs, tree ipa_get_indirect_edge_target (struct cgraph_edge *ie, ipa_call_arg_values *avals, bool *speculative); -tree ipa_get_indirect_edge_target (struct cgraph_edge *ie, - ipa_auto_call_arg_values *avals, - bool *speculative); struct cgraph_edge *ipa_make_edge_direct_to_target (struct cgraph_edge *, tree, bool speculative = false); tree ipa_impossible_devirt_target (struct cgraph_edge *, tree); @@ -1204,9 +1104,8 @@ ipa_bits *ipa_get_ipa_bits_for_value (const widest_int &value, void ipa_analyze_node (struct cgraph_node *); /* Aggregate jump function related functions. */ -tree ipa_find_agg_cst_for_param (const ipa_agg_value_set *agg, tree scalar, - HOST_WIDE_INT offset, bool by_ref, - bool *from_global_constant = NULL); +tree ipa_find_agg_cst_from_init (tree scalar, HOST_WIDE_INT offset, + bool by_ref); bool ipa_load_from_parm_agg (struct ipa_func_body_info *fbi, vec *descriptors, gimple *stmt, tree op, int *index_p, @@ -1243,6 +1142,8 @@ void ipcp_read_transformation_summaries (void); int ipa_get_param_decl_index (class ipa_node_params *, tree); tree ipa_value_from_jfunc (class ipa_node_params *info, struct ipa_jump_func *jfunc, tree type); +tree ipa_agg_value_from_jfunc (ipa_node_params *info, cgraph_node *node, + const ipa_agg_jf_item *item); unsigned int ipcp_transform_function (struct cgraph_node *node); ipa_polymorphic_call_context ipa_context_from_jfunc (ipa_node_params *, cgraph_edge *, @@ -1250,9 +1151,10 @@ ipa_polymorphic_call_context ipa_context_from_jfunc (ipa_node_params *, ipa_jump_func *); value_range ipa_value_range_from_jfunc (ipa_node_params *, cgraph_edge *, ipa_jump_func *, tree); -ipa_agg_value_set ipa_agg_value_set_from_jfunc (ipa_node_params *, - cgraph_node *, - ipa_agg_jump_function *); +void ipa_push_agg_values_from_jfunc (ipa_node_params *info, cgraph_node *node, + ipa_agg_jump_function *agg_jfunc, + unsigned dst_index, + vec *res); void ipa_dump_param (FILE *, class ipa_node_params *info, int i); void ipa_release_body_info (struct ipa_func_body_info *); tree ipa_get_callee_param_type (struct cgraph_edge *e, int i); -- 2.37.2