From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 18110 invoked by alias); 13 Feb 2010 18:04:15 -0000 Received: (qmail 17002 invoked by uid 22791); 13 Feb 2010 18:04:00 -0000 X-SWARE-Spam-Status: No, hits=-4.3 required=5.0 tests=AWL,BAYES_00 X-Spam-Check-By: sourceware.org Received: from cantor2.suse.de (HELO mx2.suse.de) (195.135.220.15) by sourceware.org (qpsmtpd/0.43rc1) with ESMTP; Sat, 13 Feb 2010 18:03:51 +0000 Received: from relay1.suse.de (charybdis-ext.suse.de [195.135.221.2]) by mx2.suse.de (Postfix) with ESMTP id EA4C38891E for ; Sat, 13 Feb 2010 19:03:47 +0100 (CET) Resent-From: Martin Jambor Resent-Date: Sat, 13 Feb 2010 19:04:41 +0100 Resent-Message-ID: <20100213180441.GC6441@alvy.suse.cz> Resent-To: GCC Patches Message-Id: <20100213180157.129177066@alvy.suse.cz> User-Agent: quilt/0.48-1 Date: Sat, 13 Feb 2010 18:04:00 -0000 From: Martin Jambor To: GCC Patches Cc: Jan Hubicka Subject: [PATCH 2/6] Indirect call graph edges References: <20100213180136.555197900@alvy.suse.cz> Content-Disposition: inline; filename=indirect_cgraph_edges.diff X-IsSubscribed: yes Mailing-List: contact gcc-patches-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Archive: List-Post: List-Help: Sender: gcc-patches-owner@gcc.gnu.org X-SW-Source: 2010-02/txt/msg00515.txt.bz2 Hi, this is a re-post of the same patch I already sent here in January. There are only minor differences. I have renamed the flag indirect_call to indirect_inlining_edge and changed its purpose to be exactly that, an edge made direct by indirect inlining (it may go away completely, I currently use it only for asserts in ipa-cp devirtualization) and I have added a few things to cgraph node dumping. Most importantly, I have re-worked streaming so that information for indirect inlining associated with nodes is written to the jump functions section again in order to allow the cgraph stream to remain simple and concise (which means it cannot store trees). The patch adds a new kind of edge to the call graph to represent indirect calls with unknown callees (which we hope we can determine at some stage) and uses them to replace ipa_param_call_notes in ipa-prop.h. My immediate reason to do this is to move all the information there from the notes because I will need them to be in memory managed by the garbage collector (and did not want ipa_node_params to become garbage collected) but in a broader sense this is the right thing to do because it facilitates a more general approach to keep information about such calls (ipa-cp jump functions and the like) that is also consistent with how we do this for direct edges. The data structure layout (a flag and a pointer in struct cgraph_edge to a another special structure) was really chosen by Honza. The subsequent patches add some more stuff to cgraph_indirect_call_info so it won't look this silly for long. I have successfully bootstrapped and tested this beast on x86_84-linux and would like to commit something along these lines when stage1 opens again. Obviously, I welcome any comments. Thanks, Martin 2010-02-10 Martin Jambor * cgraph.h (struct cgraph_node): New field indirect_calls. (struct cgraph_indirect_call_info): New type. (struct cgraph_edge): Removed field indirect_call. New fields indirect_info, indirect_inlining_edge and indirect_unknown_callee. (cgraph_create_indirect_edge): Declare. (cgraph_make_edge_direct): Likewise. (enum LTO_cgraph_tags): New item LTO_cgraph_indirect_edge. * ipa-prop.h (struct ipa_param_call_note): Removed. (struct ipa_node_params): Removed field param_calls. * cgraph.c (cgraph_add_edge_to_call_site_hash): New function. (cgraph_edge): Search also among the indirect edges, use cgraph_add_edge_to_call_site_hash to add edges to the call site hash. (cgraph_set_call_stmt): Possibly turn an indirect edge into a direct one, use cgraph_add_edge_to_call_site_hash to add edges to the call site hash. (initialize_inline_failed): Assign a reason to indirect edges. (cgraph_create_edge_1): New function. (cgraph_create_edge): Moved some functionality to cgraph_create_edge_1. (cgraph_create_indirect_edge): New function. (cgraph_edge_remove_callee): Add an assert checking for non-indirectness. (cgraph_edge_remove_caller): Special-case indirect edges. (cgraph_remove_edge): Likewise. (cgraph_set_edge_callee): New function. (cgraph_redirect_edge_callee): Use cgraph_set_edge_callee. (cgraph_make_edge_direct): New function. (cgraph_update_edges_for_call_stmt_node): Do nothing only when also the declaration of the call statement matches. (cgraph_node_remove_callees): Special-case indirect edges. (cgraph_clone_edge): Likewise. (cgraph_clone_node): Clone also the indirect edges. (dump_cgraph_node): Dump indirect_inlining_edge flag instead of indirect_call, dump count of indirect_calls edges. * ipa-prop.c (ipa_note_param_call): Create indirect edges instead of creating notes. New parameter node. (ipa_analyze_call_uses): New parameter node, pass it on to ipa_note_param_call. (ipa_analyze_stmt_uses): Likewise. (ipa_analyze_params_uses): Pass node to ipa_analyze_stmt_uses. (print_edge_addition_message): Work on edges rather than on notes. (update_call_notes_after_inlining): Likewise. (ipa_free_node_params_substructures): Do not free notes. (ipa_node_duplication_hook): Do not duplicate notes. (ipa_write_param_call_note): Removed. (ipa_read_param_call_note): Removed. (ipa_write_indirect_edge_info): New function. (ipa_read_indirect_edge_info): Likewise. (ipa_write_node_info): Do not stream notes, do stream information in indirect edges. (ipa_read_node_info): Likewise. (lto_ipa_fixup_call_notes): Removed. * ipa-cp.c (pass_ipa_cp): Set stmt_fixup to NULL. * ipa-inline.c (pass_ipa_inline): Likewise. * cgraphunit.c (verify_cgraph_node): Check also indirect edges. * cif-code.def (INDIRECT_UNKNOWN_CALL): New reason. * tree-inline.c (copy_bb): Removed an unnecessary double check for is_gimple_call. * tree-inline.c (get_indirect_callee_fndecl): Do not consider indirect edges. * lto-cgraph.c (output_outgoing_cgraph_edges): New function. (output_cgraph): Stream also indirect edges. (lto_output_edge): Added capability to stream indirect edges. (input_edge): Likewise. (input_cgraph_1): Likewise. * testsuite/gcc.dg/lto/20091209-1_0.c: New testcase. Index: icln/gcc/cgraph.h =================================================================== --- icln.orig/gcc/cgraph.h +++ icln/gcc/cgraph.h @@ -184,6 +184,9 @@ struct GTY((chain_next ("%h.next"), chai struct cgraph_edge *callers; struct cgraph_node *next; struct cgraph_node *previous; + /* List of edges representing indirect calls with a yet undetermined + caller. */ + struct cgraph_edge *indirect_calls; /* For nested functions points to function the node is nested in. */ struct cgraph_node *origin; /* Points to first nested function, if any. */ @@ -311,6 +314,17 @@ typedef enum { CIF_N_REASONS } cgraph_inline_failed_t; +/* Structure containing additional information about an indirect call. */ + +struct GTY(()) cgraph_indirect_call_info +{ + /* Index of the parameter that is called. */ + int param_index; + + /* Whether this edge has already been looked at by indirect inlining. */ + unsigned int inlining_processed : 1; +}; + struct GTY((chain_next ("%h.next_caller"), chain_prev ("%h.prev_caller"))) cgraph_edge { /* Expected number of executions: calculated in profile.c. */ gcov_type count; @@ -321,6 +335,8 @@ struct GTY((chain_next ("%h.next_caller" struct cgraph_edge *prev_callee; struct cgraph_edge *next_callee; gimple call_stmt; + /* Additional information about an indirect call. Can be NULL for */ + struct cgraph_indirect_call_info *indirect_info; PTR GTY ((skip (""))) aux; /* When equal to CIF_OK, inline this call. Otherwise, points to the explanation why function was not inlined. */ @@ -336,8 +352,12 @@ struct GTY((chain_next ("%h.next_caller" int uid; /* Depth of loop nest, 1 means no loop nest. */ unsigned short int loop_nest; - /* Whether this edge describes a call that was originally indirect. */ - unsigned int indirect_call : 1; + /* Whether this edge was created by indirect inlining. */ + unsigned int indirect_inlining_edge : 1; + /* Whether this edge describes an inderect call with an undetermined + callee. */ + unsigned int indirect_unknown_callee : 1; + /* Whether this edge is still a danglig */ /* True if the corresponding CALL stmt cannot be inlined. */ unsigned int call_stmt_cannot_inline_p : 1; /* Can this call throw externally? */ @@ -435,7 +455,8 @@ void cgraph_node_remove_callees (struct struct cgraph_edge *cgraph_create_edge (struct cgraph_node *, struct cgraph_node *, gimple, gcov_type, int, int); - +struct cgraph_edge *cgraph_create_indirect_edge (struct cgraph_node *, gimple, + gcov_type, int, int); struct cgraph_node * cgraph_get_node (tree); struct cgraph_node *cgraph_node (tree); bool cgraph_same_body_alias (tree, tree); @@ -461,6 +482,7 @@ struct cgraph_node * cgraph_clone_node ( int, bool, VEC(cgraph_edge_p,heap) *); void cgraph_redirect_edge_callee (struct cgraph_edge *, struct cgraph_node *); +void cgraph_make_edge_direct (struct cgraph_edge *, struct cgraph_node *); struct cgraph_asm_node *cgraph_add_asm_node (tree); @@ -628,6 +650,7 @@ enum LTO_cgraph_tags LTO_cgraph_overwritable_node, LTO_cgraph_unavail_node, LTO_cgraph_edge, + LTO_cgraph_indirect_edge, /* edges with indirect_unknown_callee set */ LTO_cgraph_last_tag }; Index: icln/gcc/ipa-prop.h =================================================================== --- icln.orig/gcc/ipa-prop.h +++ icln/gcc/ipa-prop.h @@ -134,32 +134,6 @@ struct ipcp_lattice tree constant; }; -/* Each instance of the following structure describes a statement that calls a - function parameter. Those referring to statements within the same function - are linked in a list. */ -struct ipa_param_call_note -{ - /* Expected number of executions: calculated in profile.c. */ - gcov_type count; - /* Linked list's next */ - struct ipa_param_call_note *next; - /* Statement that contains the call to the parameter above. */ - gimple stmt; - /* When in LTO, we the above stmt will be NULL and we need an uid. */ - unsigned int lto_stmt_uid; - /* Index of the parameter that is called. */ - int formal_id; - /* Expected frequency of executions within the function. see cgraph_edge in - cgraph.h for more on this. */ - int frequency; - /* Depth of loop nest, 1 means no loop nest. */ - unsigned short int loop_nest; - /* Set when we have already found the target to be a compile time constant - and turned this into an edge or when the note was found unusable for some - reason. */ - bool processed; -}; - /* Structure describing a single formal parameter. */ struct ipa_param_descriptor { @@ -185,8 +159,6 @@ struct ipa_node_params /* Pointer to an array of structures describing individual formal parameters. */ struct ipa_param_descriptor *params; - /* List of structures enumerating calls to a formal parameter. */ - struct ipa_param_call_note *param_calls; /* Only for versioned nodes this field would not be NULL, it points to the node that IPA cp cloned from. */ struct cgraph_node *ipcp_orig_node; Index: icln/gcc/cgraph.c =================================================================== --- icln.orig/gcc/cgraph.c +++ icln/gcc/cgraph.c @@ -721,6 +721,19 @@ edge_eq (const void *x, const void *y) return ((const struct cgraph_edge *) x)->call_stmt == y; } +/* Add call graph edge E to call site hash of its caller. */ + +static inline void +cgraph_add_edge_to_call_site_hash (struct cgraph_edge *e) +{ + void **slot; + slot = htab_find_slot_with_hash (e->caller->call_site_hash, + e->call_stmt, + htab_hash_pointer (e->call_stmt), + INSERT); + gcc_assert (!*slot); + *slot = e; +} /* Return the callgraph edge representing the GIMPLE_CALL statement CALL_STMT. */ @@ -741,26 +754,28 @@ cgraph_edge (struct cgraph_node *node, g solution. It is not good idea to add pointer into CALL_EXPR itself because we want to make possible having multiple cgraph nodes representing different clones of the same body before the body is actually cloned. */ - for (e = node->callees; e; e= e->next_callee) + for (e = node->callees; e; e = e->next_callee) { if (e->call_stmt == call_stmt) break; n++; } + if (!e) + for (e = node->indirect_calls; e; e = e->next_callee) + { + if (e->call_stmt == call_stmt) + break; + n++; + } + if (n > 100) { node->call_site_hash = htab_create_ggc (120, edge_hash, edge_eq, NULL); for (e2 = node->callees; e2; e2 = e2->next_callee) - { - void **slot; - slot = htab_find_slot_with_hash (node->call_site_hash, - e2->call_stmt, - htab_hash_pointer (e2->call_stmt), - INSERT); - gcc_assert (!*slot); - *slot = e2; - } + cgraph_add_edge_to_call_site_hash (e2); + for (e2 = node->indirect_calls; e2; e2 = e2->next_callee) + cgraph_add_edge_to_call_site_hash (e2); } return e; @@ -772,26 +787,31 @@ cgraph_edge (struct cgraph_node *node, g void cgraph_set_call_stmt (struct cgraph_edge *e, gimple new_stmt) { + tree decl; + if (e->caller->call_site_hash) { htab_remove_elt_with_hash (e->caller->call_site_hash, e->call_stmt, htab_hash_pointer (e->call_stmt)); } + e->call_stmt = new_stmt; + if (e->indirect_unknown_callee + && (decl = gimple_call_fndecl (new_stmt))) + { + /* Constant propagation (and possibly also inlining?) can turn an + indirect call into a direct one. */ + struct cgraph_node *new_callee = cgraph_node (decl); + + cgraph_make_edge_direct (e, new_callee); + } + push_cfun (DECL_STRUCT_FUNCTION (e->caller->decl)); e->can_throw_external = stmt_can_throw_external (new_stmt); pop_cfun (); if (e->caller->call_site_hash) - { - void **slot; - slot = htab_find_slot_with_hash (e->caller->call_site_hash, - e->call_stmt, - htab_hash_pointer - (e->call_stmt), INSERT); - gcc_assert (!*slot); - *slot = e; - } + cgraph_add_edge_to_call_site_hash (e); } /* Like cgraph_set_call_stmt but walk the clone tree and update all @@ -893,7 +913,9 @@ initialize_inline_failed (struct cgraph_ { struct cgraph_node *callee = e->callee; - if (!callee->analyzed) + if (e->indirect_unknown_callee) + e->inline_failed = CIF_INDIRECT_UNKNOWN_CALL; + else if (!callee->analyzed) e->inline_failed = CIF_BODY_NOT_AVAILABLE; else if (callee->local.redefined_extern_inline) e->inline_failed = CIF_REDEFINED_EXTERN_INLINE; @@ -905,15 +927,16 @@ initialize_inline_failed (struct cgraph_ e->inline_failed = CIF_FUNCTION_NOT_CONSIDERED; } -/* Create edge from CALLER to CALLEE in the cgraph. */ - -struct cgraph_edge * -cgraph_create_edge (struct cgraph_node *caller, struct cgraph_node *callee, - gimple call_stmt, gcov_type count, int freq, int nest) +/* Allocate a cgraph_edge structure and fill it with data according to the + parameters ow which only CALLEE can be NULL (when creating an indirect call + edge). */ + +static struct cgraph_edge * +cgraph_create_edge_1 (struct cgraph_node *caller, struct cgraph_node *callee, + gimple call_stmt, gcov_type count, int freq, int nest) { struct cgraph_edge *edge; - /* LTO does not actually have access to the call_stmt since these have not been loaded yet. */ if (call_stmt) @@ -939,47 +962,84 @@ cgraph_create_edge (struct cgraph_node * } edge->aux = NULL; - edge->caller = caller; edge->callee = callee; + edge->prev_caller = NULL; + edge->next_caller = NULL; + edge->prev_callee = NULL; + edge->next_callee = NULL; + + edge->count = count; + gcc_assert (count >= 0); + edge->frequency = freq; + gcc_assert (freq >= 0); + gcc_assert (freq <= CGRAPH_FREQ_MAX); + edge->loop_nest = nest; + edge->call_stmt = call_stmt; push_cfun (DECL_STRUCT_FUNCTION (caller->decl)); edge->can_throw_external = call_stmt ? stmt_can_throw_external (call_stmt) : false; pop_cfun (); - edge->prev_caller = NULL; + edge->call_stmt_cannot_inline_p = + (call_stmt ? gimple_call_cannot_inline_p (call_stmt) : false); + if (call_stmt && caller->call_site_hash) + cgraph_add_edge_to_call_site_hash (edge); + + edge->indirect_info = NULL; + edge->indirect_inlining_edge = 0; + + return edge; +} + +/* Create edge from CALLER to CALLEE in the cgraph. */ + +struct cgraph_edge * +cgraph_create_edge (struct cgraph_node *caller, struct cgraph_node *callee, + gimple call_stmt, gcov_type count, int freq, int nest) +{ + struct cgraph_edge *edge = cgraph_create_edge_1 (caller, callee, call_stmt, + count, freq, nest); + + edge->indirect_unknown_callee = 0; + initialize_inline_failed (edge); + edge->next_caller = callee->callers; if (callee->callers) callee->callers->prev_caller = edge; - edge->prev_callee = NULL; edge->next_callee = caller->callees; if (caller->callees) caller->callees->prev_callee = edge; caller->callees = edge; callee->callers = edge; - edge->count = count; - gcc_assert (count >= 0); - edge->frequency = freq; - gcc_assert (freq >= 0); - gcc_assert (freq <= CGRAPH_FREQ_MAX); - edge->loop_nest = nest; - edge->indirect_call = 0; - edge->call_stmt_cannot_inline_p = - (call_stmt ? gimple_call_cannot_inline_p (call_stmt) : false); - if (call_stmt && caller->call_site_hash) - { - void **slot; - slot = htab_find_slot_with_hash (caller->call_site_hash, - edge->call_stmt, - htab_hash_pointer - (edge->call_stmt), - INSERT); - gcc_assert (!*slot); - *slot = edge; - } + return edge; +} + + +/* Create an indirect edge with a yet-undetermined callee where the call + statement destination is a formal parameter of the caller with index + PARAM_INDEX. */ + +struct cgraph_edge * +cgraph_create_indirect_edge (struct cgraph_node *caller, gimple call_stmt, + gcov_type count, int freq, int nest) +{ + struct cgraph_edge *edge = cgraph_create_edge_1 (caller, NULL, call_stmt, + count, freq, nest); + + edge->indirect_unknown_callee = 1; initialize_inline_failed (edge); + edge->indirect_info = GGC_NEW (struct cgraph_indirect_call_info); + edge->indirect_info->param_index = -1; + edge->indirect_info->inlining_processed = 0; + + edge->next_callee = caller->indirect_calls; + if (caller->indirect_calls) + caller->indirect_calls->prev_callee = edge; + caller->indirect_calls = edge; + return edge; } @@ -988,6 +1048,7 @@ cgraph_create_edge (struct cgraph_node * static inline void cgraph_edge_remove_callee (struct cgraph_edge *e) { + gcc_assert (!e->indirect_unknown_callee); if (e->prev_caller) e->prev_caller->next_caller = e->next_caller; if (e->next_caller) @@ -1006,7 +1067,12 @@ cgraph_edge_remove_caller (struct cgraph if (e->next_callee) e->next_callee->prev_callee = e->prev_callee; if (!e->prev_callee) - e->caller->callees = e->next_callee; + { + if (e->indirect_unknown_callee) + e->caller->indirect_calls = e->next_callee; + else + e->caller->callees = e->next_callee; + } if (e->caller->call_site_hash) htab_remove_elt_with_hash (e->caller->call_site_hash, e->call_stmt, @@ -1035,8 +1101,9 @@ cgraph_remove_edge (struct cgraph_edge * /* Call all edge removal hooks. */ cgraph_call_edge_removal_hooks (e); - /* Remove from callers list of the callee. */ - cgraph_edge_remove_callee (e); + if (!e->indirect_unknown_callee) + /* Remove from callers list of the callee. */ + cgraph_edge_remove_callee (e); /* Remove from callees list of the callers. */ cgraph_edge_remove_caller (e); @@ -1045,6 +1112,20 @@ cgraph_remove_edge (struct cgraph_edge * cgraph_free_edge (e); } +/* Set callee of call graph edge E and add it to the corresponding set of + callers. */ + +static void +cgraph_set_edge_callee (struct cgraph_edge *e, struct cgraph_node *n) +{ + e->prev_caller = NULL; + if (n->callers) + n->callers->prev_caller = e; + e->next_caller = n->callers; + n->callers = e; + e->callee = n; +} + /* Redirect callee of E to N. The function does not update underlying call expression. */ @@ -1055,12 +1136,37 @@ cgraph_redirect_edge_callee (struct cgra cgraph_edge_remove_callee (e); /* Insert to callers list of the new callee. */ - e->prev_caller = NULL; - if (n->callers) - n->callers->prev_caller = e; - e->next_caller = n->callers; - n->callers = e; - e->callee = n; + cgraph_set_edge_callee (e, n); +} + +/* Make an indirect EDGE with an unknown callee an ordinary edge leading to + CALLEE. */ + +void +cgraph_make_edge_direct (struct cgraph_edge *edge, struct cgraph_node *callee) +{ + edge->indirect_unknown_callee = 0; + + /* Get the edge out of the indirect edge list. */ + if (edge->prev_callee) + edge->prev_callee->next_callee = edge->next_callee; + if (edge->next_callee) + edge->next_callee->prev_callee = edge->prev_callee; + if (!edge->prev_callee) + edge->caller->indirect_calls = edge->next_callee; + + /* Put it into the normal callee list */ + edge->prev_callee = NULL; + edge->next_callee = edge->caller->callees; + if (edge->caller->callees) + edge->caller->callees->prev_callee = edge; + edge->caller->callees = edge; + + /* Insert to callers list of the new callee. */ + cgraph_set_edge_callee (edge, callee); + + /* We need to re-determine the inlining status of the edge. */ + initialize_inline_failed (edge); } @@ -1091,7 +1197,7 @@ cgraph_update_edges_for_call_stmt_node ( { /* See if the call is already there. It might be because of indirect inlining already found it. */ - if (new_call && e->callee->decl == new_call) + if (new_call && e->callee && e->callee->decl == new_call) return; /* Otherwise remove edge and create new one; we can't simply redirect @@ -1169,7 +1275,8 @@ cgraph_node_remove_callees (struct cgrap { f = e->next_callee; cgraph_call_edge_removal_hooks (e); - cgraph_edge_remove_callee (e); + if (!e->indirect_unknown_callee) + cgraph_edge_remove_callee (e); cgraph_free_edge (e); } node->callees = NULL; @@ -1624,6 +1731,8 @@ void dump_cgraph_node (FILE *f, struct cgraph_node *node) { struct cgraph_edge *edge; + int indirect_calls_count = 0; + fprintf (f, "%s/%i(%i)", cgraph_node_name (node), node->uid, node->pid); dump_addr (f, " @", (void *)node); @@ -1697,8 +1806,8 @@ dump_cgraph_node (FILE *f, struct cgraph edge->frequency / (double)CGRAPH_FREQ_BASE); if (!edge->inline_failed) fprintf(f, "(inlined) "); - if (edge->indirect_call) - fprintf(f, "(indirect) "); + if (edge->indirect_inlining_edge) + fprintf(f, "(indirect_inlining) "); if (edge->can_throw_external) fprintf(f, "(can throw external) "); } @@ -1710,8 +1819,8 @@ dump_cgraph_node (FILE *f, struct cgraph edge->callee->uid); if (!edge->inline_failed) fprintf(f, "(inlined) "); - if (edge->indirect_call) - fprintf(f, "(indirect) "); + if (edge->indirect_inlining_edge) + fprintf(f, "(indirect_inlining) "); if (edge->count) fprintf (f, "("HOST_WIDEST_INT_PRINT_DEC"x) ", (HOST_WIDEST_INT)edge->count); @@ -1725,6 +1834,12 @@ dump_cgraph_node (FILE *f, struct cgraph } fprintf (f, "\n"); + for (edge = node->indirect_calls; edge; edge = edge->next_callee) + indirect_calls_count++; + if (indirect_calls_count) + fprintf (f, " has %i outgoing edges for indirect calls.\n", + indirect_calls_count); + if (node->same_body) { struct cgraph_node *n; @@ -1844,11 +1959,30 @@ cgraph_clone_edge (struct cgraph_edge *e freq = e->frequency * (gcov_type) freq_scale / CGRAPH_FREQ_BASE; if (freq > CGRAPH_FREQ_MAX) freq = CGRAPH_FREQ_MAX; - new_edge = cgraph_create_edge (n, e->callee, call_stmt, count, freq, - e->loop_nest + loop_nest); + + if (e->indirect_unknown_callee) + { + tree decl; + + if (call_stmt && (decl = gimple_call_fndecl (call_stmt))) + { + struct cgraph_node *callee = cgraph_node (decl); + new_edge = cgraph_create_edge (n, callee, call_stmt, count, freq, + e->loop_nest + loop_nest); + } + else + { + new_edge = cgraph_create_indirect_edge (n, call_stmt, count, freq, + e->loop_nest + loop_nest); + new_edge->indirect_info->param_index = e->indirect_info->param_index; + } + } + else + new_edge = cgraph_create_edge (n, e->callee, call_stmt, count, freq, + e->loop_nest + loop_nest); new_edge->inline_failed = e->inline_failed; - new_edge->indirect_call = e->indirect_call; + new_edge->indirect_inlining_edge = e->indirect_inlining_edge; new_edge->lto_stmt_uid = stmt_uid; if (update_original) { @@ -1918,6 +2052,10 @@ cgraph_clone_node (struct cgraph_node *n cgraph_clone_edge (e, new_node, e->call_stmt, e->lto_stmt_uid, count_scale, freq, loop_nest, update_original); + for (e = n->indirect_calls; e; e = e->next_callee) + cgraph_clone_edge (e, new_node, e->call_stmt, e->lto_stmt_uid, + count_scale, freq, loop_nest, update_original); + new_node->next_sibling_clone = n->clones; if (n->clones) n->clones->prev_sibling_clone = new_node; Index: icln/gcc/ipa-prop.c =================================================================== --- icln.orig/gcc/ipa-prop.c +++ icln/gcc/ipa-prop.c @@ -735,41 +735,36 @@ ipa_is_ssa_with_stmt_def (tree t) return false; } -/* Creates a new note describing a call to a parameter number FORMAL_ID and - attaches it to the linked list of INFO. It also sets the called flag of the - parameter. STMT is the corresponding call statement. */ +/* Create a new indirect call graph edge describing a call to a parameter + number FORMAL_ID and and set the called flag of the parameter. NODE is the + caller and is described by INFO. STMT is the corresponding call + statement. */ static void -ipa_note_param_call (struct ipa_node_params *info, int formal_id, - gimple stmt) +ipa_note_param_call (struct cgraph_node *node, struct ipa_node_params *info, + int formal_id, gimple stmt) { - struct ipa_param_call_note *note; + struct cgraph_edge *cs; basic_block bb = gimple_bb (stmt); + int freq; info->params[formal_id].called = 1; - note = XCNEW (struct ipa_param_call_note); - note->formal_id = formal_id; - note->stmt = stmt; - note->lto_stmt_uid = gimple_uid (stmt); - note->count = bb->count; - note->frequency = compute_call_stmt_bb_frequency (current_function_decl, bb); - note->loop_nest = bb->loop_depth; - - note->next = info->param_calls; - info->param_calls = note; - - return; + freq = compute_call_stmt_bb_frequency (current_function_decl, bb); + cs = cgraph_create_indirect_edge (node, stmt, bb->count, freq, + bb->loop_depth); + cs->indirect_info->param_index = formal_id; } -/* Analyze the CALL and examine uses of formal parameters of the caller +/* Analyze the CALL and examine uses of formal parameters of the caller NODE (described by INFO). Currently it checks whether the call calls a pointer that is a formal parameter and if so, the parameter is marked with the - called flag and a note describing the call is created. This is very simple - for ordinary pointers represented in SSA but not-so-nice when it comes to - member pointers. The ugly part of this function does nothing more than - tries to match the pattern of such a call. An example of such a pattern is - the gimple dump below, the call is on the last line: + called flag and a an indirect call graph edge describing the call is + created. This is very simple for ordinary pointers represented in SSA but + not-so-nice when it comes to member pointers. The ugly part of this + function does nothing more than tries to match the pattern of such a call. + An example of such a pattern is the gimple dump below, the call is on the + last line: : f$__delta_5 = f.__delta; @@ -809,7 +804,8 @@ ipa_note_param_call (struct ipa_node_par */ static void -ipa_analyze_call_uses (struct ipa_node_params *info, gimple call) +ipa_analyze_call_uses (struct cgraph_node *node, struct ipa_node_params *info, + gimple call) { tree target = gimple_call_fn (call); gimple def; @@ -830,7 +826,7 @@ ipa_analyze_call_uses (struct ipa_node_p /* assuming TREE_CODE (var) == PARM_DECL */ index = ipa_get_param_decl_index (info, var); if (index >= 0) - ipa_note_param_call (info, index, call); + ipa_note_param_call (node, info, index, call); return; } @@ -927,20 +923,21 @@ ipa_analyze_call_uses (struct ipa_node_p index = ipa_get_param_decl_index (info, rec); if (index >= 0 && !ipa_is_param_modified (info, index)) - ipa_note_param_call (info, index, call); + ipa_note_param_call (node, info, index, call); return; } -/* Analyze the statement STMT with respect to formal parameters (described in - INFO) and their uses. Currently it only checks whether formal parameters - are called. */ +/* Analyze the call statement STMT with respect to formal parameters (described + in INFO) of caller given by NODE. Currently it only checks whether formal + parameters are called. */ static void -ipa_analyze_stmt_uses (struct ipa_node_params *info, gimple stmt) +ipa_analyze_stmt_uses (struct cgraph_node *node, struct ipa_node_params *info, + gimple stmt) { if (is_gimple_call (stmt)) - ipa_analyze_call_uses (info, stmt); + ipa_analyze_call_uses (node, info, stmt); } /* Scan the function body of NODE and inspect the uses of formal parameters. @@ -965,7 +962,7 @@ ipa_analyze_params_uses (struct cgraph_n for (gsi = gsi_start_bb (bb); !gsi_end_p (gsi); gsi_next (&gsi)) { gimple stmt = gsi_stmt (gsi); - ipa_analyze_stmt_uses (info, stmt); + ipa_analyze_stmt_uses (node, info, stmt); } } @@ -1021,9 +1018,8 @@ update_jump_functions_after_inlining (st by JFUNC. NODE is the node where the call is. */ static void -print_edge_addition_message (FILE *f, struct ipa_param_call_note *nt, - struct ipa_jump_func *jfunc, - struct cgraph_node *node) +print_edge_addition_message (FILE *f, struct cgraph_edge *e, + struct ipa_jump_func *jfunc) { fprintf (f, "ipa-prop: Discovered an indirect call to a known target ("); if (jfunc->type == IPA_JF_CONST_MEMBER_PTR) @@ -1034,8 +1030,8 @@ print_edge_addition_message (FILE *f, st else print_node_brief(f, "", jfunc->value.constant, 0); - fprintf (f, ") in %s: ", cgraph_node_name (node)); - print_gimple_stmt (f, nt->stmt, 2, TDF_SLIM); + fprintf (f, ") in %s: ", cgraph_node_name (e->caller)); + print_gimple_stmt (f, e->call_stmt, 2, TDF_SLIM); } /* Update the param called notes associated with NODE when CS is being inlined, @@ -1049,37 +1045,43 @@ update_call_notes_after_inlining (struct struct cgraph_node *node, VEC (cgraph_edge_p, heap) **new_edges) { - struct ipa_node_params *info = IPA_NODE_REF (node); struct ipa_edge_args *top = IPA_EDGE_REF (cs); - struct ipa_param_call_note *nt; + struct cgraph_edge *ie, *next_ie; bool res = false; - for (nt = info->param_calls; nt; nt = nt->next) + ipa_check_create_edge_args (); + + for (ie = node->indirect_calls; ie; ie = next_ie) { + struct cgraph_indirect_call_info *ici = ie->indirect_info; struct ipa_jump_func *jfunc; - if (nt->processed) + next_ie = ie->next_callee; + if (ici->inlining_processed) continue; + /* If we ever use indirect edges for anything other than indirect + inlining, we will need to skip those with negative param_indices. */ + gcc_assert (ici->param_index >= 0); + /* We must check range due to calls with variable number of arguments: */ - if (nt->formal_id >= ipa_get_cs_argument_count (top)) + if (ici->param_index >= ipa_get_cs_argument_count (top)) { - nt->processed = true; + ici->inlining_processed = true; continue; } - jfunc = ipa_get_ith_jump_func (top, nt->formal_id); + jfunc = ipa_get_ith_jump_func (top, ici->param_index); if (jfunc->type == IPA_JF_PASS_THROUGH && jfunc->value.pass_through.operation == NOP_EXPR) - nt->formal_id = jfunc->value.pass_through.formal_id; + ici->param_index = jfunc->value.pass_through.formal_id; else if (jfunc->type == IPA_JF_CONST || jfunc->type == IPA_JF_CONST_MEMBER_PTR) { struct cgraph_node *callee; - struct cgraph_edge *new_indirect_edge; tree decl; - nt->processed = true; + ici->inlining_processed = true; if (jfunc->type == IPA_JF_CONST_MEMBER_PTR) decl = jfunc->value.member_cst.pfn; else @@ -1097,16 +1099,12 @@ update_call_notes_after_inlining (struct res = true; if (dump_file) - print_edge_addition_message (dump_file, nt, jfunc, node); + print_edge_addition_message (dump_file, ie, jfunc); - new_indirect_edge = cgraph_create_edge (node, callee, nt->stmt, - nt->count, nt->frequency, - nt->loop_nest); - new_indirect_edge->lto_stmt_uid = nt->lto_stmt_uid; - new_indirect_edge->indirect_call = 1; - ipa_check_create_edge_args (); + cgraph_make_edge_direct (ie, callee); + ie->indirect_inlining_edge = 1; if (new_edges) - VEC_safe_push (cgraph_edge_p, heap, *new_edges, new_indirect_edge); + VEC_safe_push (cgraph_edge_p, heap, *new_edges, ie); top = IPA_EDGE_REF (cs); } else @@ -1114,9 +1112,10 @@ update_call_notes_after_inlining (struct /* Ancestor jum functions and pass theoughs with operations should not be used on parameters that then get called. */ gcc_assert (jfunc->type == IPA_JF_UNKNOWN); - nt->processed = true; + ici->inlining_processed = true; } } + return res; } @@ -1208,13 +1207,6 @@ ipa_free_node_params_substructures (stru if (info->params) free (info->params); - while (info->param_calls) - { - struct ipa_param_call_note *note = info->param_calls; - info->param_calls = note->next; - free (note); - } - memset (info, 0, sizeof (*info)); } @@ -1314,7 +1306,6 @@ ipa_node_duplication_hook (struct cgraph __attribute__((unused)) void *data) { struct ipa_node_params *old_info, *new_info; - struct ipa_param_call_note *note; int param_count; ipa_check_create_node_params (); @@ -1328,17 +1319,6 @@ ipa_node_duplication_hook (struct cgraph sizeof (struct ipa_param_descriptor) * param_count); new_info->ipcp_orig_node = old_info->ipcp_orig_node; new_info->count_scale = old_info->count_scale; - - for (note = old_info->param_calls; note; note = note->next) - { - struct ipa_param_call_note *nn; - - nn = (struct ipa_param_call_note *) - xcalloc (1, sizeof (struct ipa_param_call_note)); - memcpy (nn, note, sizeof (struct ipa_param_call_note)); - nn->next = new_info->param_calls; - new_info->param_calls = nn; - } } /* Register our cgraph hooks if they are not already there. */ @@ -1964,40 +1944,40 @@ ipa_read_jump_function (struct lto_input } } -/* Stream out a parameter call note. */ +/* Stream out parts of cgraph_indirect_call_info corresponding to CS that are + relevant to indirect inlining to OB. */ static void -ipa_write_param_call_note (struct output_block *ob, - struct ipa_param_call_note *note) +ipa_write_indirect_edge_info (struct output_block *ob, + struct cgraph_edge *cs) { - gcc_assert (!note->processed); - lto_output_uleb128_stream (ob->main_stream, gimple_uid (note->stmt)); - lto_output_sleb128_stream (ob->main_stream, note->formal_id); - lto_output_sleb128_stream (ob->main_stream, note->count); - lto_output_sleb128_stream (ob->main_stream, note->frequency); - lto_output_sleb128_stream (ob->main_stream, note->loop_nest); + struct cgraph_indirect_call_info *ii = cs->indirect_info; + struct bitpack_d *bp; + + lto_output_sleb128_stream (ob->main_stream, ii->param_index); + bp = bitpack_create (); + bp_pack_value (bp, ii->inlining_processed, 1); + lto_output_bitpack (ob->main_stream, bp); + bitpack_delete (bp); } -/* Read in a parameter call note. */ +/* Read in parts of cgraph_indirect_call_info corresponding to CS that are + relevant to indirect inlining from IB. */ static void -ipa_read_param_call_note (struct lto_input_block *ib, - struct ipa_node_params *info) - +ipa_read_indirect_edge_info (struct lto_input_block *ib, + struct data_in *data_in ATTRIBUTE_UNUSED, + struct cgraph_edge *cs) { - struct ipa_param_call_note *note = XCNEW (struct ipa_param_call_note); - - note->lto_stmt_uid = (unsigned int) lto_input_uleb128 (ib); - note->formal_id = (int) lto_input_sleb128 (ib); - note->count = (gcov_type) lto_input_sleb128 (ib); - note->frequency = (int) lto_input_sleb128 (ib); - note->loop_nest = (int) lto_input_sleb128 (ib); + struct cgraph_indirect_call_info *ii = cs->indirect_info; + struct bitpack_d *bp; - note->next = info->param_calls; - info->param_calls = note; + ii->param_index = (int) lto_input_sleb128 (ib); + bp = lto_input_bitpack (ib); + ii->inlining_processed = bp_unpack_value (bp, 1); + bitpack_delete (bp); } - /* Stream out NODE info to OB. */ static void @@ -2009,8 +1989,6 @@ ipa_write_node_info (struct output_block int j; struct cgraph_edge *e; struct bitpack_d *bp; - int note_count = 0; - struct ipa_param_call_note *note; encoder = ob->decl_state->cgraph_node_encoder; node_ref = lto_cgraph_encoder_encode (encoder, node); @@ -2024,10 +2002,7 @@ ipa_write_node_info (struct output_block gcc_assert (!info->node_enqueued); gcc_assert (!info->ipcp_orig_node); for (j = 0; j < ipa_get_param_count (info); j++) - { - bp_pack_value (bp, info->params[j].modified, 1); - bp_pack_value (bp, info->params[j].called, 1); - } + bp_pack_value (bp, info->params[j].modified, 1); lto_output_bitpack (ob->main_stream, bp); bitpack_delete (bp); for (e = node->callees; e; e = e->next_callee) @@ -2039,12 +2014,8 @@ ipa_write_node_info (struct output_block for (j = 0; j < ipa_get_cs_argument_count (args); j++) ipa_write_jump_function (ob, ipa_get_ith_jump_func (args, j)); } - - for (note = info->param_calls; note; note = note->next) - note_count++; - lto_output_uleb128_stream (ob->main_stream, note_count); - for (note = info->param_calls; note; note = note->next) - ipa_write_param_call_note (ob, note); + for (e = node->indirect_calls; e; e = e->next_callee) + ipa_write_indirect_edge_info (ob, e); } /* Srtream in NODE info from IB. */ @@ -2057,7 +2028,6 @@ ipa_read_node_info (struct lto_input_blo int k; struct cgraph_edge *e; struct bitpack_d *bp; - int i, note_count; ipa_initialize_node_params (node); @@ -2071,10 +2041,7 @@ ipa_read_node_info (struct lto_input_blo } info->node_enqueued = false; for (k = 0; k < ipa_get_param_count (info); k++) - { - info->params[k].modified = bp_unpack_value (bp, 1); - info->params[k].called = bp_unpack_value (bp, 1); - } + info->params[k].modified = bp_unpack_value (bp, 1); bitpack_delete (bp); for (e = node->callees; e; e = e->next_callee) { @@ -2090,10 +2057,8 @@ ipa_read_node_info (struct lto_input_blo for (k = 0; k < ipa_get_cs_argument_count (args); k++) ipa_read_jump_function (ib, ipa_get_ith_jump_func (args, k), data_in); } - - note_count = lto_input_uleb128 (ib); - for (i = 0; i < note_count; i++) - ipa_read_param_call_note (ib, info); + for (e = node->indirect_calls; e; e = e->next_callee) + ipa_read_indirect_edge_info (ib, data_in, e); } /* Write jump functions for nodes in SET. */ @@ -2218,29 +2183,3 @@ ipa_update_after_lto_read (void) } } } - -/* Walk param call notes of NODE and set their call statements given the uid - stored in each note and STMTS which is an array of statements indexed by the - uid. */ - -void -lto_ipa_fixup_call_notes (struct cgraph_node *node, gimple *stmts) -{ - struct ipa_node_params *info; - struct ipa_param_call_note *note; - - ipa_check_create_node_params (); - info = IPA_NODE_REF (node); - note = info->param_calls; - /* If there are no notes or they have already been fixed up (the same fixup - is called for both inlining and ipa-cp), there's nothing to do. */ - if (!note || note->stmt) - return; - - do - { - note->stmt = stmts[note->lto_stmt_uid]; - note = note->next; - } - while (note); -} Index: icln/gcc/ipa-cp.c =================================================================== --- icln.orig/gcc/ipa-cp.c +++ icln/gcc/ipa-cp.c @@ -1332,7 +1332,7 @@ struct ipa_opt_pass_d pass_ipa_cp = ipcp_write_summary, /* write_summary */ ipcp_read_summary, /* read_summary */ NULL, /* function_read_summary */ - lto_ipa_fixup_call_notes, /* stmt_fixup */ + NULL, /* stmt_fixup */ 0, /* TODOs */ NULL, /* function_transform */ NULL, /* variable_transform */ Index: icln/gcc/ipa-inline.c =================================================================== --- icln.orig/gcc/ipa-inline.c +++ icln/gcc/ipa-inline.c @@ -2066,7 +2066,7 @@ struct ipa_opt_pass_d pass_ipa_inline = inline_write_summary, /* write_summary */ inline_read_summary, /* read_summary */ NULL, /* function_read_summary */ - lto_ipa_fixup_call_notes, /* stmt_fixup */ + NULL, /* stmt_fixup */ 0, /* TODOs */ inline_transform, /* function_transform */ NULL, /* variable_transform */ Index: icln/gcc/cgraphunit.c =================================================================== --- icln.orig/gcc/cgraphunit.c +++ icln/gcc/cgraphunit.c @@ -607,6 +607,24 @@ verify_cgraph_node (struct cgraph_node * error ("Inline clone is needed"); error_found = true; } + for (e = node->indirect_calls; e; e = e->next_callee) + { + if (e->aux) + { + error ("aux field set for indirect edge from %s", + identifier_to_locale (cgraph_node_name (e->caller))); + error_found = true; + } + if (!e->indirect_unknown_callee + || !e->indirect_info) + { + error ("An indirect edge from %s is not marked as indirect or has " + "associated indirect_info, the corresponding statement is: ", + identifier_to_locale (cgraph_node_name (e->caller))); + debug_gimple_stmt (e->call_stmt); + error_found = true; + } + } for (e = node->callers; e; e = e->next_caller) { if (e->count < 0) @@ -733,10 +751,10 @@ verify_cgraph_node (struct cgraph_node * gsi_next (&gsi)) { gimple stmt = gsi_stmt (gsi); - tree decl; - if (is_gimple_call (stmt) && (decl = gimple_call_fndecl (stmt))) + if (is_gimple_call (stmt)) { struct cgraph_edge *e = cgraph_edge (node, stmt); + tree decl = gimple_call_fndecl (stmt); if (e) { if (e->aux) @@ -745,25 +763,38 @@ verify_cgraph_node (struct cgraph_node * debug_gimple_stmt (stmt); error_found = true; } - if (e->callee->same_body_alias) + if (!e->indirect_unknown_callee) { - error ("edge points to same body alias:"); - debug_tree (e->callee->decl); - error_found = true; + if (e->callee->same_body_alias) + { + error ("edge points to same body alias:"); + debug_tree (e->callee->decl); + error_found = true; + } + else if (!node->global.inlined_to + && !e->callee->global.inlined_to + && decl + && !clone_of_p (cgraph_node (decl), + e->callee)) + { + error ("edge points to wrong declaration:"); + debug_tree (e->callee->decl); + fprintf (stderr," Instead of:"); + debug_tree (decl); + error_found = true; + } } - else if (!node->global.inlined_to - && !e->callee->global.inlined_to - && !clone_of_p (cgraph_node (decl), e->callee)) + else if (decl) { - error ("edge points to wrong declaration:"); - debug_tree (e->callee->decl); - fprintf (stderr," Instead of:"); - debug_tree (decl); + error ("an indirect edge with unknown callee " + "corresponding to a call_stmt with " + "a known declaration:"); error_found = true; + debug_gimple_stmt (e->call_stmt); } e->aux = (void *)1; } - else + else if (decl) { error ("missing callgraph edge for call stmt:"); debug_gimple_stmt (stmt); @@ -779,7 +810,7 @@ verify_cgraph_node (struct cgraph_node * for (e = node->callees; e; e = e->next_callee) { - if (!e->aux && !e->indirect_call) + if (!e->aux) { error ("edge %s->%s has no corresponding call_stmt", identifier_to_locale (cgraph_node_name (e->caller)), @@ -787,6 +818,17 @@ verify_cgraph_node (struct cgraph_node * debug_gimple_stmt (e->call_stmt); error_found = true; } + e->aux = 0; + } + for (e = node->indirect_calls; e; e = e->next_callee) + { + if (!e->aux) + { + error ("an indirect edge from %s has no corresponding call_stmt", + identifier_to_locale (cgraph_node_name (e->caller))); + debug_gimple_stmt (e->call_stmt); + error_found = true; + } e->aux = 0; } } Index: icln/gcc/cif-code.def =================================================================== --- icln.orig/gcc/cif-code.def +++ icln/gcc/cif-code.def @@ -84,3 +84,7 @@ DEFCIFCODE(MISMATCHED_ARGUMENTS, N_("mis /* Call was originally indirect. */ DEFCIFCODE(ORIGINALLY_INDIRECT_CALL, N_("originally indirect function call not considered for inlining")) + +/* Ths edge represents an indirect edge with a yet-undetermined callee . */ +DEFCIFCODE(INDIRECT_UNKNOWN_CALL, + N_("indirect function call with a yet undetermined callee")) Index: icln/gcc/tree-inline.c =================================================================== --- icln.orig/gcc/tree-inline.c +++ icln/gcc/tree-inline.c @@ -1674,9 +1674,8 @@ copy_bb (copy_body_data *id, basic_block /* Constant propagation on argument done during inlining may create new direct call. Produce an edge for it. */ if ((!edge - || (edge->indirect_call + || (edge->indirect_inlining_edge && id->transform_call_graph_edges == CB_CGE_MOVE_CLONES)) - && is_gimple_call (stmt) && (fn = gimple_call_fndecl (stmt)) != NULL) { struct cgraph_node *dest = cgraph_node (fn); @@ -3452,7 +3451,7 @@ get_indirect_callee_fndecl (struct cgrap struct cgraph_edge *cs; cs = cgraph_edge (node, stmt); - if (cs) + if (cs && !cs->indirect_unknown_callee) return cs->callee->decl; return NULL_TREE; @@ -3535,7 +3534,7 @@ expand_call_inline (basic_block bb, gimp /* If this call was originally indirect, we do not want to emit any inlining related warnings or sorry messages because there are no guarantees regarding those. */ - if (cg_edge->indirect_call) + if (cg_edge->indirect_inlining_edge) goto egress; if (lookup_attribute ("always_inline", DECL_ATTRIBUTES (fn)) Index: icln/gcc/lto-cgraph.c =================================================================== --- icln.orig/gcc/lto-cgraph.c +++ icln/gcc/lto-cgraph.c @@ -139,15 +139,21 @@ lto_output_edge (struct lto_simple_outpu intptr_t ref; struct bitpack_d *bp; - lto_output_uleb128_stream (ob->main_stream, LTO_cgraph_edge); + if (edge->indirect_unknown_callee) + lto_output_uleb128_stream (ob->main_stream, LTO_cgraph_indirect_edge); + else + lto_output_uleb128_stream (ob->main_stream, LTO_cgraph_edge); ref = lto_cgraph_encoder_lookup (encoder, edge->caller); gcc_assert (ref != LCC_NOT_FOUND); lto_output_sleb128_stream (ob->main_stream, ref); - ref = lto_cgraph_encoder_lookup (encoder, edge->callee); - gcc_assert (ref != LCC_NOT_FOUND); - lto_output_sleb128_stream (ob->main_stream, ref); + if (!edge->indirect_unknown_callee) + { + ref = lto_cgraph_encoder_lookup (encoder, edge->callee); + gcc_assert (ref != LCC_NOT_FOUND); + lto_output_sleb128_stream (ob->main_stream, ref); + } lto_output_sleb128_stream (ob->main_stream, edge->count); @@ -157,7 +163,7 @@ lto_output_edge (struct lto_simple_outpu bp_pack_value (bp, edge->inline_failed, HOST_BITS_PER_INT); bp_pack_value (bp, edge->frequency, HOST_BITS_PER_INT); bp_pack_value (bp, edge->loop_nest, 30); - bp_pack_value (bp, edge->indirect_call, 1); + bp_pack_value (bp, edge->indirect_inlining_edge, 1); bp_pack_value (bp, edge->call_stmt_cannot_inline_p, 1); bp_pack_value (bp, edge->can_throw_external, 1); lto_output_bitpack (ob->main_stream, bp); @@ -371,6 +377,24 @@ output_profile_summary (struct lto_simpl lto_output_uleb128_stream (ob->main_stream, 0); } +/* Output all callees or indirect outgoing edges. EDGE must be the first such + edge. */ + +static void +output_outgoing_cgraph_edges (struct cgraph_edge *edge, + struct lto_simple_output_block *ob, + lto_cgraph_encoder_t encoder) +{ + if (!edge) + return; + + /* Output edges in backward direction, so the reconstructed callgraph match + and it is easy to associate call sites in the IPA pass summaries. */ + while (edge->next_callee) + edge = edge->next_callee; + for (; edge; edge = edge->prev_callee) + lto_output_edge (ob, edge, encoder); +} /* Output the part of the cgraph in SET. */ @@ -460,16 +484,8 @@ output_cgraph (cgraph_node_set set) for (csi = csi_start (set); !csi_end_p (csi); csi_next (&csi)) { node = csi_node (csi); - if (node->callees) - { - /* Output edges in backward direction, so the reconstructed callgraph - match and it is easy to associate call sites in the IPA pass summaries. */ - edge = node->callees; - while (edge->next_callee) - edge = edge->next_callee; - for (; edge; edge = edge->prev_callee) - lto_output_edge (ob, edge, encoder); - } + output_outgoing_cgraph_edges (node->callees, ob, encoder); + output_outgoing_cgraph_edges (node->indirect_calls, ob, encoder); } lto_output_uleb128_stream (ob->main_stream, 0); @@ -657,11 +673,14 @@ input_node (struct lto_file_decl_data *f } -/* Read an edge from IB. NODES points to a vector of previously read - nodes for decoding caller and callee of the edge to be read. */ +/* Read an edge from IB. NODES points to a vector of previously read nodes for + decoding caller and callee of the edge to be read. If INDIRECT is true, the + edge being read is indirect (in the sense that it has + indirect_unknown_callee set). */ static void -input_edge (struct lto_input_block *ib, VEC(cgraph_node_ptr, heap) *nodes) +input_edge (struct lto_input_block *ib, VEC(cgraph_node_ptr, heap) *nodes, + bool indirect) { struct cgraph_node *caller, *callee; struct cgraph_edge *edge; @@ -677,9 +696,14 @@ input_edge (struct lto_input_block *ib, if (caller == NULL || caller->decl == NULL_TREE) internal_error ("bytecode stream: no caller found while reading edge"); - callee = VEC_index (cgraph_node_ptr, nodes, lto_input_sleb128 (ib)); - if (callee == NULL || callee->decl == NULL_TREE) - internal_error ("bytecode stream: no callee found while reading edge"); + if (!indirect) + { + callee = VEC_index (cgraph_node_ptr, nodes, lto_input_sleb128 (ib)); + if (callee == NULL || callee->decl == NULL_TREE) + internal_error ("bytecode stream: no callee found while reading edge"); + } + else + callee = NULL; count = (gcov_type) lto_input_sleb128 (ib); @@ -697,10 +721,14 @@ input_edge (struct lto_input_block *ib, || caller_resolution == LDPR_PREEMPTED_IR) return; - edge = cgraph_create_edge (caller, callee, NULL, count, freq, nest); + if (indirect) + edge = cgraph_create_indirect_edge (caller, NULL, count, freq, nest); + else + edge = cgraph_create_edge (caller, callee, NULL, count, freq, nest); + + edge->indirect_inlining_edge = bp_unpack_value (bp, 1); edge->lto_stmt_uid = stmt_id; edge->inline_failed = inline_failed; - edge->indirect_call = bp_unpack_value (bp, 1); edge->call_stmt_cannot_inline_p = bp_unpack_value (bp, 1); edge->can_throw_external = bp_unpack_value (bp, 1); bitpack_delete (bp); @@ -723,7 +751,9 @@ input_cgraph_1 (struct lto_file_decl_dat while (tag) { if (tag == LTO_cgraph_edge) - input_edge (ib, nodes); + input_edge (ib, nodes, false); + else if (tag == LTO_cgraph_indirect_edge) + input_edge (ib, nodes, true); else { node = input_node (file_data, ib, tag); Index: icln/gcc/lto-streamer-in.c =================================================================== --- icln.orig/gcc/lto-streamer-in.c +++ icln/gcc/lto-streamer-in.c @@ -1220,6 +1220,8 @@ fixup_call_stmt_edges_1 (struct cgraph_n struct cgraph_edge *cedge; for (cedge = node->callees; cedge; cedge = cedge->next_callee) cedge->call_stmt = stmts[cedge->lto_stmt_uid]; + for (cedge = node->indirect_calls; cedge; cedge = cedge->next_callee) + cedge->call_stmt = stmts[cedge->lto_stmt_uid]; } /* Fixup call_stmt pointers in NODE and all clones. */ Index: icln/gcc/testsuite/gcc.dg/lto/20091209-1_0.c =================================================================== --- /dev/null +++ icln/gcc/testsuite/gcc.dg/lto/20091209-1_0.c @@ -0,0 +1,23 @@ +/* Stream an indirect edge in and out. */ + +/* { dg-lto-do link } */ +/* { dg-lto-options {{ -O3 -fno-early-inlining -flto }} } */ + +volatile int something; + +static void hooray () +{ + something = 1; +} + +static void hiphip (void (*f)()) +{ + something = 2; + f (); +} + +int main (int argc, int *argv[]) +{ + hiphip (hooray); + return 0; +}