[gcc(refs/users/marxin/heads/pgo-reproducibility-test)] ipa: fix handling of multiple speculations (PR93318)

[Martin Liska <marxin@gcc.gnu.org>]

https://gcc.gnu.org/g:40d240315a6e5ff2010769bfa1c19264217b5f1c

commit 40d240315a6e5ff2010769bfa1c19264217b5f1c
Author: Jan Hubicka <jh@suse.cz>
Date:   Tue Jan 28 21:30:14 2020 +0100

    ipa: fix handling of multiple speculations (PR93318)
    
    This patch started as work to resole Richard's comment on quadratic lookups
    in resolve_speculation. While doing it I however noticed multiple problems
    in the new speuclative call code which made the patch quite big. In
    particular:
     1) Before applying speculation we consider only targets with at lest
        probability 1/2.
        If profile is sane at most two targets can have probability greater or
        equal to 1/2. So the new multi-target speculation code got enabled only
        in very special scenario when there ae precisely two target with precise
        probability 1/2 (which is tested by the single testcase).
    
        As a conseuqence the multiple target logic got minimal test coverage and
        this made us to miss several ICEs.
     2) Profile updating in profile merging, tree-inline and indirect call
        expansion was wrong which led to inconsistent profiles (as already seen
        on the testcase).
     3) Code responsible to turn speculative call to direct call was broken for
        anything with more than one target.
     4) There were multiple cases where call_site_hash went out of sync which
        eventually leads to an ICE..
     5) Some code expects that all speculative call targets forms a sequence in
        the callee linked list but there is no code to maintain that invariant
        nor a verifier.
    Fixing this it became obvious that the current API of speculative_call_info is
    not useful because it really builds on fact tht there are precisely three
    components (direct call, ref and indirect call) in every speculative call
    sequence.  I ended up replacing it with iterator API for direct call
    (first_speculative_call_target, next_speculative_call_target) and accessors for
    the other coponents updating comment in cgraph.h.
    
    Finally I made the work with call site hash more effetive by updating edge
    manipulation to keep them in sequence. So first one can be looked up from the
    hash and then they can be iterated by callee.
    
    There are other things that can be improved (for example the speculation should
    start with most common target first), but I will try to keep that for next
    stage1. This patch is mostly about getting rid of ICE and profile corruption
    which is a regression from GCC 9.
    
    Honza
    
    gcc/ChangeLog:
    
    2020-01-28  Jan Hubicka  <hubicka@ucw.cz>
    
    	PR lto/93318
    	* cgraph.c (cgraph_add_edge_to_call_site_hash): Update call site
    	hash only when edge is first within the sequence.
    	(cgraph_edge::set_call_stmt): Update handling of speculative calls.
    	(symbol_table::create_edge): Do not set target_prob.
    	(cgraph_edge::remove_caller): Watch for speculative calls when updating
    	the call site hash.
    	(cgraph_edge::make_speculative): Drop target_prob parameter.
    	(cgraph_edge::speculative_call_info): Remove.
    	(cgraph_edge::first_speculative_call_target): New member function.
    	(update_call_stmt_hash_for_removing_direct_edge): New function.
    	(cgraph_edge::resolve_speculation): Rewrite to new API.
    	(cgraph_edge::speculative_call_for_target): New member function.
    	(cgraph_edge::make_direct): Rewrite to new API; fix handling of
    	multiple speculation targets.
    	(cgraph_edge::redirect_call_stmt_to_callee): Likewise; fix updating
    	of profile.
    	(verify_speculative_call): Verify that targets form an interval.
    	* cgraph.h (cgraph_edge::speculative_call_info): Remove.
    	(cgraph_edge::first_speculative_call_target): New member function.
    	(cgraph_edge::next_speculative_call_target): New member function.
    	(cgraph_edge::speculative_call_target_ref): New member function.
    	(cgraph_edge;:speculative_call_indirect_edge): New member funtion.
    	(cgraph_edge): Remove target_prob.
    	* cgraphclones.c (cgraph_node::set_call_stmt_including_clones):
    	Fix handling of speculative calls.
    	* ipa-devirt.c (ipa_devirt): Fix handling of speculative cals.
    	* ipa-fnsummary.c (analyze_function_body): Likewise.
    	* ipa-inline.c (speculation_useful_p): Use new speculative call API.
    	* ipa-profile.c (dump_histogram): Fix formating.
    	(ipa_profile_generate_summary): Watch for overflows.
    	(ipa_profile): Do not require probablity to be 1/2; update to new API.
    	* ipa-prop.c (ipa_make_edge_direct_to_target): Update to new API.
    	(update_indirect_edges_after_inlining): Update to new API.
    	* ipa-utils.c (ipa_merge_profiles): Rewrite merging of speculative call
    	profiles.
    	* profile-count.h: (profile_probability::adjusted): New.
    	* tree-inline.c (copy_bb): Update to new speculative call API; fix
    	updating of profile.
    	* value-prof.c (gimple_ic_transform): Rename to ...
    	(dump_ic_profile): ... this one; update dumping.
    	(stream_in_histogram_value): Fix formating.
    	(gimple_value_profile_transformations): Update.
    
    gcc/testsuite/ChangeLog:
    
    2020-01-28  Jan Hubicka  <hubicka@ucw.cz>
    
    	* g++.dg/tree-prof/indir-call-prof.C: Update template.
    	* gcc.dg/tree-prof/crossmodule-indircall-1.c: Add more targets.
    	* gcc.dg/tree-prof/crossmodule-indircall-1a.c: Add more targets.
    	* gcc.dg/tree-prof/indir-call-prof.c: Update template.

Diff:
---
 gcc/ChangeLog           | 46 ++++++++++++++++++++++++++++++++++++++++++++++
 gcc/testsuite/ChangeLog |  7 +++++++
 2 files changed, 53 insertions(+)

diff --git a/gcc/ChangeLog b/gcc/ChangeLog
index 05f3b72..d124c4f 100644
--- a/gcc/ChangeLog
+++ b/gcc/ChangeLog
@@ -1,3 +1,49 @@
+2020-01-28  Jan Hubicka  <hubicka@ucw.cz>
+
+	PR lto/93318	
+	* cgraph.c (cgraph_add_edge_to_call_site_hash): Update call site
+	hash only when edge is first within the sequence.
+	(cgraph_edge::set_call_stmt): Update handling of speculative calls.
+	(symbol_table::create_edge): Do not set target_prob.
+	(cgraph_edge::remove_caller): Watch for speculative calls when updating
+	the call site hash.
+	(cgraph_edge::make_speculative): Drop target_prob parameter.
+	(cgraph_edge::speculative_call_info): Remove.
+	(cgraph_edge::first_speculative_call_target): New member function.
+	(update_call_stmt_hash_for_removing_direct_edge): New function.
+	(cgraph_edge::resolve_speculation): Rewrite to new API.
+	(cgraph_edge::speculative_call_for_target): New member function.
+	(cgraph_edge::make_direct): Rewrite to new API; fix handling of
+	multiple speculation targets.
+	(cgraph_edge::redirect_call_stmt_to_callee): Likewise; fix updating
+	of profile.
+	(verify_speculative_call): Verify that targets form an interval.
+	* cgraph.h (cgraph_edge::speculative_call_info): Remove.
+	(cgraph_edge::first_speculative_call_target): New member function.
+	(cgraph_edge::next_speculative_call_target): New member function.
+	(cgraph_edge::speculative_call_target_ref): New member function.
+	(cgraph_edge;:speculative_call_indirect_edge): New member funtion.
+	(cgraph_edge): Remove target_prob.
+	* cgraphclones.c (cgraph_node::set_call_stmt_including_clones):
+	Fix handling of speculative calls.
+	* ipa-devirt.c (ipa_devirt): Fix handling of speculative cals.
+	* ipa-fnsummary.c (analyze_function_body): Likewise.
+	* ipa-inline.c (speculation_useful_p): Use new speculative call API.
+	* ipa-profile.c (dump_histogram): Fix formating.
+	(ipa_profile_generate_summary): Watch for overflows.
+	(ipa_profile): Do not require probablity to be 1/2; update to new API.
+	* ipa-prop.c (ipa_make_edge_direct_to_target): Update to new API.
+	(update_indirect_edges_after_inlining): Update to new API.
+	* ipa-utils.c (ipa_merge_profiles): Rewrite merging of speculative call
+	profiles.
+	* profile-count.h: (profile_probability::adjusted): New.
+	* tree-inline.c (copy_bb): Update to new speculative call API; fix
+	updating of profile.
+	* value-prof.c (gimple_ic_transform): Rename to ...
+	(dump_ic_profile): ... this one; update dumping.
+	(stream_in_histogram_value): Fix formating.
+	(gimple_value_profile_transformations): Update.
+
 2020-01-28  H.J. Lu  <hongjiu.lu@intel.com>
 
 	PR target/91461
diff --git a/gcc/testsuite/ChangeLog b/gcc/testsuite/ChangeLog
index 37ab4b9..8610164 100644
--- a/gcc/testsuite/ChangeLog
+++ b/gcc/testsuite/ChangeLog
@@ -1,3 +1,10 @@
+2020-01-28  Jan Hubicka  <hubicka@ucw.cz>
+
+	* g++.dg/tree-prof/indir-call-prof.C: Update template.
+	* gcc.dg/tree-prof/crossmodule-indircall-1.c: Add more targets.
+	* gcc.dg/tree-prof/crossmodule-indircall-1a.c: Add more targets.
+	* gcc.dg/tree-prof/indir-call-prof.c: Update template.
+
 2020-01-28  H.J. Lu  <hongjiu.lu@intel.com>
 
 	PR target/91461